Scheduling period job using systemd

cmonty14 · 2024-07-10 10:08:44

Hello,
I have a backup job (using rsnapshot) that must be executed daily, weekly and monthly.
Therefore I created these systemd-timers:

# /etc/systemd/system/rsnapshot-daily.timer
[Unit]
Description=rsnapshot daily backup

[Timer]
OnCalendar=daily
RandomizedDelaySec=10m
Persistent=true
Unit=rsnapshot@daily.service

[Install]
WantedBy=timers.target

# /etc/systemd/system/rsnapshot-weekly.timer
[Unit]
Description=rsnapshot weekly backup

[Timer]
OnCalendar=weekly
RandomizedDelaySec=10m
Persistent=true
Unit=rsnapshot@weekly.service

[Install]
WantedBy=timers.target

# /etc/systemd/system/rsnapshot-monthly.timer
[Unit]
Description=rsnapshot monthly backup

[Timer]
OnCalendar=monthly
RandomizedDelaySec=10m
Persistent=true
Unit=rsnapshot@monthly.service

[Install]
WantedBy=timers.target

On 07/01/2024 all 3 jobs have been triggered in parallel.
However, rsnapshot can run only once, and therefore 2 jobs have failed (rsnapshot creates a lockfile /var/run/rsnapshot.pid).

This means, one must ensure that these scheduled jobs run sequentially in this order:
monthly - weekly - daily

Can you please advise how to modify the relevant systemd-timer config file to ensure the sequential job execution?

THX

dakota · 2024-07-10 10:30:22

What if you specified the hour-of-day for each to run, and not simply daily/weekly/monthly?

Cheers,

bulletmark · 2024-07-10 10:51:50

Just use one timer to run a simple custom script which then runs each of the 3 rsnapshot commands in sequence.

cmonty14 · 2024-07-10 11:41:21

bulletmark wrote:

Just use one timer to run a simple custom script which then runs each of the 3 rsnapshot commands in sequence.

This is certainly not the solution because I don't want to run 3 snapshot commands in sequence every day.

cmonty14 · 2024-07-10 11:42:58

dakota wrote:

What if you specified the hour-of-day for each to run, and not simply daily/weekly/monthly?
Cheers,

This is a potential solution but only if I know the runtime of each rsnapshot run.
If the forecast is incorrect there's still a probability that this would fail, too.

bulletmark · 2024-07-10 13:30:05

cmonty14 wrote:

This is certainly not the solution because I don't want to run 3 snapshot commands in sequence every day.

Perhaps I wasn't explicit enough. I mean your script would run the daily job every day, plus the weekly job each week, etc.

I had a similar cron script running my rsnapshot backup for many years (but many years ago because there are much better solutions than rsnapshot nowdays).

seth · 2024-07-10 15:01:06

cmonty14 wrote:

bulletmark wrote:
Just use one timer to run a simple custom script which then runs each of the 3 rsnapshot commands in sequence.
This is certainly not the solution because I don't want to run 3 snapshot commands in sequence every day.

It's either that or you add something like an inotifywait to your script to wait until the pending rsnapshot jobs disappear.
The run of a month and a week tend to happen [checks calender… yup, checks out] on a day.

Trilby · 2024-07-10 15:12:37

Do you even want three (nearly*) identical snapshots on those days? I'm not familiar with rsnapshot itself, but when I was just using rsync for backups it was easy to keep daily backups, weekly, and monthly, where each of the weekly or monthly directories were just hard links to the appropriate daily. This way "dailies" of a certain age could simply be deleted, but the weekly and monthly would remain because there was another hardlink to that data.

(*they should be identical, but if you run three backups in sequence there may be changes between them which would also be a bit logically odd: the monthly backup for Jan 1 could have different content than the daily backup for Jan 1!)

Alternatively, if you are backing up different content at different intervals, then stick with what you are doing, but use a different pid lock file (I suspect rsnapshot would have an optional flag for this - every bit of software that I've seen that uses lock files does).

Last edited by Trilby (2024-07-10 15:15:24)

cmonty14 · 2024-07-12 06:56:20

bulletmark wrote:

cmonty14 wrote:
This is certainly not the solution because I don't want to run 3 snapshot commands in sequence every day.
Perhaps I wasn't explicit enough. I mean your script would run the daily job every day, plus the weekly job each week, etc.
I had a similar cron script running my rsnapshot backup for many years (but many years ago because there are much better solutions than rsnapshot nowdays).

Do you mind sharing some info about better solutions than rsnapshot?

cmonty14 · 2024-07-12 07:02:47

seth wrote:

cmonty14 wrote:
bulletmark wrote:
Just use one timer to run a simple custom script which then runs each of the 3 rsnapshot commands in sequence.
This is certainly not the solution because I don't want to run 3 snapshot commands in sequence every day.
It's either that or you add something like an inotifywait to your script to wait until the pending rsnapshot jobs disappear.
The run of a month and a week tend to happen [checks calender… yup, checks out] on a day.

There's no script (or whatsoever) in this backup functionality.
It's only rsnapshot and systemd-timer involved.

seth · 2024-07-12 07:29:41

There's no script (or whatsoever) in this backup functionality.

Yes. So? That's the point.
The point is, you cannot guarantee that your daily run doesn't conflict w/ the weekly or monthly one using systemd timers because they'll fire rsnapshot and forget.
=> Whatever is invoked by the timer needs to ensure that either
a) there's no pending rsnapshot instance (otherwise wait for its termination)
b) if mulitple rsnapshot runs need to happen this day, just invoke them sequentially.

wrt (b) and Trilby's comment, you could probably just run the daily snapshot and then copy it into the weekly/monthly backup if it's the first day of a week/month? Saving the additional rsnapshot invocations (assuming they're equivalent to the daily ones)

bulletmark · 2024-07-13 02:35:42

cmonty14 wrote:

Do you mind sharing some info about better solutions than rsnapshot?

I suggest you go through the Synchronization and backup programs wiki because there are a plethora of options.

I used rsnapshot for 16 years(!) and then switched to borg for the last 6.5 years which I evaluated as best for my needs at the time. At my home, I use systemd timers to backup 2 computers to a large disk on a mini PC server and then it also rclones the encrypted backup to Backblaze B2 every night which is really cheap cloud storage. This arrangement has worked well for me ever since so I have not reevaluated any other newer options.

Last edited by bulletmark (2024-07-13 11:17:29)

Arch Linux

#1 2024-07-10 10:08:44

Scheduling period job using systemd

#2 2024-07-10 10:30:22

Re: Scheduling period job using systemd

#3 2024-07-10 10:51:50

Re: Scheduling period job using systemd

#4 2024-07-10 11:41:21

Re: Scheduling period job using systemd

#5 2024-07-10 11:42:58

Re: Scheduling period job using systemd

#6 2024-07-10 13:30:05

Re: Scheduling period job using systemd

#7 2024-07-10 15:01:06

Re: Scheduling period job using systemd

#8 2024-07-10 15:12:37

Re: Scheduling period job using systemd

#9 2024-07-12 06:56:20

Re: Scheduling period job using systemd

#10 2024-07-12 07:02:47

Re: Scheduling period job using systemd

#11 2024-07-12 07:29:41

Re: Scheduling period job using systemd

#12 2024-07-13 02:35:42

Re: Scheduling period job using systemd

Board footer