You are not logged in.

#1 2021-07-02 06:01:07

LuX
Member
From: France
Registered: 2010-06-14
Posts: 79

[SOLVED] Rsnaphot and shut down

Hello!

I'm trying to automate incremental back ups of $HOME on my personal computer. Rsnaphot seems to be the right tool for that, but I'm puzzled by the scheduling. Indeed, my computer is shut down and restarted every day, often several time a day, and this will mess up all the rotation process, I presume.

Indeed, the wiki page of rsnapshot suggests to schedule daily rotation of back ups by means of a systemd service /etc/systemd/system/rsnapshot-daily.timer:

[Unit]
Description=rsnapshot daily backup

[Timer]
# 05:30 is the clock time when to start it
OnCalendar=05:30
Persistent=true
Unit=rsnapshot@daily.service

[Install]
WantedBy=timers.target

But since my computer will be down at 05:30, this shouldn't work. In addition, for speed up reason, I would like to avoid any back up at boot time (in case I turn on my computer just at 05:30 for example).

Is there a way to run automatically 'rsnapshot daily' either:
- 10 minutes after each first boot of the day, or;
- when the previous daily back up is more than 24h old, or;
- (more cumbersone, I guess) when the computer has been used 24h times since the last rotation?

Best,

Last edited by LuX (2021-07-03 15:38:15)

Offline

#2 2021-07-02 06:07:34

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 30,426
Website

Re: [SOLVED] Rsnaphot and shut down

man systemd.timer | less -p OnBootSec

Arch + dwm   •   Mercurial repos  •   Surfraw

Registered Linux User #482438

Offline

#3 2021-07-02 10:06:00

LuX
Member
From: France
Registered: 2010-06-14
Posts: 79

Re: [SOLVED] Rsnaphot and shut down

So, systemd.timer has a man page. Many thanks for pointing out!

This page gives instructions, but nevertheless a few more examples would be helpfull too.

I have created /etc/systemd/system/rsnapshot-hourly.timer with the following content. Can anyone confirm that this will:
- start the unit 10 minutes after booting (this is clear), and;
- restart it 40, 70, 100... and so on, minutes after booting (this is less clear to me)

--- EDIT : After some testing with shorter intervals of time, I can confirm that it works as expected. ---

[Unit]
Description=rsnapshot hourly backup

[Timer]
# Start first 10 minutes after booting.
OnBootSec=10min
# Re-start then every 30 minutes later, and so on.
OnUnitActiveSec=30min
Unit=rsnapshot@hourly.service

[Install]
WantedBy=timers.target

--- EDIT : New question below ---

Also, I have created /etc/systemd/system/rsnapshot-daily.timer below as in the wiki.

[Unit]
Description=rsnapshot daily backup

[Timer]
# 05:30 is the clock time when to start it
OnCalendar=05:30
Persistent=true
Unit=rsnapshot@daily.service

[Install]
WantedBy=timers.target

The man page says that "when the timer is activated, the service unit is triggered immediately if it would have been triggered at least once during the time when the timer was inactive".

After some tests, it appears that it doesn't work as I understand it: I have replaced 05:30 in "OnCalendar=05:30" by some hour HH:MM, enabled both daily and hourly services, restarted my computer a few minutes before HH:MM, and waited a few minutes after HH:MM. Nothing happened. The hourly.[0-9] directories were created and filled as expected, but the daily.0 directory was not created at HH:MM.

Before HH:MM I had:

$ systemctl status rsnapshot-daily.timer
● rsnapshot-daily.timer - rsnapshot daily backup
     Loaded: loaded (/etc/systemd/system/rsnapshot-daily.timer; enabled; vendor preset: disa>
     Active: active (waiting) since Fri 2021-07-02 15:26:21 CEST; 56s ago
    Trigger: Fri 2021-07-02 15:32:00 CEST; 4min 42s left
   Triggers: ● rsnapshot@daily.service

juil. 02 15:26:21 seth systemd[1]: Started rsnapshot daily backup.

After HH:MM I had:

$ systemctl status rsnapshot-daily.timer
● rsnapshot-daily.timer - rsnapshot daily backup
     Loaded: loaded (/etc/systemd/system/rsnapshot-daily.timer; enabled; vendor preset: disa>
     Active: active (waiting) since Fri 2021-07-02 15:26:21 CEST; 6min ago
    Trigger: Sat 2021-07-03 15:32:00 CEST; 23h left
   Triggers: ● rsnapshot@daily.service

juil. 02 15:26:21 seth systemd[1]: Started rsnapshot daily backup.

What am I missing here?

Last edited by LuX (2021-07-02 14:07:59)

Offline

#4 2021-07-02 14:18:45

willemw
Member
Registered: 2013-02-19
Posts: 116

Re: [SOLVED] Rsnaphot and shut down

systemctl list-timers

will give you some insight when timers are going to run. It will show you the last and the next timer run times.

For daily snapshots, only once a day and not running immediately on boot, do something like:

[Timer]
OnCalendar=02:00
AccuracySec=30min
Persistent=true

That is, without OnBootSec=.
AccuracySec= is there to delay the timer on boot.

Offline

#5 2021-07-02 16:08:17

LuX
Member
From: France
Registered: 2010-06-14
Posts: 79

Re: [SOLVED] Rsnaphot and shut down

"systemctl list-timers" says that rsnaphot-daily actualy ran, apparently. I don't understand why hourly.0 directory was not created, though. It is created when I run "rsnapshot daily" by hand.

Nevertheless, I will follow your advice and add "AccuracySec=MM" with various MM in rsnaphot-[daily|weekly|monthly].timer. I will use it in real conditions for some weeks (with the real schedules and directories) and see what happens.

Thank you for your help. smile

PS: I mark the discussion as SOLVED, but will unmark it if some problem arise in the next weeks.

Last edited by LuX (2021-07-02 16:10:16)

Offline

#6 2021-07-02 16:52:22

willemw
Member
Registered: 2013-02-19
Posts: 116

Re: [SOLVED] Rsnaphot and shut down

You still run the risk that a second rsnapshot instance will try to start a backup. It will fail, because rsnapshot uses a lock during a backup.

What you can do is to run all the rsnapshot intervals sequentually.

Something like this:

RSNAPSHOTDIR=...

# Reverse time order
echo "$(date +'%Y-%m-%d %H:%M'): INFO: Started rsnapshot" > "$RSNAPSHOTDIR/rsnapshot.log"
if [ "$(date +%-u)" -eq 6 ] && [ "$(date +%-d)" -le 07 ]; then
  rsnapshot.sh -c /etc/rsnapshot.conf monthly | grep -v "not present (yet), nothing to "
elif [ "$(date +%-u)" -eq 6 ]; then
  rsnapshot.sh -c /etc/rsnapshot.conf weekly  | grep -v "not present (yet), nothing to "
else
  rsnapshot.sh -c /etc/rsnapshot.conf daily
fi
echo "$(date +'%Y-%m-%d %H:%M'): INFO: Finished rsnapshot" >> "$RSNAPSHOTDIR/rsnapshot.log"

in a script invoked by a daily running service.

Offline

#7 2021-07-02 20:00:16

LuX
Member
From: France
Registered: 2010-06-14
Posts: 79

Re: [SOLVED] Rsnaphot and shut down

I don't know if it is because of a conflict between two rsnapshot instances as suggested by willemw (honestly, I don't think so, not always), but it is a matter of fact that it doesn't work. The first hourly snapshot worked, and that's all. It is supposed to increment the hourly snapshots every 30mns, but it didn't, even after 3 hours.

$ systemctl list-timers --all
NEXT                         LEFT          LAST                         PASSED       UNIT                         ACTIVATES                     
Sat 2021-07-03 00:00:00 CEST 2h 17min left Fri 2021-07-02 06:02:13 CEST 15h ago      man-db.timer                 man-db.service
Sat 2021-07-03 00:00:00 CEST 2h 17min left Fri 2021-07-02 06:02:13 CEST 15h ago      shadow.timer                 shadow.service
Sat 2021-07-03 05:30:00 CEST 7h left       Fri 2021-07-02 16:59:00 CEST 4h 43min ago rsnapshot-daily.timer        rsnapshot@daily.service
Sat 2021-07-03 18:28:30 CEST 20h left      Fri 2021-07-02 18:28:30 CEST 3h 14min ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service
n/a                          n/a           Fri 2021-07-02 18:33:30 CEST 3h 9min ago  rsnapshot-hourly.timer       rsnapshot@hourly.service

I have just uncommented the line in /etc/rsnapshot.conf which allows writing log files. I'm going to reboot and see if I can get some information from it.

@willemw: Thank you for the script. I will use it if I can't make any progress on this.

Offline

#8 2021-07-02 20:19:39

willemw
Member
Registered: 2013-02-19
Posts: 116

Re: [SOLVED] Rsnaphot and shut down

Are all the relevant "retain" lines uncommented in rsnapshot.conf?

If you, in the script, echo/printf to stdout/stderr instead of to a log file, then I think the output will be logged in journald. You can monitor the logs by, for instance:

journalctl -f -u rsnapshot-daily.service

and test by waiting for the timer or by running:

sudo systemctl start rsnapshot-daily.service

There is also /var/log/rsnapshot.

Offline

#9 2021-07-02 22:27:01

LuX
Member
From: France
Registered: 2010-06-14
Posts: 79

Re: [SOLVED] Rsnaphot and shut down

@willemw: Yes, the retain lines are uncommented.

$ cat /etc/rsnapshot.conf | grep retain
retain    hourly    9
retain    daily    7
retain    weekly    4
retain    monthly    9

Nothing relevant was written into /var/log/rsnapshot. It ends after the first (and only) hourly rsnapshot without reporting any error:

[2021-07-02T22:25:49] /usr/bin/rsync -a --safe-links --one-file-system --delete --numeric-ids --relative --delete-excluded --exclude=/*.iso --exclude=/**/.cache/* --exclude=/**/.thumbnails/* --exclude=/**/Trash/* --link-dest=/mnt/tera1/snapshots/hourly.1/arch/ /etc/ /mnt/tera1/snapshots/hourly.0/arch/
[2021-07-02T22:25:50] touch /mnt/tera1/snapshots/hourly.0/
[2021-07-02T22:25:50] rm -f /var/run/rsnapshot.pid
[2021-07-02T22:25:50] /usr/bin/rsnapshot hourly: completed successfully

On the other hand I have the following warning:

$ systemctl status rsnapshot-hourly.timer
● rsnapshot-hourly.timer - rsnapshot hourly backup
     Loaded: loaded (/etc/systemd/system/rsnapshot-hourly.timer; enabled; vendor preset: disabled)
     Active: active (elapsed) since Fri 2021-07-02 22:05:23 CEST; 2h 6min ago
    Trigger: n/a
   Triggers: ● rsnapshot@hourly.service

juil. 02 22:05:23 seth systemd[1]: Started rsnapshot hourly backup.
juil. 02 22:59:57 seth systemd[1]: /etc/systemd/system/rsnapshot-hourly.timer:1: Assignment outside of section. Ignoring.
juil. 02 22:59:57 seth systemd[1]: /etc/systemd/system/rsnapshot-hourly.timer:10: Failed to parse timer value, ignoring: 30mn
juil. 03 00:12:12 seth systemd[1]: /etc/systemd/system/rsnapshot-hourly.timer:1: Assignment outside of section. Ignoring.

Nevertheless, running "rsnapshot hourly" after that does the job smoothly (which proves that /etc/rsnapshot.conf is fine).

Any suggestion?

Offline

#10 2021-07-03 05:59:18

willemw
Member
Registered: 2013-02-19
Posts: 116

Re: [SOLVED] Rsnaphot and shut down

Could try increasing the "verbose" level in rsnapshot.conf.

Should be 30min. (Check: systemd-analyze timespan 30mn).

It looks like the lines in the .timer files are not all inside a section or correct section ([Unit], [Timer], ...).

Note: the "started"/"finished" echo statements in my if-then-else example are only additional logging for convenience. They can be left out. The output of rsnapshot will still be logged.

Offline

#11 2021-07-03 10:31:40

LuX
Member
From: France
Registered: 2010-06-14
Posts: 79

Re: [SOLVED] Rsnaphot and shut down

I have removed all the *.timer files and the previous snapshots, to restart from scratch without OnBoot=, OnUnitActiveSec= and AccuracySec= lines in the *.timer files.

The *[daily|weekly|monthly].timer files are the same as in the wiki. The *hourly.timer file is almost the same, but there are 3 OnCalendar= lines:

$ cat /etc/systemd/system/rsnapshot-hourly.timer
/etc/systemd/system/rsnapshot-hourly.timer

[Unit]
Description=rsnapshot hourly backup

[Timer]
# Se lance toutes les 20mn.
OnCalendar=*-*-* *:00:00
OnCalendar=*-*-* *:20:00
OnCalendar=*-*-* *:40:00
Persistent=true
Unit=rsnapshot@hourly.service

[Install]
WantedBy=timers.target

It works like a charm: hourly.[0-9] directories are created/rotated every 20minutes (at HH:00, HH:20, HH:40). I don't know yet if the daily.[0-9] directories will be created too, but I'm optimistic.

It seems that these OnBoot=, OnUnitActiveSec= and AccuracySec= lines were confusing, although I can't see any reason why.

I still have this error, though:

$ systemctl status rsnapshot-hourly.timer
* rsnapshot-hourly.timer - rsnapshot hourly backup
     Loaded: loaded (/etc/systemd/system/rsnapshot-hourly.timer; enabled; vendor preset: disabled)
     Active: active (waiting) since Sat 2021-07-03 10:36:26 CEST; 1h 40min ago
    Trigger: Sat 2021-07-03 12:20:00 CEST; 3min 16s left
   Triggers: * rsnapshot@hourly.service

Jul 03 10:36:26 seth systemd[1]: Started rsnapshot hourly backup.
Jul 03 10:37:42 seth systemd[1]: /etc/systemd/system/rsnapshot-hourly.timer:1: Assignment outside of section. Ignoring.

Any idea?

willemw wrote:

It looks like the lines in the .timer files are not all inside a section or correct section ([Unit], [Timer], ...).

Could you tell me if my *hourly.timer file above is correct?

willemw wrote:

Could try increasing the "verbose" level in rsnapshot.conf.

Should be 30min. (Check: systemd-analyze timespan 30mn).

I have set the verbosity level to 5. The /var/log/rsnapshot file is now 96390 lines long, but doesn't report any warning or error. The systemd-analyse command doesn't work.

$ LANG=C ; systemd-analyze timespan 30mn
Failed to parse time span '30mn': Invalid argument

I am tempted to keep it as is, if the daily rotation works tomorrow. Anyway, if sombody can explain the errors I had previously with the OnBoot= and OnUnitActiveSec= lines, I would be interrested.

Note: The "Automation" section of the wiki which gives these working examples of *[hourly|daily|weekly|monthly].timer files is marked as "needing language" because "Too much code for an article". I disagree: such examples are pretty much useful, and adding more working examples for different situations would even be better. A reference to the man page of systemd.timer could be usefull too.

Offline

#12 2021-07-03 11:39:58

willemw
Member
Registered: 2013-02-19
Posts: 116

Re: [SOLVED] Rsnaphot and shut down

Wow, so many timers in that wiki and only for one program. I only have only 3 custom timers (and 3 scripts) I made myself on my systems: boot.timer, hourly.timer and daily.timer. The daily script also does weekly and monthly tasks. Of course I have more timers running.

/etc/systemd/system/rsnapshot-hourly.timer:1

means line 1 is incorrect. Remove that line. That "/etc/systemd/system/rsnapshot-hourly.timer" is the header of that code section (in BBCode), it is not part of the file content. In HTML you cannot see the difference between the header and the content of a code section.

Change 30mn to 30min or 30m.

I think you can reduce all the OnCalendar= lines to: OnCalendar=*-*-* *:*/20:00

*/20 meaning every 20 minutes.

See https://www.freedesktop.org/software/sy … .time.html for more time examples.

Never used OnUnitActiveSec=. See my [Timer] example above. Used that for years without problems.

My hourly timer is something like this (actually runs once every 2 hours):

[Unit]
Description=Run hourly
After=boot.timer

[Timer]
# Every two hours
#OnCalendar=00/2:00:00

OnCalendar=hourly
AccuracySec=30min
Persistent=true

[Install]
WantedBy=timers.target

Starting from scratch, I would first only test the .service file (sudo systemctl start ...service) and make that working. Then enable 1 timer and make that working.

Offline

#13 2021-07-03 12:54:16

LuX
Member
From: France
Registered: 2010-06-14
Posts: 79

Re: [SOLVED] Rsnaphot and shut down

willemw wrote:

Wow, so many timers in that wiki and only for one program. I only have only 3 custom timers (and 3 scripts) I made myself on my systems: boot.timer, hourly.timer and daily.timer. The daily script also does weekly and monthly tasks.

Many timers for one program, that's true (although 4 is not really a large number...), but rsnapshot is a program which needs several timers by essence. So this section of the wiki seems consistent to me (and helpfull).

On the other hand, this section might be reduced if a reference to the wiki page dedicated to systemd timers would be added. This would be a good idea anyway: it is the first time I'm using timers, only because they are required by rsnapshot, so I didn't imagine that timers could be a subject in itself, with it's own wiki page and man page, until now. I guess that other newbies may think the same.

willemw wrote:
/etc/systemd/system/rsnapshot-hourly.timer:1

means line 1 is incorrect. Remove that line. That "/etc/systemd/system/rsnapshot-hourly.timer" is the header of that code section (in BBCode), it is not part of the file content. In HTML you cannot see the difference between the header and the content of a code section.

You're right, captain obvious! big_smile
I did not even SEE this line above [Unit] because it is not part of the structure. Funny...

Thank you very much for your help. I'll mark this topic as SOLVED.

--- EDIT July, 4th ---
Note: Even if I'm not doing anything to avoid snapshots services to run at boot time, it seems that systemd always starts them at least a few minutes after the boot. So there's no need to OnBoot= and OnUnitActiveSec= lines for me (apparently).

Not at all! With the timers given in the wiki, when I shut down my computer, and restart it, if the time of a snapshot has passed in the mean time, systemd runs this snapshot during the boot, which suspends it for a long time. No good.

So using OnBoot= is mandatory. But this option has other side effects which I would prefer to avoid (see below).

Last edited by LuX (2021-07-04 17:16:41)

Offline

#14 2021-07-03 13:34:28

willemw
Member
Registered: 2013-02-19
Posts: 116

Re: [SOLVED] Rsnaphot and shut down

It's good to learn about timers and use them. But you can also run rsnapshot from cron, the "old" way.

That multiple timer approach in the rsnapshot Wiki: it introduces, unnecessarily in my opinion, possible backup conflicts (backups in parallel).

Other backup programs I know of that provide hourly, daily, weekly, etc. backups (Btrbk, Sanoid), they only provide a single timer (and a single service) and therefore no backup conflicts from .timers files are possible. For rsnapshot, that if-then-else example I gave above is a way to avoid backup conflicts.

Offline

#15 2021-07-03 15:37:55

LuX
Member
From: France
Registered: 2010-06-14
Posts: 79

Re: [SOLVED] Rsnaphot and shut down

Hum... you are meaning that systemd won't be aware that these different timers related to rsnapshot have a hierarchy, right?

I heard that systemd is precisely a smart way to deal with several processes, possibly contestant. It would be disappointing if it cannot deal properly with several timers on the same program. But even if it is so, /etc/rsnapsot.conf has the following option, which is enabled by default. Doesn't it prevent what you're talking about?

# If enabled, rsnapshot will write a lockfile to prevent two instances
# from running simultaneously (and messing up the snapshot_root).
# If you enable this, make sure the lockfile directory is not world
# writable. Otherwise anyone can prevent the program from running.
#
lockfile        /var/run/rsnapshot.pid

Last edited by LuX (2021-07-03 15:44:13)

Offline

#16 2021-07-03 15:53:37

willemw
Member
Registered: 2013-02-19
Posts: 116

Re: [SOLVED] Rsnaphot and shut down

systemd only knows about its units (.service files, etc.), not about the rsnapshot program. Only the exit return code of a program.

I already mentioned rsnapshot lock. If there is a conflict, the second instance of rsnapshot will fail. The changes are small, I know.

For example, if your monthly backup fails and weekly backups get pruned after that, then your oldest backup may be less than a month old. Then your backups don't meet your expectations/requirements.

Offline

#17 2021-07-04 17:07:46

LuX
Member
From: France
Registered: 2010-06-14
Posts: 79

Re: [SOLVED] Rsnaphot and shut down

@willemw: You convinced me!

In the mean time I have experienced that (contrary to my first impression), with the timers in the wiki it happens sometimes (quite often, actually) that a snapshot is run during the boot, which suspends it during more than a minute. So the OnBoot= option is mandatory. But then, if I shut down and restart my computer 4 times in 20 minutes for exemple (which can happen if I have a problem, or if I'm doing some test), a hourly snapshot will be done every time I reboot, which is not the expected behaviour.

So I'm considering to write a script, call it 5 minutes after the boot with a single timer, and let it manage the various calls to rsnapshot hourly|daily|weekly|monthly (while doing several tests, not only on the current time, but also on the timestamp of the last snapshots, which I couldn't do only with timers). This is what you suggested, but the example script you gave is only a starting point for what I'm aiming at.

By the way, what is

rsnapshot.sh -c /etc/rsnapshot.conf monthly

in your script? Shouldn't it be

/usr/bin/rsnapshot monthly

instead?

Last edited by LuX (2021-07-04 17:13:58)

Offline

#18 2021-07-04 18:49:49

willemw
Member
Registered: 2013-02-19
Posts: 116

Re: [SOLVED] Rsnaphot and shut down

The automation example in the rsnapshot Wiki is fine, I guess, but not ideal. Maybe run conflicts can be avoided with only using .timer and .service files (After=, Before=, ...).



What is suspended for more than one minute?

Are you going to add yet another timer (5min after boot)? And check the rsnapshot timestamps? Sounds a bit overly complex.

How do you know a backup completed successfully and was not interrupted? Are you going to test that as well?

Create the backup manually the first time (systemctl start), so that subsequent backups will take less time.

I only do daily, weekly, monthly backups with a script that runs uninterrupted once a day during the night. No hourly backups or backups immediately after boot.

But I do have other hourly tasks (with a similar if-then-else statement as above). If a (single) task should not start immediately after boot, then I currently solve that by putting a "sleep 2m" before running that task. This is done inside a script. I am not adding any timers. You could maybe do the same.

There is one issue. I have "After=boot.timer" in my hourly.timer and my boot.timer is something like:

[Unit]
Description=Run at boot up

[Timer]
OnBootSec=4min
#AccuracySec=1min

Persistent=true

[Install]
WantedBy=timers.target

so in theory that "sleep 2m" should not be necessary because of the "After=boot.timer", I would have thought. The first hourly service run should not start before 4 minutes after boot. Let my know if you find a solution without adding another timer.

Note that only boot.timer has "OnBootSec=". The other timers should not have that, otherwise they will run at boot time (https://www.freedesktop.org/software/sy … timer.html).

"rsnapshot.sh -c /etc/rsnapshot.conf monthly" and "/usr/bin/rsnapshot monthly" are the same. Actually, I have "rsnapshot.sh -c /etc/rsnapshot-checksum.conf", which contains an extra "--checksum" on the "rsync_long_args" line.



Another thing. On the Arch rsnapshot Wiki: why is there "IOSchedulingClass=idle"?

IOPRIO_CLASS_IDLE (3)
This is the idle scheduling class.  Processes running at
this level get I/O time only when no one else needs the
disk.  The idle class has no class data.  Attention is
required when assigning this priority class to a process,
since it may become starved if higher priority processes
are constantly accessing the disk.

Is this not bad if you want do to something important like save your data to disk?

BTW, most of my backups are now on Btrfs and ZFS (snapshots and send/receive replication).

Last edited by willemw (2021-07-04 18:51:48)

Offline

#19 2021-07-05 08:32:16

LuX
Member
From: France
Registered: 2010-06-14
Posts: 79

Re: [SOLVED] Rsnaphot and shut down

willemw wrote:

Let my know if you find a solution without adding another timer.

I'll post my script here as soon as it is finished (and tested).

The point is to avoid all the previous mentioned problems:
- No rsnapshot during the boot.
- No conflict between rsnsapshot instances.
- No reduced time intervals between snapshots in case of frequent reboots.

The script is working, and does not require any timer at all. But I still need to improve it a little bit.

willemw wrote:

Another thing. On the Arch rsnapshot Wiki: why is there "IOSchedulingClass=idle"?

I have no idea on this.

But I do have another question. My script is run on boot as a service. It will later call rsnapshot repeatedly (and sequentially, so no more conflict), not as a service, but as a command: '/usr/bin/rsnapshot hourly|daily|etc'. What happens if I shutdown the computer while rsnapshot is running? I guess that it could mess up the snapshots. How can I teach systemd to delay the shut down until the current instance of rsnapshot is finished?

willemw wrote:

BTW, most of my backups are now on Btrfs and ZFS (snapshots and send/receive replication).

I did't know about Btrfs and ZFS. This Bonwick's blog article comparing the (non-)limitations of ZFS with the mass/energy of the ocean, is amazing. lol

Offline

#20 2021-07-05 09:07:51

LuX
Member
From: France
Registered: 2010-06-14
Posts: 79

Re: [SOLVED] Rsnaphot and shut down

For memory, running rsnapshot with just the timers of the wiki can cause some delay at boot but also some errors. When I turned on my PC this morning, it must have tried to run rsnapshot-daily service for the first time, which caused a long delay, after which the following error message (still readable on tty1) was written.

Starting version 248.3-2-arch
arch: clean, 306306/1281120 files, 2483191/512000 blocks
[ TIME ] Timed out waiting for device /dev/sdb4.
[DEPEND] Dependency failed for /dev/sdb4.
[DEPEND] Dependency failed for Swap.
[FAILED] Failed to start rsnapshot (daily) backup.

The daily.0 directory is not yet created (although it should be), which confirms the above message.

This is to say that some work-around is needed (using OnBoot= or other options in timers), if not using a devoted script as you suggested.

Offline

#21 2021-07-05 09:43:15

willemw
Member
Registered: 2013-02-19
Posts: 116

Re: [SOLVED] Rsnaphot and shut down

I would set it to IOSchedulingClass=low or not set it at all.

No hourly backups avoid most of these issues. The only backups I do during the day are manual backups and by a pacman hook which runs before an install, etc.



Ext4 is fine.

Btrfs and ZFS, they checksum all the files. With other filesystem, your files may be corrupt and you won't know about it. Your backups will also contain the same corrupt files.

Also, with Btrfs and ZFS, you don't have to plan your partition sizes beforehand.

For backups with a large amount of (small) files, Btrfs/ZFS snapshots and send/receive replication are so much better. Must faster and far less harddisk head seeking (movements). Also less (network) I/O.



I think rsnapshot creates the new hourly.0, daily.0, etc. snapshot only after a successful backup. Not sure which folder is the one used during backup. But there are different scenarios (local filesystem, rsync), so maybe don't rely on this fact.

Have not used it because I only do nightly backups, but there is also "rsnapshot sync" and the "sync_first" option to stage the current backup: "This allows better recovery in the event that rsnapshot is interrupted in the middle of a sync operation, since the sync step and rotation steps are separated."

Or maybe making the lockfile persistent will help recognizing incomplete backups: lockfile /var/tmp/rsnapshot.pid

Offline

#22 2021-07-05 09:44:18

willemw
Member
Registered: 2013-02-19
Posts: 116

Re: [SOLVED] Rsnaphot and shut down

Have you added path conditions to your .service file? That's not mentioned in the rsnapshot Wiki. You could do that. Or do before calling rsnapshot:

DIR=...
if ! mountpoint --quiet "$DIR"; then
  ...
  return
fi

Don't use /dev/sd*. It can change after a reboot.

Offline

#23 2021-07-05 12:15:04

LuX
Member
From: France
Registered: 2010-06-14
Posts: 79

Re: [SOLVED] Rsnaphot and shut down

I'm answering to myself.

LuX wrote:

How can I teach systemd to delay the shut down until the current instance of rsnapshot is finished?

Following this discussion, systemd uses to wait for services to be stopped before shutting down. Hence it suffices to call rsnapshot as a service instead of a command (e.g. 'systemd start rsnapshot@hourly' instead of '/usr/bin/rsnapshot hourly). This is a good reason to do it, contrary to what I thought first.

Of course this won't help in case of hard reboot, but nothing can be done in that case, I'm afraid ('rsnapshot rsync' may reduce the damage, but probably not fully avoid it).

Last edited by LuX (2021-07-05 22:31:32)

Offline

#24 2021-07-05 12:25:29

willemw
Member
Registered: 2013-02-19
Posts: 116

Re: [SOLVED] Rsnaphot and shut down

No, you can only delay shutdown. Have you ever seen "stop jobs" during shutdown? Services will be killed after a timeout when shutting down.

/etc/systemd/system.conf

#DefaultTimeoutStopSec=90s

Offline

#25 2021-07-05 12:30:39

willemw
Member
Registered: 2013-02-19
Posts: 116

Re: [SOLVED] Rsnaphot and shut down

However, I have seen with ZFS that a service can hang forever.

Offline

Board footer

Powered by FluxBB