rsync: how to restore diff backup?

XxTriviumxX · 2023-04-04 17:33:18

Hi! This is a question about rsync i've been asking myself since yesterday.. I tried to find an answer, but i cant find a direct answer. Maybe im just blind or something..

I'm currently doing monthly system backups of my personal system like so:

sudo rsync -aAXH --info=progress2 --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found","/swapfile","/home/user/.local/share/Trash/*"} / /path/to/external/drive/april

And if my system fails badly, i restore it by rotating the paths:

sudo rsync -aAXH --info=progress2 --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found","/swapfile","/home/user/.local/share/Trash/*"} /path/to/external/drive/april /broken/linux/restore/point

Tested it multiple times and it rocks!

Now i want to do weekly backups that won't take as much space as the montly backup..like a differencial backup...

from my understanding i can use "--compare-dest=/path/to/external/drive/april" to compare the current state of my system against the april backup... (correct me if i'm wrong)

that would make mean these commands can be used to achieve this form of backup: (again, correct me if im wrong)

sudo mkdir /path/to/external/drive/diff1

sudo rsync -aAXH --info=progress2 --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found","/swapfile","/home/user/.local/share/Trash/*"} --compare-dest=/path/to/external/drive/april / /path/to/external/drive/diff1

The diff1 directory would be smaller than the april directory...right?
What i've been trying to find is... how do I restore my system using what's been saved in the diff1 directory? What are the flags to use? From that i read, i have to restore the april backup and then restore the content of diff1 and then... i can't seem to find a tutorial that shows me the flags or the entire command to use..

Thanks

Last edited by XxTriviumxX (2023-05-04 16:08:12)

graysky · 2023-04-04 17:55:00

Doesn't specifically answer your question: suggest you use a util like backintime in the AUR. Nice GUI and incremental backups.

XxTriviumxX · 2023-04-04 18:09:28

graysky wrote:

Doesn't specifically answer your question: suggest you use a util like backintime in the AUR. Nice GUI and incremental backups.

Thanks for the reply! That's actually my plan B!! It looks cool, but i'd rather be the guy who says "i backup my system monthly and weekly with rsync" LOL

Last edited by XxTriviumxX (2023-04-04 18:28:02)

Trilby · 2023-04-04 20:34:31

I wasn't familiar with the --compare-dest option, I believe it's new, and it seems awkward and not very useful to me. Traditionally one would do incremental backups with the --link-dest option so that ever "new" backup directory includes every file, but any of them that have not changed will not take up any disk space as they are just hard-links to the existing content.

Then restoring from one of those incrememental backups is identical to restoring from any (other) full backup. I imagine with --compare-dest you'd have to do two restoration steps: one from the initial backup then another from the diff content.

When I was doing good / reliable backups, I just did an incremental (via links) every day in effect getting the same results as the Mac "time machine" and likely what "back in time" does: you can then restore to any previous date yet most of those backups don't take up much space.

Last edited by Trilby (2023-04-04 20:38:14)

XxTriviumxX · 2023-04-05 17:13:52

Thanks for the info! From what i read, if i backup (diff/incr) my system with this command:

sudo rsync -aAXH --info=progress2 --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found","/swapfile","/home/user/.local/share/Trash/*"} --link-dest=/path/to/external/drive/april / /path/to/external/drive/diff1

I would have to use the following command in order to restore the diff1 backup right?

sudo rsync -aAXH --info=progress2 --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found","/swapfile","/home/user/.local/share/Trash/*"} --link-dest=/path/to/external/drive/april /path/to/external/drive/diff1 /path/to/restoration/point

Can you let me know if the restoration command makes sense?

Last edited by XxTriviumxX (2023-05-04 16:08:27)

Trilby · 2023-04-05 18:13:44

You shouldn't need the link-dest argument or /drive/april path - that's the whole point. The /drive/diff1 directory would contain (a link to) every file and be complete on it's own.

XxTriviumxX · 2023-04-06 15:06:36

Thanks Trilby!

For those curious i'm just gonna drop my restore command here:

sudo rsync -aAXH --info=progress2 --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found","/swapfile","/home/user/.local/share/Trash/*"}  /path/to/external/drive/diff1 /path/to/restoration/point

I have another question about rsync. It's about the "--delete" flag.

Let's say that I have file1 in my monthly backup i created using the command mentioned earlier.

I then create file2 and I delete file1 (on purpose, as if i hate file1) before doing a backup with "--link-dest".

After i backup with "--link-dest", if i have to restore that backup, will I get both file1 and file2 restored, or do i need a "--delete" argurment to get rid of file1 from the "--link-dest" backup restoration process?

Last edited by XxTriviumxX (2023-05-04 16:08:37)

Trilby · 2023-04-06 20:18:58

First, there's also no point in the --exclude parameter(s) in the restore command. Those paths don't exist in the backup, so they can't be excluded. It's harmless, but it makes a very short and simple command look long and complicated.

For your most recent question, I'm not sure I understand. The process you describe would result in file1 not even being present in the most recent backup - so of course it wouldn't be restored if you restored from that backup.

XxTriviumxX · 2023-04-07 01:07:57

Trilby wrote:

First, there's also no point in the --exclude parameter(s) in the restore command. Those paths don't exist in the backup, so they can't be excluded. It's harmless, but it makes a very short and simple command look long and complicated.

I followed the wiki literally on that one!

https://wiki.archlinux.org/title/rsync#Restore_a_backup

"If you wish to restore a backup, use the same rsync command that was executed but with the source and destination reversed."

Glad to hear i can just get rid of the exclude section!

Trilby wrote:

For your most recent question, I'm not sure I understand. The process you describe would result in file1 not even being present in the most recent backup - so of course it wouldn't be restored if you restored from that backup.

That right there just answered my question! File1 will remain deleted as i wished in the most recent backup. But it said in the rsync wiki page section 3.1 that "--delete: means files deleted on the source are to be deleted on the backup as well".... so thats why i was confused. It's useful mostly for running the backup multiple times in the same directory, as stated near the end of section 3.7.

Thanks for all those pointers Trilby! I hope this will help anyone interested in backing up their system with rsync. Really amazing tool! I also have the impression that rsync is more stable than say, backintime.. the reason being that backintime is more complex than rsync...i may be wrong though.

Last edited by XxTriviumxX (2023-04-07 01:08:17)

Trilby · 2023-04-07 04:15:35

backintime is just a front-end for rsync - it gives you a gui to create an rsync command that it runs behind the scenes.

Arch Linux

#1 2023-04-04 17:33:18

rsync: how to restore diff backup?

#2 2023-04-04 17:55:00

Re: rsync: how to restore diff backup?

#3 2023-04-04 18:09:28

Re: rsync: how to restore diff backup?

#4 2023-04-04 20:34:31

Re: rsync: how to restore diff backup?

#5 2023-04-05 17:13:52

Re: rsync: how to restore diff backup?

#6 2023-04-05 18:13:44

Re: rsync: how to restore diff backup?

#7 2023-04-06 15:06:36

Re: rsync: how to restore diff backup?

#8 2023-04-06 20:18:58

Re: rsync: how to restore diff backup?

#9 2023-04-07 01:07:57

Re: rsync: how to restore diff backup?

#10 2023-04-07 04:15:35

Re: rsync: how to restore diff backup?

Board footer