You are not logged in.

#1 2013-10-30 06:10:53

ngoonee
Forum Fellow
From: Between Thailand and Singapore
Registered: 2009-03-17
Posts: 7,336

Rsync or other backup solution- single source multiple target?

Seems from http://www.linuxquestions.org/questions … ly-774225/ that I can't do what I want, but I thought I'd ask as well.

My laptop hard disc is going to be backed up to two external hard discs (regular script). What I do right now is run two separate rsync commands, but of course this means that the data (all 300+ GB of it, currently) needs to be read twice. Seems it'd be much more efficient if the data were read once and then written to both external hard discs.

It doesn't seem from the above link and other google resources that this is possible. Can anyone suggest an alternative short of scripting a file-by-file cache?

Last edited by ngoonee (2013-10-31 01:33:22)


Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.

Offline

#2 2013-10-30 06:15:47

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 30,192
Website

Re: Rsync or other backup solution- single source multiple target?

Have you looked at rsync running in batch mode? It sounds like what you are after:

man rsync wrote:

       Generating  the  batch  file once saves having to perform the file status, checksum, and data block generation more than once when updating multiple
       destination trees. Multicast transport protocols can be used to transfer the batch update files in parallel to many hosts at once, instead of  send‐
       ing the same data to every host individually.

That section is pretty detailed and should give you enough to test for your setup.


Arch + dwm   •   Mercurial repos  •   Surfraw

Registered Linux User #482438

Offline

#3 2013-10-30 09:58:14

ngoonee
Forum Fellow
From: Between Thailand and Singapore
Registered: 2009-03-17
Posts: 7,336

Re: Rsync or other backup solution- single source multiple target?

Why yes, I believe that does sound like exactly what's needed. Thanks jason smile


Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.

Offline

#4 2013-10-31 01:34:47

ngoonee
Forum Fellow
From: Between Thailand and Singapore
Registered: 2009-03-17
Posts: 7,336

Re: Rsync or other backup solution- single source multiple target?

Hmm, looking through it it only seems to work if both backup directories are exact copies (of each other). However I'm looking to have one of them (on the larger hard disc) have history as well (using --backup-dir) and the other not to.

Will have to test it out though. Still running the initial backup, currently....


Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.

Offline

#5 2013-10-31 03:34:14

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 6,238

Re: Rsync or other backup solution- single source multiple target?

Doesn't this partially undermine the point of doing two backups? I realise one is off site and that they are on separate devices which rules out some sources of corruption/loss but if you run the backup as you wish and something bad happens to the source during the backup, you are likely to end up with three corrupted copies rather than only two.

I don't manage to do this as I only have one complete backup but I thought that best practice involved not touching one backup while the other backup was being created (or restored or...). That way, you always have one "known good" backup whatever happens.

EDIT: not "whatever happens" but "whichever of any of a larger number of possible catastrophes occurs". Obviously, the third copy could be on a device that dies or explodes or gets drowned by a peeved goldfish at just that moment when mice eat your source during a backup, thus corrupting your other two copies. But you can only plan so far...

Last edited by cfr (2013-10-31 03:38:24)


How To Ask Questions The Smart Way | Help Vampires

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Corporation Wireless 8265 / 8275 | US keyboard with Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#6 2013-10-31 06:30:41

ngoonee
Forum Fellow
From: Between Thailand and Singapore
Registered: 2009-03-17
Posts: 7,336

Re: Rsync or other backup solution- single source multiple target?

cfr wrote:

Doesn't this partially undermine the point of doing two backups? I realise one is off site and that they are on separate devices which rules out some sources of corruption/loss but if you run the backup as you wish and something bad happens to the source during the backup, you are likely to end up with three corrupted copies rather than only two.

I don't manage to do this as I only have one complete backup but I thought that best practice involved not touching one backup while the other backup was being created (or restored or...). That way, you always have one "known good" backup whatever happens.

EDIT: not "whatever happens" but "whichever of any of a larger number of possible catastrophes occurs". Obviously, the third copy could be on a device that dies or explodes or gets drowned by a peeved goldfish at just that moment when mice eat your source during a backup, thus corrupting your other two copies. But you can only plan so far...

This isn't my complete backup solution. I have two hard discs in one location (what I'm asking about here) and another one at a different location. The reason I have two is simply because I have a spare which doesn't have any other use, and it may as well be put to use in this way.

Besides, if something happens to the source, using --backup-dir means I'll still have the last known good copy anyway (not taking into account the backup which is at the different location.


Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.

Offline

#7 2013-11-07 10:21:57

Panzerino
Member
Registered: 2011-08-17
Posts: 9

Re: Rsync or other backup solution- single source multiple target?

I use Synkron from AUR. It is very good for me for simple backup.

Last edited by Panzerino (2013-11-07 10:22:50)

Offline

#8 2013-11-09 10:01:58

ngoonee
Forum Fellow
From: Between Thailand and Singapore
Registered: 2009-03-17
Posts: 7,336

Re: Rsync or other backup solution- single source multiple target?

Panzerino wrote:

I use Synkron from AUR. It is very good for me for simple backup.

While it is good that you want to help, you don't seem to understand the question, and simple 'I use program X' doesn't generally help anyway.


Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.

Offline

Board footer

Powered by FluxBB