Arch Linux

Spider.007

Member

Registered: 2004-06-20

Posts: 1,175

[fixed] rsync - not updating existing files?

I am using rsync extensively to backup various machines to a central server (all archlinux, latest versions on ext4). Yesterday I needed one of those backups, and I found out it is as good as useless, since a lot of files never got updated. Even when I enable checksumming the file is not updated. I am pretty sure I am missing something, but am not sure what it is.

The command I use:

rsync --stats --verbose -xaz --append --delete-during --delete-excluded \
        --exclude=[cC]ache/ --exclude=.thumbnails --exclude=.Trash*/ --exclude=*.mp3 \
        --exclude=qemu/ /home/user /etc \
        backup@servername:clientname/

new files get transferred fine, and some files get updated as well. But not everything; here are some examples:

-rw-r--r--  1 backup nobody   1445 Jan 11  2009 profile
-rw-r--r--  1 backup nobody    618 Jan 30  2008 profile.bash
drwxr-xr-x  2 backup nobody   4096 May  5 10:38 profile.d
-rw-r--r--  1 backup nobody   2510 Jan 11  2009 protocols
-rw-------  1 backup nobody      0 Apr  6  2008 .pwd.lock
-rw-r--r--  1 backup nobody   3304 Feb 28 11:17 rc.conf
drwxr-xr-x  3 backup nobody   4096 May 14 09:54 rc.d
-rwxr-xr-x  1 backup nobody    109 Nov 11  2005 rc.local
-rwxr-xr-x  1 backup nobody    187 Apr  6  2008 rc.local.shutdown
-rwxr-xr-x  1 backup nobody    503 Jan 24 15:35 rc.multi
-rwxr-xr-x  1 backup nobody   3976 May  8 12:51 rc.shutdown
-rwxr-xr-x  1 backup nobody   1970 Jan 24 15:35 rc.single
-rwxr-xr-x  1 backup nobody  12872 Mar 10  2009 rc.sysinit
-rw-r--r--  1 backup nobody    160 Mar 14  2009 resolv.conf

And this is what the client has:

-rw-r--r--  1 root root    1419 Nov  1  2009 profile
-rw-r--r--  1 root root     330 Oct 30  2009 profile.bash
drwxr-xr-x  2 root root    4096 May  5 10:38 profile.d
-rw-r--r--  1 root root    2510 Feb 26 05:56 protocols
-rw-------  1 root root       0 Mar 16  2009 .pwd.lock
-rw-r--r--  2 root root    3304 Feb 28 11:17 rc.conf
drwxr-xr-x  3 root root    4096 May 14 09:54 rc.d
-rwxr-xr-x  1 root root      96 Feb 20 15:59 rc.local
-rwxr-xr-x  1 root root      66 May 17  2008 rc.local.shutdown
-rwxr-xr-x  1 root root     503 May  8 12:51 rc.multi
-rwxr-xr-x  1 root root    3976 May  8 12:51 rc.shutdown
-rwxr-xr-x  1 root root    1913 May  8 12:51 rc.single
-rwxr-xr-x  1 root root   12256 May  8 12:51 rc.sysinit
-rw-r--r--  2 root root      19 May 14 09:22 resolv.conf

Here is an excerpt from the logfile (filtered full log):

[sender] i=66 / etc/profile mode=0100644 len=1419 uid=0 gid=0 flags=0
[sender] i=67 / etc/profile.bash mode=0100644 len=330 uid=0 gid=0 flags=0
[sender] i=68 / etc/protocols mode=0100644 len=2510 uid=0 gid=0 flags=0
[sender] i=69 / etc/rc.conf mode=0100644 len=3304 uid=0 gid=0 flags=0
[sender] i=70 / etc/rc.local mode=0100755 len=96 uid=0 gid=0 flags=0
[sender] i=71 / etc/rc.local.shutdown mode=0100755 len=66 uid=0 gid=0 flags=0
[sender] i=72 / etc/rc.multi mode=0100755 len=503 uid=0 gid=0 flags=0
[sender] i=73 / etc/rc.shutdown mode=0100755 len=3976 uid=0 gid=0 flags=0
[sender] i=74 / etc/rc.single mode=0100755 len=1913 uid=0 gid=0 flags=0
[sender] i=75 / etc/rc.sysinit mode=0100755 len=12256 uid=0 gid=0 flags=0
[sender] i=76 / etc/resolv.conf mode=0100644 len=19 uid=0 gid=0 flags=0
[...]
[generator] i=66 2 etc/profile mode=0100644 len=1419 gid=(0) flags=400
[generator] i=67 2 etc/profile.bash mode=0100644 len=330 gid=(0) flags=400
[generator] i=68 2 etc/protocols mode=0100644 len=2510 gid=(0) flags=400
[generator] i=69 2 etc/rc.conf mode=0100644 len=3304 gid=(0) flags=400
[generator] i=70 2 etc/rc.local mode=0100755 len=96 gid=(0) flags=400
[generator] i=71 2 etc/rc.local.shutdown mode=0100755 len=66 gid=(0) flags=400
[generator] i=72 2 etc/rc.multi mode=0100755 len=503 gid=(0) flags=400
[generator] i=73 2 etc/rc.shutdown mode=0100755 len=3976 gid=(0) flags=400
[generator] i=74 2 etc/rc.single mode=0100755 len=1913 gid=(0) flags=400
[generator] i=75 2 etc/rc.sysinit mode=0100755 len=12256 gid=(0) flags=400
[generator] i=76 2 etc/resolv.conf mode=0100644 len=19 gid=(0) flags=400
[...]
[generator] i=62 0 etc/profile mode=0100644 len=1445 gid=99 flags=0
[generator] i=63 0 etc/profile.bash mode=0100644 len=618 gid=99 flags=0
[generator] i=64 0 etc/protocols mode=0100644 len=2510 gid=99 flags=0
[generator] i=65 0 etc/rc.conf mode=0100644 len=3304 gid=99 flags=0
[generator] i=66 0 etc/rc.local mode=0100755 len=109 gid=99 flags=0
[generator] i=67 0 etc/rc.local.shutdown mode=0100755 len=187 gid=99 flags=0
[generator] i=68 0 etc/rc.multi mode=0100755 len=503 gid=99 flags=0
[generator] i=69 0 etc/rc.shutdown mode=0100755 len=3976 gid=99 flags=0
[generator] i=70 0 etc/rc.single mode=0100755 len=1970 gid=99 flags=0
[generator] i=71 0 etc/rc.sysinit mode=0100755 len=12872 gid=99 flags=0
[generator] i=72 0 etc/resolv.conf mode=0100644 len=160 gid=99 flags=0

obviously I can remove everything on the server-side, but I'd rather like to know what causes this problem.

Last edited by Spider.007 (2010-05-16 11:37:09)

Spider.007

Member

Registered: 2004-06-20

Posts: 1,175

Re: [fixed] rsync - not updating existing files?

Okay, the solution was pretty simple, I was using --append, but that is only meant for files that you know will increase in size. Since that isn't the case when backup up /etc/, that flag isn't very useful.