You are not logged in.

#1 2008-10-07 23:57:08

zatricky
Member
From: Stockholm
Registered: 2008-09-03
Posts: 55
Website

rsync or cp to local disk without "overwriting" previous content

Hi all

I've got a USB memory stick with a cache of my /var/cache/pacman/pkg folder to make it quicker to install on remote pcs with very slow (or sometimes just plain expensive) Internet access.

After running yaourt -Sc on my system to clean the main cache, I'd like to sync this to the memory stick. There might be only, for example, 50MB of updates but the cache is 1.1GB. I found rsync's --delete option nice in that it will remove files from the memory stick that no longer exist on my main system. rsync assumes that when doing a local-only copy that I want the --whole-file option (copy files whole (w/o delta-xfer algorithm) which makes some sense. However this means that instead of copying 50MB of updates to the memory stick, it copies 1.1GB to the memory stick, overwriting the old-yet-actually-current data that was there. So updating the memory stick is a procedure I sometimes have to actually wait for. sad

Ideally, I'd like it to just check that the file exists and skip it - or just merely consider file-size comparison to be "good enough" and then just move on to the next file. This would make the entire procedure only a few seconds long.

As a kludge, I've found that forcing the delta transfer by syncing "remotely" to tricky@localhost:/media/usbdisk/ is slightly faster. But this is only because the read speed on the memory stick is faster than the write speed.

Is there a cleaner way to do this, perhaps a flag I'm missing or maybe even a different command entirely that I should try?


pacman russian roulette: yes | pacman -Rcs $(pacman -Q | LANG=C sort -R | head -n $((RANDOM % 10)))
(yes, I know its broken)

Offline

#2 2008-10-08 00:06:34

zatricky
Member
From: Stockholm
Registered: 2008-09-03
Posts: 55
Website

Re: rsync or cp to local disk without "overwriting" previous content

This brightspark here just noticed a flag called "--size-only" in rsync's man pages.

Just shows that sometimes the right search terms are all thats needed. smile

rsync -rvv --size-only --delete /var/cache/pacman/pkg /media/usbdisk/arch/var/cache/pacman/
pkg/a52dec-0.7.4-4-x86_64.pkg.tar.gz is uptodate
pkg/abs-2.2-1-x86_64.pkg.tar.gz is uptodate
pkg/acl-2.2.47-1-x86_64.pkg.tar.gz is uptodate
pkg/akonadi-1.0.0-2-x86_64.pkg.tar.gz is uptodate
pkg/alsa-lib-1.0.17a-1-x86_64.pkg.tar.gz is uptodate
.....
.....
.....
pkg/xorg-res-utils-1.0.3-2.pkg.tar.gz is uptodate
pkg/xorg-server-1.4.2-2-x86_64.pkg.tar.gz is uptodate
pkg/xorg-server-utils-7.3-1-x86_64.pkg.tar.gz is uptodate
pkg/xorg-twm-1.0.4-1-x86_64.pkg.tar.gz is uptodate
pkg/xorg-utils-7.3-1-x86_64.pkg.tar.gz is uptodate

All in the space of a few seconds. smile

Last edited by zatricky (2008-10-08 17:06:16)


pacman russian roulette: yes | pacman -Rcs $(pacman -Q | LANG=C sort -R | head -n $((RANDOM % 10)))
(yes, I know its broken)

Offline

Board footer

Powered by FluxBB