You are not logged in.
Hi,
I have an rsync script to backup my data to my external drive when I plug it in. It 's a simple :
rsync -a --delete --backup --backup-dir="$ARCHIVE/$DATE" -f "merge $FILTERS" $SOURCE $DESTINATION
I recently installed Syncthing to sync files between my computer and my phone, and apparently in the process Syncthing updated the modification times of 20 Gb worth of pictures.
Now Rsync wants to copy all those pictures again to the backup HDD, even though they are the same, because the modification times differ.
Number of files: 30,811 (reg: 29,539, dir: 1,272)
Number of created files: 2,802 (reg: 2,738, dir: 64)
Number of deleted files: 114 (reg: 107, dir: 7)
Number of regular files transferred: 5,084
Total file size: 185,523,093,911 bytes
Total transferred file size: 23,957,034,587 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 65,532
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 709,625
Total bytes received: 23,064
sent 709,625 bytes received 23,064 bytes 488,459.33 bytes/sec
total size is 185,523,093,911 speedup is 253,208.52 (DRY RUN)
How can I proceed with my backup without copying all those identical files, but without missing files which really have been modified ?
Thanks !
Last edited by Cvlc (2021-11-27 12:01:29)
Offline
You could possibly try passing the
--size-only
Eenie meenie, chili beanie, the spirits are about to speak -- Bullwinkle J. Moose
It's a big club...and you ain't in it -- George Carlin
Registered Linux user #149839
perl -e 'print$i=pack(c5,(41*2),sqrt(7056),(unpack(c,H)-2),oct(115),10); '
Offline
would that update files which have been modified but are the same size though ?
Offline
i highly doubt that files that have been modified would be the EXACT same size, i ran into this once and "--size-only" did the trick.
Offline
i highly doubt that files that have been modified would be the EXACT same size, i ran into this once and "--size-only" did the trick.
What about a plain text file or a script ? If I ever modify a value to another of the same length, wouldn't that be the exact same size ? and then the file wouldn't copied over. Seems very risky to me.
Offline
What about a plain text file or a script ? If I ever modify a value to another of the same length, wouldn't that be the exact same size ? and then the file wouldn't copied over. Seems very risky to me.
You're moving away from what you were asking about in your OP.
If text files/scripts are that big of a risk to you, then just bite the bullet, and do the complete rsync. Problem solved.
Eenie meenie, chili beanie, the spirits are about to speak -- Bullwinkle J. Moose
It's a big club...and you ain't in it -- George Carlin
Registered Linux user #149839
perl -e 'print$i=pack(c5,(41*2),sqrt(7056),(unpack(c,H)-2),oct(115),10); '
Offline
Thanks for your answer
Well my computer is a normal one and has both text files / scripts and media files.
To rephrase the question then, is there anyway to update modification times without doing the complete rsync ?
Offline
this seems to work:
#!/bin/bash
find $1 > /tmp/list_of_files
while IFS= read -r line
do
touch -r "$line" "${line/$1/$2}"
done < /tmp/list_of_files
rm /tmp/list_of_files
exit 0
so if the script is called "timefix" you would run
timefix /path/to/files_with_correct_times /path/to/files_to_correct_times
or in your case use the source and destination paths:
timefix $source $destination
hope that makes sense
[EDIT] updated script using bash variable substitution instead of sed !
[EDIT2] updated script to actually work when path/filenames have spaces in them..... SORRY my bad
Last edited by jonno2002 (2021-11-25 13:40:36)
Offline
Thanks very much, exactly what I was looking for !
I thought "touch" was only for creating empty files....
Will try it out and report back
[edit]
there's only a slight problem, it creates empty files for files that are not on the laptop anymore. But that's pretty easy to fix
Other than that it worked, thanks !
Last edited by Cvlc (2021-11-25 16:16:02)
Offline
o right well heres an update to the script which uses a list of files generated from the destination instead of the source and checks to make sure the same file exists in the source before updating the timestamp, and also lets you know if the file doesnt exist and cannot update the timestamp.
#!/bin/bash
find $2 > /tmp/list_of_files
while IFS= read -r line
do
[ -f "${line/$2/$1}" ] || [ -d "${line/$2/$1}" ] && touch -r "${line/$2/$1}" "$line" || echo "${line/$2/$1}" "doesnt exist in the source path so cannot update timestamp"
done < /tmp/list_of_files
rm /tmp/list_of_files
exit 0
i have tested it and it seems to work fine but please let me know if i messed somthing up and ill fix it.
[EDIT] sorry i messed it up again, its fixed now, it was updating the timestamps the wrong way !
Last edited by jonno2002 (2021-11-26 01:46:58)
Offline
Great thank you very much !
have a great day
Offline