You are not logged in.
I have written a simple rsync backup script. I find the backup very slow:
#!/bin/su root
# check if drive is mounted
DIR=/run/media/brightchip/plug
if grep -qs $DIR /proc/mounts; then
echo "Directory $DIR mounted. Continuing..."
echo "****** Backing up system ******"
rsync -aAXogv --exclude={"/home/datapool","/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} --delete / $DIR
echo "****** System backup completed ******"
exit 0
else
echo "^^^^ Error: $DIR is not mounted. Exiting ^^^^"
exit 7
fi
Is there a way of speeding up rsync?
Last edited by d_fajardo (2022-07-27 11:12:24)
Offline
Is the rsync command only slow in thwt script or also if run in shell?
Have you built that -aA... line yourself or is it copypasta. If copypasta, decypher that for yourself first.
Offline
If copypasta, decypher that for yourself first.
It is copy paste and I most probably changed it too but I see what you mean. The -a flag shouldn't be there.
I noticed most of the slowness happens backing up home dot directories. That's probably where it conflicts.
Will report if it improves.
EDIT: I've only run it from shell.
Last edited by d_fajardo (2022-07-21 13:23:03)
Offline
The -a flag shouldn't be there.
And why not?
Check the rsync manpage on what "-a" equals to.
You've two of the options listed explicitly (those could go) and which of the other ones do you think you can/should spare on a system backup (let alone whether that would speed up the process)
I noticed most of the slowness happens backing up home dot directories
Ie. on many small files… I'd rather suspect -A and -X to be the slowing factor.
You probably want them for the system but can spare them on $HOME and in that case I'd run two cycles - one for the system and one for $HOME
Edit: I'd also check top/iotop to figure what's the limiting factor here (CPU or IO)
Last edited by seth (2022-07-21 14:04:36)
Online
You may also want to exclude directories with large(r) files whose content (the file itself that is) doesn't change (such as audio/video files and pictures, *.pkg.tar.zst files - if you build a lot of aur/local packages), and run a second instance of rsync with '--size-only' on them.
Last edited by dogknowsnx (2022-07-21 14:11:43)
Notifications for Arch Linux package updates
RI - Rest your Eyes and Self
"We are eternal, all this pain is an illusion" - Maynard James Keenan
Offline
rsync should™ default to size + mod time comparism, "-c" is explicit; so I'm not sure skipping only the mod time will help with esp. few large files.
Online
You're right, but in my experience it ('--size-only') is still faster if/when '--verbose' is used.
Last edited by dogknowsnx (2022-07-21 14:34:09)
Notifications for Arch Linux package updates
RI - Rest your Eyes and Self
"We are eternal, all this pain is an illusion" - Maynard James Keenan
Offline
And why not?
Check the rsync manpage on what "-a" equals to.
Yes I did change my mind after reading the docs further.
I'll have a look at the suggestions.
Offline
In the end what really made the difference in speed is hardware upgrade. I invested on an NVME drive and enclosure which plugs to a USB C bus and the difference is night and day. Now it takes minutes to backup what used to be hours.
So conclusion is rsync runs fine. It's my choice of bus that is slow. Solved.
Offline