[SOLVED] Bizarre rsync behaviour or user error?

fabertawe · 2018-09-04 12:32:12

Hi folks,

I'm loathe to post this as I'm almost certain it's bound to be something blindingly obvious I'm missing! Anyway...

I have a backup script (Bash) that I've used for a few years to rsync various things and sometime in the last couple of weeks it's stopped functioning "correctly", or as it used to, perhaps I should say.

backup_from="/"
backup_to="/media/Array/.backup/root"
sudo rsync -aAXv --log-file="$log_temp" --delete-excluded --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*/*","/lost+found","/var/lib/pacman/sync/*","/var/cache/pacman/pkg/*","$HOME/Private/*","$HOME/.gvfs","$HOME/.local/share/Trash/*","$HOME/.myGtkMenu_lockfile","$HOME/build/aurbuild/*"} "$backup_from" "$backup_to"

What happens now is that unless I put "cd /" at the beginning it copies my home folder contents into the root of the "$backup_to" folder! I exited before it run it's full course (forgive me). Like I said, I've been using this for ages and it worked previously without the "cd /".

Any thoughts?

Last edited by fabertawe (2018-09-04 14:02:57)

drcouzelis · 2018-09-04 12:51:57

Ooooh, I LOVE rsync! Let's see if we can figure this out...

So you're saying, the directory

/home/fabertawe

gets copied to

/media/Array/.backup/root/fabertawe

but the expected destination is

/media/Array/.backup/root/home/fabertawe

?

Trilby · 2018-09-04 13:00:19

I read it a bit differently, as if the source directory was the user's home directory (the contents of the OP's home folder is copied), in other words `/home/fabertawe/catphotos -> /media/Array.backup/root/catphotos`.

fabertawe · 2018-09-04 13:37:39

Tribly understands it correctly.

I think I have a "bigger" issue than first thought though. Just tried backing up the Wife's laptop and that's throwing an error now where it was working previously...

Unexpected remote arg: caroline@192.168.66.2:/
rsync error: syntax or usage error (code 1) at main.c(1354) [sender=3.1.3]

BUT it works from the shell directly, just not the script. Hmmm.

Trilby · 2018-09-04 13:42:32

Please post the *full* script as what is in your first post seems to be just an excerpt.

Is your system also fully up to date?

Last edited by Trilby (2018-09-04 13:43:23)

fabertawe · 2018-09-04 13:59:57

SOLVED. You're going to love this...

I'm using a variable to pass as secondary parameter for dry runs...

if [[ $2 = '-n' || $2 = '-d' || $2 = '--dry-run' ]]; then
  passarg='--dry-run'
elif [[ $2 = '' ]]; then
  passarg=''
else
  echo 'Unknown secondary parameter passed! Exiting...'
  exit 1
fi

So for some reason, possibly OCD of some kind, I decided to start quoting "$passarg" to rsync recently and so it was failing

sudo rsync -aAXv --log-file="$log_temp" "$passarg" --delete-excluded --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*/*","/lost+found","/var/lib/pacman/sync/*","/var/cache/pacman/pkg/*","$HOME/Private/*","$HOME/.gvfs","$HOME/.local/share/Trash/*","$HOME/.myGtkMenu_lockfile","$HOME/build/aurbuild/*"} "$backup_from" "$backup_to"

And yes, I should know better than to remove that from my first post for clarity. The very thing I thought inconsequential was the problem! And so it was indeed user error

Do you know why rsync would choke on an empty quote?

alphaniner · 2018-09-04 16:36:41

Lots of things choke on an empty quote:

$ ls ""
ls: cannot access '': No such file or directory

Quoting variables is (almost?) always good practice. In this case though I would use an array. Define it with your fixed flags then add '--dry-run' if necessary:

passarg=('-a' '-A' '-X' '-v' "--log-file='$log_temp'" '--delete-excluded')
if [[ $2 = '-n' || $2 = '-d' || $2 = '--dry-run' ]]; then
  passarg+=('--dry-run')
fi

Should be possible to put the --exclude in the array too if properly quoted.

Last edited by alphaniner (2018-09-04 16:42:23)

fabertawe · 2018-09-04 17:33:25

alphaniner wrote:

Lots of things choke on an empty quote:
$ ls ""
ls: cannot access '': No such file or directory

Fair enough. It's the little things that always get me!

alphaniner wrote:

Quoting variables is (almost?) always good practice. In this case though I would use an array.

That's why I went overboard with the quoting I always use arrays when applicable but it's overkill in this case.

Trilby · 2018-09-04 20:36:56

In my first reply I was going to comment on the excessive quotes. I disagree about it being (almost) always good practice. Knowing the benefits of quotes and knowing when they are needed is good practice. Blindly using quotes just because (just like blindly doing anything just because) is silly and potentially harmful.

But as writen your script used lots of quotes it didn't need, but skipped the ones that are necessary: if there were not at least 2 parameters provided, that whole if block would fail and error out. In such if conditions, you should quote the "$2". Alternatively (and preferable in this case) use a case statement:

case $2 in
   -n|-d|--dry-run) passarg='--dry-run' ;;
esac

eschwartz · 2018-09-04 21:44:50

Trilby wrote:

In my first reply I was going to comment on the excessive quotes. I disagree about it being (almost) always good practice. Knowing the benefits of quotes and knowing when they are needed is good practice. Blindly using quotes just because (just like blindly doing anything just because) is silly and potentially harmful.
But as writen your script used lots of quotes it didn't need, but skipped the ones that are necessary: if there were not at least 2 parameters provided, that whole if block would fail and error out. In such if conditions, you should quote the "$2". Alternatively (and preferable in this case) use a case statement:
case $2 in
   -n|-d|--dry-run) passarg='--dry-run' ;;
esac

I agree that knowing when they are needed is good practice.

Did you know that variables don't need to be quoted in GNU bash [[ ]] keywords, unlike the POSIX [ ] builtin (even when used in bash)?

No more than they need to be quoted in a POSIX case statement.

Granted that instead quoting the string literals on the other side of the [[ is kind of... odd.

...

Anyway I'm not sure why you think this is contingent on the value of $#

You could get the same failure for:

$ set -- 'foo' ''
$ printf '<%s> ' "$@"; printf '\n'
<foo> <>

where $2 is indeed defined, and given the literal value of ''

It would be far more accurate *and* natural-sounding, to say "if $2 is empty".

Not that it matters because this is bash, taking advantage of the features of bash.

Trilby · 2018-09-04 22:23:12

Oops, I tested before posting and confirmed an unqoted $2 in that if statement was problematic. But I guess muscle memory must have left the "ba" out of the shebang.

eschwartz · 2018-09-04 22:26:39

Sure, and using that code in a POSIX sh shell without muscle memory would result in syntax errors
sh: 1: [[: not found

Muscle memory vs. copy-paste strikes again, eh?

Last edited by eschwartz (2018-09-04 22:27:44)

Trilby · 2018-09-04 22:53:41

I dont use '[[' in my scripts, but busybox's ash currently accepts it. I think there were plans to ditch it, but not as of the build I'm using.

eschwartz · 2018-09-04 23:25:43

Trilby wrote:

I dont use '[[' in my scripts, but busybox's ash currently accepts it. I think there were plans to ditch it, but not as of the build I'm using.

That's completely insane...

$ busybox sh -c '[[ $foo = var ]]'
sh: var: unknown operand

I'm super disappointed, but at least dash treats this properly.

If you're going to provide aliases for GNU bashisms, at least, like, implement it properly. Or just save us all the indignity and treat it as the syntax error it is.

Arch Linux

#1 2018-09-04 12:32:12

[SOLVED] Bizarre rsync behaviour or user error?

#2 2018-09-04 12:51:57

Re: [SOLVED] Bizarre rsync behaviour or user error?

#3 2018-09-04 13:00:19

Re: [SOLVED] Bizarre rsync behaviour or user error?

#4 2018-09-04 13:37:39

Re: [SOLVED] Bizarre rsync behaviour or user error?

#5 2018-09-04 13:42:32

Re: [SOLVED] Bizarre rsync behaviour or user error?

#6 2018-09-04 13:59:57

Re: [SOLVED] Bizarre rsync behaviour or user error?

#7 2018-09-04 16:36:41

Re: [SOLVED] Bizarre rsync behaviour or user error?

#8 2018-09-04 17:33:25

Re: [SOLVED] Bizarre rsync behaviour or user error?

#9 2018-09-04 20:36:56

Re: [SOLVED] Bizarre rsync behaviour or user error?

#10 2018-09-04 21:44:50

Re: [SOLVED] Bizarre rsync behaviour or user error?

#11 2018-09-04 22:23:12

Re: [SOLVED] Bizarre rsync behaviour or user error?

#12 2018-09-04 22:26:39

Re: [SOLVED] Bizarre rsync behaviour or user error?

#13 2018-09-04 22:53:41

Re: [SOLVED] Bizarre rsync behaviour or user error?

#14 2018-09-04 23:25:43

Re: [SOLVED] Bizarre rsync behaviour or user error?

Board footer