I need help making a backup script

Allamgir · 2009-07-06 22:28:00

I'm trying to create this full system backup script that I plan to run manually. I have some commands to compress and extract my root directory and everything in it, but they're not complete because there are a few other features I would like to have. The problem is that I have no idea how to implement these specific features, and my googling of bash scripting documentation only left me confused and at square 1. This is what I have so far (just commands I can run in the terminal):

To compress:

sudo tar -cpf ~/archive.tar.gz / --exclude=/proc

To extract & restore:

sudo tar -xpf /home/agi/archive.tar.gz -C /

I want this script to:

1. allows me to run as

sh ~/.bin/backup.sh [directory to back up]

What I mean is there should be some sort of parameter in the script so that when I run it, I can choose whatever directory I want to compress rather than just be limited to just /.

2. puts finished .tar.gz in my home directory (I plan to transfer the archive to external media or online later; this gives me the freedom to not need to have a specific drive connected to run the script). I already have this in the commands, but I just wanted to point out that this is important.

3. names finished .tar.gz DD-MM-YY-backup.tar.gz where DD is day, MM is month, YY is year.

4. the "sudo" part works correctly. That is, the script, when run, asks for a password and can be granted root permissions (or is granted root permissions in some other way) so that I can back up all parts of my system without these sorts of errors.

5. shows some sort of progress indicator. Either a progress bar, shows the tar program output, or something that gives me an indication that the script is working and perhaps how long it will take. This one is really optional, for as long as the process is working properly then I'm ok. But if it is possible and not ridiculously complicated or difficult, it would be very nice.

Bregol · 2009-07-06 22:57:51

1. When you write a bash script, use $1, $2, $3, etc. as variables that mean word 1, word 2, etc that you type when running your script. For example:

#/bin/bash
sudo tar -cpf $1 / --exclude=/proc

say for example you name this script "mybackup.sh", then you just run

./mybackup.sh ~/backup.tar.gz

and the second part of what you tell it to run (~/backup.tar.gz) becomes $1 in the script. So this script would tar / into whatever $1 you tell it to tar it into.

I guess you were asking about changing what to tar rather than where to tar it to, but that's the same thing: whatever you replace by $1, $2, etc in the script is what you specify when running the script. Another example:

#/bin/bash
sudo tar -cpf $1 $2 --exclude=/proc

will tar up $2 into $1. So if you run

./mybackup.sh foo.tar.gz /bar

it will tar up /bar into foo.tar.gz

2. Then you just add a mv command to the script:

#/bin/bash
sudo tar -cpf ~/archive.tar.gz $1 --exclude=/proc
mv ~/archive.tar.gz /wherever/you/want/it/archive.tar.gz

3.

date +%d-%m-%y

gives you the date format you want. If you put it in backwards single quotes, it will give that output to another command. For example,

mv backup.tar.gz `date +%d-%m-%y`-backup.tar.gz

should do what you are looking for.

4. ok

5. try -v in your tar command - it will show what it is tarring (verbose mode). Doesn't give you a % done, but will tell you what it is working on and that it is working.

DevoidOfWindows · 2009-07-06 22:58:46

#!/bin/bash

#Change this to an absolute path (e.g., "/home/user") if you are not using Sudo.)
archive_root="$HOME"

if [[ "$UID" != '0' ]]; then
  echo "$0:  Must be executed as root user."
  exit 1
fi

if [[ -z "$1" ]]; then
  echo "$0:  Syntax:"
  echo "  $0 '/dir/to/src'"
  echo "  $0 -x '/path/to/archive/' '/path/to/dest'"
else
  if [[ "$1" == '-x' ]]; then
    shift
    sudo tar -xpvf "$1" -C "$2"
  else
    sudo tar -cpf "$archive_root/$(date +'%d-%m-%y')-backup.tar.gz" "$1" --exclude=/proc --exclude="$archive_root/*-backup.tar.gz"
  fi
fi

Last edited by DevoidOfWindows (2009-07-06 23:03:14)

brenix · 2009-07-06 23:01:47

What I mean is there should be some sort of parameter in the script so that when I run it, I can choose whatever directory I want to compress rather than just be limited to just /.

On my backup script, I specify $* which should accept any arguments passed to the script.. ex: $ ./backup.sh /source/folder /source/folder2.

in the script:

#!/bin/sh
tar -cpf $HOME/$(date +%d%m%Y)-backup.tar.gz $* --exclude=/proc

2. puts finished .tar.gz in my home directory (I plan to transfer the archive to external media or online later; this gives me the freedom to not need to have a specific drive connected to run the script). I already have this in the commands, but I just wanted to point out that this is important.

In the above script, that is what the ~ or $HOME should do..

3. names finished .tar.gz DD-MM-YY-backup.tar.gz where DD is day, MM is month, YY is year.

Same thing, the above script should do the trick, you might want to modify the date parameters (check the date manual)

4. the "sudo" part works correctly. That is, the script, when run, asks for a password and can be granted root permissions (or is granted root permissions in some other way) so that I can back up all parts of my system without these sorts of errors.

You could put something to prevent the script from running unless you are root, or you could have it prompt for a password and use sudo within the script. I would recommend the first. Here is a code snippet:

#Check for root
if [ $(whoami) != "root" ]; then
      echo "Error: You cannot perform this operation unless you are root"
      exit 1
fi

5. shows some sort of progress indicator. Either a progress bar, shows the tar program output, or something that gives me an indication that the script is working and perhaps how long it will take. This one is really optional, for as long as the process is working properly then I'm ok. But if it is possible and not ridiculously complicated or difficult, it would be very nice.

You might be able to use a similar program to tar, but with progress. Maybe someone else could shed some light here..

Hope this helps...

arkham · 2009-07-06 23:04:25

Are you sure you want to compress your backup?
Google for the "rsync / cp -al" approach, I prefer to use that even if it requires to have an external drive attached / network connection since it is much faster and lets you create incremental backups

Allamgir · 2009-07-06 23:51:59

Thank you so much! With all of your help I put together a simple script that does exactly what I wanted. Here is my ~/.bin/backup.sh (constructive criticism welcome):

#/bin/bash
if [[ "$UID" != '0' ]]; then
      echo "You cannot perform this operation unless you are root"
      exit 1
else
tar -cpvf /home/agi/$(date +'%d-%m-%y')-backup.tar.gz $1 --exclude=/proc
fi

Then, when I want to restore, I just run:

tar -xpvf ~/06-07-09-backup.tar.gz -C /

(or whatever the filename of the backup package is)

arkham got me wondering -- is there something bad about compressing my backups? I really don't mind making a full backup each time as opposed to just backing up the changes, for I've got a good size external drive and the wait isn't so bad. Is it hurting my files or anything?

Bregol · 2009-07-07 00:00:17

its not hurting anything to compress them as far as i know. its just that some (including myself) prefer rsync so it only transfers over what has changed rather than the whole thing. but if you don't mind the time and processor power to put it into a tar.gz, i don't see anything terrible about that.

arkham · 2009-07-07 00:31:26

Well no, it doesn't hurt your files, but there are some disadvantadges:
- if some file hasn't changed in the last year and you backup every week, you will have 52 copies of the same file
- if you want to see what is inside of a backup you will have to extract it first

Instead the rsync / cp -al approach saves only one copy of a file if it hasn't changed since the last backup and you won't have to extract anything to check the contents of a backup.
Here is the script that I use if you are curious:

#!/bin/bash
DEST="/media/disk/backup/acerus"
rsync -aP --delete --exclude "git/" --exclude ".mozilla/" --exclude ".opera/" $HOME/ $DEST/current/
cp -al $DEST/current /$DEST/`date +%d-%b-%y`
pacman -Qet > $DEST/pkglist.txt

In this way I have in my external disk a folder with weekly incremental backups; note that I can access any file in any backup due to the magical property of "cp -al"

Read more about this technique:
http://www.mikerubel.org/computers/rsync_snapshots/
http://fluff.info/blog/arch/00000204.htm

Last edited by arkham (2009-07-07 00:33:04)

eldragon · 2009-07-07 03:01:41

check rsnapshot.

marciorp · 2010-06-22 05:27:29

arkham wrote:

Instead the rsync / cp -al approach saves only one copy of a file if it hasn't changed since the last backup and you won't have to extract anything to check the contents of a backup.
Here is the script that I use if you are curious:
#!/bin/bash
DEST="/media/disk/backup/acerus"
rsync -aP --delete --exclude "git/" --exclude ".mozilla/" --exclude ".opera/" $HOME/ $DEST/current/
cp -al $DEST/current /$DEST/`date +%d-%b-%y`
pacman -Qet > $DEST/pkglist.txt
In this way I have in my external disk a folder with weekly incremental backups; note that I can access any file in any backup due to the magical property of "cp -al"

Man, i've been checking on this approach, and i believe i have almost everything figured out already. There is only one thing nagging me: how to avoid creating a snapshot in case no changes have actually been made?

Solution 1 would be parsing rsync's --stat, but that just sounds painful. Solution 2 would be to --write-batch and check for the size, but my initial checks show that it can be terribly imprecise. Ideally, i would like to run rsync with --link-dest and a switch that tells it to do nothing in case there is nothing to do...

Any ideas? Please? Pretty please with sugar on top?

Arch Linux

#1 2009-07-06 22:28:00

I need help making a backup script

#2 2009-07-06 22:57:51

Re: I need help making a backup script

#3 2009-07-06 22:58:46

Re: I need help making a backup script

#4 2009-07-06 23:01:47

Re: I need help making a backup script

#5 2009-07-06 23:04:25

Re: I need help making a backup script

#6 2009-07-06 23:51:59

Re: I need help making a backup script

#7 2009-07-07 00:00:17

Re: I need help making a backup script

#8 2009-07-07 00:31:26

Re: I need help making a backup script

#9 2009-07-07 03:01:41

Re: I need help making a backup script

#10 2010-06-22 05:27:29

Re: I need help making a backup script

Board footer