rsnapshot full system restore command

Xelank · 2015-10-13 13:56:48

Hi Guys,

I'm setting up my full system backup with rsnapshot and it has ran correctly, backing it up to /backup.

Now I want to write a script so that when a full system upgrade breaks my system I can just boot in using the live cd and get my old system back with no issues.

I've scouted the wiki and funny enough information on restoration seems to be quite poor

I have got something together, and would like to have a few more pair of eyes on this before I run it

The idea is that you mount all the drives first like this

    mkdir /mnt
    mkdir /mnt/boot
    mkdir /mnt/home
    mkdir /mnt/backup
    mount /dev/sda1 /mnt/boot
    mount /dev/sda2 /mnt
    mount /dev/sda3 /mnt/home
    #sdb2 is where you stored your backups using rsnapshot
    mount /dev/sdb2 /mnt/backup

then

#!/bin/sh
# you can do a dry run by adding the -n flag
rsync -av \
--exclude=/mnt/backup/* \
--exclude=/mnt/dev/* \
--exclude=/mnt/proc/* \
--exclude=/mnt/run/* \
--exclude=/mnt/sys/* \
--exclude=/mnt/tmp/* \
--exclude=/mnt/var/tmp/* \
--exclude=/mnt/media/* \
/mnt/boot/YOUR_LATEST_GOOD_BACKUP/localhost \
/mnt

The reason for all the excludes is so that it doesn't actually go into, say, /var/tmp and wipe everything out. The excludes are pulled directly from my exclude settings in /etc/rsnapshot.conf
you need to put a /mnt/ path before all of them because you're in the Live CD environment, not in your actual filesystem.

Is is looking ok?

For reference, this is my /etc/rsnapshot.conf

#################################################
# rsnapshot.conf - rsnapshot configuration file #
#################################################
#                                               #
# PLEASE BE AWARE OF THE FOLLOWING RULE:        #
#                                               #
# This file requires tabs between elements      #
#                                               #
#################################################

#######################
# CONFIG FILE VERSION #
#######################

config_version	1.2

###########################
# SNAPSHOT ROOT DIRECTORY #
###########################

# All snapshots will be stored under this root directory.
#
snapshot_root	/backup/

# If no_create_root is enabled, rsnapshot will not automatically create the
# snapshot_root directory. This is particularly useful if you are backing
# up to removable media, such as a FireWire or USB drive.
#
#no_create_root	1

#################################
# EXTERNAL PROGRAM DEPENDENCIES #
#################################

# LINUX USERS:   Be sure to uncomment "cmd_cp". This gives you extra features.
# EVERYONE ELSE: Leave "cmd_cp" commented out for compatibility.
#
# See the README file or the man page for more details.
#
cmd_cp		/usr/bin/cp

# uncomment this to use the rm program instead of the built-in perl routine.
#
cmd_rm		/usr/bin/rm

# rsync must be enabled for anything to work. This is the only command that
# must be enabled.
#
cmd_rsync	/usr/bin/rsync

# Uncomment this to enable remote ssh backups over rsync.
#
#cmd_ssh	/usr/bin/ssh

# Comment this out to disable syslog support.
#
cmd_logger	/usr/bin/logger

# Uncomment this to specify the path to "du" for disk usage checks.
# If you have an older version of "du", you may also want to check the
# "du_args" parameter below.
#
#cmd_du		/usr/bin/du

# Uncomment this to specify the path to rsnapshot-diff.
#
#cmd_rsnapshot_diff	/usr/local/bin/rsnapshot-diff

# Specify the path to a script (and any optional arguments) to run right
# before rsnapshot syncs files
#
#cmd_preexec	/path/to/preexec/script

# Specify the path to a script (and any optional arguments) to run right
# after rsnapshot syncs files
#
#cmd_postexec	/path/to/postexec/script

# Paths to lvcreate, lvremove, mount and umount commands, for use with
# Linux LVMs.
#
#linux_lvm_cmd_lvcreate	/path/to/lvcreate
#linux_lvm_cmd_lvremove	/path/to/lvremove
#linux_lvm_cmd_mount	/usr/bin/mount
#linux_lvm_cmd_umount	/usr/bin/umount

#########################################
#     BACKUP LEVELS / INTERVALS         #
# Must be unique and in ascending order #
# e.g. alpha, beta, gamma, etc.         #
#########################################

retain	manual	5
retain	alpha	6
retain	beta	7
retain	gamma	4
#retain	delta	3

############################################
#              GLOBAL OPTIONS              #
# All are optional, with sensible defaults #
############################################

# Verbose level, 1 through 5.
# 1     Quiet           Print fatal errors only
# 2     Default         Print errors and warnings only
# 3     Verbose         Show equivalent shell commands being executed
# 4     Extra Verbose   Show extra verbose information
# 5     Debug mode      Everything
#
verbose		2

# Same as "verbose" above, but controls the amount of data sent to the
# logfile, if one is being used. The default is 3.
#
loglevel	3

# If you enable this, data will be written to the file you specify. The
# amount of data written is controlled by the "loglevel" parameter.
#
#logfile	/var/log/rsnapshot

# If enabled, rsnapshot will write a lockfile to prevent two instances
# from running simultaneously (and messing up the snapshot_root).
# If you enable this, make sure the lockfile directory is not world
# writable. Otherwise anyone can prevent the program from running.
#
lockfile	/var/run/rsnapshot.pid

# Default rsync args. All rsync commands have at least these options set.
#
#rsync_short_args	-a
#rsync_long_args	--delete --numeric-ids --relative --delete-excluded

# ssh has no args passed by default, but you can specify some here.
#
#ssh_args	-p 22

# Default arguments for the "du" program (for disk space reporting).
# The GNU version of "du" is preferred. See the man page for more details.
# If your version of "du" doesn't support the -h flag, try -k flag instead.
#
#du_args	-csh

# If this is enabled, rsync won't span filesystem partitions within a
# backup point. This essentially passes the -x option to rsync.
# The default is 0 (off).
#
#one_fs		0

# The include and exclude parameters, if enabled, simply get passed directly
# to rsync. If you have multiple include/exclude patterns, put each one on a
# separate line. Please look up the --include and --exclude options in the
# rsync man page for more details on how to specify file name patterns. 
# 
#include	???
#include	???
exclude	/backup/*
exclude	/dev/*
exclude	/proc/*
exclude	/run/*
exclude	/sys/*
exclude	/tmp/*
exclude	/var/tmp/*
exclude	/media/*


# The include_file and exclude_file parameters, if enabled, simply get
# passed directly to rsync. Please look up the --include-from and
# --exclude-from options in the rsync man page for more details.
#
#include_file	/path/to/include/file
#exclude_file	/path/to/exclude/file

# If your version of rsync supports --link-dest, consider enabling this.
# This is the best way to support special files (FIFOs, etc) cross-platform.
# The default is 0 (off).
#
#link_dest	0

# When sync_first is enabled, it changes the default behaviour of rsnapshot.
# Normally, when rsnapshot is called with its lowest interval
# (i.e.: "rsnapshot alpha"), it will sync files AND rotate the lowest
# intervals. With sync_first enabled, "rsnapshot sync" handles the file sync,
# and all interval calls simply rotate files. See the man page for more
# details. The default is 0 (off).
#
#sync_first	0

# If enabled, rsnapshot will move the oldest directory for each interval
# to [interval_name].delete, then it will remove the lockfile and delete
# that directory just before it exits. The default is 0 (off).
#
#use_lazy_deletes	0

# Number of rsync re-tries. If you experience any network problems or
# network card issues that tend to cause ssh to fail with errors like
# "Corrupted MAC on input", for example, set this to a non-zero value
# to have the rsync operation re-tried.
#
#rsync_numtries 0

# LVM parameters. Used to backup with creating lvm snapshot before backup
# and removing it after. This should ensure consistency of data in some special
# cases
#
# LVM snapshot(s) size (lvcreate --size option).
#
#linux_lvm_snapshotsize	100M

# Name to be used when creating the LVM logical volume snapshot(s).
#
#linux_lvm_snapshotname	rsnapshot

# Path to the LVM Volume Groups.
#
#linux_lvm_vgpath	/dev

# Mount point to use to temporarily mount the snapshot(s).
#
#linux_lvm_mountpath	/path/to/mount/lvm/snapshot/during/backup

###############################
### BACKUP POINTS / SCRIPTS ###
###############################

# LOCALHOST
# backup everything
backup	/	localhost/
# backup	/home/		localhost/
# backup	/etc/		localhost/
# backup	/usr/local/	localhost/
#backup	/var/log/rsnapshot		localhost/
#backup	/etc/passwd	localhost/
#backup	/home/foo/My Documents/		localhost/
#backup	/foo/bar/	localhost/	one_fs=1, rsync_short_args=-urltvpog
#backup_script	/usr/local/bin/backup_pgsql.sh	localhost/postgres/
# You must set linux_lvm_* parameters below before using lvm snapshots
#backup	lvm://vg0/xen-home/	lvm-vg0/xen-home/

# EXAMPLE.COM
#backup_exec	/bin/date "+ backup of example.com started at %c"
#backup	root@example.com:/home/	example.com/	+rsync_long_args=--bwlimit=16,exclude=core
#backup	root@example.com:/etc/	example.com/	exclude=mtab,exclude=core
#backup_exec	ssh root@example.com "mysqldump -A > /var/db/dump/mysql.sql"
#backup	root@example.com:/var/db/dump/	example.com/
#backup_exec	/bin/date "+ backup of example.com ended at %c"

# CVS.SOURCEFORGE.NET
#backup_script	/usr/local/bin/backup_rsnapshot_cvsroot.sh	rsnapshot.cvs.sourceforge.net/

# RSYNC.SAMBA.ORG
#backup	rsync://rsync.samba.org/rsyncftp/	rsync.samba.org/rsyncftp/

Ideally the restore script can be generated from a script so when you add new exclude folders in your rsnapshot configuration it will automatically be updated.

What do you guys think? Am I missing something here?

branch · 2015-10-13 17:27:13

A few notes:

1) I assume the second from last line of the script should be /mnt/backup/... instead of /mnt/boot/...

2) I would advise mounting the backup volume outside of the restore hierarchy instead of relying just on the --exclude to keep the source out of the destination, just so if there is any mistake in the script it will be less likely to corrupt the backup.

3) Note the rsync args used by rsnapshot to take the backup set.
a. Since you are restoring from a livecd environment, you likely want at least --numeric-ids on restore also.
b. I would also use --delete on restore, as the presence of certain files can affect configuration (ex. /etc/sysctl.d/*).

4) Are your backups behaving as expected (specifically the excludes)? I found I needed "**" instead of "*" for the exclude wildcards (see "FILTER RULES" section of rsync man page for the difference). This would apply to both rsnapshot.conf and the restore script.

5) I have to question your desire to retain any contents in the directories that were excluded from the backup set. While most of the ones actually listed (/sys, /proc, /run, /dev) should just be empty mount points, for the ones with persistent storage (ex. /var/tmp), you want either the contents when the backup was taken or an empty directory. What you probably do not want is whatever junk is in there from the failed state.

Xelank · 2015-10-14 12:26:39

1) Yes, good pick up thanks.

2) Good point

3) a. Didn't know about that option! Added
b. added

4) My backups are looking ok. All my excluded directories are empty (tho the top level directory does exist, which is intended as you probably want them to be there when you boot. e.g. /run or /sys. Though I guess the kernel will create them if they don't exist)

5) Good point. I was overthinking this because when you're copying things back you want a clean slate, so you don't really need to prevent folders like /var/tmp (on your restore target) from being wiped. I guess I don't need the --exclude flags set at all.

This is my mount instructions when you load the live cd (assuming all partitions are still in place)

mkdir /mnt
mkdir /mnt/boot
mkdir /mnt/home
mkdir /restore
mount /dev/sda1 /mnt/boot
mount /dev/sda2 /mnt/
mount /dev/sda3 /mnt/home
mount /dev/sdb2 /restore

and the restoration command

rsync -av --numeric-ids --delete /restore/manual.0/localhost /mnt

Where you replace manual.0 with your last good backup (e.g or daily.5 if you have daily automatic backups)

What do you think?

===

Lastly, I've changed to using exclude file (--exclude_file option in rsync/rsnapshot) since it's less error prone to add new skip directories (stupid tab separators..)

I created a new file /etc/rsnapshot/excludes and use the exclude_file option with rsnapshot. Here's the content of the file

/backup/*
/dev/*
/proc/*
/run/*
/sys/*
/tmp/*
/var/tmp/*
/media/*

and the new rsnapshot.conf:

#################################################
# rsnapshot.conf - rsnapshot configuration file #
#################################################
#                                               #
# PLEASE BE AWARE OF THE FOLLOWING RULE:        #
#                                               #
# This file requires tabs between elements      #
#                                               #
#################################################

#######################
# CONFIG FILE VERSION #
#######################

config_version	1.2

###########################
# SNAPSHOT ROOT DIRECTORY #
###########################

# All snapshots will be stored under this root directory.
#
snapshot_root	/backup/

# If no_create_root is enabled, rsnapshot will not automatically create the
# snapshot_root directory. This is particularly useful if you are backing
# up to removable media, such as a FireWire or USB drive.
#
#no_create_root	1

#################################
# EXTERNAL PROGRAM DEPENDENCIES #
#################################

# LINUX USERS:   Be sure to uncomment "cmd_cp". This gives you extra features.
# EVERYONE ELSE: Leave "cmd_cp" commented out for compatibility.
#
# See the README file or the man page for more details.
#
cmd_cp		/usr/bin/cp

# uncomment this to use the rm program instead of the built-in perl routine.
#
cmd_rm		/usr/bin/rm

# rsync must be enabled for anything to work. This is the only command that
# must be enabled.
#
cmd_rsync	/usr/bin/rsync

# Uncomment this to enable remote ssh backups over rsync.
#
#cmd_ssh	/usr/bin/ssh

# Comment this out to disable syslog support.
#
cmd_logger	/usr/bin/logger

# Uncomment this to specify the path to "du" for disk usage checks.
# If you have an older version of "du", you may also want to check the
# "du_args" parameter below.
#
#cmd_du		/usr/bin/du

# Uncomment this to specify the path to rsnapshot-diff.
#
#cmd_rsnapshot_diff	/usr/local/bin/rsnapshot-diff

# Specify the path to a script (and any optional arguments) to run right
# before rsnapshot syncs files
#
#cmd_preexec	/path/to/preexec/script

# Specify the path to a script (and any optional arguments) to run right
# after rsnapshot syncs files
#
#cmd_postexec	/path/to/postexec/script

# Paths to lvcreate, lvremove, mount and umount commands, for use with
# Linux LVMs.
#
#linux_lvm_cmd_lvcreate	/path/to/lvcreate
#linux_lvm_cmd_lvremove	/path/to/lvremove
#linux_lvm_cmd_mount	/usr/bin/mount
#linux_lvm_cmd_umount	/usr/bin/umount

#########################################
#     BACKUP LEVELS / INTERVALS         #
# Must be unique and in ascending order #
# e.g. alpha, beta, gamma, etc.         #
#########################################

retain	manual	5
retain	alpha	6
retain	beta	7
retain	gamma	4
#retain	delta	3

############################################
#              GLOBAL OPTIONS              #
# All are optional, with sensible defaults #
############################################

# Verbose level, 1 through 5.
# 1     Quiet           Print fatal errors only
# 2     Default         Print errors and warnings only
# 3     Verbose         Show equivalent shell commands being executed
# 4     Extra Verbose   Show extra verbose information
# 5     Debug mode      Everything
#
verbose		2

# Same as "verbose" above, but controls the amount of data sent to the
# logfile, if one is being used. The default is 3.
#
loglevel	3

# If you enable this, data will be written to the file you specify. The
# amount of data written is controlled by the "loglevel" parameter.
#
#logfile	/var/log/rsnapshot

# If enabled, rsnapshot will write a lockfile to prevent two instances
# from running simultaneously (and messing up the snapshot_root).
# If you enable this, make sure the lockfile directory is not world
# writable. Otherwise anyone can prevent the program from running.
#
lockfile	/var/run/rsnapshot.pid

# Default rsync args. All rsync commands have at least these options set.
#
#rsync_short_args	-a
#rsync_long_args	--delete --numeric-ids --relative --delete-excluded

# ssh has no args passed by default, but you can specify some here.
#
#ssh_args	-p 22

# Default arguments for the "du" program (for disk space reporting).
# The GNU version of "du" is preferred. See the man page for more details.
# If your version of "du" doesn't support the -h flag, try -k flag instead.
#
#du_args	-csh

# If this is enabled, rsync won't span filesystem partitions within a
# backup point. This essentially passes the -x option to rsync.
# The default is 0 (off).
#
#one_fs		0

# The include and exclude parameters, if enabled, simply get passed directly
# to rsync. If you have multiple include/exclude patterns, put each one on a
# separate line. Please look up the --include and --exclude options in the
# rsync man page for more details on how to specify file name patterns. 
# 
#include	???
#include	???


# The include_file and exclude_file parameters, if enabled, simply get
# passed directly to rsync. Please look up the --include-from and
# --exclude-from options in the rsync man page for more details.
#
#include_file	/path/to/include/file
exclude_file	/etc/rsnapshot/excludes

# If your version of rsync supports --link-dest, consider enabling this.
# This is the best way to support special files (FIFOs, etc) cross-platform.
# The default is 0 (off).
#
#link_dest	0

# When sync_first is enabled, it changes the default behaviour of rsnapshot.
# Normally, when rsnapshot is called with its lowest interval
# (i.e.: "rsnapshot alpha"), it will sync files AND rotate the lowest
# intervals. With sync_first enabled, "rsnapshot sync" handles the file sync,
# and all interval calls simply rotate files. See the man page for more
# details. The default is 0 (off).
#
#sync_first	0

# If enabled, rsnapshot will move the oldest directory for each interval
# to [interval_name].delete, then it will remove the lockfile and delete
# that directory just before it exits. The default is 0 (off).
#
#use_lazy_deletes	0

# Number of rsync re-tries. If you experience any network problems or
# network card issues that tend to cause ssh to fail with errors like
# "Corrupted MAC on input", for example, set this to a non-zero value
# to have the rsync operation re-tried.
#
#rsync_numtries 0

# LVM parameters. Used to backup with creating lvm snapshot before backup
# and removing it after. This should ensure consistency of data in some special
# cases
#
# LVM snapshot(s) size (lvcreate --size option).
#
#linux_lvm_snapshotsize	100M

# Name to be used when creating the LVM logical volume snapshot(s).
#
#linux_lvm_snapshotname	rsnapshot

# Path to the LVM Volume Groups.
#
#linux_lvm_vgpath	/dev

# Mount point to use to temporarily mount the snapshot(s).
#
#linux_lvm_mountpath	/path/to/mount/lvm/snapshot/during/backup

###############################
### BACKUP POINTS / SCRIPTS ###
###############################

# LOCALHOST
# backup everything
backup	/	localhost/
# backup	/home/		localhost/
# backup	/etc/		localhost/
# backup	/usr/local/	localhost/
#backup	/var/log/rsnapshot		localhost/
#backup	/etc/passwd	localhost/
#backup	/home/foo/My Documents/		localhost/
#backup	/foo/bar/	localhost/	one_fs=1, rsync_short_args=-urltvpog
#backup_script	/usr/local/bin/backup_pgsql.sh	localhost/postgres/
# You must set linux_lvm_* parameters below before using lvm snapshots
#backup	lvm://vg0/xen-home/	lvm-vg0/xen-home/

# EXAMPLE.COM
#backup_exec	/bin/date "+ backup of example.com started at %c"
#backup	root@example.com:/home/	example.com/	+rsync_long_args=--bwlimit=16,exclude=core
#backup	root@example.com:/etc/	example.com/	exclude=mtab,exclude=core
#backup_exec	ssh root@example.com "mysqldump -A > /var/db/dump/mysql.sql"
#backup	root@example.com:/var/db/dump/	example.com/
#backup_exec	/bin/date "+ backup of example.com ended at %c"

# CVS.SOURCEFORGE.NET
#backup_script	/usr/local/bin/backup_rsnapshot_cvsroot.sh	rsnapshot.cvs.sourceforge.net/

# RSYNC.SAMBA.ORG
#backup	rsync://rsync.samba.org/rsyncftp/	rsync.samba.org/rsyncftp/

branch · 2015-10-14 16:42:07

Xelank wrote:

This is my mount instructions when you load the live cd (assuming all partitions are still in place)
mkdir /mnt
mkdir /mnt/boot
mkdir /mnt/home
mkdir /restore
mount /dev/sda1 /mnt/boot
mount /dev/sda2 /mnt/
mount /dev/sda3 /mnt/home
mount /dev/sdb2 /restore

Mount /mnt before making subdirectories and submount points, so:

mkdir -p /mnt
mount /dev/sda2 /mnt
mkdir -p /mnt/boot
mkdir -p /mnt/home
mount /dev/sda1 /mnt/boot
mount /dev/sda3 /mnt/home
mkdir /restore
mount /dev/sdb2 /restore

The rest looks good to me.

Arch Linux

#1 2015-10-13 13:56:48

rsnapshot full system restore command

#2 2015-10-13 17:27:13

Re: rsnapshot full system restore command

#3 2015-10-14 12:26:39

Re: rsnapshot full system restore command

#4 2015-10-14 16:42:07

Re: rsnapshot full system restore command

Board footer