Backup script for rsnapshot

LeeF · 2010-09-18 09:56:28

Hello everyone, just thought I would share this script, like the topic says, it's a backup script for rsnapshot. It's not really that complex, but it gets the job done. I had to write this after cron couldn't execute my "complicated" timing scheme. To clarify, this script runs a certain set of commands based on the time and date. I've also included a little script that makes my server beep once it's completed the backup task.

Update: The Bash script didn't work for weekly, monthly or yearly backups, so I rewrote it in PHP, a language I'm more familiar with; since I ended up installing PHP for a different application anyways, I had only used Bash to begin with because I didn't want excess software installed on the machine.

Also I really hate bash scripting and try to avoid it where I can, so if anyone sees something that could be done better in this script, please let me know.

#!/usr/bin/php
<?php
#
# /usr/local/bin/backup ---- An automated backup script utilizing rsnapshot
# v2.0 - 10/08/10 -- 15:38 GMT
#
#
# This is a backup script that is designed to be run twice a day at midnight and noon.
# This script only checks the hour, so it can be run anytime between midnight and 1 AM.
# Although designed to run twice a day, it will still function correctly if ran hourly,
# as long as /etc/rsnapshot.conf is configured accordingly.
#
# Example crontab entry:
# 0 0,12 * * *    /usr/local/bin/backup
#
# Backups occur on Jan 1 at midnight(yearly), the first of the month at midnight(monthly),
# Sundays at midnight(weekly), running the script at any other time produces an hourly backup.
#
#
# NOTE: This script assumes rsnapshot is configured correctly, and will always exit without
#    producing an error. It should also be run as root if you are backing up system files.
#
#
# Rleased under the BSD license.
# http://opensource.org/licenses/bsd-license.php
#
# Copyright (c) 2010, LeeF.
# All rights reserved.
# leef (a) hushmail (dot) com
#
#
#
#
# LeeF
#

# Modify this variable accordingly.
$rsnapshot = "/usr/bin/rsnapshot";

# Modify these variables if needed. (Refer to your /etc/rsnapshot.conf)
$hourly = "hourly";
$daily = "daily";
$weekly = "weekly";
$monthly = "monthly";
$yearly = "yearly";


# Do not modify below this line.


# Set current date variables.
$hour = date('G');
$day = date('j');
$weekday = date('w');
$month = date('n');

# Yearly Backups
if ($month == 1 && $day == 1 && $hour == 0){
    exec("$rsnapshot $hourly");
    exec("$rsnapshot $daily");
    # Check if it's Sunday.
    if ($weekday == 0) exec("$rsnapshot $weekly");
    exec("$rsnapshot $monthly");
    exec("$rsnapshot $yearly");
    exit;
}
# Monthly Backups
elseif ($day == 1 && $hour == 0){
    exec("$rsnapshot $hourly");
    exec("$rsnapshot $daily");
    # Check if it's Sunday.
    if ($weekday == 0) exec("$rsnapshot $weekly");
    exec("$rsnapshot $monthly");
    exit;
}
# Weekly Backups
elseif ($weekday == 0 && $hour == 0){
    exec("$rsnapshot $hourly");
    exec("$rsnapshot $daily");
    exec("$rsnapshot $weekly");
    exit;
}
# Daily Backups
elseif ($hour == 0){
    exec("$rsnapshot $hourly");
    exec("$rsnapshot $daily");
    exit;
}
# Hourly Backups
else {
    exec("$rsnapshot $hourly");
    exit;
}
?>

And my backup beeper:

#!/bin/bash
# File:  /usr/local/bin/backup_success
#
# Add the following line to /etc/rsnapshot.conf
# cmd_postexec    /usr/local/bin/backup_success

beep -f 500 -l 100 -r 6 -d 0
beep -f 1000 -l 100 -r 3 -d 0
beep -f 2000 -l 100 -r 1 -d 0

Last edited by LeeF (2010-10-10 11:31:43)

falconindy · 2010-09-18 12:11:49

Bash critique, if you don't mind. If you add a /bin/bash shebang, script like you mean it!

Don't run date 4 times. Just run it once and call the 'read' builtin along with it:

read HOUR DAY WEEKDAY MONTH < <(date +'%k %d %w %m')

You should use (( )) instead of [ ] for numerical comparison:

if (( MONTH == 1 && DAY == 1 && HOUR == 0)); then ...

fukawi2 · 2010-09-18 12:34:57

Why couldn't cron do this?

0 0 1 jan * rsnapshot yearly
0 0 1 * *   rsnapshot monthly
0 0 * * sun rsnapshot weekly
0 0 * * *   rsnapshot daily

I feel I'm missing something this script does that the above does, apart from beeping?

Last edited by fukawi2 (2010-09-18 12:35:46)

LeeF · 2010-09-18 14:39:34

falconindy wrote:

Bash critique, if you don't mind. If you add a /bin/bash shebang, script like you mean it!
Don't run date 4 times. Just run it once and call the 'read' builtin along with it:
read HOUR DAY WEEKDAY MONTH < <(date +'%k %d %w %m')
You should use (( )) instead of [ ] for numerical comparison:
if (( MONTH == 1 && DAY == 1 && HOUR == 0)); then ...

By all means, please critique, Bash isn't my native nor preferred language, all advice is welcome. I was under the impression that "==" was for string comparison, not numeric, and therefore, 01 != 1. Can you please explain the syntactical difference between (( )) and []? I had tried to use them in a previous iteration of my script, but it had produced an error that I couldn't reconcile.

The read function is pretty sweet, I'm going to have to implement that in my script. Thank you!

P.S. Love the whole shebang and script like you mean it line, gave me a good chuckle.

fukawi2 wrote:

Why couldn't cron do this?
0 0 1 jan * rsnapshot yearly
0 0 1 * *   rsnapshot monthly
0 0 * * sun rsnapshot weekly
0 0 * * *   rsnapshot daily
I feel I'm missing something this script does that the above does, apart from beeping?

Clearly, there is something your missing, it's this thread. Please re-read my first post, I mentioned it there, but the link is somewhat hard to notice. I'd also recommend carefully reading my script, if you scroll down, you'll realize it executes a series of backup commands depending on the date. Lastly, it's the second script that beeps, not the first. Maybe your like me and not very familiar with bash scripting?

However you did offer me some new information, I didn't know I could use the abbreviation of the day or month in my crontab, so thanks for that.

I also want to mention this is my first time using rsnapshot for backups, I was under the impression that the commands had to be run in sequence, I'd appreciate any additional information on this. I've been spoiled using Symantec Backup Exec's remote agents in an enterprise environment for too long, or I've had backups managed for me because they were servers hosted in a datacenter, this however, is for my home server.

Thank you both again for your input, it is most welcome.

falconindy · 2010-09-18 18:01:03

LeeF wrote:

By all means, please critique, Bash isn't my native nor preferred language, all advice is welcome. I was under the impression that "==" was for string comparison, not numeric, and therefore, 01 != 1. Can you please explain the syntactical difference between (( )) and []? I had tried to use them in a previous iteration of my script, but it had produced an error that I couldn't reconcile.
The read function is pretty sweet, I'm going to have to implement that in my script. Thank you!
P.S. Love the whole shebang and script like you mean it line, gave me a good chuckle.

The difference isn't the equality operator, the difference is the parenthesis versus brackets. (( )) is used for arithmetic comparison, while [[ ]] is typically used for string comparison. Inside the (( )), 01 will correctly equal 1. There are some slightly different rules when dealing with (( )) over [[ ]], but I'll just leave you with these bookmarks:

http://mywiki.wooledge.org/BashGuide
http://mywiki.wooledge.org/BashFAQ

fukawi2 · 2010-09-18 23:22:28

LeeF wrote:

Clearly, there is something your missing, it's this thread. Please re-read my first post, I mentioned it there, but the link is somewhat hard to notice. I'd also recommend carefully reading my script, if you scroll down, you'll realize it executes a series of backup commands depending on the date. Lastly, it's the second script that beeps, not the first. Maybe your like me and not very familiar with bash scripting?

I'm very familiar with bash scripting; I see that it executes multiple commands depend on the date ($DAY =eq 1), that's what the first 2 crontab lines do.

You might want to read the rsnapshot docs closer, esp http://rsnapshot.org/howto/1.2/rsnapsho … automation
I use it without needing any scripting to schedule it using this:

00 */4 * * *    /usr/bin/rsnapshot hourly ; sync
30 3 * * *    /usr/bin/rsnapshot daily ; sync
00 3 * * mon    /usr/bin/rsnapshot weekly ; sync
30 2 1 * *    /usr/bin/rsnapshot monthly ; sync

LeeF · 2010-09-19 02:40:01

falconindy wrote:

The difference isn't the equality operator, the difference is the parenthesis versus brackets. (( )) is used for arithmetic comparison, while [[ ]] is typically used for string comparison. Inside the (( )), 01 will correctly equal 1. There are some slightly different rules when dealing with (( )) over [[ ]], but I'll just leave you with these bookmarks:
http://mywiki.wooledge.org/BashGuide
http://mywiki.wooledge.org/BashFAQ

Ah, thanks for the explanation, it clears up a lot. Also, thanks for those links, those are much better than the guide I had been going off of.

~~Forgot to add this: in your last example using (( )), you referenced variables without using "$", was that a typo, or was it intentional? If it was intentional, can you please explain it?~~ It works, although, if you don't mind a short explaination as to why "$" isn't needed would help me out.

Thanks again for you patience and help, they're greatly appreciated.

EDIT: I now get this error when using (( )): /usr/local/bin/backup: line 57: ((: 09: value too great for base (error token is "09")

fukawi2 wrote:

I'm very familiar with bash scripting; I see that it executes multiple commands depend on the date ($DAY =eq 1), that's what the first 2 crontab lines do.
You might want to read the rsnapshot docs closer, esp http://rsnapshot.org/howto/1.2/rsnapsho … automation
I use it without needing any scripting to schedule it using this:
00 */4 * * *    /usr/bin/rsnapshot hourly ; sync
30 3 * * *    /usr/bin/rsnapshot daily ; sync
00 3 * * mon    /usr/bin/rsnapshot weekly ; sync
30 2 1 * *    /usr/bin/rsnapshot monthly ; sync

Now like I said, I'm not really familiar with rsnapshot, but I see a potential problem with that setup you mention. What if your weekly backup job hasn't completed before your daily job is started, would this cause an error or worse, a race condition? And if it didn't cause an error, would the result be a a duplicate snapshot that isn't using links and therefore double the size?

This is a real concern for me since during the periods between "hourly" backup jobs, my backups could easily grow between 30 and 40 Gigs, and take longer than 30 minutes to complete a backup task, on my server, rsnapshot takes about 1 minute per Gig. Saying that the task *should* be done by the time the other backup task starts running just isn't good enough for me, I like certainty and precision. I also don't like spreading my backups across different times, I want them done within about an hour window, start to finish(unless enough data is added to make it take longer). I picked a 12 hour interval because the time it takes for new datasets to be fully propagated is approximately 5-8 hours, which adds a ~50% margin to the average.

After more carefully reading the rsnapshot automation howto, I get the impression that I should run the commands in the reverse order, ending with the hourly interval in my command sequence as opposed starting with it. Is that correct?

So just to reiterate, if the two tasks could run concurrently and not cause any errors or data duplication, then I will go with your solution, but I need more information fist. Because after re-reading that howto, my impression is that it seems like any two rsnapshot backup tasks running concurrently would produce a race condition.

Thanks for your continued input it's most appreciated and helpful.

Last edited by LeeF (2010-09-19 03:37:18)

fukawi2 · 2010-09-19 03:38:31

To create the next higher level of backup, rsnapshot just takes the latest of the lower level and hard links it AFAIK. The lowest level of backup (hourly) is what actually transfers data from the remote hosts.

So daily backups are just a hard-link of the latest daily, weekly is the latest daily, etc

Hardlinking is very fast, so it shouldn't take long at all to complete.

There is a lockfile to prevent multiple instances:

# if we're using a lockfile, try to add it
# (the program will bail if one exists and it's not stale)
add_lockfile();

Dieter@be · 2010-09-19 19:48:55

fukawi2 wrote:

Hardlinking is very fast, so it shouldn't take long at all to complete.

In my case, it takes about an hour. To give you an idea of the dataset size: the copying takes 2 hours.

LeeF · 2010-10-08 14:53:24

I've rewritten this in PHP because only daily and hourly backups worked in the bash version.

Arch Linux

#1 2010-09-18 09:56:28

Backup script for rsnapshot

#2 2010-09-18 12:11:49

Re: Backup script for rsnapshot

#3 2010-09-18 12:34:57

Re: Backup script for rsnapshot

#4 2010-09-18 14:39:34

Re: Backup script for rsnapshot

#5 2010-09-18 18:01:03

Re: Backup script for rsnapshot

#6 2010-09-18 23:22:28

Re: Backup script for rsnapshot

#7 2010-09-19 02:40:01

Re: Backup script for rsnapshot

#8 2010-09-19 03:38:31

Re: Backup script for rsnapshot

#9 2010-09-19 19:48:55

Re: Backup script for rsnapshot

#10 2010-10-08 14:53:24

Re: Backup script for rsnapshot

Board footer