[SOLVED]Date -I -d string issue with HR format in shell script?

Thme · 2012-09-09 15:16:27

Edit:Solution at bottom
OK first off I'm new bash scripting and have an issue with using date -I -d 'anystring' as a variable.
the following script is for rsync to create inremental snapshots on a daily basis. Older blogs on this topic had scripts and expainations that, although the information helped me get a script that finally works, the usage of "date" in order to create a dated incremental snapshot directory each day was "old". strong quoting and escaping doesn't seem to resolve this issue.
here's a sample of the script I wrote:

#!/bin/bash
DAY0=`date -I`
DAY1=`date -I -d yesterday`
SRC="/*"
TRG="/home/maat/snapshots/$DAY0/"
LNK="/home/maat/snapshots/$DAY1/"
#The hv options are here for testing from a terminal until I put it on a cron job 
OPT="-aAXhv --delete --link-dest=$LNK"
rsync $OPT $SRC $TRG --exclude={/dev/,/proc/,/sys/,/tmp/,/run/,/mnt/,/media/,/lost+found,/home/}
exit 0

The problem is that i need to use a third DAY30 variable when completing this script so I can have it automatically rm -r a snapshot after 30 days have passed thus creating a rotating snapshot script.
The terminal command "date -I -d '30 days ago' outputs the correct date i need to pass to another local variable similar to the TRG and LNK ones I'm using. I tried escaping the hard quotes and all but can't seem to pass this string to date correctly. As can be seen the DAY0 variable passes fine since it doesn't need to be quoted to give the right output.
Any suggestions would be appreciated. I'm a bit new to using bash and writing scripts so be easy... If there's room for improvement here as well I'm open to suggestions (with a little explanation if possible) on that as well.
Solution:
Well not sure why everywhere I looked used strong quoting inside thier date variables when doing something similar to what I was trying to achieve but weak quoting did the trick...
so a variable that prints the date 30 days ago properly would use
date -d "30 days ago"
instead of
date -d '30 days ago'
when used as a variable.
in my case I'd use it to get the correct path to pass to rm -r when running the script to clean out the snapshot from 30 days back once rsync was done. creating another.

Last edited by Thme (2012-09-09 20:29:55)

rockin turtle · 2012-09-09 20:08:46

I'm not exactly sure what you are asking. You could do:

DAY30=`date -I -d '30 days ago'`

but _MUCH_ better would be to use the $(...) syntax instead of the `...` syntax.

DAY0=$(date -I)
DAY1=$(date -I -d yesterday)
DAY30=$(date -I -d '30 days ago')

Last edited by rockin turtle (2012-09-09 20:15:42)

Trilby · 2012-09-09 20:18:32

It works fine here:

$ ./script
2012-09-09, 2012-09-08, 2012-08-10

$ cat script
#!/bin/bash
DAY0=`date -I`
DAY1=`date -I -d yesterday`
DAY30=`date -I -d "30 days ago"`
echo "$DAY0, $DAY1, $DAY30"

If you include the actual script code that fails and the output it produces, we could probably identify what went wrong.

EDIT: Note that there may be another issue in the logic of this script. AFAIK, as currently described, if you have a file that was created 30 days ago (and perhaps much important information was added to it) but it has not changed since, your script will delete the original source of the file from 30 days ago. As this file has not changed, all of the more recent backups will now only include a link to the now-deleted backup from 30 days ago.

If my assessment is correct, you could use this type of backup for the daily snapshots, but use a full backup (no linking to previous backup) periodically. Or periodic (eg monthly) backups could use a logic exactly like the daily backups, but link to the previously monthly backup rather than the previous daily.

EDIT: just reread the post below: thanks for the clarification - thats good to know about how the hardlinks work. I figured I'd just as this as an edit as there is no point in bumping the thread for this.

Last edited by Trilby (2012-09-12 22:31:47)

Thme · 2012-09-09 22:37:06

not at my computer at the moment with internet access but to answer you trilby I decided to create a test play script pointing to a directory I wanted to test in. so I used the same layout as the rsync script. focusing on date -I -d 'x days ago' as the variable to pass to mkdir to create some dated directories inside The test dir. as a result date kept throwing missing option after -d and as a result mkdir couldn't create the dated directories. when I included $DAYx at the end of the path following mkdir like this
mkdir /path/to/test/dir/$DAYx/
I'll post the test script when I can as an example but I did find that weak quoting passed "n days ago" to date -I -d correctly. In other words, the example you used that was the one that worked for me but replacing " with ' didn't.
And to rockin turtle, I haven't thought about parentheses... I'm still learning the general syntax with scripting. 2 days ago I had no clue what a variable was in bash. I'm teaching myself the hard way: trial and error.

edit: also trilby the rsync script creates incremental backups with hardlinks when using the --link-dest= option and comparing it to the last recent snapshot. this gives the illusion that they are full snapshots are far as any other application are concerned. Even rsync when reading that data unless you tell it to specifically treat hard links as links with -H. a single file can have multiple hardlinks to the same inodes where that data is stored. removing the oldest will only remove the oldest data that had changed and not the rest. the other snapshots will still point to the same data. for 30 days worth of backups on my current install (which is about 7.5 gigs) ill only need around 15 - 20max to keep 30 days worth of updates and be able to roll back if something seriously goes wrong(have an external drive for keeping a duplicate of the snapshots offsite.)

Last edited by Thme (2012-09-10 06:17:43)

Arch Linux

#1 2012-09-09 15:16:27

[SOLVED]Date -I -d string issue with HR format in shell script?

#2 2012-09-09 20:08:46

Re: [SOLVED]Date -I -d string issue with HR format in shell script?

#3 2012-09-09 20:18:32

Re: [SOLVED]Date -I -d string issue with HR format in shell script?

#4 2012-09-09 22:37:06

Re: [SOLVED]Date -I -d string issue with HR format in shell script?

Board footer