[solved] Need help in creating a specialized "backup"-script

vuotz · 2010-09-05 07:28:42

Hey there,

I'm fairly new to Linux, having used the system for a few years now and then, but never really cared about the tools provided. So, this one here will be my first real Script that has got something to do with my system and my files I have little programming experience, namely some PHP and a little bit of Perl, but that was years ago.

So, without further ado: I want to copy new files from Dir1 to Dir2. Would be kinda easy with rsync, but here's the problem: The Files will be altered in Dir2, so a comparison with rsync would lead to doubled files. I thought about it, and came to this solution: I will create a list of files in Dir1 with find, Dirlist1. Now, everytime this script is run, it creates a new list: Dirlist2. The script then uses Diff to find out, where Dirlist2 differs from Dirlist1. These lines are the new files in Dir1, and can then be copied safely to Dir2.

So let's say, Dirlist1 looks like this:

/bla/blafile1

And Dirlist2:

/bla/blafile1
/bla/blafile2

I'd like the diff output to be

/bla/blafile2

so I can start the cp-command without much further editing.

Like I said, i'm very new to tools like diff, find etc. so I need some assistance. Is this even possible? Is there a better solution?

Edit: I've searched around a bit, and now think that comm would be a better solution.

comm Dirlist 1 Dirlist2
    /bla/blafile2
        /bla/blafile1

So, the files i need are in the second column. Therefore, i need to filter all lines that have 0 or 2 /t's in front. regexp?

Edit2: Ugh, screw that. manpages ftw. comm -13 is what I need.

Last edited by vuotz (2010-09-05 10:54:18)

litemotiv · 2010-09-05 09:02:36

Is plain cp -n not just what you need?

Moving to Workstation User.

vuotz · 2010-09-05 09:13:25

No, it is exactly what i do not want.

Say, i have file /dir1/file1. This is going to be copied to /dir2/file1. Now, in the course of time, i rename /dir2/file1 to /dir2/foobar. If i would do cp -n, i'd have file1 again. And this one would be identical to foobar! So, i want this script to "remember" if it already has copied this file, and then automatically skip it.

graysky · 2010-09-05 09:28:43

@op - What you're asking is beyond me with filenames changing. Have you considered back in time from the AUR. It is an incremental backup solution with a nice GUI.

http://aur.archlinux.org/packages.php?ID=23002

Last edited by graysky (2010-09-05 09:30:02)

litemotiv · 2010-09-05 09:29:21

vuotz wrote:

No, it is exactly what i do not want.
Say, i have file /dir1/file1. This is going to be copied to /dir2/file1. Now, in the course of time, i rename /dir2/file1 to /dir2/foobar. If i would do cp -n, i'd have file1 again. And this one would be identical to foobar! So, i want this script to "remember" if it already has copied this file, and then automatically skip it.

You said the files would be altered in Dir2, you didn't say moved or renamed.

Sometimes, it might be better to change your workflow than to create complex solutions to work around it. A file with changes could be prone to error, since losing it for instance would cause all your files to be re-copied, making a big mess of the target folder. This is entirely up to you though.

What i would do if i really wanted to achieve your workflow, is create a temp directory under 1 for copying:

Dir1/
Dir1/_new
Dir2/

Drop new files in _new and use a script that copies them to Dir2/, then moves them to Dir1.

Not a fan of solutions like this though.

quigybo · 2010-09-05 09:31:39

As fun as scripting is, I believe unison will do what you want.

vuotz · 2010-09-05 09:38:19

Thanks for the replies. litemotiv, the problem is, that the files in Dir1 come from transmission, so i don't want to touch them, once they've been placed there.

@graysky Thanks for the tip, but i don't think that I want a real backup utility. But I'll look into it.

@quigybo Thank you too, will look into it.

litemotiv · 2010-09-05 10:04:04

vuotz wrote:

the problem is, that the files in Dir1 come from transmission, so i don't want to touch them, once they've been placed there.

You can also only copy the files from Dir1 that were added or modified since you last ran the script, something like:

$ find /dir1/* -mtime -60 -exec cp {} /dir2/ \;

If you run the script with regular intervals mtime can be a fixed number, but you can also echo the date to a timestamp.txt that you use on the next run.

vuotz · 2010-09-05 10:27:39

Hey, that is a very nice idea! I think this could solve the problem. Also, there would be no fear of losing the index of already copied files. Would this script also maintain the folder structure? Thats another thing i'm afraif of. If you list the directory with find, you would get the complete file name with directories, and cp would copy just the file by the file name straight into Dir2? *opening man page*

vuotz · 2010-09-05 10:47:37

Surprise, the solution is extremely easy.

#!/bin/sh
cd $TR_TORRENT_DIR
cp "$TR_TORRENT_NAME" /dir2/

Set Transmission to run this script on every completed torrent. Works. Now i'm going to hit the wall with my head. Hard.

Thank you all for your time.

litemotiv · 2010-09-05 10:48:56

vuotz wrote:

Surprise, the solution is extremely easy.
#!/bin/sh
cd $TR_TORRENT_DIR
cp "$TR_TORRENT_NAME" /dir2/
Set Transmission to run this script on every completed torrent. Works. Now i'm going to hit the wall with my head. Hard.
Thank you all for your time.

Please mark your topic [solved].

Arch Linux

#1 2010-09-05 07:28:42

[solved] Need help in creating a specialized "backup"-script

#2 2010-09-05 09:02:36

Re: [solved] Need help in creating a specialized "backup"-script

#3 2010-09-05 09:13:25

Re: [solved] Need help in creating a specialized "backup"-script

#4 2010-09-05 09:28:43

Re: [solved] Need help in creating a specialized "backup"-script

#5 2010-09-05 09:29:21

Re: [solved] Need help in creating a specialized "backup"-script

#6 2010-09-05 09:31:39

Re: [solved] Need help in creating a specialized "backup"-script

#7 2010-09-05 09:38:19

Re: [solved] Need help in creating a specialized "backup"-script

#8 2010-09-05 10:04:04

Re: [solved] Need help in creating a specialized "backup"-script

#9 2010-09-05 10:27:39

Re: [solved] Need help in creating a specialized "backup"-script

#10 2010-09-05 10:47:37

Re: [solved] Need help in creating a specialized "backup"-script

#11 2010-09-05 10:48:56

Re: [solved] Need help in creating a specialized "backup"-script

Board footer