Arch Linux

oib111

Member

Registered: 2009-06-20

Posts: 87

Copy based on File Differences

Is there a program that will inspect two directories (an up to date and out of date), then if certain files aren't the same or aren't there they will copy from the up to date directory the out of date directory?

kgas

Member

From: Qatar

Registered: 2008-11-08

Posts: 718

Re: Copy based on File Differences

rsync will do this.

oib111

Member

Registered: 2009-06-20

Posts: 87

Re: Copy based on File Differences

Isn't rsync for backing up files?

Last edited by oib111 (2009-08-17 17:21:37)

Trent

Member

From: Baltimore, MD (US)

Registered: 2009-04-16

Posts: 990

Re: Copy based on File Differences

Yes, exactly.  Keeping two copies of the same information in two different places, need a tool to keep them synchronized... sounds like a backup to me.  If not, what else do you need it for?

brisbin33

Member

From: boston, ma

Registered: 2008-07-24

Posts: 1,796

Website

Re: Copy based on File Differences

rsync [SRC] [DEST].  this will sync DEST so it looks like SRC.  great for backups.  i don't think there is any way to pass two directories to rsync have it sort out on its own which is SRC and which is DEST based on up-to-date-ness-of-directory-contents.

you could have a shell script to check mod times on the parent directories via ls, then pass them to rsync in the proper order.  but then what happens if you edit ten files in dirA/ then touch some other file in dirB/... same problem if you look at the contents of dir{A,B}, say you edit ten files in dirA/dir1/ then touch some other file in dirB/dir1/... it's turtles all the way down...

anyway, if rsync can do it i'd love to hear about it.  but if i read the OP correctly, its a more involved task than backing up some files.

/edit: oh, and as for this:

Keeping two copies of the same information in two different places, need a tool to keep them synchronized...

i'd suggest a bind mount or ln -s for that situation.  'backup', to me, implies a one-way sync.

Last edited by brisbin33 (2009-08-17 19:21:31)

---

//github/

ngoonee

Forum Fellow

From: Between Thailand and Singapore

Registered: 2009-03-17

Posts: 7,356

Re: Copy based on File Differences

I think he's not so much doing a one-way sync as two-way synchronization...

So, for example, he has machines A and B, any changes in A are reflected to B, any changes in B are reflected to A.

I use Unison for this, its in the repos. rsync and the GUI for it, Grsync, are only one-way and for my backups, but Unison does full two-way, pretty stable as well.

---

Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.

oib111

Member

Registered: 2009-06-20

Posts: 87

Re: Copy based on File Differences

I read about unison, didn't check if it was in the repos though, thanks! At the moment though I only need one way synchronization, so I'm going to use rsync. I'm having some problems though. I'm running rsync with sudo and I'm getting these errors:

rsync: recv_generator: mkdir "/mnt/windows/d/My Music/Beirut" failed: Operation not permitted (1)
*** Skipping any contents from this failed directory ***
rsync: recv_generator: mkdir "/mnt/windows/d/My Music/Bush/BUSH - DISCOGRAPHY" failed: Operation not permitted (1)
*** Skipping any contents from this failed directory ***
rsync: recv_generator: mkdir "/mnt/windows/d/My Music/Fuel/Fuel - Discography" failed: Operation not permitted (1)
*** Skipping any contents from this failed directory ***
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Citizen Cope/.Citizen Cope-SelfTitled-1-Intro.mp3.bdMhUV" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Citizen Cope/.Citizen Cope-SelfTitled-10-Appetite (For Lightin' Dynamite).mp3.MegJ6A" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Citizen Cope/.Citizen Cope-SelfTitled-11-Theresa Prelude.mp3.lLx5xg" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Citizen Cope/.Citizen Cope-SelfTitled-12-Theresa.mp3.4qUw4V" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Citizen Cope/.Citizen Cope-SelfTitled-13-Hold'On.mp3.BwusUB" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Citizen Cope/.Citizen Cope-SelfTitled-14-Mandy.mp3.SlxFki" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Citizen Cope/.Citizen Cope-SelfTitled-2-Contact.mp3.v08HbZ" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Citizen Cope/.Citizen Cope-SelfTitled-3-If There's Love.mp3.iibEvG" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Citizen Cope/.Citizen Cope-SelfTitled-4-Let The Drummer Kick.mp3.vBS9ro" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Citizen Cope/.Citizen Cope-SelfTitled-5-Mistaken I.D..mp3.SvJuf7" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Citizen Cope/.Citizen Cope-SelfTitled-6-200,000 (In Counterfeit 50 Dollar Bills).mp3.DWTYyQ" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Citizen Cope/.Citizen Cope-SelfTitled-7-Salvation.mp3.WobehA" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Citizen Cope/.Citizen Cope-SelfTitled-8-Hands Of The Saints.mp3.VaIswk" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Citizen Cope/.Citizen Cope-SelfTitled-9-Comin' Back.mp3.inFL54" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Every Waking Moment/.01 Back Together.mp3.hNGA2P" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Every Waking Moment/.02 Every Waking Moment.mp3.4Yn2cB" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Every Waking Moment/.03 Friendly Fire.mp3.7cNaEm" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Every Waking Moment/.04 More Than It Seems.mp3.mbKyl8" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Every Waking Moment/.05 Brother Lee.mp3.LmzaqU" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Every Waking Moment/.06 107 Degrees.mp3.80xmLG" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Every Waking Moment/.07 Somehow.mp3.zlg1st" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Every Waking Moment/.08 John Lennon.mp3.UQe2tg" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Every Waking Moment/.09 All Dressed Up.mp3.XyKFS3" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Every Waking Moment/.10 Awe.mp3.KQrvFR" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - Every Waking Moment/.11 Left For Dead.mp3.x2Pj0F" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - The Clarence Greenwood Recordings/.Citizen Cope-Clarence Greenwood Recordings-1-Nite Becomes Day.mp3.qRMayu" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - The Clarence Greenwood Recordings/.Citizen Cope-Clarence Greenwood Recordings-10-Fame.mp3.r6tLIj" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - The Clarence Greenwood Recordings/.Citizen Cope-Clarence Greenwood Recordings-11-Deep.mp3.8Wvrl9" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - The Clarence Greenwood Recordings/.Citizen Cope-Clarence Greenwood Recordings-2-Pablo Picasso.mp3.3S8nmZ" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - The Clarence Greenwood Recordings/.Citizen Cope-Clarence Greenwood Recordings-3-My Way Home.mp3.uTg1QP" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - The Clarence Greenwood Recordings/.Citizen Cope-Clarence Greenwood Recordings-4-Son's Gonna Rise.mp3.v8eaBG" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - The Clarence Greenwood Recordings/.Citizen Cope-Clarence Greenwood Recordings-5-Sideways.mp3.u4vwCx" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - The Clarence Greenwood Recordings/.Citizen Cope-Clarence Greenwood Recordings-6-Penitentiary.mp3.fVfQ0o" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - The Clarence Greenwood Recordings/.Citizen Cope-Clarence Greenwood Recordings-7-Hurricane Waters.mp3.mzIPGg" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - The Clarence Greenwood Recordings/.Citizen Cope-Clarence Greenwood Recordings-8-D'Artagnan's Theme.mp3.LSkFF8" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/Citizen Cope - The Clarence Greenwood Recordings/.Citizen Cope-Clarence Greenwood Recordings-9-Bullet and a Target.mp3.ohe0Z0" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/The Clarence Greenwood Recordings/.Citizen Cope - 01 - Nite Becomes Day.flac.dic1ET" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/The Clarence Greenwood Recordings/.Citizen Cope - 02 - Pablo Picasso.flac.4BtKaP" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/The Clarence Greenwood Recordings/.Citizen Cope - 05 - Sideways.flac.VRpdQM" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/The Clarence Greenwood Recordings/.Citizen Cope - 06 - Penitentiary.flac.g3D8xM" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/The Clarence Greenwood Recordings/.Citizen Cope - 07 - Hurricane Waters.flac.BWcmUO" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/The Clarence Greenwood Recordings/.Citizen Cope - 08 - D'Artagnan's Theme.flac.0v7PfT" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/The Clarence Greenwood Recordings/.Citizen Cope - 09 - Bullet And A Target.flac.xZvsZZ" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/The Clarence Greenwood Recordings/.Citizen Cope - 10 - Fame.flac.QDnOl9" failed: Permission denied (13)
rsync: mkstemp "/mnt/windows/d/My Music/Citizen Cope/The Clarence Greenwood Recordings/.Citizen Cope - 11 - Deep.flac.tMgjzk" failed: Permission denied (13)
rsync: recv_generator: mkdir "/mnt/windows/d/My Music/Led Zeppelin" failed: Operation not permitted (1)
*** Skipping any contents from this failed directory ***
rsync: recv_generator: mkdir "/mnt/windows/d/My Music/Live/LIVE - DISCOGRAPHY [CHANNEL NEO]" failed: Operation not permitted (1)
*** Skipping any contents from this failed directory ***
rsync: recv_generator: mkdir "/mnt/windows/d/My Music/Live/Live - Songs From Black Mountain.NLT-Release" failed: Operation not permitted (1)
*** Skipping any contents from this failed directory ***
rsync: recv_generator: mkdir "/mnt/windows/d/My Music/Ludo/Ludo" failed: Operation not permitted (1)
*** Skipping any contents from this failed directory ***
rsync: recv_generator: mkdir "/mnt/windows/d/My Music/Nine Inch Nails/NIN" failed: Operation not permitted (1)
*** Skipping any contents from this failed directory ***
rsync: recv_generator: mkdir "/mnt/windows/d/My Music/Queens of the Stoneage" failed: Operation not permitted (1)
*** Skipping any contents from this failed directory ***
rsync: recv_generator: mkdir "/mnt/windows/d/My Music/Seether/SEETHER - DISCOGRAPHY [CHANNEL NEO]" failed: Operation not permitted (1)
*** Skipping any contents from this failed directory ***
rsync: recv_generator: mkdir "/mnt/windows/d/My Music/Social Distortion/Social Distortion Complete Collection" failed: Operation not permitted (1)
*** Skipping any contents from this failed directory ***
rsync: recv_generator: mkdir "/mnt/windows/d/My Music/The Who/The Who - Discography (Mp3@320Kbps)" failed: Operation not permitted (1)
*** Skipping any contents from this failed directory ***
rsync: recv_generator: mkdir "/mnt/windows/d/My Music/Zeppelin" failed: Operation not permitted (1)
*** Skipping any contents from this failed directory ***
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1039) [sender=3.0.6]

Last edited by oib111 (2009-08-19 02:56:15)

Statix

Member

From: Hangzhou, China

Registered: 2008-02-16

Posts: 240

Re: Copy based on File Differences

Are you trying to copy to a NTFS partition?

The default ntfs driver used by the mount command is read-only. You need to install the ntfs-3g package and mount it with ntfs-3g as the filesystem.

For example

mount -t ntfs-3g /dev/sda3 /mnt/windows

---

Madly in love with Arch64, Openbox, DotA, and of course... penguins!
Happy to help if you're not a Help Vampire. Use your wonderful resources like ArchWiki, Google, and our wonderful search page.

oib111

Member

Registered: 2009-06-20

Posts: 87

Re: Copy based on File Differences

I'll try that out, thanks!