Arch Linux

thom_raindog

Member

Registered: 2008-04-29

Posts: 175

Comparing files in a folder

Not sure this is a Newbie Corner type question, but I did not see where else to go, so here it is...
Scenario: I have two seperate folders that contain a lot of files, some again in sub-folders. Now I need to find out which files happen to be in both folders, and they must be identical as well. I know how to run diff to compare two files, but that barely scratches the surface of what to do here, seeing as we talk about several dozen files in both folders, possibly even arranged in folders named differently.

I am looking for an elegant solution to this that goes beyond comparing them visually in dolphin.... Any hints?

jwbirdsong

Member

From: Western KS USA

Registered: 2006-11-25

Posts: 173

Re: Comparing files in a folder

Meld from extra has a compare dir (folders) options.  See if that will fit the bill for what you need.

---

PLEASE read and try to FIX/FILE BUGS instead of assuming other have/will.

davvil

Member

Registered: 2008-05-06

Posts: 165

Re: Comparing files in a folder

You can use find and diff for this, something in the line of (only partially tested)

for f in `find $dir1 -type f`; do find $dir2 -name `basename $f` -exec diff -q $f {} \;; done

stqn

Member

Registered: 2010-03-19

Posts: 1,191

Website

Re: Comparing files in a folder

"diff -qr dir1 dir2" works fine, except for special files like symlinks that point to nonexistent locations (I discovered this yesterday when checking my new arch copy...) diff will report that it can't open the file in such case.

I used meld to avoid this problem, but it used a lot of memory for some files and that made linux close all running programs, which was bad.

Edit: ah, if "possibly even arranged in folders named differently" means what I think it means, then davvil's solution is closer to what you need, yeah...

Last edited by stqn (2010-11-18 01:15:40)

ewaller

Administrator

From: Pasadena, CA

Registered: 2009-07-13

Posts: 19,774

Re: Comparing files in a folder

I kind of like:

find . -exec md5sum {} \;  | sort 

start in the directory of interest, or replace the '.' with the starting path

Edit: Actually, this depends on the directories being nested.  If they are not, run the find on each directory, concatanate the results, and pipe that through sort.   If you don't want to comb through all of this, create a sed program to look for the duplicates.  I leave this as an exercise for the reader

Last edited by ewaller (2010-11-18 05:13:24)

---

Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way