You are not logged in.

#1 2012-09-28 14:26:42

alphaniner
Member
From: Ancapistan
Registered: 2010-07-12
Posts: 2,810

[Solved] grep -f vs grep in loop

I have a file with 16k+ md5sums (all.dups).  The sums are duplicates between two full md5sum hash files (drive1.md5 and drive2.md5), but it contains just the sums.  When I run

grep -f all.dups drive1.md5 >drive1.dups

grep uses 100% cpu, ram usage increases constantly, and nothing seems to happen.  I killed it after ~6mins and nothing was written to drive1.dups.

However, when I run

for i in "${A[@]}"        #array A contains contents of all.dups
do
grep "$i" drive1.md5 >>drive1.dups
done

the operation finished in about a minute.  Am I using the -f flag wrong or ... ???

Last edited by alphaniner (2012-09-28 14:57:14)


But whether the Constitution really be one thing, or another, this much is certain - that it has either authorized such a government as we have had, or has been powerless to prevent it. In either case, it is unfit to exist.
-Lysander Spooner

Offline

#2 2012-09-28 14:51:14

skanky
Member
From: WAIS
Registered: 2009-10-23
Posts: 1,847

Re: [Solved] grep -f vs grep in loop


"...one cannot be angry when one looks at a penguin."  - John Ruskin
"Life in general is a bit shit, and so too is the internet. And that's all there is." - scepticisle

Offline

#3 2012-09-28 14:56:24

alphaniner
Member
From: Ancapistan
Registered: 2010-07-12
Posts: 2,810

Re: [Solved] grep -f vs grep in loop

Thanks.


But whether the Constitution really be one thing, or another, this much is certain - that it has either authorized such a government as we have had, or has been powerless to prevent it. In either case, it is unfit to exist.
-Lysander Spooner

Offline

Board footer

Powered by FluxBB