Arch Linux

alphaniner

Member

From: Ancapistan

Registered: 2010-07-12

Posts: 2,810

[Solved] grep -f vs grep in loop

I have a file with 16k+ md5sums (all.dups).  The sums are duplicates between two full md5sum hash files (drive1.md5 and drive2.md5), but it contains just the sums.  When I run

grep -f all.dups drive1.md5 >drive1.dups

grep uses 100% cpu, ram usage increases constantly, and nothing seems to happen.  I killed it after ~6mins and nothing was written to drive1.dups.

However, when I run

for i in "${A[@]}"        #array A contains contents of all.dups
do
grep "$i" drive1.md5 >>drive1.dups
done

the operation finished in about a minute.  Am I using the -f flag wrong or ... ???

Last edited by alphaniner (2012-09-28 14:57:14)

---

But whether the Constitution really be one thing, or another, this much is certain - that it has either authorized such a government as we have had, or has been powerless to prevent it. In either case, it is unfit to exist.
-Lysander Spooner

skanky

Member

From: WAIS

Registered: 2009-10-23

Posts: 1,847

Re: [Solved] grep -f vs grep in loop

It's a known bug and has been around for a while:

http://www.mail-archive.com/bug-grep%40 … 00628.html
http://www.fam.tuwien.ac.at/~schamane/_ … tern_files

---

"...one cannot be angry when one looks at a penguin."  - John Ruskin
"Life in general is a bit shit, and so too is the internet. And that's all there is." - scepticisle

alphaniner

Member

From: Ancapistan

Registered: 2010-07-12

Posts: 2,810

Re: [Solved] grep -f vs grep in loop

Thanks.

---

But whether the Constitution really be one thing, or another, this much is certain - that it has either authorized such a government as we have had, or has been powerless to prevent it. In either case, it is unfit to exist.
-Lysander Spooner