Arch Linux

wifon

Member

Registered: 2013-02-03

Posts: 14

grep doesn't filter everything?

Hi
I've got a question on grep. I'm long time linux user, yet the question and my problem is quite stupid

I tried to filter pacman's log today, looking for a cinnamon update. I've put:

cat /var/log/pacman.log | grep cinnamon

in terminal and surpisingly the whole output is:

[2017-02-01 16:19] [PACMAN] Running 'pacman -Syu cinnamon'
[2017-02-01 16:20] [ALPM] installed cinnamon-desktop (3.2.4-2)
[2017-02-01 16:20] [ALPM] installed cinnamon-settings-daemon (3.2.1-2)
[2017-02-01 16:20] [ALPM] installed cinnamon-session (3.2.0-1)
[2017-02-01 16:20] [ALPM] installed cinnamon-translations (3.2.2-1)
[2017-02-01 16:20] [ALPM] installed cinnamon-menus (3.2.0-1)
[2017-02-01 16:20] [ALPM] installed cinnamon-control-center (3.2.1-1)
[2017-02-01 16:20] [ALPM] installed cinnamon-screensaver (3.2.13-3)
[2017-02-01 16:20] [ALPM] installed cinnamon (3.2.8-1)
Binary file (standard input) matches

While if I just cat the file, I also see:

[2018-04-28 08:20] [ALPM] upgraded cinnamon-screensaver (3.6.1-1 -> 3.8.0-1)
[2018-04-28 08:20] [ALPM] upgraded cinnamon-session (3.8.0-1 -> 3.8.1-1)
(...)
[2018-04-28 08:20] [ALPM] upgraded cinnamon (3.6.7-2 -> 3.8.0-1)
[2018-04-28 08:20] [ALPM] upgraded cinnamon-translations (3.6.4-1 -> 3.8.0-1)

Have I been using grep wrong for my entire life?

Last edited by wifon (2018-04-29 08:58:11)

loqs

Member

Registered: 2014-03-06

Posts: 19,113

Re: grep doesn't filter everything?

Binary file (standard input) matches

cat /var/log/pacman.log | grep -a cinnamon
grep -a cinnamon /var/log/pacman.log

wifon

Member

Registered: 2013-02-03

Posts: 14

Re: grep doesn't filter everything?

OK, so if I see "Binary file (standard input) matches", should I use -a switch, right?

loqs

Member

Registered: 2014-03-06

Posts: 19,113

Re: grep doesn't filter everything?

The -a switch will class the file as text but it seems strange that grep found something that made it think the log was binary.

       --binary-files=TYPE
              If  a  file's  data  or metadata indicate that the file contains
              binary data, assume that the file is  of  type  TYPE.   Non-text
              bytes  indicate  binary data; these are either output bytes that
              are improperly encoded for the current  locale,  or  null  input
              bytes when the -z option is not given.

              By  default, TYPE is binary, and when grep discovers that a file
              is binary it suppresses any further output, and instead  outputs
              either  a one-line message saying that a binary file matches, or
              no message if there is no match.

Lone_Wolf

Administrator

From: Netherlands, Europe

Registered: 2005-10-04

Posts: 15,557

Re: grep doesn't filter everything?

When i open pacman.log with krusader internal viewer, it complains like this ;

The file /var/log/pacman.log was opened with UTF-8 econcoding but contained invalid characters.

Perhaps grep and krusader choke on the same thing ?

---

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky

seth

Member

From: Won't reply 2 private help req

Registered: 2012-09-03

Posts: 78,849

Re: grep doesn't filter everything?

file /var/log/pacman.log  
/var/log/pacman.log: UTF-8 Unicode text, with very long lines, with escape sequences
grep --color='auto' -P -n "[\x80-\xFF]" /var/log/pacman.log

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

Lone_Wolf

Administrator

From: Netherlands, Europe

Registered: 2005-10-04

Posts: 15,557

Re: grep doesn't filter everything?

Looks like it's indeed those codes, all the symbols below come from names with special lettering. 

The last 3 in the list come from this line in pacman.log .

21724:[2018-04-25 13:08] [ALPM-SCRIPTLET] Certificate added: C=TR, L=Ankara, O=TÜRKTRUST Bilgi İletişim ve Bilişim Güvenliği Hizmetleri A.Ş., CN=TÜRKTRUST Elektronik Sertifika Hizmet Sağlayıcısı H5

$ grep -ao -P -n "[\x80-\xFF]" /var/log/pacman.log
12321: 
12321: 
12322: 
12322: 
12322: 
12322: 
12323: 
12323: 
15958:ä
15962:á
15963:ø
15964:ó
15964:ö
15986:ü
15986:ö
15995:é
16004:é
16007:ä
16011:ö
16501:ä
16505:á
16506:ø
16507:ó
16507:ö
16530:ü
16530:ö
16539:é
16548:é
16551:ä
16555:ö
18010:ä
18015:á
18016:ø
18017:ó
18017:ö
18040:ü
18040:ö
18049:é
18058:é
18061:ä
18065:ö
19252:ä
19257:á
19258:ø
19259:ó
19259:ö
19283:ü
19283:ö
19292:é
19301:é
19304:ä
19308:ö
21081:ä
21086:á
21087:ø
21088:ó
21088:ö
21114:ü
21114:ö
21123:é
21132:é
21135:ä
21139:ö
22656:ä
22661:á
22662:ø
22663:ó
22663:ö
22689:ü
22689:ö
22698:é
22703:é
22704:é
22705:é
22709:é
22713:ä
22717:ö
23146:ä
23151:á
23152:ø
23153:ó
23153:ö
23179:ü
23179:ö
23188:é
23193:é
23197:é
23201:ä
23205:ö
24102:ä
24108:á
24109:ø
24110:ó
24110:ö
24137:ü
24137:ö
24146:é
24151:é
24155:é
24159:ä
24163:ö
121540:ú
121540:í
121540:á
121540:ó
121540:ú
121540:í
121540:á
121584:Ü
121584:ü
121584:Ü
121680:ú
121680:í
121680:á
121680:ó
121680:ú
121680:í
121680:á
121724:Ü
121724:ü
121724:Ü
$

---

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky