Format of pacman.log

alphaniner · 2014-01-08 22:58:47

What factors can contribute to the format of the log? Another thread prompted me to run file on mine, which returned "Non-ISO extended-ASCII text". If I cut it in half, the second half is reported as "UTF-8 Unicode text". Another poster said his was reported as simple "ASCII text". And the OP's is reported as "data".

jasonwryan · 2014-01-09 00:20:46

┌─[Shiv ~ ]
└─╼ file /var/log/pacman.log
/var/log/pacman.log: UTF-8 Unicode text, with very long lines

┌─[Centurion ~: «SSH»]
└─╼ file /var/log/pacman.log
/var/log/pacman.log: ASCII text, with escape sequences

skunktrader · 2014-01-09 00:32:13

$ file /var/log/pacman.log
/var/log/pacman.log: data

karol · 2014-01-09 01:52:26

'data' isn't so good, it means 'file' doesn't know what it is:

man file wrote:

Any file that cannot be identified as having been written in any of the character sets listed above is simply said to be “data”.

Inxsible · 2014-01-09 04:29:09

[inxsible ~ ]$ file /var/log/pacman.log 
/var/log/pacman.log: UTF-8 Unicode text
[inxsible ~ ]$

ewaller · 2014-01-09 04:33:09

ewaller$@$odin ~ 1020 %file /var/log/pacman.log 
/var/log/pacman.log: ASCII text, with very long lines
ewaller$@$odin ~ 1021 %

I like to be different.

WonderWoofy · 2014-01-09 05:06:04

ewaller wrote:

ewaller$@$odin ~ 1020 %file /var/log/pacman.log 
/var/log/pacman.log: ASCII text, with very long lines
ewaller$@$odin ~ 1021 %

I like to be different.

Nah, I get that on my machine too.

% file /var/log/pacman.log                                                                                                                                                                     
/var/log/pacman.log: ASCII text, with very long lines

But on my file server...

% file /var/log/pacman.log                                                                                                                                                                     
/var/log/pacman.log: UTF-8 Unicode text

lolilolicon · 2014-01-09 05:45:34

Mine says ASCII too, and "with very long lines"... merely because of a 300+ character line.

For those not getting ASCII, does this produce anything?

grep -Pn '[^\x00-\x7f]' /var/log/pacman.log

EDIT: better code...

Last edited by lolilolicon (2014-01-09 05:53:29)

WonderWoofy · 2014-01-09 05:48:22

lolilolicon wrote:

Mine says ASCII too, and "with very long lines"... merely because of a 300+ character line.
For those not getting ASCII, does this produce anything?
grep -Pv '[\x00-\x7f]' /var/log/pacman.log

Nope. (UTF-8 Unicode text)

karol · 2014-01-09 05:48:51

$ grep -Pv '[\x00-\x7f]' /var/log/pacman.log
$ file /var/log/pacman.log
/var/log/pacman.log: UTF-8 Unicode text, with very long lines

x33a · 2014-01-09 06:11:48

karol wrote:

'data' isn't so good, it means 'file' doesn't know what it is:
man file wrote:
Any file that cannot be identified as having been written in any of the character sets listed above is simply said to be “data”.

Woah, I was wondering how skunktrader got 'data', but even I get 'data' too! Though with perl-file-mimeinfo, I get

$ mimetype /var/log/pacman.log
/var/log/pacman.log: text/x-log

lolilolicon wrote:

For those not getting ASCII, does this produce anything?
grep -Pn '[^\x00-\x7f]' /var/log/pacman.log

Yeah, I get

1923:[2012-11-16 09:33] gpg: key 824B18E8: "Thomas Bächler (Arch Linux Master Key) <thomas@master-key.archlinux.org>" not changed
1928:[2012-11-16 09:33] gpg: key EC133BAD: "Angel Velásquez <angvp@archlinux.org>" not changed
1929:[2012-11-16 09:33] gpg: key 5CF9C8D4: "Alexander Rødseth <rodseth@gmail.com>" not changed
1930:[2012-11-16 09:33] gpg: key 0901C163: "Balló György <ballogyor@gmail.com>" not changed
1956:[2012-11-16 09:33] gpg: key 589874AB: "Jürgen Hötzel <juergen@hoetzel.info>" not changed
1965:[2012-11-16 09:33] gpg: key 2D1493D2: "Rémy Oudompheng <remy@archlinux.org>" not changed
1966:[2012-11-16 09:33] gpg: key 70E80477: "Роман Кирилич (Roman Kyrylych) <roman@archlinux.org>" not changed
1974:[2012-11-16 09:33] gpg: key AB441196: "Stéphane Gaudreault <stephane@archlinux.org>" not changed
1977:[2012-11-16 09:33] gpg: key 8E4B1A25: "Thomas Bächler <thomas@archlinux.org>" not changed
1981:[2012-11-16 09:33] gpg: key 295AFBF4: "Thorsten Töpper <atsutane@freethoughts.de>" not changed
2889:[2013-01-15 17:01] gpg: key 824B18E8: "Thomas Bächler (Arch Linux Master Key) <thomas@master-key.archlinux.org>" not changed
2894:[2013-01-15 17:01] gpg: key EC133BAD: "Angel Velásquez <angvp@archlinux.org>" not changed
2895:[2013-01-15 17:01] gpg: key 5CF9C8D4: "Alexander Rødseth <rodseth@gmail.com>" not changed
2896:[2013-01-15 17:01] gpg: key 0901C163: "Balló György <ballogyor@gmail.com>" not changed
2922:[2013-01-15 17:01] gpg: key 589874AB: "Jürgen Hötzel <juergen@hoetzel.info>" not changed
2931:[2013-01-15 17:01] gpg: key 2D1493D2: "Rémy Oudompheng <remy@archlinux.org>" not changed
2932:[2013-01-15 17:01] gpg: key 70E80477: "Роман Кирилич (Roman Kyrylych) <roman@archlinux.org>" not changed
2936:[2013-01-15 17:01] gpg: key 2072D77A: "Sébastien Luttringer <seblu@seblu.net>" not changed
2940:[2013-01-15 17:01] gpg: key AB441196: "Stéphane Gaudreault <stephane@archlinux.org>" not changed
2944:[2013-01-15 17:01] gpg: key 8E4B1A25: "Thomas Bächler <thomas@archlinux.org>" not changed
2948:[2013-01-15 17:01] gpg: key 295AFBF4: "Thorsten Töpper <atsutane@freethoughts.de>" not changed
3175:[2013-02-01 13:00] gpg: key 824B18E8: "Thomas Bächler (Arch Linux Master Key) <thomas@master-key.archlinux.org>" not changed
3180:[2013-02-01 13:00] gpg: key EC133BAD: "Angel Velásquez <angvp@archlinux.org>" not changed
3181:[2013-02-01 13:00] gpg: key 5CF9C8D4: "Alexander Rødseth <rodseth@gmail.com>" not changed
3182:[2013-02-01 13:00] gpg: key 0901C163: "Balló György <ballogyor@gmail.com>" not changed
3208:[2013-02-01 13:00] gpg: key 589874AB: "Jürgen Hötzel <juergen@hoetzel.info>" not changed
3217:[2013-02-01 13:00] gpg: key 2D1493D2: "Rémy Oudompheng <remy@archlinux.org>" not changed
3218:[2013-02-01 13:00] gpg: key 70E80477: "Роман Кирилич (Roman Kyrylych) <roman@archlinux.org>" not changed
3222:[2013-02-01 13:00] gpg: key 2072D77A: "Sébastien Luttringer <seblu@seblu.net>" not changed
3226:[2013-02-01 13:00] gpg: key AB441196: "Stéphane Gaudreault <stephane@archlinux.org>" not changed
3230:[2013-02-01 13:00] gpg: key 8E4B1A25: "Thomas Bächler <thomas@archlinux.org>" not changed
3234:[2013-02-01 13:00] gpg: key 295AFBF4: "Thorsten Töpper <atsutane@freethoughts.de>" 4 new signatures
3726:[2013-02-26 22:16] gpg: key 824B18E8: "Thomas Bächler (Arch Linux Master Key) <thomas@master-key.archlinux.org>" not changed
3732:[2013-02-26 22:16] gpg: key EC133BAD: "Angel Velásquez <angvp@archlinux.org>" not changed
3733:[2013-02-26 22:16] gpg: key 5CF9C8D4: "Alexander Rødseth <rodseth@gmail.com>" not changed
3734:[2013-02-26 22:16] gpg: key 0901C163: "Balló György <ballogyor@gmail.com>" not changed
3761:[2013-02-26 22:16] gpg: key 589874AB: "Jürgen Hötzel <juergen@hoetzel.info>" not changed
3770:[2013-02-26 22:16] gpg: key 2D1493D2: "Rémy Oudompheng <remy@archlinux.org>" not changed
3771:[2013-02-26 22:16] gpg: key 70E80477: "Роман Кирилич (Roman Kyrylych) <roman@archlinux.org>" not changed
3775:[2013-02-26 22:16] gpg: key 2072D77A: "Sébastien Luttringer <seblu@seblu.net>" not changed
3779:[2013-02-26 22:16] gpg: key AB441196: "Stéphane Gaudreault <stephane@archlinux.org>" not changed
3783:[2013-02-26 22:16] gpg: key 8E4B1A25: "Thomas Bächler <thomas@archlinux.org>" not changed
3787:[2013-02-26 22:16] gpg: key 295AFBF4: "Thorsten Töpper <atsutane@freethoughts.de>" not changed

loafer · 2014-01-09 06:36:16

$  file /var/log/pacman.log 
/var/log/pacman.log: ISO-8859 text

$  grep -Pn '[^\x00-\x7f]' /var/log/pacman.log
3753:[2012-11-12 12:34] gpg: key 824B18E8: "Thomas Bchler (Arch Linux Master Key) <thomas@master-key.archlinux.org>" not changed

...

lolilolicon · 2014-01-09 06:46:43

loafer wrote:

$  file /var/log/pacman.log 
/var/log/pacman.log: ISO-8859 text

$  grep -Pn '[^\x00-\x7f]' /var/log/pacman.log
3753:[2012-11-12 12:34] gpg: key 824B18E8: "Thomas Bchler (Arch Linux Master Key) <thomas@master-key.archlinux.org>" not changed

...

Apparently FluxBB has eaten the ISO-8859-1 encoded "ä". So the answer is the text encoding in pacman.log really depends on your locale settings? Maybe people who get "data" changed locale at some point (making pacman.log a mixed bag of different text encodings)?

x33a · 2014-01-09 06:59:41

lolilolicon wrote:

So the answer is the text encoding in pacman.log really depends on your locale settings? Maybe people who get "data" changed locale at some point (making pacman.log a mixed bag of different text encodings)?

I haven't changed my locale, but either I manually edited pacman.log at some time, or I used Xyne's pacman.log compacting script (I can't remember if I just tested it, or did replace my log with the compacted one).

alphaniner · 2014-01-09 14:05:27

lolilolicon wrote:

grep -Pn '[^\x00-\x7f]' /var/log/pacman.log

Same with others, a lot of "gpg: key" lines. But I also noticed that the encoding apparently changed between my 2012-07-02 and 2012-08-16 updates:

3160:[2012-07-02 17:57] gpg: key AB441196: "St#phane Gaudreault <stephane@archlinux.org>" not changed
...
3749:[2012-08-16 18:02] gpg: key AB441196: "Stéphane Gaudreault <stephane@archlinux.org>" not changed

In the terminal the # was a 'box'. FF showed it as a question mark in diamond but the forum ate it. All the other non-ASCII characters before 08-16 were 'boxes' too. I guess that explains the disparity between the reported type of the two halves of my log.

Trilby · 2014-01-09 14:10:48

So ~~Allan~~Stephane broke it?

Now, we need a new t-shirt line.

alphaniner · 2014-01-09 14:16:53

Apologies to Stéphane for dragging his good (as long as you're not using ASCII ) name into this...

fsckd · 2014-01-09 14:43:25

The pacman.log from an old computer of mine,

ASCII text, with CR, LF line terminators, with escape sequences

I have no idea how it got those line terminators.

halo · 2014-01-09 18:01:47

I get data, but have never changed my locale or edited the log. Is there any ramifications to having a data formatted log? It still seems to do everything it should be doing... ???

This outputs nothing:

grep -Pn '[^\x00-\x7f]' /var/log/pacman.log

And this looks fine to me:

[halo@ArchNemesis ~]$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Last edited by halo (2014-01-09 23:13:08)

karol · 2014-01-09 18:35:07

It's just a log.
It may be interesting to find out why 'file' categorizes it differently for different users, but it's nothing to worry about; sorry if my comment https://bbs.archlinux.org/viewtopic.php … 7#p1368897 scared you.

halo · 2014-01-09 18:39:38

^I suppose that is OK, but I guess with Arch and it's philosophy of KISS and correctness, one sometimes sees what looks anomalous and wants get to the bottom of the why and figure out how. It's an awfully pedantic distro (not without merit), so I sometimes wonder if things like this signal other issues and if this issue will effect other parts of the OS and so on.

I'm not going to sweat it. Arch can make you paranoid though lol.

Last edited by halo (2014-01-09 18:55:47)

lolilolicon · 2014-01-09 19:05:43

halo wrote:

I get data [...] and this outputs nothing:
grep -Pn '[^\x00-\x7f]' /var/log/pacman.log

Gets me curious... What if you run `file` on the log line by line?

n=1; while read -r line; do [[ -z $line ]] && continue; printf "%s: %s\n" "$(( n++ ))" "$(printf '%s\n' "$line" | file -bi -)"; done < /var/log/pacman.log

Is there any line that's not "text/plain; charset=us-ascii" or "text/plain; charset=utf-8"?

Last edited by lolilolicon (2014-01-09 19:18:27)

halo · 2014-01-09 19:21:55

lolilolicon wrote:

halo wrote:
I get data [...] and this outputs nothing:
grep -Pn '[^\x00-\x7f]' /var/log/pacman.log
Gets me curious... What if you run `file` on the log line by line?
n=1; while read -r line; do [[ -z $line ]] && continue; printf "%s: %s\n" "$(( n++ ))" "$(printf '%s\n' "$line" | file -bi -)"; done < /var/log/pacman.log
Is there any line that's not "text/plain; charset=us-ascii" or "text/plain; charset=utf-8"?

I get about 2000 lines of this:

...
1933: text/plain; charset=us-ascii
1934: text/plain; charset=us-ascii
1935: text/plain; charset=us-ascii
1936: text/plain; charset=us-ascii
1937: text/plain; charset=us-ascii
1938: text/plain; charset=us-ascii
1939: text/plain; charset=us-ascii
1940: text/plain; charset=us-ascii
1941: text/plain; charset=us-ascii
1942: text/plain; charset=us-ascii
1943: text/plain; charset=us-ascii
1944: text/plain; charset=us-ascii
1945: text/plain; charset=us-ascii
1946: text/plain; charset=us-ascii
1947: text/plain; charset=us-ascii
1948: text/plain; charset=us-ascii
1949: text/plain; charset=us-ascii
1950: text/plain; charset=us-ascii
1951: text/plain; charset=us-ascii
1952: text/plain; charset=us-ascii
1953: text/plain; charset=us-ascii
1954: text/plain; charset=us-ascii
1955: text/plain; charset=us-ascii
1956: text/plain; charset=us-ascii
1957: text/plain; charset=us-ascii
1958: text/plain; charset=us-ascii
1959: text/plain; charset=us-ascii
1960: text/plain; charset=us-ascii
1961: text/plain; charset=us-ascii
1962: text/plain; charset=us-ascii
1963: text/plain; charset=us-ascii
1964: text/plain; charset=us-ascii
1965: text/plain; charset=us-ascii
1966: text/plain; charset=us-ascii
1967: text/plain; charset=us-ascii
1968: text/plain; charset=us-ascii
[halo@ArchNemesis ~]$

Last edited by halo (2014-01-09 19:22:35)

lolilolicon · 2014-01-09 19:30:43

No worries, that's a small log file you got there
Hmm, are you sure they're all us-ascii? I'd redirect the output to a file and do some grep on it.

halo · 2014-01-09 19:31:48

Yes they are all us-ascii. I scrolled through.

Last edited by halo (2014-01-09 23:15:01)

Arch Linux

#1 2014-01-08 22:58:47

Format of pacman.log

#2 2014-01-09 00:20:46

Re: Format of pacman.log

#3 2014-01-09 00:32:13

Re: Format of pacman.log

#4 2014-01-09 01:52:26

Re: Format of pacman.log

#5 2014-01-09 04:29:09

Re: Format of pacman.log

#6 2014-01-09 04:33:09

Re: Format of pacman.log

#7 2014-01-09 05:06:04

Re: Format of pacman.log

#8 2014-01-09 05:45:34

Re: Format of pacman.log

#9 2014-01-09 05:48:22

Re: Format of pacman.log

#10 2014-01-09 05:48:51

Re: Format of pacman.log

#11 2014-01-09 06:11:48

Re: Format of pacman.log

#12 2014-01-09 06:36:16

Re: Format of pacman.log

#13 2014-01-09 06:46:43

Re: Format of pacman.log

#14 2014-01-09 06:59:41

Re: Format of pacman.log

#15 2014-01-09 14:05:27

Re: Format of pacman.log

#16 2014-01-09 14:10:48

Re: Format of pacman.log

#17 2014-01-09 14:16:53

Re: Format of pacman.log

#18 2014-01-09 14:43:25

Re: Format of pacman.log

#19 2014-01-09 18:01:47

Re: Format of pacman.log

#20 2014-01-09 18:35:07

Re: Format of pacman.log

#21 2014-01-09 18:39:38

Re: Format of pacman.log

#22 2014-01-09 19:05:43

Re: Format of pacman.log

#23 2014-01-09 19:21:55

Re: Format of pacman.log

#24 2014-01-09 19:30:43

Re: Format of pacman.log

#25 2014-01-09 19:31:48

Re: Format of pacman.log

Board footer