journald filling up /var/log/journal with random user IDs

foowtf · 2016-12-05 14:44:30

I noticed that journald deletes lots of my syslog messages. It cannot even keep a page full of messages from recent boots. Now I understand that journald does not allow its log files to grow larger than SystemMaxUse, and indeed this limit is reached on my machine. But looking into /var/log/journal:

8.1M    system.journal
8.1M    user-3067491928.journal
8.1M    user-3067496024.journal
8.1M    user-3067500120.journal
8.1M    user-3067508312.journal
8.1M    user-3067520600.journal
8.1M    user-3067524696.journal
8.1M    user-3067536984.journal
8.1M    user-3067545176.journal
8.1M    user-3067553368.journal
8.1M    user-3067557464.journal
8.1M    user-3067561560.journal
8.1M    user-3067569752.journal
8.1M    user-3067586136.journal
8.1M    user-3067590232.journal
8.1M    user-3067594328.journal
8.1M    user-3067598424.journal
8.1M    user-3067602520.journal
8.1M    user-3067606616.journal
...

It goes on for this for 190 lines. Needless to say, there are no such user IDs on my system. The user-* files I checked each contain a tiny log message from a cron job and otherwise mostly zeroes. It seems to me journald makes a new file for each cron job run, even though those are configured to run as root.

1. How does journald generate those user IDs?
2. Even if the log folder is full, journald still seems to have allocated 8.1M for my system log. Why does it still keep deleting the messages? Running

# journalctl -o verbose -a | wc --bytes

gives me 1177239 = 1.12M and

# cat system.journal | xz | wc --bytes

133792 = 131K. So the file is clearly not full.

berbae · 2016-12-05 15:36:49

Post the beginning (50 lines) of the output of:

journalctl --file=user-3067491928.journal

And give details about the cron job you are speaking about.

Also the output of:

journalctl --disk-usage

and the content of /etc/systemd/journald.conf

foowtf · 2016-12-05 23:13:48

As I said, the user-* files each contain only one message from the cron job, a single line. The job is configured as a one-line bash script in /etc/cron.hourly which calls another script, so the job is sure to be run as root. A random detail is that its syslog messages are generated by calling the util-linux logger program. I am sure there are no setuid(2) calls or anything in this chain.

journalctl --disk-usage is very interesting:

Archived and active journals take up 1.5G on disk.

It is the same as du -sh /var/log/journal gives. The size of / (and /var) is 15G, so it is full.

The journald.conf is the completely default Arch one.

HiImTye · 2016-12-06 00:27:46

foowtf wrote:

As I said, the user-* files each contain only one message from the cron job, a single line. The job is configured as a one-line bash script in /etc/cron.hourly which calls another script, so the job is sure to be run as root.

it seems like you've sorted it yourself, unless there's more information you're not telling us. sort out the cron job and your problem will be fixed

seth · 2016-12-06 08:19:36

In case you need more help, you'll have to provide the actual information, ie. the actual line in the journals and the actual cronjob and the actual script it's calling. Otherwise you're asking every to *guess* what your problem is, let alone what causes it.
That's not how it works.

Arch Linux

#1 2016-12-05 14:44:30

journald filling up /var/log/journal with random user IDs

#2 2016-12-05 15:36:49

Re: journald filling up /var/log/journal with random user IDs

#3 2016-12-05 23:13:48

Re: journald filling up /var/log/journal with random user IDs

#4 2016-12-06 00:27:46

Re: journald filling up /var/log/journal with random user IDs

#5 2016-12-06 08:19:36

Re: journald filling up /var/log/journal with random user IDs

Board footer