Arch Linux

pietje_puk

Member

Registered: 2016-01-10

Posts: 6

[SOLVED] Journalctl showing no entries, but journald is logging

My system seems to be logging, but some functions of journalctl are not working.

For example, running "journalctl -k" shows no entries:

 [root@fback ~]# journalctl -k
-- No entries --

The same happens, e.g., for "journalctl -u sshd". However, just running "journalctl" does show me lots of output:

 Jan 10 13:12:39 fback systemd-journald[529]: Time spent on flushing to /var is 530us for 2 entries.
Jan 10 13:12:39 fback kernel: microcode: CPU0 microcode updated early to revision 0x1c, date = 2014-07-03
Jan 10 13:12:39 fback kernel: Initializing cgroup subsys cpuset
Jan 10 13:12:39 fback kernel: Initializing cgroup subsys cpu
Jan 10 13:12:39 fback kernel: Initializing cgroup subsys cpuacct
Jan 10 13:12:39 fback kernel: Linux version 4.3.3-2-ARCH (builduser@tobias) (gcc version 5.3.0 (GCC) ) #1 SMP PREEMPT Wed Dec 23 20:09:18 CET 2015
Jan 10 13:12:39 fback kernel: Command line: initrd=\intel-ucode.img initrd=\initramfs-linux.img root=/dev/mapper/fhrp-lvolroot rw
Jan 10 13:12:39 fback kernel: x86/fpu: xstate_offset[2]: 0240, xstate_sizes[2]: 0100
Jan 10 13:12:39 fback kernel: x86/fpu: Supporting XSAVE feature 0x01: 'x87 floating point registers'
Jan 10 13:12:39 fback kernel: x86/fpu: Supporting XSAVE feature 0x02: 'SSE registers'
Jan 10 13:12:39 fback kernel: x86/fpu: Supporting XSAVE feature 0x04: 'AVX registers'
Jan 10 13:12:39 fback kernel: x86/fpu: Enabled xstate features 0x7, context size is 0x340 bytes, using 'standard' format.
Jan 10 13:12:39 fback kernel: x86/fpu: Using 'eager' FPU context switches.
Jan 10 13:12:39 fback kernel: e820: BIOS-provided physical RAM map:

etc...

I've already verified that the journal file is consistent using "journalctl --verify" and the service "systemd-journald.service" is active.

This behaviour occured when I changed my hostname (two weeks after installation). Could this be the cause? If so what to do?

After I noticed that journalctl was not working properly I deleted my /var/log/journal and after that I changed Storage=persistent in /etc/systemd/journald.conf .

Any suggestions?

Last edited by pietje_puk (2016-01-13 09:50:36)

ewaller

Administrator

From: Pasadena, CA

Registered: 2009-07-13

Posts: 20,402

Re: [SOLVED] Journalctl showing no entries, but journald is logging

Try it as sudo journalctl
What is the output of groups.  You may want to look at paragraph 7 under Description in man journalctl

After you set the Storage as persistent, did you restart the systemd-journald service?  Did your journal files reappear?

---

Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
The shortest way to ruin a country is to give power to demagogues.— Dionysius of Halicarnassus
---
How to Ask Questions the Smart Way

pietje_puk

Member

Registered: 2016-01-10

Posts: 6

Re: [SOLVED] Journalctl showing no entries, but journald is logging

Using sudo journalctl makes no difference on the output (I was running it as root before)

I only have one account on my system in the wheel group, which has the privileges to see the journal files  (=output of groups)

Yes, I've restarted systemd-journal.service a dozen times on every reboot; the problem remains over reboots.

The problem is not so much that the files aren't there or I can't access them. The command journalctl just doesn't show an output with certain options.

Last edited by pietje_puk (2016-01-10 20:13:05)

loafer

Member

From: the pub

Registered: 2009-04-14

Posts: 1,772

Re: [SOLVED] Journalctl showing no entries, but journald is logging

I assume the new hostname is "fback"?  Does anything show as the old name?

Last edited by loafer (2016-01-10 20:37:41)

---

All men have stood for freedom...
For freedom is the man that will turn the world upside down.
Gerrard Winstanley.

pietje_puk

Member

Registered: 2016-01-10

Posts: 6

Re: [SOLVED] Journalctl showing no entries, but journald is logging

Yes, that's the new hostname. All the new entries only show 'fback'. The weird thing is that there are 'kernel' entries in the logs, but if I specifically ask for them through journalctl -k the response is '-- No entries --'

One other thing I noticed is that the journal is not working as before. Before, I would get lots of unknown IP addresses trying to connect to me over SSH and I could see those tries with journalctl. Moreover, if I was logging in via SSH I could find an entry of my own login in the journal (something like 'successful connection to...') and after logout something like 'disconnected from...'. Now I only see my own disconnects (no more successful connections) in the journal. And there are no failed attempts from unknown IP addresses any more. What explains this behaviour?

Last edited by pietje_puk (2016-01-10 22:20:33)

pietje_puk

Member

Registered: 2016-01-10

Posts: 6

Re: [SOLVED] Journalctl showing no entries, but journald is logging

After some more effort I've found the cause of this problem and a "fix".

During shutdown I could see that my /var and /var/log mounts were not unmounting correctly. This discussion addresses that problem. Basically it boils down to "Systemd can not unmount /var becuase journald is still writing log files to /var/log/journal". I tried adding the "shutdown" hook to my "initramfs" but to no effect (I believe this function is deprecated).

So, as a "fix" I decided to change the "Storage" parameter back from "persistent" to "volatile" in journald.conf. However, now my journals will be saved on /run/log which is not preserved over reboots, right? Is there a way to write these logs to a more permanent location?

This did solve the problem I had with not being able to see kernel/sshd logs in journalctl

Last edited by pietje_puk (2016-01-11 13:47:20)