Arch Linux

Strykar

Member

Registered: 2018-02-17

Posts: 56

Website

systemd-journald crashes incessantly

The systemd-journald.service keeps crashing causing prometheus.service to crash amongst others.
How do I troubleshoot this?

(gdb) core-file /tmp/core.systemd-journal.0.e495ebde75b2481ea63f992f74204f6e.66159.1640931347000000
[New LWP 66159]
Core was generated by `/usr/lib/systemd/systemd-journald'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007f3f10714d22 in ?? ()
(gdb) bt
#0  0x00007f3f10714d22 in ?? ()
#1  0x0000000400004a02 in ?? ()
#2  0x000055d4578ec03a in ?? ()
#3  0x0000000000000398 in ?? ()
#4  0x00007f3f107e4195 in ?? ()
#5  0x0000003000000030 in ?? ()
#6  0x00007ffde5211270 in ?? ()
#7  0x00007ffde52111b0 in ?? ()
#8  0x78634a37864b2f00 in ?? ()
#9  0x00007ffde5212f16 in ?? ()
#10 0x00007f3f0ff3a8b8 in ?? ()
#11 0xffffffffffffffff in ?? ()
#12 0x0000000000000000 in ?? ()
(gdb) 

Jan 01 21:02:04 i5 systemd-journald[173190]: File /var/log/journal/54d81479709b4ba3b3f7ca246c45a40f/system.journal corrupted or uncleanly shut down, renaming and replacing.
Jan 01 21:02:04 i5 systemd-journald[173190]: Journal started
Jan 01 21:02:04 i5 systemd-journald[173190]: System Journal (/var/log/journal/54d81479709b4ba3b3f7ca246c45a40f) is 800.3M, max 4.0G, 3.2G free.
Jan 01 21:02:04 i5 systemd[1]: systemd-journald.service: Main process exited, code=dumped, status=6/ABRT
Jan 01 21:02:04 i5 systemd[1]: systemd-journald.service: Failed with result 'core-dump'.
Jan 01 21:02:05 i5 systemd-journald[173190]: File /var/log/journal/54d81479709b4ba3b3f7ca246c45a40f/user-1000.journal corrupted or uncleanly shut down, renaming and replacing.
Jan 01 21:02:08 i5 systemd-journald[173202]: Journal started
Jan 01 21:02:08 i5 systemd-journald[173202]: System Journal (/var/log/journal/54d81479709b4ba3b3f7ca246c45a40f) is 800.3M, max 4.0G, 3.2G free.
Jan 01 21:02:08 i5 systemd-journald[173202]: File /var/log/journal/54d81479709b4ba3b3f7ca246c45a40f/system.journal corrupted or uncleanly shut down, renaming and replacing.
Jan 01 21:02:08 i5 systemd[1]: systemd-journald.service: Main process exited, code=dumped, status=6/ABRT
Jan 01 21:02:08 i5 systemd[1]: systemd-journald.service: Failed with result 'core-dump'.
Jan 01 21:02:13 i5 systemd-journald[173218]: Journal started
Jan 01 21:02:13 i5 systemd-journald[173218]: System Journal (/var/log/journal/54d81479709b4ba3b3f7ca246c45a40f) is 800.3M, max 4.0G, 3.2G free.
Jan 01 21:02:13 i5 systemd-journald[173218]: File /var/log/journal/54d81479709b4ba3b3f7ca246c45a40f/system.journal corrupted or uncleanly shut down, renaming and replacing.
Jan 01 21:02:13 i5 systemd[1]: systemd-journald.service: Main process exited, code=dumped, status=6/ABRT
Jan 01 21:02:13 i5 systemd[1]: systemd-journald.service: Failed with result 'core-dump'.
Jan 01 21:02:15 i5 systemd-journald[173226]: File /var/log/journal/54d81479709b4ba3b3f7ca246c45a40f/system.journal corrupted or uncleanly shut down, renaming and replacing.
Jan 01 21:02:15 i5 systemd-journald[173226]: Journal started
Jan 01 21:02:15 i5 systemd-journald[173226]: System Journal (/var/log/journal/54d81479709b4ba3b3f7ca246c45a40f) is 800.3M, max 4.0G, 3.2G free.
Jan 01 21:02:15 i5 systemd[1]: systemd-journald.service: Main process exited, code=dumped, status=6/ABRT
Jan 01 21:02:15 i5 systemd[1]: systemd-journald.service: Failed with result 'core-dump'.
Jan 01 21:02:18 i5 systemd-journald[173240]: File /var/log/journal/54d81479709b4ba3b3f7ca246c45a40f/system.journal corrupted or uncleanly shut down, renaming and replacing.
Jan 01 21:02:18 i5 systemd-journald[173240]: Journal started
Jan 01 21:02:18 i5 systemd-journald[173240]: System Journal (/var/log/journal/54d81479709b4ba3b3f7ca246c45a40f) is 800.3M, max 4.0G, 3.2G free.
Jan 01 21:02:18 i5 systemd[1]: systemd-journald.service: Main process exited, code=dumped, status=6/ABRT
Jan 01 21:02:18 i5 systemd[1]: systemd-journald.service: Failed with result 'core-dump'.
Jan 01 21:02:35 i5 systemd-journald[173240]: File /var/log/journal/54d81479709b4ba3b3f7ca246c45a40f/user-1000.journal corrupted or uncleanly shut down, renaming and replacing.
Jan 01 21:03:05 i5 systemd-journald[173372]: Journal started
Jan 01 21:03:05 i5 systemd-journald[173372]: System Journal (/var/log/journal/54d81479709b4ba3b3f7ca246c45a40f) is 800.3M, max 4.0G, 3.2G free.
Jan 01 21:03:05 i5 systemd-journald[173372]: File /var/log/journal/54d81479709b4ba3b3f7ca246c45a40f/system.journal corrupted or uncleanly shut down, renaming and replacing.
Jan 01 21:03:05 i5 systemd-journald[173372]: File /var/log/journal/54d81479709b4ba3b3f7ca246c45a40f/user-1000.journal corrupted or uncleanly shut down, renaming and replacing.
Jan 01 21:03:05 i5 systemd[1]: systemd-journald.service: Main process exited, code=dumped, status=6/ABRT
Jan 01 21:03:05 i5 systemd[1]: systemd-journald.service: Failed with result 'core-dump'.
Jan 01 21:03:16 i5 systemd-journald[173398]: File /var/log/journal/54d81479709b4ba3b3f7ca246c45a40f/system.journal corrupted or uncleanly shut down, renaming and replacing.
Jan 01 21:03:16 i5 systemd-journald[173398]: Journal started
Jan 01 21:03:16 i5 systemd-journald[173398]: System Journal (/var/log/journal/54d81479709b4ba3b3f7ca246c45a40f) is 800.3M, max 4.0G, 3.2G free.
Jan 01 21:03:16 i5 systemd[1]: systemd-journald.service: Main process exited, code=dumped, status=6/ABRT
Jan 01 21:03:16 i5 systemd[1]: systemd-journald.service: Failed with result 'core-dump'.
Jan 01 21:03:18 i5 systemd-journald[173407]: Journal started
Jan 01 21:03:18 i5 systemd-journald[173407]: System Journal (/var/log/journal/54d81479709b4ba3b3f7ca246c45a40f) is 800.3M, max 4.0G, 3.2G free.
Jan 01 21:03:18 i5 systemd-journald[173407]: File /var/log/journal/54d81479709b4ba3b3f7ca246c45a40f/system.journal corrupted or uncleanly shut down, renaming and replacing.
Jan 01 21:03:18 i5 systemd[1]: systemd-journald.service: Main process exited, code=dumped, status=6/ABRT
Jan 01 21:03:18 i5 systemd[1]: systemd-journald.service: Failed with result 'core-dump'.
Jan 01 21:03:35 i5 systemd-journald[173407]: File /var/log/journal/54d81479709b4ba3b3f7ca246c45a40f/user-1000.journal corrupted or uncleanly shut down, renaming and replacing.

Stopping systemd-journald and wiping /var/log/journal did not help as suggested by Namarrgon on IRC.

Last edited by Strykar (2022-01-02 00:59:03)

Head_on_a_Stick

Member

From: The Wirral

Registered: 2014-02-20

Posts: 8,580

Website

Re: systemd-journald crashes incessantly

I think you'll have to rebuild systemd with the debug symbols enabled to get a meaningful backtrace.

Perhaps also check the health of the drive.

---

Jin, Jiyan, Azadî