[SOLVED] systemd-timesyncd fails to start after recent update

CounterSlash · 2024-10-02 20:06:17

Hi all,

I'm using systemd-timesyncd for time sync and after a system update it started failing with a SIGSEGV. It looks like maybe it isn't timesyncd failing specifically, but something else on the system (sd-resolve?). I'd really apreciate some help solving this crash.
Nothing else seems to be broken on the system except for this one unit.

Updated packages:

[2024-10-01T23:13:48+0300] [ALPM] transaction started
[2024-10-01T23:13:48+0300] [ALPM] upgraded btrfs-progs (6.10.1-2 -> 6.11-1)
[2024-10-01T23:13:48+0300] [ALPM] upgraded ca-certificates-mozilla (3.104-1 -> 3.105-1)
[2024-10-01T23:13:48+0300] [ALPM] upgraded nss (3.104-1 -> 3.105-1)
[2024-10-01T23:13:48+0300] [ALPM] upgraded gtk-update-icon-cache (1:4.16.2-1 -> 1:4.16.2-2)
[2024-10-01T23:13:49+0300] [ALPM] upgraded nvidia-utils (560.35.03-3 -> 560.35.03-4)
[2024-10-01T23:13:49+0300] [ALPM-SCRIPTLET] If you run into trouble with CUDA not being available, run nvidia-modprobe first.
[2024-10-01T23:13:49+0300] [ALPM-SCRIPTLET] If you use GDM on Wayland, you might have to run systemctl enable --now nvidia-resume.service
[2024-10-01T23:13:50+0300] [ALPM] upgraded discord (0.0.69-1 -> 0.0.70-1)
[2024-10-01T23:13:50+0300] [ALPM] upgraded firefox (130.0.1-2 -> 131.0-1)
[2024-10-01T23:13:50+0300] [ALPM] upgraded gdb-common (15.1-1 -> 15.2-1)
[2024-10-01T23:13:50+0300] [ALPM] upgraded gdb (15.1-1 -> 15.2-1)
[2024-10-01T23:13:50+0300] [ALPM] upgraded gtk4 (1:4.16.2-1 -> 1:4.16.2-2)
[2024-10-01T23:13:50+0300] [ALPM] upgraded krita (5.2.5-2 -> 5.2.6-1)
[2024-10-01T23:13:50+0300] [ALPM] upgraded lib32-nss (3.104-1 -> 3.105-1)
[2024-10-01T23:13:50+0300] [ALPM] upgraded libqmi (1.34.0-1 -> 1.34.0-2)
[2024-10-01T23:13:51+0300] [ALPM] upgraded linux (6.10.10.arch1-1 -> 6.11.1.arch1-1)
[2024-10-01T23:13:52+0300] [ALPM] upgraded linux-headers (6.10.10.arch1-1 -> 6.11.1.arch1-1)
[2024-10-01T23:13:52+0300] [ALPM] upgraded net-tools (2.10-2 -> 2.10-3)
[2024-10-01T23:13:52+0300] [ALPM] upgraded netpbm (10.86.42-1 -> 10.86.43-1)
[2024-10-01T23:13:52+0300] [ALPM] upgraded nvidia (560.35.03-6 -> 560.35.03-9)
[2024-10-01T23:13:52+0300] [ALPM] upgraded python-aiohappyeyeballs (2.4.2-1 -> 2.4.3-1)
[2024-10-01T23:13:52+0300] [ALPM] upgraded qt6-base (6.7.3-1 -> 6.7.3-2)
[2024-10-01T23:13:52+0300] [ALPM] upgraded qbittorrent (4.6.7-1 -> 5.0.0-1)
[2024-10-01T23:13:52+0300] [ALPM] upgraded rpi-imager (1.8.5-4 -> 1.9.0-2)
[2024-10-01T23:13:52+0300] [ALPM] upgraded webkit2gtk-4.1 (2.46.0-2 -> 2.46.1-1)
[2024-10-01T23:13:53+0300] [ALPM] upgraded webkitgtk-6.0 (2.46.0-2 -> 2.46.1-1)
[2024-10-01T23:13:53+0300] [ALPM] transaction completed

System log:

...
Oct 01 23:15:13 Mirai systemd[1]: systemd-hostnamed.service: Deactivated successfully.
Oct 01 23:15:13 Mirai kernel: sd-resolve[944]: segfault at 0 ip 0000000000000000 sp 0000771b4d3fab60 error 14 likely on CPU 8 (core 20, socket 0)
Oct 01 23:15:13 Mirai kernel: Code: Unable to access opcode bytes at 0xffffffffffffffd6.
Oct 01 23:15:13 Mirai systemd-coredump[2307]: Process 937 (systemd-timesyn) of user 975 terminated abnormally with signal 11/SEGV, processing...
Oct 01 23:15:13 Mirai systemd[1]: Created slice Slice /system/drkonqi-coredump-processor.
Oct 01 23:15:13 Mirai systemd[1]: Created slice Slice /system/systemd-coredump.
Oct 01 23:15:13 Mirai systemd[1]: Started Process Core Dump (PID 2307/UID 0).
Oct 01 23:15:13 Mirai systemd[1]: Started Pass systemd-coredump journal entries to relevant user for potential DrKonqi handling.
Oct 01 23:15:13 Mirai systemd-coredump[2309]: Process 937 (systemd-timesyn) of user 975 dumped core.
                                              
                                              Stack trace of thread 944:
                                              #0  0x0000000000000000 n/a (n/a + 0x0)
                                              #1  0x0000771b4dc7539d n/a (libc.so.6 + 0x9439d)
                                              #2  0x0000771b4dcfa2a4 __clone (libc.so.6 + 0x1192a4)
                                              
                                              Stack trace of thread 937:
                                              #0  0x0000771b4dcfa8b2 epoll_wait (libc.so.6 + 0x1198b2)
                                              #1  0x0000771b4e10e9b0 sd_event_wait (libsystemd-shared-256.6-1.so + 0x30e9b0)
                                              #2  0x0000771b4e1101bc sd_event_run (libsystemd-shared-256.6-1.so + 0x3101bc)
                                              #3  0x0000771b4e110308 sd_event_loop (libsystemd-shared-256.6-1.so + 0x310308)
                                              #4  0x00005781effa7b61 n/a (systemd-timesyncd + 0x3b61)
                                              #5  0x0000771b4dc06e08 n/a (libc.so.6 + 0x25e08)
                                              #6  0x0000771b4dc06ecc __libc_start_main (libc.so.6 + 0x25ecc)
                                              #7  0x00005781effa80e5 n/a (systemd-timesyncd + 0x40e5)
                                              ELF object binary architecture: AMD x86-64
Oct 01 23:15:13 Mirai systemd[1]: systemd-coredump@0-2307-0.service: Deactivated successfully.
Oct 01 23:15:14 Mirai systemd[1]: systemd-timesyncd.service: Main process exited, code=dumped, status=11/SEGV
Oct 01 23:15:14 Mirai systemd[1]: systemd-timesyncd.service: Failed with result 'core-dump'.
Oct 01 23:15:14 Mirai systemd[1]: systemd-timesyncd.service: Scheduled restart job, restart counter is at 1.
Oct 01 23:15:14 Mirai systemd[1]: Starting Network Time Synchronization...
Oct 01 23:15:14 Mirai systemd[1]: Started Network Time Synchronization.
Oct 01 23:15:14 Mirai kernel: sd-resolve[2322]: segfault at 0 ip 0000000000000000 sp 00007891c61fab60 error 14 likely on CPU 8 (core 20, socket 0)
Oct 01 23:15:14 Mirai kernel: Code: Unable to access opcode bytes at 0xffffffffffffffd6.
Oct 01 23:15:14 Mirai drkonqi-coredump-processor[2310]: "/usr/lib/systemd/systemd-timesyncd" 937 "/var/lib/systemd/coredump/core.systemd-timesyn.975.c3ed17f0b3a54470bbdb05ecea836b19.937.1727813713000000.zst"
Oct 01 23:15:14 Mirai drkonqi-coredump-processor[2310]: The socket path doesn't exist @ "/run/user/975/drkonqi-coredump-launcher"
Oct 01 23:15:14 Mirai systemd-coredump[2323]: Process 2321 (systemd-timesyn) of user 975 terminated abnormally with signal 11/SEGV, processing...
Oct 01 23:15:14 Mirai systemd[1]: drkonqi-coredump-processor@0-2307-0.service: Deactivated successfully.
Oct 01 23:15:14 Mirai systemd[1]: Started Process Core Dump (PID 2323/UID 0).
Oct 01 23:15:14 Mirai systemd[1]: Started Pass systemd-coredump journal entries to relevant user for potential DrKonqi handling.
Oct 01 23:15:14 Mirai systemd-coredump[2327]: Process 2321 (systemd-timesyn) of user 975 dumped core.
                                              
                                              Stack trace of thread 2322:
                                              #0  0x0000000000000000 n/a (n/a + 0x0)
                                              #1  0x00007891c6bb439d n/a (libc.so.6 + 0x9439d)
                                              #2  0x00007891c6c392a4 __clone (libc.so.6 + 0x1192a4)
                                              
                                              Stack trace of thread 2321:
                                              #0  0x00007891c6c398b2 epoll_wait (libc.so.6 + 0x1198b2)
                                              #1  0x00007891c710e9b0 sd_event_wait (libsystemd-shared-256.6-1.so + 0x30e9b0)
                                              #2  0x00007891c71101bc sd_event_run (libsystemd-shared-256.6-1.so + 0x3101bc)
                                              #3  0x00007891c7110308 sd_event_loop (libsystemd-shared-256.6-1.so + 0x310308)
                                              #4  0x00005667c16b2b61 n/a (systemd-timesyncd + 0x3b61)
                                              #5  0x00007891c6b45e08 n/a (libc.so.6 + 0x25e08)
                                              #6  0x00007891c6b45ecc __libc_start_main (libc.so.6 + 0x25ecc)
                                              #7  0x00005667c16b30e5 n/a (systemd-timesyncd + 0x40e5)
                                              ELF object binary architecture: AMD x86-64
Oct 01 23:15:14 Mirai systemd[1]: systemd-coredump@1-2323-0.service: Deactivated successfully.
Oct 01 23:15:14 Mirai systemd[1]: systemd-timesyncd.service: Main process exited, code=dumped, status=11/SEGV
Oct 01 23:15:14 Mirai systemd[1]: systemd-timesyncd.service: Failed with result 'core-dump'.
Oct 01 23:15:14 Mirai systemd[1]: systemd-timesyncd.service: Scheduled restart job, restart counter is at 2.
Oct 01 23:15:14 Mirai systemd[1]: Starting Network Time Synchronization...
Oct 01 23:15:14 Mirai systemd[1]: Started Network Time Synchronization.
Oct 01 23:15:14 Mirai kernel: sd-resolve[2339]: segfault at 0 ip 0000000000000000 sp 00007baab93fab60 error 14 likely on CPU 6 (core 12, socket 0)
...
(systemd-timesyncd goes on to crash another 4 times before calling it quits)

Last edited by CounterSlash (2024-10-07 07:23:20)

-thc · 2024-10-03 07:00:27

I would first try to isolate the cause. If your /etc/system/timesyncd.conf contains a line

NTP=my.time.server

replace it with the corresponding IP address.
If your etc/system/timesyncd.conf is unchanged, add a line

NTP=78.47.168.188

(one of the four actual IPv4 addresses of "0.arch.pool.ntp.org")
and restart the timesyncd service.

Does it work when no DNS resolving is involved?

CounterSlash · 2024-10-03 12:12:16

-thc wrote:

Does it work when no DNS resolving is involved?

I added "NTP=162.159.200.1" to the timesyncd.conf file and the same exact thing happens.
The sd-resolve segfault still appears and the systemd-timesyncd stack trace is identical.
There is no DNS network activity, unlike when no IP is set (with no IP, it gets as far as making a single request to 1.1.1.1, which is answered, before it crashes)

... (sample of crash)
Oct 03 15:03:42 Mirai systemd[1]: systemd-timesyncd.service: Scheduled restart job, restart counter is at 4.
Oct 03 15:03:42 Mirai systemd[1]: Starting Network Time Synchronization...
Oct 03 15:03:42 Mirai systemd[1]: Started Network Time Synchronization.
Oct 03 15:03:42 Mirai systemd-coredump[5375]: Process 5368 (systemd-timesyn) of user 975 terminated abnormally with signal 11/SEGV, processing...
Oct 03 15:03:42 Mirai kernel: sd-resolve[5374]: segfault at 0 ip 0000000000000000 sp 000070c5c3bfab60 error 14 likely on CPU 10 (core 22, socket 0)
Oct 03 15:03:42 Mirai kernel: Code: Unable to access opcode bytes at 0xffffffffffffffd6.
Oct 03 15:03:42 Mirai systemd[1]: Started Process Core Dump (PID 5375/UID 0).
Oct 03 15:03:42 Mirai systemd[1]: Started Pass systemd-coredump journal entries to relevant user for potential DrKonqi handling.
Oct 03 15:03:42 Mirai systemd-coredump[5376]: [?] Process 5368 (systemd-timesyn) of user 975 dumped core.
                                              
                                              Stack trace of thread 5374:
                                              #0  0x0000000000000000 n/a (n/a + 0x0)
                                              #1  0x000070c5c43b439d n/a (libc.so.6 + 0x9439d)
                                              #2  0x000070c5c44392a4 __clone (libc.so.6 + 0x1192a4)
                                              
                                              Stack trace of thread 5368:
                                              #0  0x000070c5c44398b2 epoll_wait (libc.so.6 + 0x1198b2)
                                              #1  0x000070c5c490e9b0 sd_event_wait (libsystemd-shared-256.6-1.so + 0x30e9b0)
                                              #2  0x000070c5c49101bc sd_event_run (libsystemd-shared-256.6-1.so + 0x3101bc)
                                              #3  0x000070c5c4910308 sd_event_loop (libsystemd-shared-256.6-1.so + 0x310308)
                                              #4  0x000055d6b55adb61 n/a (systemd-timesyncd + 0x3b61)
                                              #5  0x000070c5c4345e08 n/a (libc.so.6 + 0x25e08)
                                              #6  0x000070c5c4345ecc __libc_start_main (libc.so.6 + 0x25ecc)
                                              #7  0x000055d6b55ae0e5 n/a (systemd-timesyncd + 0x40e5)
                                              ELF object binary architecture: AMD x86-64
Oct 03 15:03:42 Mirai systemd[1]: systemd-coredump@40-5375-0.service: Deactivated successfully.
Oct 03 15:03:42 Mirai drkonqi-coredump-processor[5355]: "/usr/lib/systemd/systemd-timesyncd" 5349 "/var/lib/systemd/coredump/core.systemd-timesyn.975.19c53ac70db14679b553e492c79aeb51.>
Oct 03 15:03:42 Mirai drkonqi-coredump-processor[5377]: "/usr/lib/systemd/systemd-timesyncd" 5368 "/var/lib/systemd/coredump/core.systemd-timesyn.975.19c53ac70db14679b553e492c79aeb51.>
Oct 03 15:03:42 Mirai drkonqi-coredump-processor[5355]: The socket path doesn't exist @ "/run/user/975/drkonqi-coredump-launcher"
Oct 03 15:03:42 Mirai drkonqi-coredump-processor[5377]: The socket path doesn't exist @ "/run/user/975/drkonqi-coredump-launcher"
Oct 03 15:03:42 Mirai systemd[1]: drkonqi-coredump-processor@39-5353-0.service: Deactivated successfully.
Oct 03 15:03:42 Mirai systemd[1]: drkonqi-coredump-processor@40-5375-0.service: Deactivated successfully.
Oct 03 15:03:42 Mirai systemd[1]: systemd-timesyncd.service: Main process exited, code=dumped, status=11/SEGV
Oct 03 15:03:42 Mirai systemd[1]: systemd-timesyncd.service: Failed with result 'core-dump'.
Oct 03 15:03:42 Mirai systemd[1]: systemd-timesyncd.service: Scheduled restart job, restart counter is at 5.
...

-thc · 2024-10-03 15:26:45

Strange. Normally I would try a rollback of the program itself but systemd was last updated 9/10.

seth · 2024-10-03 19:51:30

pacman -Qikk systemd ca-certificates-mozilla

The latter is the main contender here.

CounterSlash · 2024-10-05 07:42:48

seth wrote:

pacman -Qikk systemd ca-certificates-mozilla
The latter is the main contender here.

By contender, do you mean that ca-certificates-mozilla might be the problem? I tried downgrading it to the previous version but the error persists.
I also tried executing what the .service file executes (/usr/lib/systemd/systemd-timesyncd) in a terminal with sudo and it does not crash.

pacman -Qikk systemd ca-certificates-mozilla output:

Name            : systemd
Version         : 256.6-1
Description     : system and service manager
Architecture    : x86_64
URL             : https://www.github.com/systemd/systemd
Licenses        : LGPL-2.1-or-later  CC0-1.0  GPL-2.0-or-later  MIT-0
Groups          : None
Provides        : nss-myhostname  systemd-tools=256.6  udev=256.6
Depends On      : systemd-libs=256.6  acl  libacl.so=1-64  bash  cryptsetup  libcryptsetup.so=12-64  dbus  dbus-units  kbd  kmod  hwdata  libcap  libcap.so=2-64  libgcrypt  libxcrypt
                  libcrypt.so=2-64  libidn2  lz4  pam  libelf  libseccomp  libseccomp.so=2-64  util-linux  libblkid.so=1-64  libmount.so=1-64  xz  pcre2  audit  libaudit.so=1-64
                  openssl  libcrypto.so=3-64  libssl.so=3-64
Optional Deps   : libmicrohttpd: systemd-journal-gatewayd and systemd-journal-remote
                  quota-tools: kernel-level quota management
                  systemd-sysvcompat: symlink package to provide sysvinit binaries [installed]
                  systemd-ukify: combine kernel and initrd into a signed Unified Kernel Image
                  polkit: allow administration as unprivileged user [installed]
                  curl: systemd-journal-upload, machinectl pull-tar and pull-raw [installed]
                  gnutls: systemd-journal-gatewayd and systemd-journal-remote [installed]
                  qrencode: show QR codes [installed]
                  iptables: firewall features [installed]
                  libarchive: convert DDIs to tarballs [installed]
                  libbpf: support BPF programs [installed]
                  libpwquality: check password quality
                  libfido2: unlocking LUKS2 volumes with FIDO2 token
                  libp11-kit: support PKCS#11 [installed]
                  tpm2-tss: unlocking LUKS2 volumes with TPM2 [installed]
Required By     : accountsservice  base  bolt  cups  flatpak  intel-gpu-tools  libcolord  libgudev  libmbim  libpulse  mdadm  media-player-info  mkinitcpio  modemmanager  pacman
                  phodav  plocate  profile-sync-daemon  rtkit  systemd-resolvconf  systemd-sysvcompat  vte3  xdg-desktop-portal  xdg-user-dirs
Optional For    : airgeddon
Conflicts With  : nss-myhostname  systemd-tools  udev
Replaces        : nss-myhostname  systemd-tools  udev
Installed Size  : 32.45 MiB
Packager        : Christian Hesse <eworm@archlinux.org>
Build Date      : Tue 10 Sep 2024 06:06:16 PM EEST
Install Date    : Sun 15 Sep 2024 11:44:18 AM EEST
Install Reason  : Installed as a dependency for another package
Install Script  : Yes
Validated By    : Signature

backup file: systemd: /etc/systemd/timesyncd.conf (Modification time mismatch)
backup file: systemd: /etc/systemd/timesyncd.conf (Size mismatch)
backup file: systemd: /etc/systemd/timesyncd.conf (SHA256 checksum mismatch)
warning: systemd: /var/log/journal (GID mismatch)
systemd: 1547 total files, 1 altered file
Name            : ca-certificates-mozilla
Version         : 3.105-1
Description     : Mozilla's set of trusted CA certificates
Architecture    : x86_64
URL             : https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS
Licenses        : MPL-2.0
Groups          : None
Provides        : None
Depends On      : ca-certificates-utils>=20181109-3
Optional Deps   : None
Required By     : ca-certificates
Optional For    : None
Conflicts With  : None
Replaces        : None
Installed Size  : 1069.00 KiB
Packager        : Jan Alexander Steffens (heftig) <heftig@archlinux.org>
Build Date      : Sat 28 Sep 2024 09:00:51 PM EEST
Install Date    : Fri 04 Oct 2024 04:15:39 PM EEST
Install Reason  : Installed as a dependency for another package
Install Script  : No
Validated By    : Signature

seth · 2024-10-05 12:43:32

By contender, do you mean that ca-certificates-mozilla might be the problem?

Yup, it's the only updated dependency (but despite the crucial tail of the pacman output is missing in your post it does not seem to be the cause?)

Alternatively this is triggered by a previous problem (eg. b/c of the kernel update)
=> Do you get the same w/ the LTS kernel?

CounterSlash · 2024-10-05 19:14:02

seth wrote:

Alternatively this is triggered by a previous problem (eg. b/c of the kernel update)
=> Do you get the same w/ the LTS kernel?

I installed linux-lts 6.6.52-1 and timesyncd stops crashing! Downgrading linux and linux-headers to their pre-update versions fixes it as well.
I guess it's a kernel thing, but as for debugging further, I have no idea what to do. (6.11.1 in general was quite weird on my laptop as, aside from this, I even got some new sensors showing up)

Last edited by CounterSlash (2024-10-05 19:16:12)

seth · 2024-10-05 19:31:33

Please post your complete system journal for a boot w/ the affected kernel, eg. :

sudo journalctl -b -1 | curl -F 'file=@-' 0x0.st

for the previous one

CounterSlash · 2024-10-05 20:16:03

Here's the log of a fresh boot: https://0x0.st/XEjK.txt
(the large nvidia warnings have been happening since 6.10)

seth · 2024-10-05 20:25:21

Oct 05 23:04:03 Mirai dhcpcd[919]: no interfaces have a carrier
Oct 05 23:04:03 Mirai dhcpcd[919]: enp8s0: waiting for carrier
Oct 05 23:04:03 Mirai NetworkManager[914]: <info>  [1728158643.9696] dhcp: init: Using DHCP client 'internal'

Disable dhcpcd.

https://bbs.archlinux.org/viewtopic.php?id=299462
https://bbs.archlinux.org/viewtopic.php?id=299736
But there's no indication of a connection issue.

CounterSlash · 2024-10-05 20:30:33

seth wrote:

Disable dhcpcd.

Nothing changes, it still crashes. I have no issues with internet connectivity either.

Last edited by CounterSlash (2024-10-05 20:31:06)

seth · 2024-10-06 07:01:07

Since it segfaults w/ remarkable consistency in __clone() I'd guess it's either realted to the nvidia warningss (or other OOT drivers) or a misbehaving network cause an infinite recursion.

* Does the network generally work w/ 6.11?
* Do you get the same when using nouveau instead of nvidia?

CounterSlash · 2024-10-06 09:06:17

seth wrote:

* Does the network generally work w/ 6.11?

Yes, I haven't had any network problems in 6.11, nor ever since installing linux on this machine.

seth wrote:

* Do you get the same when using nouveau instead of nvidia?

The crash still occurs even with nouveau getting control of the GPU and all nvidia driver related packages removed (nvidia-utils, lib32-nvidia-utils, nvidia-prime).
I also removed the other OOT modules (the virtbox and lenovolegionlinux ones) and no luck

cslash@Mirai ~> lspci -k | grep -A 3 NVIDIA
01:00.0 VGA compatible controller: NVIDIA Corporation AD107M [GeForce RTX 4050 Max-Q / Mobile] (rev a1)
        Subsystem: Lenovo Device 3c90
        Kernel driver in use: nouveau
        Kernel modules: nouveau
01:00.1 Audio device: NVIDIA Corporation AD107 High Definition Audio Controller (rev a1)
        Subsystem: Lenovo Device 3c90
        Kernel driver in use: snd_hda_intel
        Kernel modules: snd_hda_intel
cslash@Mirai ~> sudo journalctl -b | grep tree
cslash@Mirai ~ [0|1]>

Sample of crash:

Oct 06 11:55:25 Mirai systemd[1]: NetworkManager-dispatcher.service: Deactivated successfully.
Oct 06 11:55:28 Mirai systemd[1]: systemd-hostnamed.service: Deactivated successfully.
Oct 06 11:55:30 Mirai kernel: sd-resolve[874]: segfault at 0 ip 0000000000000000 sp 000072288c7fab60 error 14 likely on CPU 4 (core 8, socket 0)
Oct 06 11:55:30 Mirai kernel: Code: Unable to access opcode bytes at 0xffffffffffffffd6.
Oct 06 11:55:30 Mirai systemd-coredump[2204]: Process 870 (systemd-timesyn) of user 975 terminated abnormally with signal 11/SEGV, processing...
Oct 06 11:55:30 Mirai systemd[1]: Created slice Slice /system/drkonqi-coredump-processor.
Oct 06 11:55:30 Mirai systemd[1]: Created slice Slice /system/systemd-coredump.
Oct 06 11:55:30 Mirai systemd[1]: Started Process Core Dump (PID 2204/UID 0).
Oct 06 11:55:30 Mirai systemd[1]: Started Pass systemd-coredump journal entries to relevant user for potential DrKonqi handling.
Oct 06 11:55:31 Mirai systemd-coredump[2205]: [?] Process 870 (systemd-timesyn) of user 975 dumped core.
                                              
                                              Stack trace of thread 874:
                                              #0  0x0000000000000000 n/a (n/a + 0x0)
                                              #1  0x000072288d1b439d n/a (libc.so.6 + 0x9439d)
                                              #2  0x000072288d2392a4 __clone (libc.so.6 + 0x1192a4)
                                              
                                              Stack trace of thread 870:
                                              #0  0x000072288d2398b2 epoll_wait (libc.so.6 + 0x1198b2)
                                              #1  0x000072288d70e9b0 sd_event_wait (libsystemd-shared-256.6-1.so + 0x30e9b0)
                                              #2  0x000072288d7101bc sd_event_run (libsystemd-shared-256.6-1.so + 0x3101bc)
                                              #3  0x000072288d710308 sd_event_loop (libsystemd-shared-256.6-1.so + 0x310308)
                                              #4  0x0000628f1c069b61 n/a (systemd-timesyncd + 0x3b61)
                                              #5  0x000072288d145e08 n/a (libc.so.6 + 0x25e08)
                                              #6  0x000072288d145ecc __libc_start_main (libc.so.6 + 0x25ecc)
                                              #7  0x0000628f1c06a0e5 n/a (systemd-timesyncd + 0x40e5)
                                              ELF object binary architecture: AMD x86-64
Oct 06 11:55:31 Mirai systemd[1]: systemd-coredump@0-2204-0.service: Deactivated successfully.
Oct 06 11:55:31 Mirai systemd[1]: systemd-timesyncd.service: Main process exited, code=dumped, status=11/SEGV
Oct 06 11:55:31 Mirai systemd[1]: systemd-timesyncd.service: Failed with result 'core-dump'.
Oct 06 11:55:31 Mirai systemd[1]: systemd-timesyncd.service: Scheduled restart job, restart counter is at 1.
Oct 06 11:55:31 Mirai systemd[1]: Starting Network Time Synchronization...
Oct 06 11:55:31 Mirai systemd[1]: Started Network Time Synchronization.
Oct 06 11:55:31 Mirai drkonqi-coredump-processor[2206]: "/usr/lib/systemd/systemd-timesyncd" 870 "/var/lib/systemd/coredump/core.systemd-timesyn.975.8c0c8fceefc44b9894909fd049206fbb.8>
Oct 06 11:55:31 Mirai drkonqi-coredump-processor[2206]: The socket path doesn't exist @ "/run/user/975/drkonqi-coredump-launcher"
Oct 06 11:55:31 Mirai kernel: sd-resolve[2218]: segfault at 0 ip 0000000000000000 sp 00007ac045bfab60 error 14 likely on CPU 8 (core 20, socket 0)
... (continues)

seth · 2024-10-06 14:19:54

"ibt=off intel_iommu=on"
"ibt=off" should no longer be required and it's not a good sign when you have to force the intel_iommu
What if you skip that (and vfio)?
And also disable https://wiki.archlinux.org/title/Zswap ?

CounterSlash · 2024-10-06 19:51:37

seth wrote:

"ibt=off" should no longer be required

In my case, waydroid fails to start so I have to keep IBT disabled. Was ibt=off needed for anything in the past?

seth wrote:

it's not a good sign when you have to force the intel_iommu

The page about PCI passthrough does mention that it may not be needed. I'll try to run my VMs without it to see if they work.

seth wrote:

What if you skip that (and vfio)?

Even with ibt reenabled, iommu=on removed, vfio pci ids removed and zswap disabled, the crash still occurs. I did actually have some issues before with zswap and had disabled it but recently I re-enabled it and the bug(?) seems to have gone away at some point.

Flat · 2024-10-07 01:18:22

Do you happen to use OpenSnitch? If so that is likely the culprit. https://github.com/systemd/systemd/issues/34615

afader · 2024-10-07 02:30:05

[edit] yeah it's opensnitch

I seem to have this problem too. Not sure when it started. I also use nvidia and latest 6.11.2 arch kernel. I tried putting my pacsave systemd-timesync conf back but it still crashes

❯ cat /etc/systemd/timesyncd.conf.pacsave
[Time]
NTP=time-a-g.nist.gov time-b-g.nist.gov 
FallbackNTP=0.pool.ntp.org 1.pool.ntp.org 0.fr.pool.ntp.org

systemctl status systemd-timesyncd.service
  1 × systemd-timesyncd.service - Network Time Synchronization
  2      Loaded: loaded (/usr/lib/systemd/system/systemd-timesyncd.service; enabled; preset: enabled)
  3     Drop-In: /usr/lib/systemd/system/systemd-timesyncd.service.d
  4              └─apparmor.conf
  5      Active: failed (Result: core-dump) since Sun 2024-10-06 22:26:19 EDT; 2min 15s ago
  6    Duration: 173ms
  7  Invocation: 837813f81e8b48a2bfe32324ed574da9
  8        Docs: man:systemd-timesyncd.service(8)
  9     Process: 53228 ExecStart=/usr/lib/systemd/systemd-timesyncd (code=dumped, signal=SEGV)
 10    Main PID: 53228 (code=dumped, signal=SEGV)
 11      Status: "Idle."
 12 
 13 Oct 06 22:26:19 obsidian systemd[1]: systemd-timesyncd.service: Scheduled restart job, restart counter is at 5.
 14 Oct 06 22:26:19 obsidian systemd[1]: systemd-timesyncd.service: Start request repeated too quickly.
 15 Oct 06 22:26:19 obsidian systemd[1]: systemd-timesyncd.service: Failed with result 'core-dump'.
 16 Oct 06 22:26:19 obsidian systemd[1]: Failed to start Network Time Synchronization.


 ✘  sudo coredumpctl gdb
           PID: 53228 (systemd-timesyn)
           UID: 192 (systemd-timesync)
           GID: 192 (systemd-timesync)
        Signal: 11 (SEGV)
     Timestamp: Sun 2024-10-06 22:26:19 EDT (3min 47s ago)
  Command Line: /usr/lib/systemd/systemd-timesyncd
    Executable: /usr/lib/systemd/systemd-timesyncd
 Control Group: /system.slice/systemd-timesyncd.service
          Unit: systemd-timesyncd.service
         Slice: system.slice
       Boot ID: a52e3db8220c4a078dd787d1ca4f5b88
    Machine ID: 2e13b14e07a644dd8d93b702997bc463
      Hostname: obsidian
       Storage: /var/lib/systemd/coredump/core.systemd-timesyn.192.a52e3db8220c4a078dd787d1ca4f5b88.53228.1728267979000000.zst (present)
  Size on Disk: 1M
       Message: Process 53228 (systemd-timesyn) of user 192 dumped core.
                
                Stack trace of thread 53229:
                #0  0x0000000000000000 n/a (n/a + 0x0)
                #1  0x00007dcd6cf3039d n/a (libc.so.6 + 0x9439d)
                #2  0x00007dcd6cfb52a4 __clone (libc.so.6 + 0x1192a4)
                
                Stack trace of thread 53228:
                #0  0x00007dcd6cfb58b2 epoll_wait (libc.so.6 + 0x1198b2)
                #1  0x00007dcd6d50e9b0 sd_event_wait (libsystemd-shared-256.6-1.so + 0x30e9b0)
                #2  0x00007dcd6d5101bc sd_event_run (libsystemd-shared-256.6-1.so + 0x3101bc)
                #3  0x00007dcd6d510308 sd_event_loop (libsystemd-shared-256.6-1.so + 0x310308)
                #4  0x0000614b2bcafb61 n/a (systemd-timesyncd + 0x3b61)
                #5  0x00007dcd6cec1e08 n/a (libc.so.6 + 0x25e08)
                #6  0x00007dcd6cec1ecc __libc_start_main (libc.so.6 + 0x25ecc)
                #7  0x0000614b2bcb00e5 n/a (systemd-timesyncd + 0x40e5)
                ELF object binary architecture: AMD x86-64

Last edited by afader (2024-10-10 21:58:57)

CounterSlash · 2024-10-07 07:22:39

Flat wrote:

Do you happen to use OpenSnitch? If so that is likely the culprit.

Yes, I do and indeed it is the culprit! After removing opensnitch-ebpf-module, the service starts and operates normally. There's an issue opened on the opensnitch github as well.
I'm marking this thread as solved, since all I can do now is wait for the module to be updated.

SimpleTechGuy · 2024-11-09 00:38:15

Hi, thanks for mentioning this opensnitch issue. Have any of you using opensnitch run into an issue after that latest pacman -Syyu update where there is a 2 minute delay after logging into your system before anything loads up? It's happening on all of my arch systems that have been updated in the past few days...

Arch Linux

#1 2024-10-02 20:06:17

[SOLVED] systemd-timesyncd fails to start after recent update

#2 2024-10-03 07:00:27

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#3 2024-10-03 12:12:16

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#4 2024-10-03 15:26:45

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#5 2024-10-03 19:51:30

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#6 2024-10-05 07:42:48

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#7 2024-10-05 12:43:32

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#8 2024-10-05 19:14:02

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#9 2024-10-05 19:31:33

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#10 2024-10-05 20:16:03

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#11 2024-10-05 20:25:21

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#12 2024-10-05 20:30:33

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#13 2024-10-06 07:01:07

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#14 2024-10-06 09:06:17

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#15 2024-10-06 14:19:54

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#16 2024-10-06 19:51:37

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#17 2024-10-07 01:18:22

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#18 2024-10-07 02:30:05

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#19 2024-10-07 07:22:39

Re: [SOLVED] systemd-timesyncd fails to start after recent update

#20 2024-11-09 00:38:15

Re: [SOLVED] systemd-timesyncd fails to start after recent update

Board footer