Systemd issues after upgrading to 244

VVL · 2019-12-08 00:14:37

After upgrading arch system to last version (last update wasnt old, about 10 days), it received systemd 244. Rebooting > have much systemd services errors.
Journalctl shows:

Dec 08 02:58:16 systemd-resolved[19295]: Positive Trust Anchors:
Dec 08 02:58:16 systemd-resolved[19295]: . IN DS 19036 8 2 49aac11d7b6f6446702e54a1607371607a1a41855200fd2ce1cdde32f24e8fb5
Dec 08 02:58:16 systemd-resolved[19295]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
Dec 08 02:58:16 systemd-resolved[19295]: Negative trust anchors: 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.i>
Dec 08 02:58:16 systemd-resolved[19295]: Using system hostname 'trololo'.
Dec 08 02:58:16 systemd-resolved[19295]: Could not create manager: Permission denied
Dec 08 02:58:16 systemd[1]: systemd-resolved.service: Main process exited, code=exited, status=1/FAILURE
Dec 08 02:58:16 systemd[1]: systemd-resolved.service: Failed with result 'exit-code'.
Dec 08 02:58:16 systemd[1]: Failed to start Network Name Resolution.
Dec 08 02:58:16 systemd[1]: systemd-resolved.service: Scheduled restart job, restart counter is at 5.
Dec 08 02:58:16 systemd[1]: Stopped Network Name Resolution.
Dec 08 02:58:16 systemd[1]: systemd-resolved.service: Start request repeated too quickly.
Dec 08 02:58:16 systemd[1]: systemd-resolved.service: Failed with result 'exit-code'.

Dec 08 02:20:58 systemd[1]: Starting Network Time Synchronization...
Dec 08 02:20:59 systemd-timesyncd[890]: Failed to allocate manager: Permission denied
Dec 08 02:20:59 systemd[1]: systemd-timesyncd.service: Main process exited, code=exited, status=1/FAILURE
Dec 08 02:20:59 systemd[1]: systemd-timesyncd.service: Failed with result 'exit-code'.
Dec 08 02:20:59 systemd[1]: Failed to start Network Time Synchronization.

When trying to run this services in bash directly, seems same error:

root ~ # /usr/lib/systemd/systemd-timesyncd
Failed to allocate manager: Permission denied

What going wrong?

xerxes_ · 2019-12-08 12:26:37

Post output of command:

systemctl status systemd-resolved

VVL · 2019-12-09 06:32:00

● systemd-resolved.service - Network Name Resolution
     Loaded: loaded (/etc/systemd/system/systemd-resolved.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/systemd-resolved.service.d
             └─override.conf
     Active: inactive (dead) since Sun 2019-12-08 02:58:16 MSK; 1 day 6h ago
       Docs: man:systemd-resolved.service(8)
             https://www.freedesktop.org/wiki/Software/systemd/resolved
             https://www.freedesktop.org/wiki/Software/systemd/writing-network-configuration-managers
             https://www.freedesktop.org/wiki/Software/systemd/writing-resolver-clients
    Process: 19295 ExecStart=/usr/lib/systemd/systemd-resolved (code=exited, status=1/FAILURE)
   Main PID: 19295 (code=exited, status=1/FAILURE)

Dec 08 02:58:16 Eviron systemd[1]: systemd-resolved.service: Scheduled restart job, restart counter is at 5.
Dec 08 02:58:16 Eviron systemd[1]: Stopped Network Name Resolution.
Dec 08 02:58:16 Eviron systemd[1]: systemd-resolved.service: Start request repeated too quickly.
Dec 08 02:58:16 Eviron systemd[1]: systemd-resolved.service: Failed with result 'exit-code'.
Dec 08 02:58:16 Eviron systemd[1]: Failed to start Network Name Resolution.

CarbonChauvinist · 2019-12-09 12:54:15

output of

systemctl cat systemd-resolved

and for good measure

systemctl list-unit-files --state enabled

and

systemd-delta

Last edited by CarbonChauvinist (2019-12-09 18:35:17)

VVL · 2019-12-10 12:59:48

~ # systemctl cat systemd-resolved
# /etc/systemd/system/systemd-resolved.service
#  SPDX-License-Identifier: LGPL-2.1+
#
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

[Unit]
Description=Network Name Resolution
Documentation=man:systemd-resolved.service(8)
Documentation=https://www.freedesktop.org/wiki/Software/systemd/resolved
Documentation=https://www.freedesktop.org/wiki/Software/systemd/writing-network-configuration-managers
Documentation=https://www.freedesktop.org/wiki/Software/systemd/writing-resolver-clients
DefaultDependencies=no
After=systemd-sysusers.service systemd-networkd.service
Before=network.target nss-lookup.target shutdown.target
Conflicts=shutdown.target
Wants=nss-lookup.target

[Service]
AmbientCapabilities=CAP_SETPCAP CAP_NET_RAW CAP_NET_BIND_SERVICE
CapabilityBoundingSet=CAP_SETPCAP CAP_NET_RAW CAP_NET_BIND_SERVICE
ExecStart=!!/usr/lib/systemd/systemd-resolved
LockPersonality=yes
MemoryDenyWriteExecute=yes
NoNewPrivileges=yes
PrivateDevices=yes
PrivateTmp=yes
ProtectControlGroups=yes
ProtectHome=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
ProtectKernelLogs=yes
ProtectSystem=strict
Restart=always
RestartSec=0
RestrictAddressFamilies=AF_UNIX AF_NETLINK AF_INET AF_INET6
RestrictNamespaces=yes
RestrictRealtime=yes
RestrictSUIDSGID=yes
RuntimeDirectory=systemd/resolve
RuntimeDirectoryPreserve=yes
SystemCallArchitectures=native
SystemCallErrorNumber=EPERM
SystemCallFilter=@system-service
Type=notify
User=systemd-resolve
WatchdogSec=3min

[Install]
WantedBy=multi-user.target
Alias=dbus-org.freedesktop.resolve1.service

~ # systemctl list-unit-files --state enabled
UNIT FILE                              STATE  
autovt@.service                        enabled
clamav-daemon.service                  enabled
dbus-org.freedesktop.network1.service  enabled
dbus-org.freedesktop.resolve1.service  enabled
dbus-org.freedesktop.timesync1.service enabled
dovecot.service                        enabled
fail2ban.service                       enabled
getty@.service                         enabled
initrd-network.service                 enabled
mariadb.service                        enabled
nftables.service                       enabled
nginx.service                          enabled
php71-fpm.service                      enabled
pm2.service                            enabled
postfix.service                        enabled
redis.service                          enabled
rngd.service                           enabled
rspamd.service                         enabled
sshd.service                           enabled
systemd-networkd-wait-online.service   enabled
systemd-networkd.service               enabled
systemd-resolved.service               enabled
systemd-timesyncd.service              enabled
clamav-daemon.socket                   enabled
systemd-networkd.socket                enabled
remote-fs.target                       enabled
t-clear_logs.timer                     enabled
t-dehydrated.timer                     enabled
t-postfix.timer                        enabled

~ # systemd-delta
[OVERRIDDEN] /etc/systemd/system/clamav-daemon.service → /usr/lib/systemd/system/clamav-daemon.service

--- /usr/lib/systemd/system/clamav-daemon.service       2019-12-06 01:55:11.000000000 +0300
+++ /etc/systemd/system/clamav-daemon.service   2018-07-31 11:57:50.000000000 +0300
@@ -11,7 +11,6 @@
 # Reload the database
 ExecReload=/bin/kill -USR2 $MAINPID
 StandardOutput=syslog
-TimeoutStartSec=420
 
 [Install]
 WantedBy=multi-user.target

1 overridden configuration files found.

CarbonChauvinist · 2019-12-10 16:53:58

Thanks. An earlier post of yours showed a drop in override file for systemd-resolved

● systemd-resolved.service - Network Name Resolution
     Loaded: loaded (/etc/systemd/system/systemd-resolved.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/systemd-resolved.service.d
             └─override.conf

I was expecting to see what exactly you were modifying, but it looks like you've since removed the override? At least based on your last post.

I don't see anything immediately evident with your enabled services WRT conflicting network managment for e.g. But you're also running a lot of services I have no experience with.

I also now see that in your first post timesyncd was failing as well as resolved?

Dec 08 02:20:59 systemd-timesyncd[890]: Failed to allocate manager: Permission denied

And you tried to start it by running as root the following?

root ~ # /usr/lib/systemd/systemd-timesyncd
Failed to allocate manager: Permission denied

Why would you try to manually run it that way? Why wouldn't you just use systemctl to start the service? Or enable --now the service? Not sure what that was meant to accomplish, but I may be missing something. <shrug>

Actually, with all that being said - I'm not sure we really know where you currently stand with your system. You've not been specific about exactly which services fail. Are there more than one failing? Are they still failing after you've made whatever changes you made since your first post?

$ systemctl --failed

On a lark, what are the permissions of the root dir?

ls -ld /

Other's more skilled at troubleshooting may chime in.

VVL · 2019-12-17 12:49:32

I was expecting to see what exactly you were modifying, but it looks like you've since removed the override? At least based on your last post.

.override exists, but it's empty. I tried to edit some preferences after fail, but restored original.

I don't see anything immediately evident with your enabled services WRT conflicting network managment for e.g. But you're also running a lot of services I have no experience with.

It doesnt matter, I already have same bug on 1 more machine, where only systemd and ssh.

I also now see that in your first post timesyncd was failing as well as resolved?

Yes.

And you tried to start it by running as root the following?

Yes.

Why would you try to manually run it that way? Why wouldn't you just use systemctl to start the service? Or enable --now the service? Not sure what that was meant to accomplish, but I may be missing something. <shrug>

Why not? I tried to run it with systemd - its failed. Then I tried to edit some preferences - its failed. Logs are not usable. Then I tried to run it manually.

Actually, with all that being said - I'm not sure we really know where you currently stand with your system. You've not been specific about exactly which services fail. Are there more than one failing? Are they still failing after you've made whatever changes you made since your first post?

Yes, still no progress

On a lark, what are the permissions of the root dir?
ls -ld /

~ # ls -ld /
drwxr-xr-x 1 root root 122 дек 13 14:32 /

Arch Linux

#1 2019-12-08 00:14:37

Systemd issues after upgrading to 244

#2 2019-12-08 12:26:37

Re: Systemd issues after upgrading to 244

#3 2019-12-09 06:32:00

Re: Systemd issues after upgrading to 244

#4 2019-12-09 12:54:15

Re: Systemd issues after upgrading to 244

#5 2019-12-10 12:59:48

Re: Systemd issues after upgrading to 244

#6 2019-12-10 16:53:58

Re: Systemd issues after upgrading to 244

#7 2019-12-17 12:49:32

Re: Systemd issues after upgrading to 244

Board footer