systemd: Yet Another Init Replacement

Viper_Scull · 2011-02-10 21:47:48

falconindy wrote:

@Viper_Scull: Your first error isn't even systemd related, and I have no idea about the latter. I've linked /etc/mtab -> /proc/self/mounts on multiple machines and I can't find that error on any of them.

I supposed it had something to do with it because booting without systemd there were no references about this. I got this error only if i boot with systemd and polkitd.service is enabled, if i disable the service then there is no error

Last edited by Viper_Scull (2011-02-10 21:52:35)

falconindy · 2011-02-10 22:48:29

polkit will start even without the service file. Disable it for now.

ngoonee · 2011-02-11 01:37:33

Anyone has experience with using bootchart (or bootchart2-git) with systemd? Is it possible to have two init lines in grub?

falconindy · 2011-02-11 02:26:31

systemd reports time spent in kernelspace and userspace to the syslog. If you want to be able to run bootchartd, i suspect you'd need to symlink init to /bin/systemd

ngoonee · 2011-02-11 04:30:16

falconindy wrote:

systemd reports time spent in kernelspace and userspace to the syslog. If you want to be able to run bootchartd, i suspect you'd need to symlink init to /bin/systemd

I just found that bootchartd2 has a .service file, enabling that gets me a nice bootchart. Thanks falconindy.

I've been working on net-auto-wired.service and net-auto-wireless.service, please let me know what you think of these 2:-

[Unit]
Description=Provides automatic netcfg wired connection
After=dbus.service

[Service]
ExecStart=/usr/sbin/ifplugd -i $WIRED_INTERFACE -r /etc/ifplugd/netcfg.action -fI -u0 -d10 || /bin/true
ExecStop=/usr/sbin/ifplugd -k -i $WIRED_INTERFACE -r /etc/ifplugd/netcfg.action || /bin/true
EnvironmentFile=/etc/rc.conf
RemainAfterExit=yes
Type=forking

[Install]
WantedBy=network.target

Sourcing rc.conf gives the $WIRED_INTERFACE set there. The arguments at the back of ExecStart are taken from /etc/ifplugd/ifplugd.conf (couldn't figure out how to source to EnvironmentFiles), I removed -w which waits for completion because I hate waiting . ifplugd can return non-0 according to /etc/rc.d/net-auto-wired, so I append /bin/true, don't know if it works though.

[Unit]
Description=Provides automatic netcfg wireless connection
After=dbus.service

[Service]
ExecStart=/usr/bin/netcfg-wpa_actiond $WIRELESS_INTERFACE
ExecStop=/usr/bin/netcfg-wpa_actiond stop $WIRELESS_INTERFACE
EnvironmentFile=/etc/rc.conf
RemainAfterExit=yes
Type=forking

[Install]
WantedBy=network.target

This one is much simpler, of course. Another question, should I enclose WIRED_INTERFACE and WIRELESS_INTERFACE in curly brackets?

falconindy · 2011-02-11 05:02:30

A few comments:

1) You can ignore errors by prefixing the command with a -, e.g.

ExecStart=-/usr/bin/ifplugd

2) Whenever possible, avoid making services of type forking. ifplugd has a -n flag for no-daemonize. Use it. systemd will fork the daemon for you and will gain better control over it than if the program forks itself. This also lets you ditch the RemainAfterExit.
3) Both units should be WantedBy multi-user.target. However, you'll want to specify, in the unit section, Before=network.target.
4) Setting a unit to run after dbus is bogus, since dbus is activated by systemd very early in the boot process. You can pretty much guarantee this condition will be met implicitly.

ngoonee · 2011-02-11 06:14:45

falconindy wrote:

2) Whenever possible, avoid making services of type forking. ifplugd has a -n flag for no-daemonize. Use it. systemd will fork the daemon for you and will gain better control over it than if the program forks itself. This also lets you ditch the RemainAfterExit.

Does it then still need ExecStop?

falconindy wrote:

3) Both units should be WantedBy multi-user.target. However, you'll want to specify, in the unit section, Before=network.target.

Should I keep WantedBy=network.target as well? From the man page (systemd.special) network.target is a dependency of any units which require a network, hence why I put it there rather than multi-user.target. Use both? Only multi-user?

falconidy wrote:

4) Setting a unit to run after dbus is bogus, since dbus is activated by systemd very early in the boot process. You can pretty much guarantee this condition will be met implicitly.

That part is copied from your arch-daemons system-generator

Dieter@be · 2011-02-11 12:50:58

is there a howto/documentation somewhere about these systemd packages?
or just https://wiki.archlinux.org/index.php/Systemd ?
I had a few minutes to spare yesterday so I installed systemd-git along with systemd-arch-units and initscripts-systemd, it got stuck somewhere (didn't get the login console), I haven't had time to properly debug the issue.
Also:
- does systemd start dbus even when I never told it to do so? (it probably should, because it relies on it, afaik)
- the wiki page recommends to use "systemctl enable <service>" to start a service during boot, but you can just as well still add it to the DAEMONS array in rc.conf, right?
- the initscripts-systemd-git install file invokes `/bin/systemctl enable $units` and `/bin/systemctl daemon-reload`; this can fail when dbus is not running (i.e. you install while having booted with the default init system), so that should be handled.
Also, why would you want to reload daemons after install/upgrade? that's not the arch way, unless I'm missing something.

falconindy · 2011-02-11 13:48:37

Dieter@be wrote:

is there a howto/documentation somewhere about these systemd packages?
or just https://wiki.archlinux.org/index.php/Systemd ?
I had a few minutes to spare yesterday so I installed systemd-git along with systemd-arch-units and initscripts-systemd, it got stuck somewhere (didn't get the login console), I haven't had time to properly debug the issue.
Also:
- does systemd start dbus even when I never told it to do so? (it probably should, because it relies on it, afaik)

Yes, it's started via socket activation on bootup. A quick perusal of ps shows this.

Dieter@be wrote:

- the wiki page recommends to use "systemctl enable <service>" to start a service during boot, but you can just as well still add it to the DAEMONS array in rc.conf, right?

No, you cannot. Tom Gunderson (tomegun) sent in a patch to remove that logic. It was hackish and doesn't follow what upstream wants us to be doing. Use native unit files, even if they're just a silly wrapper around the legacy /etc/rc.d script. If you absolutely must, you can enable the arch-daemons.target.

Dieter@be wrote:

- the initscripts-systemd-git install file invokes `/bin/systemctl enable $units` and `/bin/systemctl daemon-reload`; this can fail when dbus is not running (i.e. you install while having booted with the default init system), so that should be handled.
Also, why would you want to reload daemons after install/upgrade? that's not the arch way, unless I'm missing something.

Hiding errors isn't the Arch way, either, but I'll see if I can find a way to conditionally avoid it. The install scriptlet for systemd-arch-units forces systemd to re-read unit files. It's not restarting the daemons themselves, but I can see why the comment I placed in the install scriptlet might be interpreted that way.

falconindy · 2011-02-11 13:54:11

ngoonee wrote:

falconindy wrote:
2) Whenever possible, avoid making services of type forking. ifplugd has a -n flag for no-daemonize. Use it. systemd will fork the daemon for you and will gain better control over it than if the program forks itself. This also lets you ditch the RemainAfterExit.
Does it then still need ExecStop?

Not if SIGTERM brings it down in a friendly manner. In this case, I guess there's hooks that might want to run on shutdown so you probably want to leave it how it is (minus the || true).

ngoonee wrote:

falconindy wrote:
3) Both units should be WantedBy multi-user.target. However, you'll want to specify, in the unit section, Before=network.target.
Should I keep WantedBy=network.target as well? From the man page (systemd.special) network.target is a dependency of any units which require a network, hence why I put it there rather than multi-user.target. Use both? Only multi-user?

There's a fair bit of semi-confusing verbiage going on in unit files. A unit which requires a network connection should specify After=network.target. A unit which provides a network connection should specify Before=network.target. The idea here is that by the time network.target is satisfied, you in fact have a network connection available. I hope that makes sense.

ngoonee wrote:

falconindy wrote:
4) Setting a unit to run after dbus is bogus, since dbus is activated by systemd very early in the boot process. You can pretty much guarantee this condition will be met implicitly.
That part is copied from your arch-daemons system-generator

meh... that's not my kid.

Dieter@be · 2011-02-11 20:20:25

1)

$ cat /etc/modprobe.d/rc.conf 
# Autogenerated by systemd, do not edit
install  /bin/false

the regular init system generates quite some errors about this while booting
2) I had a look at why the boot hangs: it stops after mounting my /home (well, it says "starting /home", and afterwards nothing seems to happen). it did manage to bring up my swap. swap, home and root are all logical volumes on top of an encrypted partition, so if swap works that means the logical volumes are accessible.

falconindy · 2011-02-11 20:33:35

Dieter@be wrote:

1)
$ cat /etc/modprobe.d/rc.conf 
# Autogenerated by systemd, do not edit
install  /bin/false
the regular init system generates quite some errors about this while booting
2) I had a look at why the boot hangs: it stops after mounting my /home (well, it says "starting /home", and afterwards nothing seems to happen). it did manage to bring up my swap. swap, home and root are all logical volumes on top of an encrypted partition, so if swap works that means the logical volumes are accessible.

1) Since you didn't post your MODULES array, I'm going to assume that it's blank and make appropriations for that in the bash script handling those files. That's pushed.
2) If you need help debugging a boot issue, I again suggest adding systemd.log_level=debug to your kernel cmdline.

Dieter@be · 2011-02-11 20:52:14

falconindy wrote:

1) Since you didn't post your MODULES array, I'm going to assume that it's blank and make appropriations for that in the bash script handling those files. That's pushed.
2) If you need help debugging a boot issue, I again suggest adding systemd.log_level=debug to your kernel cmdline.

not quite:

~ grep -v ^#  /etc/rc.conf 
LOCALE="en_US.utf8"
HARDWARECLOCK="UTC"
USEDIRECTISA="no"
TIMEZONE="Europe/Brussels"
KEYMAP="dvorak"
CONSOLEFONT=
CONSOLEMAP=
USECOLOR="yes"
MOD_AUTOLOAD="yes"
MODULES=(3c59x e1000 forcedeth mii slhc autofs4 ac97_bus \
         snd-mixer-oss snd-pcm-oss snd-seq-oss snd-seq-device snd-seq-midi-event snd-seq snd-page-alloc snd-pcm snd-rawmidi snd-timer snd snd-mpu401-uart snd-mpu401 snd-ac97-codec snd-intel8x0 soundcore \
         vboxdrv loop !pcspkr !snd_pcsp nfs vboxnetflt)
USELVM="yes"
HOSTNAME="dieter-ws"
eth0="dhcp"
INTERFACES=(eth0)
gateway="default gw 192.168.1.1"
ROUTES=(gateway)
NETWORK_PERSIST="no"
DAEMONS=(syslog-ng hal network @netfs @crond @fam @openntpd @autofs @alsa @scrobby @sshd @bluetooth @lighttpd)

about issue 2, first it hangs for a minute (or more) like this: http://dieter.plaetinck.be/files/20110211_002.jpg (about the nfs mount: i have no idea why that fails, works fine with the normal init system)
then, it hangs on this: http://dieter.plaetinck.be/files/20110211_003.jpg (I don't even use raid?)
I'll try out the debug setting.

Last edited by Dieter@be (2011-02-11 20:53:23)

tomegun · 2011-02-12 20:51:13

Just a comment about this:

falconindy wrote:

2) Whenever possible, avoid making services of type forking. ifplugd has a -n flag for no-daemonize. Use it. systemd will fork the daemon for you and will gain better control over it than if the program forks itself. This also lets you ditch the RemainAfterExit.

You should keep in mind that if you do not set Type=forking (which makes it default to Type=simple), then the service will be considered started immediately. If, however, you use Type=forking, then it will only be considered started after it forks.

There might be good reasons to use Type=forking in case a service sets up some functionality (like a network connection) before it forks, otherwise ordering something "after" the service does not make sense.

tomegun · 2011-02-12 21:05:04

falconindy wrote:

ngoonee wrote:
falconindy wrote:
4) Setting a unit to run after dbus is bogus, since dbus is activated by systemd very early in the boot process. You can pretty much guarantee this condition will be met implicitly.
That part is copied from your arch-daemons system-generator
meh... that's not my kid.

To defend my kid:
The generated files are not a good starting point for creating service files, as they are, well, generated. They are (supposedly) emulating the way Arch starts daemons, but in systemd we can of course do much better. It is much better to start from the existing service files in systemd-arch-units.

The particular case you have "After=dbus.service", is just because your DAEMONS array is something along the lines of

DAEMONS=(... dbus @foo ... @bar net-auto-wired ...)

That is, dbus is the last non-backgrounded daemon before net-auto-wired.

tomegun · 2011-02-12 21:14:39

falconindy wrote:

Dieter@be wrote:
- the wiki page recommends to use "systemctl enable <service>" to start a service during boot, but you can just as well still add it to the DAEMONS array in rc.conf, right?
No, you cannot. Tom Gunderson (tomegun) sent in a patch to remove that logic. It was hackish and doesn't follow what upstream wants us to be doing. Use native unit files, even if they're just a silly wrapper around the legacy /etc/rc.d script. If you absolutely must, you can enable the arch-daemons.target.

arch-daemons.target is only supposed to support the minimum required to not lose any functionality when moving from sysvinit to systemd, that is, if having something in the DAEMONS array works with sysvinit, then it should also work with systemd. However, it is strongly discouraged to rely on it to do things beyond this. E.g. if you have a systemd service file that you want to run during boot, with no corresponding script in /etc/rc.d/, then adding it to the DAEMONS array is not supported (though it might work by accident).

Another way to see it: if your DAEMONS array has errors according to sysvinit, then its behavior is undefined under systemd.

falconindy · 2011-02-13 16:53:15

initscripts-systemd-git: fixed the issue with arch-persistent-settings generating bad /etc/modprobe.d/rc.conf files. I'm able to reproduce the hang on boot so I'll try to figure out what's causing it. In the meantime, you can get by it with SysRq-e (yes I realize that's not a solution).

Last edited by falconindy (2011-02-13 16:57:18)

tomegun · 2011-02-13 17:21:44

Dieter@be wrote:

about issue 2, first it hangs for a minute (or more) like this: http://dieter.plaetinck.be/files/20110211_002.jpg (about the nfs mount: i have no idea why that fails, works fine with the normal init system)
then, it hangs on this: http://dieter.plaetinck.be/files/20110211_003.jpg (I don't even use raid?)
I'll try out the debug setting.

I don't use nfs, so cannot reproduce, but my best bet is that your network is not up when your nfs mount is started.

A network mount should be ordered after network.target. I can think of two problems:
Maybe network.target is not pulled in at all? It is not enough to be ordered After stuff, it must be Wanted or Required by something. To check the status, do "systemctl show network.target" and "systemctl status network.target".
Alternatively, there might be something wrong with the network service. Is it enabled? Similarly, do "systemctl show network.service" and "systemctl status network.service". I notice that it is of Type=oneshot, it might be that it should be of Type=forking instead (as it will fork off a dhcpcd daemon (and maybe more), so maybe it is never considered "started").

Just my 2c.

falconindy · 2011-02-13 17:52:55

Figured out the hang on boot: removing "Conflicts=systemd-kmsg-syslogd.service" solves it. This is removed in the git repo now.

@tomegun: why did we want this line to begin with? obviously when you first added it, there were no noticeable adverse affects because of the typo. It looks like having proper socket activation of syslog-ng causes the kmsg bridge to shut down on its own. That's the path we should be supporting. Feel free to bonk me if I'm missing something.

Last edited by falconindy (2011-02-13 18:04:44)

tomegun · 2011-02-13 18:16:07

Hmmmm... interesting....
The original idea was taken from a comment by Lennart about how a service file for rsyslog should/could look like. Namely, that it should stop (not conflict, but I think conflict eventually will be the right solution, though maybe not yet) systemd-kmsg-syslogd.service before starting rsyslog.service.

The idea being that rsyslog/syslog-ng will take over the syslog socket (/dev/log), and there not being a need for more than one logging daemon. If rsyslog/syslog-ng then crashes/stops in the future, the socket will activate systemd-kmsg-syslogd again and all will be well.

This all works very nicely with your syslog-ng-systemd package (with socket activation support), at least for me (correct me if I'm wrong here, maybe you see the problem even with socket activation support?).

This is my guess at what is wrong: In the case where syslog-ng/rsyslog does not have socket activation support, then systemd-kmsg-syslogd will keep being started by the socket, but it cannot as it conflicts with an already running service.

Regardless of what the underlying cause is, I guess we should revert the Conflicts from the rsyslog/syslog-ng files for now (I think everything should work even if two daemons are listening to /dev/log, so it does not matter). We could try again once socket activation has reached the official packages (should not take long).

PS
Would be nice if you could describe a simple test case (it seems that it has nothing to do with networking) so we can check if it works or not in the future.

Oni · 2011-03-10 10:49:38

Hi,

I install Systemd and I can't shutdown my machin, it's just hang up on shutdown.
I add a pacmandb.service (I use pacmandb to grown speed pacman access to repos' local info) based on alsa.service (on start I run /etc/rc.d/pacmandb start on stop /etc/rc.d/pacmandb stop)
Maybe, it can have a link…
Otherwise, I see that on systemd's bug tracker : https://bugs.freedesktop.org/show_bug.cgi?id=33421
If, anyone have a idea about this, this can be cool
Thanks

Viper_Scull · 2011-03-10 11:59:37

Oni wrote:

Hi,
I install Systemd and I can't shutdown my machin, it's just hang up on shutdown.
I add a pacmandb.service (I use pacmandb to grown speed pacman access to repos' local info) based on alsa.service (on start I run /etc/rc.d/pacmandb start on stop /etc/rc.d/pacmandb stop)
Maybe, it can have a link…
Otherwise, I see that on systemd's bug tracker : https://bugs.freedesktop.org/show_bug.cgi?id=33421
If, anyone have a idea about this, this can be cool
Thanks

How long did u wait till press the button?

It happened to me a couple of times. Let it be for 1 minute or minute and a half. It should shutdown.

Oni · 2011-03-10 12:50:29

Viper_Scull wrote:

Oni wrote:
Hi,
I install Systemd and I can't shutdown my machin, it's just hang up on shutdown.
I add a pacmandb.service (I use pacmandb to grown speed pacman access to repos' local info) based on alsa.service (on start I run /etc/rc.d/pacmandb start on stop /etc/rc.d/pacmandb stop)
Maybe, it can have a link…
Otherwise, I see that on systemd's bug tracker : https://bugs.freedesktop.org/show_bug.cgi?id=33421
If, anyone have a idea about this, this can be cool
Thanks
How long did u wait till press the button?
It happened to me a couple of times. Let it be for 1 minute or minute and a half. It should shutdown.

I wait 3/4 minutes before "playing" with SysRq to shutdown.
Maybe I should try to see with debug/verbose, if I could have a clue

tomegun · 2011-03-10 13:29:52

Oni,

If you boot with the "verbose" kernel parameter you will probably get the right output to tell what is going on.

If I remember correctly, the standard timeout is 3 minutes (increased from 1). This means that on shutdown systemd will send the TERM signal, and if the service is still there after 3 minutes it will kill it. Most probably this is a misbehaved daemon of some sort, would be nice if we could figure out which one.

By way of comparison, the arch shutdown script will wait for only a few seconds (so you might not have noticed if there was a problem before).

tomegun · 2011-03-10 13:31:10

PS
This is of course assuming that you are not hit by the systemd bug you linked to above...

Arch Linux

#176 2011-02-10 21:47:48

Re: systemd: Yet Another Init Replacement

#177 2011-02-10 22:48:29

Re: systemd: Yet Another Init Replacement

#178 2011-02-11 01:37:33

Re: systemd: Yet Another Init Replacement

#179 2011-02-11 02:26:31

Re: systemd: Yet Another Init Replacement

#180 2011-02-11 04:30:16

Re: systemd: Yet Another Init Replacement

#181 2011-02-11 05:02:30

Re: systemd: Yet Another Init Replacement

#182 2011-02-11 06:14:45

Re: systemd: Yet Another Init Replacement

#183 2011-02-11 12:50:58

Re: systemd: Yet Another Init Replacement

#184 2011-02-11 13:48:37

Re: systemd: Yet Another Init Replacement

#185 2011-02-11 13:54:11

Re: systemd: Yet Another Init Replacement

#186 2011-02-11 20:20:25

Re: systemd: Yet Another Init Replacement

#187 2011-02-11 20:33:35

Re: systemd: Yet Another Init Replacement

#188 2011-02-11 20:52:14

Re: systemd: Yet Another Init Replacement

#189 2011-02-12 20:51:13

Re: systemd: Yet Another Init Replacement

#190 2011-02-12 21:05:04

Re: systemd: Yet Another Init Replacement

#191 2011-02-12 21:14:39

Re: systemd: Yet Another Init Replacement

#192 2011-02-13 16:53:15

Re: systemd: Yet Another Init Replacement

#193 2011-02-13 17:21:44

Re: systemd: Yet Another Init Replacement

#194 2011-02-13 17:52:55

Re: systemd: Yet Another Init Replacement

#195 2011-02-13 18:16:07

Re: systemd: Yet Another Init Replacement

#196 2011-03-10 10:49:38

Re: systemd: Yet Another Init Replacement

#197 2011-03-10 11:59:37

Re: systemd: Yet Another Init Replacement

#198 2011-03-10 12:50:29

Re: systemd: Yet Another Init Replacement

#199 2011-03-10 13:29:52

Re: systemd: Yet Another Init Replacement

#200 2011-03-10 13:31:10

Re: systemd: Yet Another Init Replacement

Board footer