systemd: Yet Another Init Replacement

Da_Coynul · 2012-02-27 18:27:59

MartinZ wrote:

I just updated my box (haven't done that in about a week) and now I can't boot. I'm currently running from a liveUSB. I use testing repo (because I need Libreoffice 3.5) and linux-ck.
Trying to boot gives me this:
Welcome to emergency mode. Use "systemctl default" or ^D to activate default mode.
However, I cannot log in nor write anything, just rebooting with ctrl+alt+supr works. This seems to be because of an update which overwrote my /etc/inittab thus misconfiguring fgetty, will test in next reboot.

Same thing happened here so I reverted to a pre-update backup, changed back to initscripts from systemd, updated and then changed back to systemd. Now everything is working but I would like to know the cause / fix just in case it happens again and I am caught without a recent backup.

tomegun · 2012-02-27 18:35:58

Da_Coynul wrote:

MartinZ wrote:
I just updated my box (haven't done that in about a week) and now I can't boot. I'm currently running from a liveUSB. I use testing repo (because I need Libreoffice 3.5) and linux-ck.
Trying to boot gives me this:
Welcome to emergency mode. Use "systemctl default" or ^D to activate default mode.
However, I cannot log in nor write anything, just rebooting with ctrl+alt+supr works. This seems to be because of an update which overwrote my /etc/inittab thus misconfiguring fgetty, will test in next reboot.
Same thing happened here so I reverted to a pre-update backup, changed back to initscripts from systemd, updated and then changed back to systemd. Now everything is working but I would like to know the cause / fix just in case it happens again and I am caught without a recent backup.

I have not seen this behavior. You mention fgetty, might that be the cause? Do you have any other non-standard packages installed that might be related? Changes to /etc/inittab should not be related, as systemd ignores that file.

Regarding fgetty, have you considered using agetty from util-linux instead? All features from mingetty has been merged into it, so it should cover the vast majority of use-cases by now.

falconindy · 2012-02-27 19:35:39

fgetty doesn't even support PAM without patching... there's no way it's going to work well with systemd.

Da_Coynul · 2012-02-27 19:47:27

tomegun wrote:

Da_Coynul wrote:
MartinZ wrote:
I just updated my box (haven't done that in about a week) and now I can't boot. I'm currently running from a liveUSB. I use testing repo (because I need Libreoffice 3.5) and linux-ck.
Trying to boot gives me this:
Welcome to emergency mode. Use "systemctl default" or ^D to activate default mode.
However, I cannot log in nor write anything, just rebooting with ctrl+alt+supr works. This seems to be because of an update which overwrote my /etc/inittab thus misconfiguring fgetty, will test in next reboot.
Same thing happened here so I reverted to a pre-update backup, changed back to initscripts from systemd, updated and then changed back to systemd. Now everything is working but I would like to know the cause / fix just in case it happens again and I am caught without a recent backup.
I have not seen this behavior. You mention fgetty, might that be the cause? Do you have any other non-standard packages installed that might be related? Changes to /etc/inittab should not be related, as systemd ignores that file.
Regarding fgetty, have you considered using agetty from util-linux instead? All features from mingetty has been merged into it, so it should cover the vast majority of use-cases by now.

I'm using agetty from util-linux and do not have fgetty installed so I don't think that is the cause.

MartinZ · 2012-02-27 21:06:50

tomegun wrote:

Da_Coynul wrote:
MartinZ wrote:
I just updated my box (haven't done that in about a week) and now I can't boot. I'm currently running from a liveUSB. I use testing repo (because I need Libreoffice 3.5) and linux-ck.
Trying to boot gives me this:
Welcome to emergency mode. Use "systemctl default" or ^D to activate default mode.
However, I cannot log in nor write anything, just rebooting with ctrl+alt+supr works. This seems to be because of an update which overwrote my /etc/inittab thus misconfiguring fgetty, will test in next reboot.
Same thing happened here so I reverted to a pre-update backup, changed back to initscripts from systemd, updated and then changed back to systemd. Now everything is working but I would like to know the cause / fix just in case it happens again and I am caught without a recent backup.
I have not seen this behavior. You mention fgetty, might that be the cause? Do you have any other non-standard packages installed that might be related? Changes to /etc/inittab should not be related, as systemd ignores that file.
Regarding fgetty, have you considered using agetty from util-linux instead? All features from mingetty has been merged into it, so it should cover the vast majority of use-cases by now.

You're right about getty. I used fgetty before, but surely after changing to systemd it started using agetty and I didn't note. So I'm stuck, I don't have a working console nor know how to get one. I currently am only able to boot from a LiveUSB.

@Da_Conyul: could you enter a console when you had the problem?

Thanks a lot for the help.

Avant-texte · 2012-02-27 21:17:44

Bringing up getty + systemd problems. Anyone know if qingy doesn't play well with systemd? I'm going to try systemd out soon and it would be nice to have a heads up since I use qingy atm.

Da_Coynul · 2012-02-27 21:35:30

MartinZ wrote:

@Da_Conyul: could you enter a console when you had the problem?
Thanks a lot for the help.

I was able to get to a console and I noticed that my /boot/grub was empty except for menu.lst. My initial attempt to repair grub was not successful and instead of spending more time on it, I just decided to restore my backup. After restoring I tried updating again while using systemd and had the same result. Then I restored one more time, changed to initscripts and rebooted successully after the update.

It seems like something wiped out GRUB. I'm not sure what it was or why it only happend when I tried to update while using systemd and not with initscripts.

@MartinZ, is your /boot/grub empty as well?

MartinZ · 2012-02-27 22:05:23

Da_Coynul wrote:

I was able to get to a console and I noticed that my /boot/grub was empty except for menu.lst. My initial attempt to repair grub was not successful and instead of spending more time on it, I just decided to restore my backup. After restoring I tried updating again while using systemd and had the same result. Then I restored one more time, changed to initscripts and rebooted successully after the update.
It seems like something wiped out GRUB. I'm not sure what it was or why it only happend when I tried to update while using systemd and not with initscripts.
@MartinZ, is your /boot/grub empty as well?

I use syslinux instead of grub, and it seems to be properly configured:

LABEL arch-ck
        MENU LABEL Arch Linux -ck
        LINUX /boot/vmlinuz-linux-ck
        APPEND root=/dev/sda3 elevator=noop ro fastboot logo.nologo quiet init=/bin/systemd vga=773
        INITRD /boot/initramfs-linux-ck.img

Da_Coynul · 2012-02-27 23:07:07

MartinZ wrote:

Da_Coynul wrote:
I was able to get to a console and I noticed that my /boot/grub was empty except for menu.lst. My initial attempt to repair grub was not successful and instead of spending more time on it, I just decided to restore my backup. After restoring I tried updating again while using systemd and had the same result. Then I restored one more time, changed to initscripts and rebooted successully after the update.
It seems like something wiped out GRUB. I'm not sure what it was or why it only happend when I tried to update while using systemd and not with initscripts.
@MartinZ, is your /boot/grub empty as well?
I use syslinux instead of grub, and it seems to be properly configured:
LABEL arch-ck
        MENU LABEL Arch Linux -ck
        LINUX /boot/vmlinuz-linux-ck
        APPEND root=/dev/sda3 elevator=noop ro fastboot logo.nologo quiet init=/bin/systemd vga=773
        INITRD /boot/initramfs-linux-ck.img

Now that I think about it, /boot/grub was probably empty because my boot partition wasn't mounted. I guess something is causing systemd to fail before mounting one or all of your partitions.

MartinZ · 2012-02-28 03:24:58

Da_Coynul wrote:

Now that I think about it, /boot/grub was probably empty because my boot partition wasn't mounted. I guess something is causing systemd to fail before mounting one or all of your partitions.

I agree.

I finally booted from an Arch iso and chrooted into my existing installation. Then I installed initscripts and understood why it wasn't installed: because of a conflict with systemd-sysvcompat, which I installed maybe 2 weeks ago. I was able to run my system with no trouble with initscripts instead of systemd. I haven't tested yet to boot again with systemd, will try and let you know. Anyway, I fear that anyone running systemd with sysvcompat may left his box unbootable as happened to me.

PS: as expected, trying to boot again with systemd (after another pacman -Syu) is also unable to boot. I think that the cause is systemd 43-3, but haven't downgraded yet to try.

Last edited by MartinZ (2012-02-28 03:49:52)

falconindy · 2012-02-28 04:01:43

systemd still foolishly relies on sulogin, which you removed by installing systemd-sysvcompat. That's why you couldn't login from the rescue prompt. I'm working on moving sulogin to util-linux, where it can be init-agnostic...

Last edited by falconindy (2012-02-28 04:01:58)

MartinZ · 2012-02-28 04:47:49

falconindy wrote:

systemd still foolishly relies on sulogin, which you removed by installing systemd-sysvcompat. That's why you couldn't login from the rescue prompt. I'm working on moving sulogin to util-linux, where it can be init-agnostic...

Thanks for the clarification. In fact, after reinstalling initscripts I can login without trouble. I downgraded systemd to 43-2, will let you know the results as soon as I reboot.

PS: Nope, 43-2 doesn't solve the problem.

Last edited by MartinZ (2012-02-28 14:13:42)

MartinZ · 2012-02-29 15:54:04

I finally fixed the problem downgrading every package but libreoffice from testing to core / extra. This is the list of packages:

[2012-02-29 12:40] upgraded mkinitcpio-busybox (1.19.4-1 -> 1.19.2-1)
[2012-02-29 12:40] upgraded util-linux (2.21-3 -> 2.20.1-2)
[2012-02-29 12:40] warning: /etc/mkinitcpio.conf installed as /etc/mkinitcpio.conf.pacnew
[2012-02-29 12:40] upgraded mkinitcpio (0.8.3-1 -> 0.8.2-3)
[2012-02-29 12:40] upgraded xcb-proto (1.7-2 -> 1.6-2)
[2012-02-29 12:40] upgraded libxcb (1.8-2 -> 1.7-2)
[2012-02-29 12:40] upgraded xf86-input-evdev (2.6.99.901-1 -> 2.6.0-4)
[2012-02-29 12:40] upgraded libglapi (8.0.1-1 -> 7.11.2-1)
[2012-02-29 12:40] upgraded khrplatform-devel (8.0.1-1 -> 7.11.2-1)
[2012-02-29 12:40] upgraded libgles (8.0.1-1 -> 7.11.2-1)
[2012-02-29 12:40] upgraded xorg-server-common (1.11.99.903-2 -> 1.11.4-1)
[2012-02-29 12:40] upgraded inputproto (2.1.99.6-1 -> 2.0.2-1)
[2012-02-29 12:40] upgraded libxi (1.5.99.3-1 -> 1.4.5-1)
[2012-02-29 12:40] upgraded xf86-input-synaptics (1.5.99-0.2 -> 1.5.0-1)
[2012-02-29 12:40] upgraded libpciaccess (0.12.902-1 -> 0.12.1-1)
[2012-02-29 12:40] upgraded xorg-server (1.11.99.903-2 -> 1.11.4-1)
[2012-02-29 12:40] upgraded libx11 (1.4.99.1-1 -> 1.4.4-1)
[2012-02-29 12:40] upgraded xorg-xinput (1.5.99.1-1 -> 1.5.3-2)
[2012-02-29 12:40] upgraded libegl (8.0.1-1 -> 7.11.2-1)
[2012-02-29 12:40] upgraded glproto (1.4.15-1 -> 1.4.14-1)
[2012-02-29 12:40] upgraded mesa (8.0.1-1 -> 7.11.2-1)

Watching the dependencies, I think that util-linux 2.21-3 may be the responsible.

tomegun · 2012-02-29 15:59:56

MartinZ wrote:

Watching the dependencies, I think that util-linux 2.21-3 may be the responsible.

If you can verify this (i.e., pure core/extra works, upgrading to testing/util-linux fails), please file a bug report (containing all the relevant info, as I have lost track of this thread).

frankieboy · 2012-02-29 16:54:07

Blµb wrote:

[SOLVED] (see the EDIT)
Just tried systemd today, and it hangs when mounting my filesystems (local-fs.service or whaterver)
The journal shows this funny nonsense:
Feb 26 22:37:23 blubpc systemd[1]: Job dev-mapper-isw_dejdfhfcag_Volume0p3.device/start timed out.
This makes no sense at all. That device is actually there, it's added in the initramfs already, and my root filesystem is mounted in the same volume-set, so, what is it doing there beside being silly?
EDIT:
Created .mount entries for the raid entries with Before=local-fs.mount and added them to /etc/systemd/system/local-fs.target.wants and now everything works fine

This didn't solve my problem, I've ocz revodrive ssd raid, with dmraid, my boot stops during fscking /home and /boot, but /root is mounted. Initscripts works fine, everything is mounted normally.

Frankieboy

Blµb · 2012-02-29 20:56:30

frankieboy wrote:

Blµb wrote:
(...)
EDIT:
Created .mount entries for the raid entries with Before=local-fs.mount and added them to /etc/systemd/system/local-fs.target.wants and now everything works fine
This didn't solve my problem, I've ocz revodrive ssd raid, with dmraid, my boot stops during fscking /home and /boot, but /root is mounted. Initscripts works fine, everything is mounted normally.
Frankieboy

I forgot to mention that I also added comment=systemd.automount to the fstab entries, try adding that as well?
Though my problem was not during fscking...
Can you post your systemd-journalctl output?

In any case, here's a .mount unit for reference just in case (it's nothing special though... so it probably won't help)

[Unit]
Description = /home
Before = local-fs.target
[Mount]
What = /dev/mapper/isw_dejdfhfcag_Volume0p6
Where = /home
Type = ext4

# ls -l /etc/systemd/system/local-fs.target.wants |grep home
lrwxrwxrwx 1 root root 13 Feb 27 12:35 home.mount -> ../home.mount

# grep isw /etc/fstab |grep home
/dev/mapper/isw_dejdfhfcag_Volume0p6    /home           ext4    defaults,comment=systemd.automount     0   1

I hope it helps

Last edited by Blµb (2012-02-29 20:57:21)

bwat47 · 2012-03-01 00:56:47

Since it seems systemd will be a dependency for gnome 3.4, Moving back to systemd yet again lol. I like the improvements since last I used it The text output during boot looks much nicer, and the systemd-sysvcompat is quite helpful

Everything seems to be working better than ever, even with initscripts removed.

frankieboy · 2012-03-01 05:38:10

Blµb wrote:

frankieboy wrote:
Blµb wrote:
(...)
EDIT:
Created .mount entries for the raid entries with Before=local-fs.mount and added them to /etc/systemd/system/local-fs.target.wants and now everything works fine
This didn't solve my problem, I've ocz revodrive ssd raid, with dmraid, my boot stops during fscking /home and /boot, but /root is mounted. Initscripts works fine, everything is mounted normally.
Frankieboy
I forgot to mention that I also added comment=systemd.automount to the fstab entries, try adding that as well?
Though my problem was not during fscking...
Can you post your systemd-journalctl output?
In any case, here's a .mount unit for reference just in case (it's nothing special though... so it probably won't help)
[Unit]
Description = /home
Before = local-fs.target
[Mount]
What = /dev/mapper/isw_dejdfhfcag_Volume0p6
Where = /home
Type = ext4
# ls -l /etc/systemd/system/local-fs.target.wants |grep home
lrwxrwxrwx 1 root root 13 Feb 27 12:35 home.mount -> ../home.mount
# grep isw /etc/fstab |grep home
/dev/mapper/isw_dejdfhfcag_Volume0p6    /home           ext4    defaults,comment=systemd.automount     0   1
I hope it helps

It's a half solution for me, because the boot process stops for a few minutes, than continues, and everything is mounted. It's interesting, that I have an md raid with 2 hdd besides the ssd, and it's mounted without time outs in systemd, only dmraid ssd has problems. Fstab has comment option already, but it doesn't help either, I'm using UUIDS in my fstab, if it matters. On systemd mailing list, they told me, that this is a distribution specific problem, I've written in this forum earlier, but nobody gave me a solution. I attach the systemd-journal output:

http://pastebin.com/NccT8NGw

Thank you for your help

Frankieboy

unikum · 2012-03-01 08:41:38

How can I set the dhcpcd daemon at boot?
I try

$ sudo systemctl enable dhcpcd@eth0.service
Failed to issue method call: No such file or directory

I can start it manually

$ sudo systemctl start dhcpcd@eth0.service
$ sudo systemctl status dhcpcd@eth0.service
dhcpcd@eth0.service - dhcpcd on eth0
          Loaded: loaded (/lib/systemd/system/dhcpcd@.service; disabled)
          Active: active (running) since Thu, 01 Mar 2012 15:40:29 +0700; 1s ago
         Process: 1684 ExecStop=/sbin/dhcpcd -k %I (code=exited, status=0/SUCCESS)
         Process: 1700 ExecStart=/sbin/dhcpcd -q -w %I (code=exited, status=0/SUCCESS)
        Main PID: 1727 (dhcpcd)
          CGroup: name=systemd:/system/dhcpcd@.service/eth0
                  └ 1727 /sbin/dhcpcd -q -w eth0

Mar 01 15:40:24 unikum-desktop dhcpcd[1700]: version 5.5.4 starting
Mar 01 15:40:24 unikum-desktop dhcpcd[1700]: eth0: sending IPv6 Router Solicitation
Mar 01 15:40:24 unikum-desktop dhcpcd[1700]: eth0: broadcasting for a lease
Mar 01 15:40:24 unikum-desktop dhcpcd[1700]: eth0: offered 192.168.1.2 from 192.168.1.1
Mar 01 15:40:24 unikum-desktop dhcpcd[1700]: eth0: acknowledged 192.168.1.2 from 192.168.1.1
Mar 01 15:40:24 unikum-desktop dhcpcd[1700]: eth0: checking for 192.168.1.2
Mar 01 15:40:28 unikum-desktop dhcpcd[1700]: eth0: sending IPv6 Router Solicitation
Mar 01 15:40:29 unikum-desktop dhcpcd[1700]: eth0: leased 192.168.1.2 for 43200 seconds

Where I can find mysqld, lighttpd and cpufreq native service files?

UPD:
About ligghtpd and mysql I found solution in the Gentoo wiki:
http://en.gentoo-wiki.com/wiki/Systemd#lighttpd
http://en.gentoo-wiki.com/wiki/Systemd#MySQL
For cpufreq I made following service file:

[Unit]
Description=CPU frequency scaling daemon

[Service]
EnvironmentFile=/etc/conf.d/cpufreq
ExecStart=/usr/bin/cpufreq-set -r -g $governor

[Install]
WantedBy=multi-user.target

It's works for me. May be worth adding them in systemd-arch-units package?

UPD2:
Issue with dhcpcd solved:

# ln -sf /lib/systemd/system/dhcpcd@.service /etc/systemd/system//multi-user.target.wants/dhcpcd@eth0.service

Last edited by unikum (2012-03-01 11:19:44)

Blµb · 2012-03-01 10:16:53

frankieboy wrote:

(...)
http://pastebin.com/NccT8NGw
Thank you for your help
Frankieboy

I just hate how systemd-journalctl cuts off lines even when piping to a file, could you use `systemd-journalctl -a` please?

Earnestly · 2012-03-01 11:18:45

Trilby wrote:

...

Ahoy Trilby,
Perhaps this has already been linked but I've recently found this tidbit on youtube which might of interest. It's the FOSDEM talk by Lennart on systemd. (http://youtu.be/TyMLi8QF6sw)

It's still a tad disconcerting to hear systemd will eventually try accomplish many other jobs which aren't related to init (e.g. session management). (This is the Unix philosophy: Write programs that do one thing and do it well.)

However, time will tell. Looking forward to see what systemd can bring. The prospect of a unified init system across all Linux distributions is a very tantalising thought.

That, and I'm a sucker when it comes to standard(isation)s

Last edited by Earnestly (2012-03-01 11:32:21)

Blµb · 2012-03-01 12:36:07

I'm having a new problem now. @frankieboy, this might interest you specifically.
I have finally managed to get my IMSM raid working with MDADM - the old "mdadm" initramfs target still segfaults, but the udev variant "mdadm_udev" works, since it uses mdadm rather than mdassemble.
However I have to add BINARIES="/sbin/mdmon" for some reason to have it start mdmon, because without it the raid stays in read-only mode (and no, sadly not just auto-read-only).

However, it now restarts resyncing the raid at every boot. The interesting part is that when I manually shutdown (ie. stopping all services, remounting all raid partitions read-only, then perform a `poweroff -fp`, the resync doesn't happen (well, not sure about that exactly, but at least it didn't restart the resync process, which otherwise it does).
What's even stranger is that when I shutdown after it's finished, and boot into windows7, this windows does not start a resync... (possibly mdmon recognizes some flags in linux partitions which windows doesn't know/care about?)

Anyway, I'll have to look into the shutdown sequence and maybe add some additional targets to properly put the raid into readonly before finally shutting down...

IMSM raids really suck
(but the only option on this hardware if I also want windows7 to be able to use it properly)

tomegun · 2012-03-01 12:39:58

Blµb wrote:

However, it now restarts resyncing the raid at every boot. The interesting part is that when I manually shutdown (ie. stopping all services, remounting all raid partitions read-only, then perform a `poweroff -fp`, the resync doesn't happen (well, not sure about that exactly, but at least it didn't restart the resync process, which otherwise it does).
What's even stranger is that when I shutdown after it's finished, and boot into windows7, this windows does not start a resync... (possibly mdmon recognizes some flags in linux partitions which windows doesn't know/care about?)
Anyway, I'll have to look into the shutdown sequence and maybe add some additional targets to properly put the raid into readonly before finally shutting down...

If your root partition is on raid, then you might need to add something to /run/initramfs/shutdown in order to tear it down properly. This can not be done properly from /etc/rc.shutdown.

Blµb · 2012-03-01 16:27:49

I don't use rc.shutdown (or at least I don't have initscripts-systemd installed)
(And my /run/initramfs is empty, will systemd use that by default if I just create shutdown in it?)

tomegun · 2012-03-01 16:29:47

Blµb wrote:

I don't use rc.shutdown (or at least I don't have initscripts-systemd installed)
(And my /run/initramfs is empty, will systemd use that by default if I just create shutdown in it?)

Replace rc.shutdown with systemd-shutdown :-) You have to enable the "shutdown" hook in mkinitcpio.conf to get /run/initramfs/shutdown. At the moment it simply unmounts whatever systemd/initscripts did not manage to unmount, but perhaps it also needs to tear down some raid stuff.

Arch Linux

#1226 2012-02-27 18:27:59

Re: systemd: Yet Another Init Replacement

#1227 2012-02-27 18:35:58

Re: systemd: Yet Another Init Replacement

#1228 2012-02-27 19:35:39

Re: systemd: Yet Another Init Replacement

#1229 2012-02-27 19:47:27

Re: systemd: Yet Another Init Replacement

#1230 2012-02-27 21:06:50

Re: systemd: Yet Another Init Replacement

#1231 2012-02-27 21:17:44

Re: systemd: Yet Another Init Replacement

#1232 2012-02-27 21:35:30

Re: systemd: Yet Another Init Replacement

#1233 2012-02-27 22:05:23

Re: systemd: Yet Another Init Replacement

#1234 2012-02-27 23:07:07

Re: systemd: Yet Another Init Replacement

#1235 2012-02-28 03:24:58

Re: systemd: Yet Another Init Replacement

#1236 2012-02-28 04:01:43

Re: systemd: Yet Another Init Replacement

#1237 2012-02-28 04:47:49

Re: systemd: Yet Another Init Replacement

#1238 2012-02-29 15:54:04

Re: systemd: Yet Another Init Replacement

#1239 2012-02-29 15:59:56

Re: systemd: Yet Another Init Replacement

#1240 2012-02-29 16:54:07

Re: systemd: Yet Another Init Replacement

#1241 2012-02-29 20:56:30

Re: systemd: Yet Another Init Replacement

#1242 2012-03-01 00:56:47

Re: systemd: Yet Another Init Replacement

#1243 2012-03-01 05:38:10

Re: systemd: Yet Another Init Replacement

#1244 2012-03-01 08:41:38

Re: systemd: Yet Another Init Replacement

#1245 2012-03-01 10:16:53

Re: systemd: Yet Another Init Replacement

#1246 2012-03-01 11:18:45

Re: systemd: Yet Another Init Replacement

#1247 2012-03-01 12:36:07

Re: systemd: Yet Another Init Replacement

#1248 2012-03-01 12:39:58

Re: systemd: Yet Another Init Replacement

#1249 2012-03-01 16:27:49

Re: systemd: Yet Another Init Replacement

#1250 2012-03-01 16:29:47

Re: systemd: Yet Another Init Replacement

Board footer