[Solved] Boot Error - systemd[1]: Failed to start Journal Service

kungsu · 2012-10-19 17:59:30

Hi,
I searched quite a lot, but could not find anything related to this problem. I didn't want a 3rd day going into trial-and-error method of getting around this. So, here goes...

I have an Arch installation running very well on my HDD. I recently bought an SSD and installed Windows 7 on it. Partitioned the drives correctly and installed Arch to an SSD partition from the existing HDD Arch using this guide. I setup the configuration files in accordance with the Native Systemd Configuration Files. I included the init=/usr/lib/systemd/systemd to my kernel command line and did a reboot. However, all I'm able to see after this are the kernel-log-like-messages repeated over and over again, eventually not booting at all.

... snip a lot of similar lines
[ 79.309480] systemd[1]: Failed to start Journal Service.
[ 79.311736] systemd[1]: Failed to start D-Bus System Message Bus
... snip a lot of similar lines

Removing the init=/usr/lib/systemd/systemd works just fine, and thanks to my SSD, It is so fast that I see my login console directly after the GRUB menu without the on-screen initialization logs. Having solved quite a few hangups with udev/mkinitcpio in the past, I am not inexperienced with Arch or Linux in general, but I seem to have a problem getting systemd sink into my head.

Root is a single partition and everything is on it. I've mounted my HDD partitions in /media. Its still a fresh system. Not many packages installed. Do let me know what other information is relevant to understand/solve it.

Last edited by kungsu (2012-10-28 10:24:06)

cfr · 2012-10-20 00:48:27

EDIT: remove misinformation.

How are you starting dbus? Have you set up a "pure" systemd configuration or a "mixed" one? (See the wiki page on systemd if this does not sound familiar.)

Whichever you are using, is dbus configured to start? (Either in rc.conf or using a service file.)

I'm guessing you are using a "mixed" system. If so, make sure dbus is in the DAEMONS line of rc.conf.

The service should be started by default anyway in the case of systemd but you could try enabling it manually just in case.

Of course check that dbus (and systemd - but I think it must be for init scrpts to work) is installed correctly.

EDIT: Is it literally printing "<timestamp>"? Or does it print the date/time? If not, maybe you need to take care of the system clock etc.

Last edited by cfr (2012-10-20 01:07:14)

kungsu · 2012-10-20 02:48:13

It is a fresh install and i'm trying to setup pure systemd before customizing further, ofcourse, by following the Systemd wiki.
Dbus is started normally in rc.conf :

DAEMONS=(dbus syslog-ng @net-profiles !network @crond @alsa)

Sorry for misleading by putting it as [<timestamp>]. It does print time values and I've updated the post to reflect that.

cfr · 2012-10-20 21:53:32

You can't both be using a "pure systemd" and using that line in rc.conf. A "pure" setup no longer uses rc.conf at all.

Note that using syslog-ng in conjunction with journald requires a specific set up. I don't think this should prevent journald running or boot but it will interfere with syslog-ng.

Does net-profiles provide a service file? I don't use this so it is not installed but I can't find an appropriate service for default systemd. So that will fall back to using the rc.d daemon. Make sure you have not also enabled any conflicting service for systemd.

Which services, if any, have you enabled native service files for? You will probably get better results if you switch what you can to the native services.

WonderWoofy · 2012-10-20 22:00:51

I believe net-profiles is a netcfg thing, and the relevent service is netcfg@.service or netcfg.service

kungsu · 2012-10-21 08:33:32

I know that a pure systemd does not need rc.conf. I am simply following the Systemd wiki instructions, from the "A mixed systemd/sysvinit/initscripts installation" section. Since the setup for syslog and netcfg as per systemd requirement comes at a later stage, I've not altered the DAEMONS array yet. Nor have I added any services through the systemctl enable service_name.service command (where service_name = syslog/syslog-ng/netcfg/netcfg@/net-auto-wired)

Move away from the deprecated initscripts configuration formats (there should be warnings at boot) to the native systemd configuration files, and reboot to verify that this works as expected with initscripts.

- Done.

Install systemd from the official repositories.

- Done.

Add init=/usr/lib/systemd/systemd to the kernel parameters in your bootloader.

- Done

Reboot.

- Getting the above 2 logs continuously after this.

I'm deliberately being verbose to outline everything that I've done.

cfr · 2012-10-21 21:12:56

Have you tried any of the strategies suggested here? (This page is linked from the wiki via a Fedora page.)

kungsu · 2012-10-23 16:44:23

I tried the strategies suggested in the link and could get something working only when I added "emergency" to the kernel command line.

I didn't know what to do and checked /var/log directory for any hints. This link is a section of /var/log/everything.log. My previous boot attempts showed something similar.

I believe that journald.service is not started during boot. Once booted into emergency, jounalctl command is not giving any output. Neither does journalctl -b.

Guybrush · 2012-10-23 17:29:13

Just curious can you post the result for
ls -l /var

?

kungsu · 2012-10-23 17:48:09

I wonder why, but here it is:

[root@archbox /]# ls -l /var
total 40
drwxr-xr-x  5 root root  4096 Oct  7 23:44 cache
drwxr-xr-x  3 root root  4096 Oct 15 21:56 db
drwxr-xr-x  2 root root  4096 Oct  4 02:02 empty
drwxrwxr-x  2 root games 4096 Oct  4 02:02 games
drwxr-xr-x 12 root root  4096 Oct 15 22:54 lib
drwxr-xr-x  2 root root  4096 Oct  4 02:02 local
lrwxrwxrwx  1 root root    11 Oct  4 02:02 lock -> ../run/lock
drwxr-xr-x  3 root root  4096 Oct 23 21:45 log
lrwxrwxrwx  1 root root    10 Oct  4 02:02 mail -> spool/mail
drwxr-xr-x  2 root root  4096 Oct  4 02:02 opt
lrwxrwxrwx  1 root root     6 Oct  4 02:02 run -> ../run
drwxr-xr-x  5 root root  4096 Oct  7 23:26 spool
drwxrwxrwt  2 root root  4096 Oct 19 14:22 tmp

Guybrush · 2012-10-23 17:50:37

Looks fine, i had also a failed journal service after pretty much a fresh install and my problem is this one: https://bbs.archlinux.org/viewtopic.php?id=151285
When i symlink /var/run everything works again.

cfr · 2012-10-25 01:56:25

What happens if you try to start the journal service yourself from the emergency shell?

kungsu · 2012-10-25 16:50:36

[root@archbox /]# systemctl enable systemd-journald.service
The unit files have no [Install] section. They are not meant to be enabled using systemctl.

kungsu · 2012-10-25 18:33:14

An update on this.
I did a pacman -Syu and it updated systemd 194-1 to 194-4 and linux from 3.6.2-1 to 3.6.3-1 among others. The DAEMONS array in /etc/rc.conf remains the same as described above.

But now, I get slightly different error. I am getting only the 2nd error message. the error about the journal is not present anymore. Besides, after about 5 seconds no more logs are shown on the screen and nothing happens till i press ctrl+alt+del to reboot.

...snip a lot of similar lines
[ 5.131376] systemd[1]: Failed to start D-Bus System Message Bus.
<no further logs on screen>

checking the everything.log, these lines seemed fishy and I'm unable to pin point what is happening.

Oct 25 22:39:41 localhost [ 4.341857] systemd[1]: Starting D-Bus System Message Bus Socket.
Oct 25 22:39:41 localhost [ 4.341950] systemd[1]: Listening on D-Bus System Message Bus Socket.
Oct 25 22:39:41 localhost [ 4.341986] systemd[1]: Starting Sockets.
Oct 25 22:39:41 localhost [ 4.342012] systemd[1]: Reached target Sockets.
Oct 25 22:39:41 localhost [ 4.342044] systemd[1]: Starting Daily Cleanup of Temporary Directories.
Oct 25 22:39:41 localhost [ 4.342081] systemd[1]: Started Daily Cleanup of Temporary Directories.
Oct 25 22:39:41 localhost [ 4.342107] systemd[1]: Starting Basic System.
Oct 25 22:39:41 localhost [ 4.342132] systemd[1]: Reached target Basic System.
Oct 25 22:39:41 localhost [ 4.342161] systemd[1]: Starting Periodic Command Scheduler...
Oct 25 22:39:41 localhost [ 4.371813] systemd[1]: Started Periodic Command Scheduler.
Oct 25 22:39:41 localhost [ 4.371898] systemd[1]: Starting Netcfg multi-profile daemon...
Oct 25 22:39:41 localhost [ 4.408427] systemd[1]: Starting System Logger Daemon...
Oct 25 22:39:41 localhost [ 4.431719] systemd[1]: Started System Logger Daemon.
Oct 25 22:39:41 localhost [ 4.431807] systemd[1]: Starting Login Service...
Oct 25 22:39:41 localhost [ 4.441642] systemd[1]: Starting Permit User Sessions...
Oct 25 22:39:41 localhost [ 4.472692] systemd[1]: Starting /etc/rc.local Compatibility...
Oct 25 22:39:41 localhost [ 4.484890] systemd[1]: Starting D-Bus System Message Bus...
Oct 25 22:39:41 localhost [ 4.515783] systemd[1]: Started D-Bus System Message Bus.
Oct 25 22:39:41 localhost [ 4.515923] systemd[1]: Starting LEGACY unit for "alsa" rc script...
Oct 25 22:39:41 localhost kernel: [ 4.522362] atl1c 0000:02:00.0: irq 41 for MSI/MSI-X
Oct 25 22:39:41 localhost dbus-daemon[262]: Failed to start message bus: Could not get UID and GID for username "dbus"
Oct 25 22:39:41 localhost echo[269]: WARNING: A legacy rc script is being used. Consider removing this from your DAEMONS array and enabling a native systemd service instead.

Searching the forum, i found this post relevant. But since it is more than 2 years old, I am wondering if the bug resurfaced. Moreover, I'm not sure if it is related to my original problem.

cfr · 2012-10-25 22:56:09

Have you tried reinstalling dbus?

kungsu · 2012-10-26 03:36:32

Yes I did reinstall dbus.

ataraxia · 2012-10-26 14:38:55

The dbus user and group are created by dbus-core, not dbus, so you might try reinstalling dbus-core as well.

kungsu · 2012-10-26 16:46:27

I did reinstall dbus-core along with all dependencies of both dbus and dbus-core.
Its been more than a week since I'm stuck here. I'll give it this weekend before going for a full wipe plus reinstall. As much fun as it is to tinker with the low level stuff, I want to experience Arch on the SSD too.

cfr · 2012-10-27 02:01:26

Do the dbus user and group exist? That is, did they get created correctly?

kungsu · 2012-10-27 07:53:17

/etc/group:

dbus:x:81:

/etc/passwd:

dbus:x:81:81:System message bus:/:/bin/false

Antoine · 2012-10-27 13:02:10

Sorry if I seem obtuse here. Have you tried removing dbus from your rc.conf daemons array, and rebooting? You don't need to enable it via systemctl.

When I moved to systemd, I went through the daemons one-by-one, removing them, enabling the corresponding service (if it needed that), rebooting, and it all went quite well.

kungsu · 2012-10-27 15:05:25

Antoine, thank you for the tip, and please dont use "sorry" when you are trying to be of some help. :-)
As I've said earlier, I'm still in the first step of moving to systemd and have not come to the step of removing the daemons. I've tried 3 things : without having any service enabled, enabling a few (one by one, as well as a few in conjunction) which I thought would be required, and disabling them all when none seemed to work. But still, I'm left without success.

Antoine · 2012-10-27 19:15:56

kungsu wrote:

As I've said earlier, I'm still in the first step of moving to systemd and have not come to the step of removing the daemons.

You're getting an error message asking you to do just that, though.

You should try removing dbus from the rc.conf daemons array and rebooting. You could also remove syslog-ng, while you're at it, and enable the equivalent service (which does need enabling). You can't really hurt anything. The thing is, systemd boots so fast, it doesn't take a lot of time to do this.

There's a list of daemons and their equivalent services here:
https://wiki.archlinux.org/index.php/Daemon/List

cfr · 2012-10-28 02:59:27

I thought that systemd was supposed to substitute the service file automatically provided the daemon and service had the same name? (Which is the case for dbus.)

Which doesn't mean that removing the daemon from the rc.conf line isn't a good idea - just I'm pretty sure it should not be necessary. (I'm certain that I got systemd to boot before removing anything at all from that line because I initially set up distinct grub entries for systemd and initscripts so I could boot either.)

kungsu · 2012-10-28 09:16:14

Okay, this is it.
I could not figure out what was the cause for this problem despite enabling/disabling daemons/services through rc.conf/systemctl.
I tried to understand what was happening, but i'm running short of time for research here. I'm nuking my current setup as I am typing this now.
I removed all packages save grub and am installing Arch afresh in a chroot environment. Since Arch has officially moved to systemd completely, I could as well start off with a new pure systemd setup.
Lets see how it goes.

Arch Linux

#1 2012-10-19 17:59:30

[Solved] Boot Error - systemd[1]: Failed to start Journal Service

#2 2012-10-20 00:48:27

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#3 2012-10-20 02:48:13

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#4 2012-10-20 21:53:32

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#5 2012-10-20 22:00:51

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#6 2012-10-21 08:33:32

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#7 2012-10-21 21:12:56

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#8 2012-10-23 16:44:23

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#9 2012-10-23 17:29:13

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#10 2012-10-23 17:48:09

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#11 2012-10-23 17:50:37

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#12 2012-10-25 01:56:25

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#13 2012-10-25 16:50:36

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#14 2012-10-25 18:33:14

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#15 2012-10-25 22:56:09

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#16 2012-10-26 03:36:32

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#17 2012-10-26 14:38:55

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#18 2012-10-26 16:46:27

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#19 2012-10-27 02:01:26

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#20 2012-10-27 07:53:17

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#21 2012-10-27 13:02:10

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#22 2012-10-27 15:05:25

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#23 2012-10-27 19:15:56

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#24 2012-10-28 02:59:27

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

#25 2012-10-28 09:16:14

Re: [Solved] Boot Error - systemd[1]: Failed to start Journal Service

Board footer