[SOLVED] lightdm won't work from systemd but will from terminal

Malvineous · 2020-04-13 04:31:46

I'm setting up a new PC and I have gotten lightdm working when I log in (via SSH) as root and run "lightdm" from the command line. This loads xorg and brings up the GTK greeter as it should.

However if I run "systemctl start lightdm" (to simulate what happens when the system boots normally) then it loads xorg, but the greeter never appears, it just sits at a black screen. I can't work out why - does anyone have any ideas?

xorg is fine, because when this black screen is visible, via the SSH connection I can run

XAUTHORITY=/run/lightdm/root/:0 DISPLAY=:0 urxvt

and a terminal window appears on the screen, just the lightdm greeter for some reason won't appear.

When I start lightdm from the command line I can press Ctrl+C to terminate it and it exits immediately, but running "systemctl stop lightdm" takes about 10 seconds to stop it. I am suspecting the lightdm-gtk-greeter process has deadlocked but I can't work out why.

Looking in /var/log/lightdm/seat0-greeter.log it just says:

** Message: 14:12:00.950: Starting lightdm-gtk-greeter 2.0.7 (Dec 17 2019, 19:36:20)
** Message: 14:12:00.952: [Configuration] Reading file: /etc/lightdm/lightdm-gtk-greeter.conf

If I run lightdm from the command line where it works, then this same log file instead says:

** (process:1764): WARNING **: 14:25:49.288: Failed to open CK session: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.ConsoleKit was not provided by any .service files
** Message: 14:25:49.304: Starting lightdm-gtk-greeter 2.0.7 (Dec 17 2019, 19:36:20)
** Message: 14:25:49.306: [Configuration] Reading file: /etc/lightdm/lightdm-gtk-greeter.conf
**
GLib:ERROR:../glib/glib/gspawn.c:1362:do_exec: assertion failed: (read_null != -1)
**
GLib:ERROR:../glib/glib/gspawn.c:1362:do_exec: assertion failed: (read_null != -1)
**
GLib:ERROR:../glib/glib/gspawn.c:1362:do_exec: assertion failed: (read_null != -1)

(lightdm-gtk-greeter:1771): Gtk-WARNING **: 14:25:50.412: Drawing a gadget with negative dimensions. Did you forget to allocate a size? (node menubar owner GreeterMenuBar)

It doesn't really tell me anything beyond the greeter apparently getting stuck at some unknown point when lightdm is launched via systemd. It's ironic that the log file with all the errors in it is the one that actually works!

Looking at the lightdm log file, when it works it shows VT7 activated and then the greeter connecting, and when it's broken it still activates VT7 but never shows the greeter having connected.

I'm kind of stuck now as I'm not sure what else to look at. Can anyone else offer any suggestions?

Last edited by Malvineous (2020-06-04 14:26:28)

Malvineous · 2020-04-13 05:55:39

I've made a little progress. I renamed /usr/bin/lightdm-gtk-greeter and put a shell script in its place, which called strace on the original binary. This allowed full system call traces to end up in /var/log/lightdm/seat0-greeter.log.

From this, I could see that when it works, lightdm-gtk-greeter uses the system dbus socket:

connect(7, {sa_family=AF_UNIX, sun_path="/var/run/dbus/system_bus_socket"}, 110) = 0
sendto(7, "AUTH EXTERNAL 363230\r\n", 22, MSG_NOSIGNAL, NULL, 0) = 22
recvfrom(7, "OK 7f94aa8da629128be07028bc5e93e"..., 4096, 0, NULL, NULL) = 37

However the broken one (launched via systemd) instead appears to use a user-specific socket instead:

connect(6, {sa_family=AF_UNIX, sun_path="/run/user/620/bus"}, 19) = 0
sendto(6, "AUTH EXTERNAL 363230\r\n", 22, MSG_NOSIGNAL, NULL, 0) = 22
poll([{fd=6, events=POLLIN}], 1, -1

That last poll() call never returns, which seems to be why it freezes. fuser tells me that the 620/bus file is owned by the "systemd --user" command (running as the lightdm user) so it appears that *something* is using the socket, it's just not replying to AUTH messages.

Weirdly if I use "su" to change to the lightdm user, I can't run lightdm as it tells me it has to run as root. Yet when when I run it as root, using the same command systemd uses (as given in /usr/lib/systemd/system/lightdm.service), it works fine. I ran it as root and checked the /run/user/ folder but there is no 620 folder for the lightdm user when I run it myself. Only running via systemd seems to create /run/user/620/ owned by the lightdm user.

I tried using socat to connect to the 620/bus but no matter what I type nothing happens. I tried connecting to 1000/bus instead (the user I am currently logged in as via SSH) and no matter what I type I immediately get disconnected. So if 1000/bus is disconnecting me but 620/bus is not responding at all, that seems to suggest the "systemd --user" process for the lightdm user is somehow broken?

Any ideas why systemd is creating this /run/user/620/bus socket, but then not responding on it when lightdm tries to use it?

andym · 2020-04-13 07:26:54

You might find some answers on this help page, and especially the the testing section.

seth · 2020-04-13 07:57:22

Did you try https://wiki.archlinux.org/index.php/Li … TTY_output

Malvineous · 2020-04-13 08:48:48

Thanks for the replies!

@andym I've been through the wiki page but nothing there shed any light on the problem, and I've been using the debug mode as a shortcut but all it does is write to stdout what you see in the log files anyway, so it doesn't give you any additional information. I tried --test-mode but it doesn't really help because as far as lightdm is concerned there is nothing wrong. It's the greeter itself that is having problems.

@seth That graphical check didn't make a difference, which I suppose makes sense because the graphics drivers load at boot, and here I am many hours later starting and stopping lightdm.

Any other ideas I can try?

Head_on_a_Stick · 2020-04-13 13:47:02

The LightDM service file conflicts with getty@tty1.service so that might cause problems if you try to start it from TTY1.

Have you actually tried enabling lightdm.service and then rebooting? You may also have to use

# systemctl set-default graphical.target

Also check that /usr/lib/systemd/system/lightdm.service is symlinked to /etc/systemd/system/display-manager.service before rebooting.

Malvineous · 2020-04-13 22:30:25

Yes when I reboot I see exactly the same behaviour as running "systemctl start lightdm", in that X starts but the screen is blank, and the logs say the greeter never connected to lightdm, despite the greeter process running. Killing the lightdm-gtk-greeter process also does nothing, I have to use kill -9 to terminate it (which triggers lightdm to exit and reload, but after reloading the behaviour is the same).

I haven't tried to start it from tty1 because it either runs at boot or I'm launching it via SSH, and there's no login prompt on tty1 so there shouldn't be a conflict there.

Head_on_a_Stick · 2020-04-14 11:33:37

Is the default set to graphical.target and is /etc/systemd/system/display-manager.service a symlink to /usr/lib/systemd/system/lightdm.service?

Have you tried different greeters? Does automatic login get you to the desktop with lightdm.service enabled?

Malvineous · 2020-04-14 12:04:17

Thanks for the suggestions.

The display-manager.service is a symlink to lightdm with the paths you have mentioned. I don't have a default target set, per the wiki advice I have run "systemctl enable lightdm.service" and that is enough to launch lightdm (and by dependency X11) during boot. I believe this part is working fine, because 1) when booting the system it loads X11 and lightdm runs, it's just the greeter spawned by lightdm that doesn't work, and 2) setting up this PC was done largely by rsync'ing from another working machine, with only network and fstab configurations changed so far after the rsync. So the same config works on the source machine, but not on this target one.

I haven't tried different greeters as they all seem to have many large dependencies, Is there a lightweight one you can recommend for testing?

I tried configuring automatic login (based on a working lightdm autologin on another PC) but this did not work. It seems lightdm loads the greeter, and the greeter handles the autologin. In my case lightdm loads the greeter successfully but the greeter apparently locks before it attempts the autologin.

Head_on_a_Stick · 2020-04-14 12:11:41

Malvineous wrote:

I haven't tried different greeters as they all seem to have many large dependencies, Is there a lightweight one you can recommend for testing?

No, sorry, I don't use LightDM. No display manager at all has always been my preference for simple systems.

Malvineous wrote:

It seems lightdm loads the greeter, and the greeter handles the autologin.

Not according to the ArchWiki:

LightDM page wrote:

It is possible to use LightDM without a greeter, but only if an automatic login is configured

https://wiki.archlinux.org/index.php/LightDM#Greeter

So perhaps your problem is with LightDM itself, probably best to post the entire log file from a failed session (with autologin enabled).

Malvineous · 2020-04-14 12:33:16

Sorry my bad, after looking at the log file with autologin enabled I realised I missed a step and forgot to add my user to the autologin group. lightdm was telling me the autologin failed and it was falling back to the greeter which is why I saw the same behaviour. Now I have configured autologin correctly, lightdm does indeed log in correctly and the user session starts successfully.

Unfortunately I don't think it's possible for me to operate without the greeter, as normally I have two sessions on the machine - a 'home' one on VT7 and a 'work' one on VT8, so that I can use Alt+Ctrl+F7 and Alt+Ctrl+F8 to quickly switch between home and work environments. Normally I log in on the boot up greeter with my home user then run 'dm-tool switch-to-greeter' to bring up another session on the next VT where I can log in as the work user. I don't think it's possible to set up two sessions like this with autologin, but let me know if I'm wrong.

Head_on_a_Stick · 2020-04-14 12:57:12

Malvineous wrote:

I don't think it's possible to set up two sessions like this with autologin, but let me know if I'm wrong.

Not sure about LightDM but it's certainly possible without a display manager:

https://bbs.archlinux.org/viewtopic.php … 8#p1887508

Malvineous · 2020-06-04 14:26:12

Just marking this thread solved. After a long series of debugging, going from LightDM via GTK via an accessibility library to dbus then systemd and finally udev, the problem was discovered by @seth in this thread.

Turns out an update to systemd-udevd caused one of my custom rules to start matching too many devices, and it applied the wrong permissions to /dev/null, /dev/urandom, and a whole bunch of other device nodes. This in turn meant that dbus-daemon couldn't open /dev/null so refused to load, meaning systemd couldn't set up a user dbus session, which meant the accessibility library froze trying to connect to dbus, which meant GTK never finished initialising, which meant LightDM couldn't start.

Adjusting the udev rule fixed everything and now LightDM works fine again. It also explains why running it as root worked, because as root the bogus permissions in /dev weren't an issue.

I sure learned a lot about debugging through this process though!

Arch Linux

#1 2020-04-13 04:31:46

[SOLVED] lightdm won't work from systemd but will from terminal

#2 2020-04-13 05:55:39

Re: [SOLVED] lightdm won't work from systemd but will from terminal

#3 2020-04-13 07:26:54

Re: [SOLVED] lightdm won't work from systemd but will from terminal

#4 2020-04-13 07:57:22

Re: [SOLVED] lightdm won't work from systemd but will from terminal

#5 2020-04-13 08:48:48

Re: [SOLVED] lightdm won't work from systemd but will from terminal

#6 2020-04-13 13:47:02

Re: [SOLVED] lightdm won't work from systemd but will from terminal

#7 2020-04-13 22:30:25

Re: [SOLVED] lightdm won't work from systemd but will from terminal

#8 2020-04-14 11:33:37

Re: [SOLVED] lightdm won't work from systemd but will from terminal

#9 2020-04-14 12:04:17

Re: [SOLVED] lightdm won't work from systemd but will from terminal

#10 2020-04-14 12:11:41

Re: [SOLVED] lightdm won't work from systemd but will from terminal

#11 2020-04-14 12:33:16

Re: [SOLVED] lightdm won't work from systemd but will from terminal

#12 2020-04-14 12:57:12

Re: [SOLVED] lightdm won't work from systemd but will from terminal

#13 2020-06-04 14:26:12

Re: [SOLVED] lightdm won't work from systemd but will from terminal

Board footer