[SOLVED] Systemd hangs on boot

str1ngd · 2023-07-16 17:01:05

I think this started happening after I made my own service which I then deleted before disabling in systemctl, every time I start my system now it hangs on tty1 after starting a couple of services on systemd. I have to manually go to tty2, start network manager and my graphical environment to use my computer. I have no idea how to debug it though

Let me know if you need anymore information

Last edited by str1ngd (2023-07-16 22:37:41)

seth · 2023-07-16 19:21:40

I'd start by looking a the system journal.

str1ngd · 2023-07-16 20:36:09

seth wrote:

I'd start by looking a the system journal.

I tried skimming through it though I am not really sure what to look at. Here is my journalctl from boot till I switch to tty2: https://bin.disroot.org/?be38c4108cae89 … YAaaod6aPe

Last edited by str1ngd (2023-07-16 20:36:29)

ewaller · 2023-07-16 21:07:15

when you log into tty2, look through /etc/systemd/system and its sub-directories looking for references to your service and remove them

seth · 2023-07-16 21:20:07

jul 17 00:24:46 thinch kernel: Linux version 6.4.3-arch1-2 (linux@archlinux) (gcc (GCC) 13.1.1 20230714, GNU ld (GNU Binutils) 2.40.0) #1 SMP PREEMPT_DYNAMIC Sat, 15 Jul 2023 19:25:49 +0000
…
jul 17 00:24:47 thinch systemd[1]: Reached target Multi-User System.
jul 17 00:24:47 thinch systemd[1]: Reached target Graphical Interface.
jul 17 00:24:47 thinch systemd[1]: Startup finished in 6.258s (firmware) + 1.720s (loader) + 2.812s (kernel) + 1.666s (userspace) = 12.458s.

Please post the output of

find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f

Also what was that "own service" and how do you "start … my graphical environment"?

str1ngd · 2023-07-16 21:20:53

ewaller wrote:

when you log into tty2, look through /etc/systemd/system and its sub-directories looking for references to your service and remove them

There was only one file in the system directory, (no sub directories, or hidden files etc) which was the file I used to enable one of my services, though I had made it completely empty. I deleted it and restarted however sadly no progress was made

Last edited by str1ngd (2023-07-16 21:21:23)

str1ngd · 2023-07-16 21:26:41

seth wrote:

jul 17 00:24:46 thinch kernel: Linux version 6.4.3-arch1-2 (linux@archlinux) (gcc (GCC) 13.1.1 20230714, GNU ld (GNU Binutils) 2.40.0) #1 SMP PREEMPT_DYNAMIC Sat, 15 Jul 2023 19:25:49 +0000
…
jul 17 00:24:47 thinch systemd[1]: Reached target Multi-User System.
jul 17 00:24:47 thinch systemd[1]: Reached target Graphical Interface.
jul 17 00:24:47 thinch systemd[1]: Startup finished in 6.258s (firmware) + 1.720s (loader) + 2.812s (kernel) + 1.666s (userspace) = 12.458s.

Please post the output of

find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f

Also what was that "own service" and how do you "start … my graphical environment"?

$ find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f
p11-kit-server.socket                    | sockets.target.wants
pipewire-pulse.socket                    | sockets.target.wants
pipewire-session-manager.service         | user
pipewire.socket                          | sockets.target.wants
wireplumber.service                      | pipewire.service.wants
xdg-user-dirs-update.service             | default.target.wants
$

I start my desktop environment (KDE) with my .xinitrc file using startx. My own services were from this reddit post, which I tried modifying as well as making it a user service instead. I might've messed something up accidentally a long the way? I've also tried reinstalling systemd just to see if it would magically resolve itself however unfortunately nothing happened.

Last edited by str1ngd (2023-07-16 21:29:24)

seth · 2023-07-16 21:43:51

Please avoid full quotes.
NM isn't enabled and there's also no getty@tty1.service or the likes.
What do you actually expect to happen when it "hangs"?
Does simply switching the VT to tty2 and back to tty1 "unhang" it?

The services there are for the suspend & hibernation targets and irrelevant to the boot.

ewaller · 2023-07-16 21:46:04

str1ngd wrote:

ewaller wrote:
when you log into tty2, look through /etc/systemd/system and its sub-directories looking for references to your service and remove them
There was only one file in the system directory, (no sub directories, or hidden files etc) which was the file I used to enable one of my services, though I had made it completely empty. I deleted it and restarted however sadly no progress was made

no 'wants' directories for the various targets?
no getty@tty1.service.d ?
no links to any .service files?

str1ngd · 2023-07-16 21:47:07

Sorry, I will stop quoting

I am not sure I fully understand, when I say it "hangs", I mean that systemd doesn't continue any further, it just sits at "[ OK ] Started X service" forever in tty1, what I expect is it to clear the screen and show me a log in screen (getty). Whenever I log in to tty2, I also have to manually enable NetworkManager and bluetooth won't work even if I manually enable the bluetooth service

str1ngd · 2023-07-16 21:48:37

ewaller wrote:

no 'wants' directories for the various targets?
no getty@tty1.service.d ?
no links to any .service files?

I think I must've messed something up in the system directory? Is it too hard to restore it, would the safest option just be to reinstall the system?

Last edited by str1ngd (2023-07-16 21:50:22)

seth · 2023-07-16 21:50:19

Whenever I log in to tty2, I also have to manually enable NetworkManager and bluetooth

Because none of these services are enabled.

ps aux | grep agetty

Edit: wait, one file *in total*? Not only pertaining to your "own" service?

ls -laR /etc/systemd

Last edited by seth (2023-07-16 21:52:08)

str1ngd · 2023-07-16 22:00:04

seth wrote:

Because none of these services are enabled.

Yeah, you seem to be completely right, I enabled NetworkManager and bluetooth and it populated the system directory again:

$ ls -a
.  ..  bluetooth.target.wants  dbus-org.bluez.service  dbus-org.freedesktop.nm-dispatcher.service  multi-user.target.wants  network-online.target.wants
$

This returned nothing other than my grep, I tried enabling "agetty" but it failed and said the unit file doesn't exist:

ps aux | grep agetty

$ ls -laR /etc/systemd
/etc/systemd:
totalt 76
drwxr-xr-x  5 root root 4096 17 jul 00.24 .
drwxr-xr-x 97 root root 4096 17 jul 01.52 ..
-rw-r--r--  1 root root  894 11 jul 14.01 coredump.conf
-rw-r--r--  1 root root  766 11 jul 14.01 homed.conf
-rw-r--r--  1 root root 1282 11 jul 14.01 journald.conf
-rw-r--r--  1 root root  894 11 jul 14.01 journal-remote.conf
-rw-r--r--  1 root root  822 11 jul 14.01 journal-upload.conf
-rw-r--r--  1 root root 1539 11 jul 14.01 logind.conf
drwxr-xr-x  2 root root 4096 10 apr 19.46 network
-rw-r--r--  1 root root  846 11 jul 14.01 networkd.conf
-rw-r--r--  1 root root  804 11 jul 14.01 oomd.conf
-rw-r--r--  1 root root  670 11 jul 14.01 pstore.conf
-rw-r--r--  1 root root 1564 11 jul 14.01 resolved.conf
-rw-r--r--  1 root root  953 11 jul 14.01 sleep.conf
drwxr-xr-x  5 root root 4096 17 jul 01.51 system
-rw-r--r--  1 root root 2127 11 jul 14.01 system.conf
-rw-r--r--  1 root root  856 11 jul 14.01 timesyncd.conf
drwxr-xr-x  5 root root 4096 16 apr 14.42 user
-rw-r--r--  1 root root 1490 11 jul 14.01 user.conf

/etc/systemd/network:
totalt 8
drwxr-xr-x 2 root root 4096 10 apr 19.46 .
drwxr-xr-x 5 root root 4096 17 jul 00.24 ..

/etc/systemd/system:
totalt 20
drwxr-xr-x 5 root root 4096 17 jul 01.51 .
drwxr-xr-x 5 root root 4096 17 jul 00.24 ..
drwxr-xr-x 2 root root 4096 17 jul 01.51 bluetooth.target.wants
lrwxrwxrwx 1 root root   41 17 jul 01.51 dbus-org.bluez.service -> /usr/lib/systemd/system/bluetooth.service
lrwxrwxrwx 1 root root   57 17 jul 01.51 dbus-org.freedesktop.nm-dispatcher.service -> /usr/lib/systemd/system/NetworkManager-dispatcher.service
drwxr-xr-x 2 root root 4096 17 jul 01.51 multi-user.target.wants
drwxr-xr-x 2 root root 4096 17 jul 01.51 network-online.target.wants

/etc/systemd/system/bluetooth.target.wants:
totalt 8
drwxr-xr-x 2 root root 4096 17 jul 01.51 .
drwxr-xr-x 5 root root 4096 17 jul 01.51 ..
lrwxrwxrwx 1 root root   41 17 jul 01.51 bluetooth.service -> /usr/lib/systemd/system/bluetooth.service

/etc/systemd/system/multi-user.target.wants:
totalt 8
drwxr-xr-x 2 root root 4096 17 jul 01.51 .
drwxr-xr-x 5 root root 4096 17 jul 01.51 ..
lrwxrwxrwx 1 root root   46 17 jul 01.51 NetworkManager.service -> /usr/lib/systemd/system/NetworkManager.service

/etc/systemd/system/network-online.target.wants:
totalt 8
drwxr-xr-x 2 root root 4096 17 jul 01.51 .
drwxr-xr-x 5 root root 4096 17 jul 01.51 ..
lrwxrwxrwx 1 root root   58 17 jul 01.51 NetworkManager-wait-online.service -> /usr/lib/systemd/system/NetworkManager-wait-online.service

/etc/systemd/user:
totalt 20
drwxr-xr-x 5 root root 4096 16 apr 14.42 .
drwxr-xr-x 5 root root 4096 17 jul 00.24 ..
drwxr-xr-x 2 root root 4096 16 apr 14.42 default.target.wants
drwxr-xr-x 2 root root 4096 16 apr 14.42 pipewire.service.wants
lrwxrwxrwx 1 root root   41 16 apr 14.42 pipewire-session-manager.service -> /usr/lib/systemd/user/wireplumber.service
drwxr-xr-x 2 root root 4096 16 apr 15.00 sockets.target.wants

/etc/systemd/user/default.target.wants:
totalt 8
drwxr-xr-x 2 root root 4096 16 apr 14.42 .
drwxr-xr-x 5 root root 4096 16 apr 14.42 ..
lrwxrwxrwx 1 root root   50 16 apr 14.42 xdg-user-dirs-update.service -> /usr/lib/systemd/user/xdg-user-dirs-update.service

/etc/systemd/user/pipewire.service.wants:
totalt 8
drwxr-xr-x 2 root root 4096 16 apr 14.42 .
drwxr-xr-x 5 root root 4096 16 apr 14.42 ..
lrwxrwxrwx 1 root root   41 16 apr 14.42 wireplumber.service -> /usr/lib/systemd/user/wireplumber.service

/etc/systemd/user/sockets.target.wants:
totalt 8
drwxr-xr-x 2 root root 4096 16 apr 15.00 .
drwxr-xr-x 5 root root 4096 16 apr 14.42 ..
lrwxrwxrwx 1 root root   43 16 apr 12.00 p11-kit-server.socket -> /usr/lib/systemd/user/p11-kit-server.socket
lrwxrwxrwx 1 root root   43 16 apr 15.00 pipewire-pulse.socket -> /usr/lib/systemd/user/pipewire-pulse.socket
lrwxrwxrwx 1 root root   37 16 apr 14.42 pipewire.socket -> /usr/lib/systemd/user/pipewire.socket

Last edited by str1ngd (2023-07-16 22:01:44)

seth · 2023-07-16 22:12:05

own service which I then deleted

If you go back in your shell history, there's probably a d'ohhh moment in there for you.

enable getty@tty1.service … and everything else you're no missing

str1ngd · 2023-07-16 22:37:49

seth wrote:

If you go back in your shell history, there's probably a d'ohhh moment in there for you.

Yeah, thank you so much guys for willing to help me, my system seems to be perfectly fine now. I didn't know it was this goofy, though it's definitely my fault since I must've accidentally deleted *every* single file in the system directory some how lol

Like I said my system seems perfectly stable and fine so I don't think I had that many services enabled anyways, I guess I will just have to find out which ones I will have to re-enable. I'll prepend [SOLVED] to the title, thank you guys again

Arch Linux

#1 2023-07-16 17:01:05

[SOLVED] Systemd hangs on boot

#2 2023-07-16 19:21:40

Re: [SOLVED] Systemd hangs on boot

#3 2023-07-16 20:36:09

Re: [SOLVED] Systemd hangs on boot

#4 2023-07-16 21:07:15

Re: [SOLVED] Systemd hangs on boot

#5 2023-07-16 21:20:07

Re: [SOLVED] Systemd hangs on boot

#6 2023-07-16 21:20:53

Re: [SOLVED] Systemd hangs on boot

#7 2023-07-16 21:26:41

Re: [SOLVED] Systemd hangs on boot

#8 2023-07-16 21:43:51

Re: [SOLVED] Systemd hangs on boot

#9 2023-07-16 21:46:04

Re: [SOLVED] Systemd hangs on boot

#10 2023-07-16 21:47:07

Re: [SOLVED] Systemd hangs on boot

#11 2023-07-16 21:48:37

Re: [SOLVED] Systemd hangs on boot

#12 2023-07-16 21:50:19

Re: [SOLVED] Systemd hangs on boot

#13 2023-07-16 22:00:04

Re: [SOLVED] Systemd hangs on boot

#14 2023-07-16 22:12:05

Re: [SOLVED] Systemd hangs on boot

#15 2023-07-16 22:37:49

Re: [SOLVED] Systemd hangs on boot

Board footer