Arch Linux

jekyllhyde

Member

Registered: 2023-11-16

Posts: 6

[Solved] Sometimes shutdown hangs

This issue happens somewhat intermittently, but usually after the computer has been on for 6+ hours and then I try to shutdown. It shows something like "Reached shut down stage/target" and then hangs forever until a hard power off. The keyboard's already disconnected at this point so there's nothing else to do.

The weird thing is, reboot always works so I've started just rebooting and then shutting down right after, which always works.

I have not seen any relevant log messages. The last message is usually something like "journalctl received terminate signal/shutting down."

Things I've tried to no avail:
- disabling non-default systemctl services (docker)
- switching from nvidia to nouveau drivers
- trying different grub configs for GRUB_CMDLINE_LINUX_DEFAULT
- increasing swap space (from 3GB to 16GB)
- switching from xorg to wayland (sway)

I just don't know a good way of trying to get to the bottom of this issue. The only remaining thing I wanted to try was disabling the ly login manager, and if that doesn't work maybe partition the drive and dual install/boot something like nixOS to see if that also would have the same issue or not.

Any suggestions?

Last edited by jekyllhyde (2023-11-29 16:28:05)

twelveeighty

Member

From: Alberta, Canada

Registered: 2011-09-04

Posts: 1,096

Re: [Solved] Sometimes shutdown hangs

Please don't paraphrase error messages and post the actual, relevant journal messages from your last failed shutdown. Make sure to use -b -1 switch to ensure you get the previous boot messages.

Also make sure you do not have 'quiet' as your kernel parameter and watch the TTY during a shutdown. It's possible that there's messages shown there that are not captured in the journal, in case the journal was closed before the error happens.

jekyllhyde

Member

Registered: 2023-11-16

Posts: 6

Re: [Solved] Sometimes shutdown hangs

So it just happened again, and these were the last messages when it hung:

...
[ OK ] Unmounted Temporary Directory /tmp.
[ OK ] Stopped target Swaps.
[ OK ] Deactivating swap /swapfile...
[ OK ] Deactivated swap /swapfile.
[ OK ] Reached target Unmount All Filesystems.
[ OK ] Stopped Remount Root and Kernel File Systems.
[ OK ] Reached target System Shutdown.
[ OK ] Reached target Late Shutdown Services.
[ OK ] Finished System Power Off.
[ OK ] Reached target System Power Off.

And the last journal messages from the last session:

Nov 16 21:20:38 computername systemd[1]: NetworkManager.service: Consumed 6.344s CPU time.
Nov 16 21:20:38 computername systemd[1]: Stopping D-Bus System Message Bus...
Nov 16 21:20:38 computername systemd[1]: dbus.service: Deactivated successfully.
Nov 16 21:20:38 computername systemd[1]: Stopped D-Bus System Message Bus.
Nov 16 21:20:38 computername systemd[1]: dbus.service: Consumed 1.968s CPU time.
Nov 16 21:20:38 computername systemd[1]: Stopped target Basic System.
Nov 16 21:20:38 computername systemd[1]: Stopped target Path Units.
Nov 16 21:20:38 computername systemd[1]: Stopped target Slice Units.
Nov 16 21:20:38 computername systemd[1]: Removed slice User and Session Slice.
Nov 16 21:20:38 computername systemd[1]: user.slice: Consumed 3h 3min 46.802s CPU time.
Nov 16 21:20:38 computername systemd[1]: Stopped target Socket Units.
Nov 16 21:20:38 computername systemd[1]: dbus.socket: Deactivated successfully.
Nov 16 21:20:38 computername systemd[1]: Closed D-Bus System Message Bus Socket.
Nov 16 21:20:38 computername systemd[1]: Stopped target System Initialization.
Nov 16 21:20:38 computername systemd[1]: proc-sys-fs-binfmt_misc.automount: Deactivated successfully.
Nov 16 21:20:38 computername systemd[1]: Unset automount Arbitrary Executable File Formats File System Automount Point.
Nov 16 21:20:38 computername systemd[1]: Stopped target Local Encrypted Volumes.
Nov 16 21:20:38 computername systemd[1]: systemd-ask-password-console.path: Deactivated successfully.
Nov 16 21:20:38 computername systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.
Nov 16 21:20:38 computername systemd[1]: systemd-ask-password-wall.path: Deactivated successfully.
Nov 16 21:20:38 computername systemd[1]: Stopped Forward Password Requests to Wall Directory Watch.
Nov 16 21:20:38 computername systemd[1]: Stopped target Local Integrity Protected Volumes.
Nov 16 21:20:38 computername systemd[1]: Stopped target Local Verity Protected Volumes.
Nov 16 21:20:38 computername systemd[1]: systemd-sysctl.service: Deactivated successfully.
Nov 16 21:20:38 computername systemd[1]: Stopped Apply Kernel Variables.
Nov 16 21:20:38 computername systemd[1]: systemd-coredump.socket: Deactivated successfully.
Nov 16 21:20:38 computername systemd[1]: Closed Process Core Dump Socket.
Nov 16 21:20:38 computername systemd[1]: systemd-modules-load.service: Deactivated successfully.
Nov 16 21:20:38 computername systemd[1]: Stopped Load Kernel Modules.
Nov 16 21:20:38 computername systemd[1]: Stopping Record System Boot/Shutdown in UTMP...
Nov 16 21:20:38 computername systemd[1]: systemd-update-utmp.service: Deactivated successfully.
Nov 16 21:20:38 computername systemd[1]: Stopped Record System Boot/Shutdown in UTMP.
Nov 16 21:20:38 computername systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.
Nov 16 21:20:38 computername systemd[1]: Stopped Create Volatile Files and Directories.
Nov 16 21:20:38 computername systemd[1]: Stopped target Local File Systems.
Nov 16 21:20:38 computername systemd[1]: efi.automount: Deactivated successfully.
Nov 16 21:20:38 computername systemd[1]: Unset automount EFI System Partition Automount.
Nov 16 21:20:38 computername systemd[1]: Stopped target Preparation for Local File Systems.
Nov 16 21:20:38 computername systemd[1]: Unmounting Temporary Directory /tmp...
Nov 16 21:20:38 computername systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.
Nov 16 21:20:38 computername systemd[1]: Stopped Create Static Device Nodes in /dev.
Nov 16 21:20:38 computername systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.
Nov 16 21:20:38 computername systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.
Nov 16 21:20:38 computername systemd[1]: tmp.mount: Deactivated successfully.
Nov 16 21:20:38 computername systemd[1]: Unmounted Temporary Directory /tmp.
Nov 16 21:20:38 computername systemd[1]: Stopped target Swaps.
Nov 16 21:20:38 computername systemd[1]: Deactivating swap /swapfile...
Nov 16 21:20:38 computername systemd[1]: swapfile.swap: Deactivated successfully.
Nov 16 21:20:38 computername systemd[1]: Deactivated swap /swapfile.
Nov 16 21:20:38 computername systemd[1]: Reached target Unmount All Filesystems.
Nov 16 21:20:38 computername systemd[1]: systemd-remount-fs.service: Deactivated successfully.
Nov 16 21:20:38 computername systemd[1]: Stopped Remount Root and Kernel File Systems.
Nov 16 21:20:38 computername systemd[1]: Reached target System Shutdown.
Nov 16 21:20:38 computername systemd[1]: Reached target Late Shutdown Services.
Nov 16 21:20:38 computername systemd[1]: systemd-poweroff.service: Deactivated successfully.
Nov 16 21:20:38 computername systemd[1]: Finished System Power Off.
Nov 16 21:20:38 computername systemd[1]: Reached target System Power Off.
Nov 16 21:20:38 computername systemd[1]: Shutting down.
Nov 16 21:20:39 computername systemd-shutdown[1]: Syncing filesystems and block devices.
Nov 16 21:20:39 computername systemd-shutdown[1]: Sending SIGTERM to remaining processes...
Nov 16 21:20:39 computername systemd-journald[237]: Journal stopped

seth

Member

Registered: 2012-09-03

Posts: 51,861

Re: [Solved] Sometimes shutdown hangs

I wanted to try was disabling the ly login manager

Have you?

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

twelveeighty

Member

From: Alberta, Canada

Registered: 2011-09-04

Posts: 1,096

Re: [Solved] Sometimes shutdown hangs

Try booting into the 'multi-user.target' target by specifying systemd.unit=multi-user.target as a kernel parameter on boot. This should bypass the login manager. Then leave the PC running for an extended time and issue the shutdown command using 'sudo systemctl poweroff'.

jekyllhyde

Member

Registered: 2023-11-16

Posts: 6

Re: [Solved] Sometimes shutdown hangs

I disabled the ly login manager but the issue remains. I also ruled out the Intel watchdog (iTCO_wdt) by disabling that. I turned off chronyd and a few "scratchpad" sway apps that were set to run at startup (a clock app and a notes app).

I'm still curious why the issue happens only for shutdown but never reboot. I believe the issue is related to something within systemd somewhere, and systemd has "modules" (?) for power off and reboot and maybe I'll try looking into each configuration to see how they differ (I haven't changed any of the defaults as it came). I do see some systemd services with timeouts of "infinity" so I might try capping those to see if that makes a difference. Everything else is a reasonable limit and since I know leaving the hanging shutdown alone without a hard reboot for 12+ hours doesn't change anything, I'm thinking it might be one of these infinite timeouts.

./system/systemd-quotacheck.service:TimeoutSec=infinity
./system/systemd-growfs@.service:TimeoutSec=infinity
./system/systemd-growfs-root.service:TimeoutSec=infinity
./system/systemd-fsck-root.service:TimeoutSec=infinity
./system/systemd-fsck@.service:TimeoutSec=infinity

Unfortunately because the issue only happens after the computer has been on for some hours, I can only test a theory once or twice per day.

seth

Member

Registered: 2012-09-03

Posts: 51,861

Re: [Solved] Sometimes shutdown hangs

Ther was https://bbs.archlinux.org/viewtopic.php?id=289301 - does it still power off w/ the LTS kernel?

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

jekyllhyde

Member

Registered: 2023-11-16

Posts: 6

Re: [Solved] Sometimes shutdown hangs

Ther was https://bbs.archlinux.org/viewtopic.php?id=289301 - does it still power off w/ the LTS kernel?

I can't seem to get linux-lts to work properly with the nouveau driver/module. When it boots up it doesn't automatically load it, so I can't start sway. If I do a

sudo modprobe nouveau

the screen goes and stays completely black, but I think it actually loads it since I can issue commands blindly to start sway and then reboot from it.

I'll try to get LTS working with nouveau (or try it with the nvidia drivers) but for the meantime I'm trying out the linux-zen kernel since that doesn't also have the issue with nouveau.

jekyllhyde

Member

Registered: 2023-11-16

Posts: 6

Re: [Solved] Sometimes shutdown hangs

The power off problem goes away with the linux-zen kernel. Thanks @seth for your help.

I'll continue to monitor (and re-enable some things I've disabled) and mark it as [Solved] in a couple days.

jekyllhyde

Member

Registered: 2023-11-16

Posts: 6

Re: [Solved] Sometimes shutdown hangs

One thing that might have helped was removing "nvidia-utils" package and setting the "nomodeset" kernel parameter.

Regardless, seems to be solved now.