Arch Linux

swansone

Member

Registered: 2016-09-20

Posts: 3

[Solved] After system upgrade getting soft lockup, and other errors.

I just finished running

pacman -Syu

and it appeared to complete successfully. However, upon rebooting, my system is quite broken. I'm going to do my best to detail the problems that I'm seeing, but it will be somewhat difficult considering I'm typing this from a different machine, and I don't know exactly how to trace the problem. I'm happy to provide more information if it would be helpful. Here's what I'm seeing:

When I reboot the machine, it seems to boot fine, and I'm presented with a tty login prompt (I usually login from tty and then run startx to start xmonad). After about 2 seconds, a bunch of error information is dumped to the screen, even before I log in. I don't have a scrollback, so I can't see all of it, but I'll type in some of what I can see:

...
---[ end trace 9cb11e3177470123 ]---
Fixing recursive fault but reboot is needed
BUG: scheduling while atomic: wl_event_handle/1206/0x00000003
Modules linked in: ...
...
CPU: 2 PID: 1206 Comm: wl_event_handle Tainted: P    D   0    4.8.10-1-ARCH #1
Hardware name: Apple Inc. MacBookAir7,1/...
....

After seeing the above, I can hit [Enter] and get a login prompt. If I login, however, I experience more strange behavior. I'm not dropped into a prompt, but instead the following message is printed out, and I have a blank cursor:

systemd-journald: File /var/log/journal/.../user-100.journal corrupted or uncleanly shut down renaming and replacing

At this point, if I press [Control-c] I can get to a bash prompt, but it doesn't seem to have read in my .bash_profile, or behave how I would expect it to. Additionally, at this point, every 22 seconds or so, an error is printed out that looks something like:

NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [pidof:1515]
Modules linked in: ...
...
CPU : 1 PID: 1515 Comm: pidof Tainted: P    D W  O L  4.8.10-1-ARCH #1
...

Finally, if I try to run something like startx or even reboot or shutdown, the system just hangs. I can hard reset, but I see the same behavior. I'm not sure what my next steps for debugging this should be, so any advice would be appreciated.

Last edited by swansone (2016-11-29 00:38:00)

seth

Member

Registered: 2012-09-03

Posts: 51,456

Re: [Solved] After system upgrade getting soft lockup, and other errors.

The info you posted points the wifi kernel module, so figure which that is (it's very much likely listed in the error messages, likely some broadcom chip) and blacklist it (eg. from the installation medium or grml etc.)

Afterwards, you need to figure the cause of the bug. A more detailed error message (eg. modprobing the module via ssh) and the actual module/chip combination should make a superb google call ;-)

The weird prompt is likely a busybox rescue shell, not of much use to fix the system (get grml ;-)

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

swansone

Member

Registered: 2016-09-20

Posts: 3

Re: [Solved] After system upgrade getting soft lockup, and other errors.

Thanks so much for the advice seth! I was able to get a working system by booting to recovery media, and removing the dynamic kernel module I had installed for wireless, broadcom-wl-dkms (in aur: (https://aur.archlinux.org/packages/broadcom-wl-dkms/). At this point, I was able to login to my normal shell, startx, etc. I was a bit worried I wouldn't be able to get wireless back, but simply updating this package and reinstalling fixed the problem.

I'm marking this as solved, because I now have a working system, but I do have a couple more questions. First, is there anything that I could have noticed in the output of pacman -Syu that could have hinted I was heading towards a broken system? Second, is there any extra work that I need to do for dynamic kernel modules installed through aur? I assumed they would play nice with a kernel upgrade (and maybe they do in general) but I didn't seem to be lucky this time.