Keyboard goes unresponsive for some reason

DutchEllie · 2022-02-25 08:50:05

Hi, sometimes the keyboard on my laptop randomly locks up. It happens when I am typing or just using the thing, never when I am away. My laptop is a ThinkPad L15 Gen 2, running 5.16.10-arch1-1.

As a total security risk, this is my output of journalctl -b -1. I removed some IP addresses, as my public ip is also logged. Termbin log

I think it may have happened around 9:08 in the log, as right after that I logged out and restarted, because that's the only fix I know of.
Here's what happens. I can't use my keyboard at all. The caps lock and numlock lights are unresponsive, so are their functions. Volume adjust doesn't work, none of the buttons work at all. I can't switch TTYs either, because no buttons work.
One key always gets stuck and keeps repeating forever until I fix it by rebooting. It's always a different key, not the same.
The touchpad keeps working just fine and I can press it's buttons and use the mouse normally. That's how I am able to reboot.

If you need any more logs or anything, I can provide.

stepnem · 2022-02-25 11:28:58

FWIW, I've been seeing something similar on my L14. If you have an external keyboard at hand, you might get away with no reboot plugging that in. At least in my case, reloading the keyboard driver (using the external keyboard) then brings the internal keyboard back, e.g.:

sudo modprobe -r atkbd; sudo modprobe atkbd

As to the cause of the issue, I have no idea. In my case there is absolutely nothing happening in the journal when it occurs. I don't see anything that would stand out as related in your journal, either.

DutchEllie · 2022-02-25 11:31:41

stepnem wrote:

FWIW, I've been seeing something similar on my L14. If you have an external keyboard at hand, you might get away with no reboot plugging that in. At least in my case, reloading the keyboard driver (using the external keyboard) then brings the internal keyboard back, e.g.:
sudo modprobe -r atkbd; sudo modprobe atkbd
As to the cause of the issue, I have no idea. In my case there is absolutely nothing happening in the journal when it occurs. I don't see anything that would stand out as related in your journal, either.

I have indeed not tried to use an external keyboard, as none were on hand when it happened. I will definitely try your command next time this happens.
I also did not notice anything in the journal around the time when it occurred, but that's why I asked here, thinking I might have missed something.

Shouldn't there be logs somewhere for the keyboard driver?

seth · 2022-02-25 12:38:43

One key always gets stuck and keeps repeating forever until I fix it by rebooting. It's always a different key, not the same.

Sounds like HW and if the key is pressed and doesn't release, you'll not be able to inject further events.
Did you press the stuck key right before this happened? What if you re-press it and wiggle the cap a bit, then release it?

Shouldn't there be logs somewhere for the keyboard driver?

The kernel module will log into dmesg which is captured by the system journal, the X11 driver will log into the xorg log.
https://wiki.archlinux.org/title/Xorg#General

Absent a second keyboard (which would likely either impact the situation or be impacted by it), you could maybe ssh into the system?

DutchEllie · 2022-02-25 12:47:04

seth wrote:

One key always gets stuck and keeps repeating forever until I fix it by rebooting. It's always a different key, not the same.
Sounds like HW and if the key is pressed and doesn't release, you'll not be able to inject further events.
Did you press the stuck key right before this happened? What if you re-press it and wiggle the cap a bit, then release it?

The key itself isn't stuck at all. It has to be a software issue, not a hardware one (not a physically stuck key anyway).

seth wrote:

Shouldn't there be logs somewhere for the keyboard driver?
The kernel module will log into dmesg which is captured by the system journal, the X11 driver will log into the xorg log.
https://wiki.archlinux.org/title/Xorg#General
Absent a second keyboard (which would likely either impact the situation or be impacted by it), you could maybe ssh into the system?

Checking the Xorg logs, there aren't any logs from the time it happened just this morning. If it happens again, I could try to find a way to SSH into the system.

seth · 2022-02-25 13:09:02

Xorg.0.log.old?
From the ssh, you want to check whether there're corresponding events in "evtest" and "libinput debug-events" - the autorepeating key might be a symptom, does restarting the X11 server help?

DutchEllie · 2022-02-25 13:10:54

seth wrote:

Xorg.0.log.old?
From the ssh, you want to check whether there're corresponding events in "evtest" and "libinput debug-events" - the autorepeating key might be a symptom, does restarting the X11 server help?

I checked the dates of those files, if that is okay. That one is from after the PC had already restarted. Currently, it's not happening, so I can't check.

DutchEllie · 2022-02-28 09:43:59

It happened again today and after plugging in an external keyboard I was able to reload my keyboard driver with stepnem's command. There is no log about it in the Xorg.0.log or any of the others. The journal also has nothing to say about it.

seth · 2022-02-28 11:05:50

Did you check whether the failure extends to evtest and/or libinput?
Because of the symptoms, you could pass "atkbd.softrepeat=1" to the kernel and see whether the issue shows up w/ that as well.

DutchEllie · 2022-02-28 12:51:52

seth wrote:

Did you check whether the failure extends to evtest and/or libinput?
Because of the symptoms, you could pass "atkbd.softrepeat=1" to the kernel and see whether the issue shows up w/ that as well.

I am not entirely sure how I would do that, I'm sorry. What do I need to do and what logs do I need to check next time it happens?

seth · 2022-02-28 12:54:09

https://wiki.archlinux.org/title/Kernel_parameters and https://bbs.archlinux.org/viewtopic.php … 9#p2024349 (both commands will require root permissions)

stepnem · 2022-03-26 15:24:50

seth wrote:

Did you check whether the failure extends to evtest and/or libinput?

evtest continues to receive keydown events for the stuck key until I reload atkbd.
libinput debug output is a bit harder to interpret due to the silly timestamping and
the fact that it only shows one key press for a held-down key, but I guess it's the same
there.
Interestingly (?) when I tried tracing atkbd_interrupt directly, e.g.

bpftrace -e 'kprobe:atkbd_interrupt { printf("data: %02x, flags: %02x\n", arg1, arg2); }'

it did seem to stop receiving any events when the key gets stuck.

Thing is, first, I don't really know what I should be looking for, and second, I don't have a way to
reproduce it other than waiting for it to happen (which it does about once a day or two).

seth wrote:

Because of the symptoms, you could pass "atkbd.softrepeat=1" to the kernel and see whether the issue shows up w/ that as well.

Happened with that set, too.

My workaround du jour:

#!/usr/bin/env -S bpftrace --unsafe

kprobe:psmouse_report_standard_buttons
{
        if (arg1 == 10) {
                @count += 1;
        }
}

interval:s:1
{
        if (@count > 2) {
                system("/script/that/reloads/atkbd");
        }
        @count = 0;
}

So whenever I press right touchpad button more than 2 times in a second, a script that reloads the driver (and restores some X keyboard adjustments) is run. Better than futzing with an external keyboard at least.

rs_31415 · 2022-06-03 07:27:34

I'm experiencing the same issue with a laptop ThinkPad L15 Gen 2 running Lubuntu 22.04 with kernel 5.15.0-33-generic. I won't try to reiterate what DutchEllie and stepnem already described perfectly. I also upgraded the kernel to 5.16 and late versions of 5.17 just in case recent improvements for AMD fixed this bug, but it didn't help.

I was wondering if any of you guys have an update on this issue. It seems to me Lenovo is still tracking bugs with this model (e.g. ThinkPad-P14s-Gen-2-AMD-trackpoint-moves-on-its-own). I also found another reference to this sort of issue here . In the meantime, I will wait for the next iteration of this issue and hopefully will be able to report back with more information.

stepnem · 2022-06-03 08:11:11

rs_31415 wrote:

I was wondering if any of you guys have an update on this issue.

No update from me. Still happens, albeit apparently less often (maybe once or twice a week).

The Ubuntu bug you link to seems useless and doesn't make any sense to me (they first say
"only the mouse keeps working", which would fit our issue, but in the next sentence they say
switching to another terminal via keyboard works).

I don't experience any system freezes, the only issue is with the internal keyboard (the atkbd module).

DutchEllie · 2022-06-03 09:30:25

rs_31415 wrote:

I was wondering if any of you guys have an update on this issue.

I don't have much to say on the issue anymore. It hasn't happened to me in a long time, last time was months ago and it hasn't happened since. Maybe some update fixed it, but I really wouldn't know which one it could have been...

rs_31415 · 2022-06-03 18:53:55

stepnem wrote:

The Ubuntu bug you link to seems useless and doesn't make any sense to me (they first say
"only the mouse keeps working", which would fit our issue, but in the next sentence they say
switching to another terminal via keyboard works).
I don't experience any system freezes, the only issue is with the internal keyboard (the atkbd module).

I think you're right. It seemed significant at first since it was the only reference to this sort of issue occurring in the same model, however, as you pointed out, the description doesn't fit very well. I was taking into account that people can describe the same problem differently and in fact, when it first happened to me I thought the system was "freezing". Anyway, in the meantime, I added the option i8042.debug in GRUB_CMDLINE_LINUX_DEFAULT, just in case something interesting is sent to the logs.

rs_31415 · 2022-06-03 19:05:04

DutchEllie wrote:

I don't have much to say on the issue anymore. It hasn't happened to me in a long time, last time was months ago and it hasn't happened since. Maybe some update fixed it, but I really wouldn't know which one it could have been...

That's good news! Out of curiosity, can you post general information about your system (kernel, distribution, etc)? Do you remember changing something (adding flags to the kernel, or removing something that could be causing issues)? I used to have issues keeping the desired layout (although changing it "by hand" in .profile with setxkbmap works).

Luffield · 2022-06-12 07:55:05

I have a similar issue on a 13 inch Gen2 ThinkBook, running Xfce. I happens every 2-3 weeks I think.
I found that I don't have to reboot to fix this, it's enough to log out and back in.

rs_31415 · 2022-06-13 19:21:15

Luffield wrote:

I have a similar issue on a 13 inch Gen2 ThinkBook, running Xfce. I happens every 2-3 weeks I think.
I found that I don't have to reboot to fix this, it's enough to log out and back in.

Are you able to click on menus with your touchpad and log out? I can't do that because the key presses are so fast and intrusive that I can't perform any other action. However, closing the lid and opening it again "fixes the issue".

By the way, after adding the i8042.debug option, the issue has occurred once, but looking at the logs was not very helpful. It showed simply a key being pressed repeatedly. I have been tinkering with lowering the rate with `xset r rate`, but I don't know if that's helpful since I have been unable to trigger the bug with a higher rate.

hdert · 2022-06-29 07:41:57

Hi, I have nearly the exact same issue on my 14-inch AMD L14 (2nd gen?). It feels like it happens more often than every 2-3 weeks, more like once every three days for me. I know it's a keyboard freeze because I can't switch TTY's but the mouse works.

I usually just suspend and unsuspend to fix the problem. Sometimes, but rarely, the keyboard freeze happens without any keys being held down (or the program I'm on doesn't accept that key, perhaps).

I'm on 5.18-arch1-1, and I've done some light log checking, but I haven't found anything significant. I'll try plugging an external keyboard in next time it happens to see if it does anything. I'm running i3-wm.

If you want any more info, feel free to ask.

rs_31415 · 2022-06-29 18:35:45

I have upgraded to 5.18.3-051803-generic and added the flag "i8042.dumbkbd" in /etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash i8042.direct i8042.dumbkbd"

After those changes, I haven't experienced this issue for a couple of weeks. I wanted to remove that flag and see if the kernel upgrade was enough, but since you said that you're experiencing the same issue on 5.18-arch1-1, maybe that's not what seems to have fixed this issue for me.

hdert · 2022-07-29 05:20:38

Hello, it's been a while. It happened again quite recently, and that time, I was able to use an external keyboard to reload the module, and that fixed it. After I confirmed the external keyboard solution worked, I added i8042.direct and i8042.dumbkbd to my kernel parameters and verified using `cat /proc/cmdline`.

Even after doing this, my internal keyboard still freezes anyway. I'm on 5.18.7-arch1-1 now, I don't think there's anything else I can do but hope that an update fixes this.

seth · 2022-07-29 07:48:32

You could check whether there're BIOS updates available for your device since it most likely operates out of spec.

rs_31415 · 2022-07-29 08:51:17

seth wrote:

You could check whether there're BIOS updates available for your device since it most likely operates out of spec.

I second that. I also installed BIOS updates before installing Linux on this laptop. I'm pretty sure this problem has been fixed since I haven't experienced this issue for 45 days.

hdert · 2022-08-03 20:24:56

Thanks for the suggestion, I had completely forgotten about BIOS updates. I've since updated from 1.19 to 1.26, there doesn't seem to be anything specific to this problem in the upgrade notes from 1.19 to 1.26, but here's hoping it fixes the problem.

Arch Linux

#1 2022-02-25 08:50:05

Keyboard goes unresponsive for some reason

#2 2022-02-25 11:28:58

Re: Keyboard goes unresponsive for some reason

#3 2022-02-25 11:31:41

Re: Keyboard goes unresponsive for some reason

#4 2022-02-25 12:38:43

Re: Keyboard goes unresponsive for some reason

#5 2022-02-25 12:47:04

Re: Keyboard goes unresponsive for some reason

#6 2022-02-25 13:09:02

Re: Keyboard goes unresponsive for some reason

#7 2022-02-25 13:10:54

Re: Keyboard goes unresponsive for some reason

#8 2022-02-28 09:43:59

Re: Keyboard goes unresponsive for some reason

#9 2022-02-28 11:05:50

Re: Keyboard goes unresponsive for some reason

#10 2022-02-28 12:51:52

Re: Keyboard goes unresponsive for some reason

#11 2022-02-28 12:54:09

Re: Keyboard goes unresponsive for some reason

#12 2022-03-26 15:24:50

Re: Keyboard goes unresponsive for some reason

#13 2022-06-03 07:27:34

Re: Keyboard goes unresponsive for some reason

#14 2022-06-03 08:11:11

Re: Keyboard goes unresponsive for some reason

#15 2022-06-03 09:30:25

Re: Keyboard goes unresponsive for some reason

#16 2022-06-03 18:53:55

Re: Keyboard goes unresponsive for some reason

#17 2022-06-03 19:05:04

Re: Keyboard goes unresponsive for some reason

#18 2022-06-12 07:55:05

Re: Keyboard goes unresponsive for some reason

#19 2022-06-13 19:21:15

Re: Keyboard goes unresponsive for some reason

#20 2022-06-29 07:41:57

Re: Keyboard goes unresponsive for some reason

#21 2022-06-29 18:35:45

Re: Keyboard goes unresponsive for some reason

#22 2022-07-29 05:20:38

Re: Keyboard goes unresponsive for some reason

#23 2022-07-29 07:48:32

Re: Keyboard goes unresponsive for some reason

#24 2022-07-29 08:51:17

Re: Keyboard goes unresponsive for some reason

#25 2022-08-03 20:24:56

Re: Keyboard goes unresponsive for some reason

Board footer