[SOLVED] Keyboard failure after bad system update

kevinae91 · 2023-11-13 04:32:18

I’ve always been a BIOS user prior to the new machine I’m having this issue with, but this time I went with a UEFI system setup. Well, I screwed up and ran a full system upgrade/update without mounting the boot partition. During the initial boot failure I was dropped into a recovery shell, and noticed the system wasn’t reading input from the keyboard. I loaded into the ISO and edited my fstab so i could get back into a normal shell, which worked fine, but still the keyboard would not work.

The keyboard is a usb wired keychron k6

I am contemplating just blowing everything up and reinstalling, not too big of an issue because this install doesn’t have much on it. However, my ideal scenario is one in which I can identify what the issue is and fix it.

Do you guys have any ideas where to start?

Last edited by kevinae91 (2023-12-01 16:23:36)

seth · 2023-11-13 09:26:40

Edited the fstab how?

an a full system upgrade/update without mounting the boot partition

You're likely (still) booting a dated kernel (compare "uname -a" and "pacman -Qs kernel") and the fix for that is to (clear the /boot mountpoint), mount the boot partition and re-install the kernel (from the iso)

kevinae91 · 2023-11-13 14:41:48

seth wrote:

Edited the fstab how?
an a full system upgrade/update without mounting the boot partition
You're likely (still) booting a dated kernel (compare "uname -a" and "pacman -Qs kernel") and the fix for that is to (clear the /boot mountpoint), mount the boot partition and re-install the kernel (from the iso)

Actually i edited it using information I found from you that you posted in someone else's thread lol. I commented out the boot partition in fstab, which worked, but then I ran into the keyboard issue. I also forgot to mention that I booted into the ISO afterwards, arch-chroot, mounted the boot partition, then ran “pacman -S linux”, which didn’t throw any errors. But that didn’t fix the issue, still no input from the keyboard at the root login.

As a side question; do I need to uncomment the boot partition line in the fstab when all is said and done, or do I just delete that line and then generate a new fstab?

Thanks for the help by the way!

Last edited by kevinae91 (2023-11-13 14:42:26)

seth · 2023-11-13 14:48:11

do I need to uncomment the boot partition line in the fstab

We shall see.
Can you reboot the keyboardless system (I assume you cannot interact w/ it, given you've no keyboard) in a clean way (short push of the power button)?
Then post the last system journal of the installed system from the installation iso?
https://wiki.archlinux.org/title/System … al_to_view (and pipe that into "curl -F 'file=@-' 0x0.st")

kevinae91 · 2023-11-16 03:10:50

seth wrote:

do I need to uncomment the boot partition line in the fstab
We shall see.
Can you reboot the keyboardless system (I assume you cannot interact w/ it, given you've no keyboard) in a clean way (short push of the power button)?
Then post the last system journal of the installed system from the installation iso?
https://wiki.archlinux.org/title/System … al_to_view (and pipe that into "curl -F 'file=@-' 0x0.st")

Hey sorry for the late reply, life is busy atm.

I can indeed reboot the system in a clean way, at root login prompt, a short push of the power button does a clean shutdown.

https://0x0.st/Hv_Q.txt

seth · 2023-11-16 10:04:38

Nov 15 22:39:09 v3lka kernel: Linux version 6.5.9-arch2-1 (linux@archlinux) (gcc (GCC) 13.2.1 20230801, GNU ld (GNU Binutils) 2.41.0) #1 SMP PREEMPT_DYNAMIC Thu, 26 Oct 2023 00:52:20 +0000
Nov 15 22:39:09 v3lka kernel: Command line: BOOT_IMAGE=../vmlinuz-linux root=/dev/nvme0n1p1 rw initrd=../initramfs-linux.img

You're still booting the 6.5.9-arch2-1 kernel (I assume you've installed a newer version and downgrading to 6.5.9-arch2-1 would "fix" your keyboard?)
"BOOT_IMAGE=../vmlinuz-linux" - are you using syslinux?

Nov 15 22:39:09 v3lka kernel: usb 1-7.2: new full-speed USB device number 5 using xhci_hcd
Nov 15 22:39:09 v3lka kernel: usb 1-7.2: New USB device found, idVendor=05ac, idProduct=024f, bcdDevice= 1.12
Nov 15 22:39:09 v3lka kernel: usb 1-7.2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
Nov 15 22:39:09 v3lka kernel: usb 1-7.2: Product: Keychron K6
Nov 15 22:39:09 v3lka kernel: usb 1-7.2: Manufacturer: Keychron
Nov 15 22:39:09 v3lka kernel: usb 1-7.3: new full-speed USB device number 7 using xhci_hcd
Nov 15 22:39:09 v3lka kernel: usb 1-7.3: New USB device found, idVendor=046d, idProduct=c52b, bcdDevice=24.11
Nov 15 22:39:09 v3lka kernel: usb 1-7.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
Nov 15 22:39:09 v3lka kernel: usb 1-7.3: Product: USB Receiver
Nov 15 22:39:09 v3lka kernel: usb 1-7.3: Manufacturer: Logitech

Keyboard principally shows up, does the logitech device work?

kevinae91 · 2023-11-17 00:39:26

seth wrote:

Nov 15 22:39:09 v3lka kernel: Linux version 6.5.9-arch2-1 (linux@archlinux) (gcc (GCC) 13.2.1 20230801, GNU ld (GNU Binutils) 2.41.0) #1 SMP PREEMPT_DYNAMIC Thu, 26 Oct 2023 00:52:20 +0000
Nov 15 22:39:09 v3lka kernel: Command line: BOOT_IMAGE=../vmlinuz-linux root=/dev/nvme0n1p1 rw initrd=../initramfs-linux.img

You're still booting the 6.5.9-arch2-1 kernel (I assume you've installed a newer version and downgrading to 6.5.9-arch2-1 would "fix" your keyboard?)
"BOOT_IMAGE=../vmlinuz-linux" - are you using syslinux?

Nov 15 22:39:09 v3lka kernel: usb 1-7.2: new full-speed USB device number 5 using xhci_hcd
Nov 15 22:39:09 v3lka kernel: usb 1-7.2: New USB device found, idVendor=05ac, idProduct=024f, bcdDevice= 1.12
Nov 15 22:39:09 v3lka kernel: usb 1-7.2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
Nov 15 22:39:09 v3lka kernel: usb 1-7.2: Product: Keychron K6
Nov 15 22:39:09 v3lka kernel: usb 1-7.2: Manufacturer: Keychron
Nov 15 22:39:09 v3lka kernel: usb 1-7.3: new full-speed USB device number 7 using xhci_hcd
Nov 15 22:39:09 v3lka kernel: usb 1-7.3: New USB device found, idVendor=046d, idProduct=c52b, bcdDevice=24.11
Nov 15 22:39:09 v3lka kernel: usb 1-7.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
Nov 15 22:39:09 v3lka kernel: usb 1-7.3: Product: USB Receiver
Nov 15 22:39:09 v3lka kernel: usb 1-7.3: Manufacturer: Logitech

Keyboard principally shows up, does the logitech device work?

Okay I can try the previous kernel, and yeah, I am using syslinux.

So yeah, I noticed that systemd recognized the keyboard, keychron k6, and also the mouse, which is the Logitech device. Which is confusing as to why it won’t receive input from the keyboard when it certainly sees it. I hadn’t gotten around to installing a UI before screwing up, so I haven’t had a chance to see if the mouse works.

EDIT—

Okay, reverting the kernel worked, thank you for your help in diagnosing this. So now, theoretically, I could correctly mount the boot partition and then reissue a full system update, and I shouldn’t run into the same issue as before, correct? If I understand it correctly, for UEFI, I just need to mount the boot partition before doing a system update. Is that it?

Thanks again!

Last edited by kevinae91 (2023-11-17 01:31:01)

seth · 2023-11-17 07:59:56

This has nothing to do w/ uefi - are you using https://wiki.archlinux.org/title/Syslinux ?

ls -lr /boot # w/ and w/o the boot partition mounted there

kevinae91 · 2023-11-17 12:43:46

seth wrote:

This has nothing to do w/ uefi - are you using https://wiki.archlinux.org/title/Syslinux ?
ls -lr /boot # w/ and w/o the boot partition mounted there

Yes as I mentioned in my previous post, I am using syslinux as my bootloader. I think I just initially misunderstood some of what you wrote in this post here: https://bbs.archlinux.org/viewtopic.php?id=269588 but I think I understand it now. This issue has nothing to do with UEFI, but with having separate boot partition, yes?

As for running that command; without /boot mounted, I get “total 0” and with /boot mounted I get a few thousand, and now the directory is populated with “vmlinuz-linux” “syslinux” “EFI” “initramfs-linux.img” and “initramfs-linux-fallback.img”.

seth · 2023-11-17 14:47:50

Sorry, I overread the syslinux statement.
(On a tagential note: please don't full-quote posts, it's just bloating the thread)

I get a few thousand

More important is "where":

sudo find /boot -name vmlinuz-linux

kevinae91 · 2023-11-18 16:57:53

seth wrote:

(On a tagential note: please don't full-quote posts, it's just bloating the thread)

My apologies.

sudo find /boot -name vmlinuz-linux

Running this command with /boot mounted returns two results:

/boot/vmlinuz-linux
/boot/EFI/vmlinuz-linux

That first result though is just a remnant of me messing around with different mount points during the initial install of the system, and I just haven’t gotten around to cleaning up what I didn’t wind up using. “/boot/EFI/…” is what is used by syslinux.efi and efibootmgr.

seth · 2023-11-18 20:25:34

That first result though is just a remnant of me messing around with different mount points during the initial install of the system

Don't delete it

“/boot/EFI/…” is what is used by syslinux.efi and efibootmgr.

Nov 15 22:39:09 v3lka kernel: Command line: BOOT_IMAGE=../vmlinuz-linux root=/dev/nvme0n1p1 rw initrd=../initramfs-linux.img

file /boot/vmlinuz-linux
sudo find /boot -name initramfs-linux.img

kevinae91 · 2023-11-18 21:18:45

seth wrote:

Nov 15 22:39:09 v3lka kernel: Command line: BOOT_IMAGE=../vmlinuz-linux root=/dev/nvme0n1p1 rw initrd=../initramfs-linux.img

Okay well, that is what it should be using, but apparently isn’t because… reasons? I guess I must not have configured it to do so, but then idk why it works correctly with the previous kernel.

file /boot/vmlinuz-linux
sudo find /boot -name initramfs-linux.img

Yeah the initramfs is in the “/boot/EFI/“ directory too, along with those other files I mentioned a couple posts back, as well as the syslinux directory. I believe that’s where they are supposed to be, unless I am wrong.

Last edited by kevinae91 (2023-11-18 21:20:01)

seth · 2023-11-18 21:25:55

Please post the output of the commands - I highly suspect hat you've been updating in /boot/EFI but the kernel is actually loaded from /boot (hence the ../vmlinuz-linux BOOT_IMAGE, syslinux steps down from /boot/EFI to /boot for that file)

kevinae91 · 2023-11-19 23:50:41

seth wrote:

Please post the output of the commands

/boot/vmlinuz-linux: Linux kernel x86 boot executable bzImage, version 6.6.1-arch1-1 (linux@archlinux) #1 SMP PREEMPT_DYNAMIC Wed, 08 Nov 2023 16:05:38 +0000, RO-rootFS, swap_dev 0XC, Normal VGA

/boot/initramfs-linux.img
/boot/EFI/initramfs-linux.img

EDIT—

Also, if syslinux is just going to load the kernel from “/boot” then what even is the purpose of “/boot/EFI” and the files which reside in it?

Last edited by kevinae91 (2023-11-20 03:45:10)

seth · 2023-11-25 11:19:17

Sorry for the delay.

https://wiki.archlinux.org/title/EFI_system_partition - esp. the note in https://wiki.archlinux.org/title/EFI_sy … unt_points
But the kernel in /boot is actually updated - either /boot/EFI is a recursive mount of /boot (you mounted the same partition into two locations) or you've meanwhile fixed the /boot mount

file /boot/EFI/vmlinuz-linux

is the dated kernel then?
Are you still booting it?

kevinae91 · 2023-11-28 18:51:48

No need to apologize; thank you for not forgetting about me.

Well anyway I managed to break the whole boot partition in a way that I couldn’t recover it, no matter what I tried

I wiped the esp and this time I’m just going to start fresh and use “/boot” instead of “/boot/efi” and/or “/efi”. That being said, I have one question in particular; does efibootmgr write the bootloader code to esp, or do I still need to run the dd command outlined in section 3.1.2.2 of the syslinux page: https://wiki.archlinux.org/title/Syslinux

Now, you’re probably thinking that this is a silly question given the fact that I originally had syslinux working and the system booted properly before my updating broke it, but, during my initial install I had actually run the syslinux install script like I have many times over the years without realizing I was actually on UEFI for this machine. Only after the script was all said and done did I realize I was using UEFI, but no errors were thrown during the install script, so I didn’t think anything of it and just skipped on ahead to section 3.2 which is where UEFI stuff is covered on that page. So my thought process currently is that although the syslinux install script isnt meant for UEFI, it probably still has some functions that are platform agnostic, like, for example, writing the bootloader code to the esp.

That’s where I’m at currently, sorry for the wall of text.

EDIT—

Okay I fixed everything after some trial and error. My question now is, do I just need to make sure /boot is mounted before I do any system upgrades? As in, is that the only prerequisite when updating the system? I at least know I can rollback the kernel if in the future I make the same mistake. I guess I can bindmount the boot partition so i don’t have to worry about it, but shouldn’t systemd and fstab automatically mount that partition?

Last edited by kevinae91 (2023-11-29 02:53:54)

seth · 2023-11-29 08:19:58

https://wiki.archlinux.org/title/Syslin … tion_table is for BIOS systems (and GPT) - unless you set the UEFI to operate in BIOS/legacy mode, that entire section should be irrelevant and you'd strictly follow https://wiki.archlinux.org/title/Syslinux#UEFI_systems

Edit:

but shouldn’t systemd and fstab automatically mount that partition?

Not sure about systemd, but if you've an fstab entry, that will certainly applied during the boot..
Just remember that the line-order in /etc/fstab matters (ie. the root partition needs to be mounted before /boot and any bind mount has to happen after the desired path is in place)

Last edited by seth (2023-11-29 08:22:35)

kevinae91 · 2023-11-29 23:08:49

seth wrote:

…
Just remember that the line-order in /etc/fstab matters (ie. the root partition needs to be mounted before /boot and any bind mount has to happen after the desired path is in place)

Gotcha. I think I should be good from here. Thank you for all of your help!

I consider this solved!

Last edited by kevinae91 (2023-11-29 23:10:23)

seth · 2023-11-30 07:18:19

Please always remember to mark resolved threads by editing your initial posts subject - so others will know that there's no task left, but maybe a solution to find.
Thanks.

Arch Linux

#1 2023-11-13 04:32:18

[SOLVED] Keyboard failure after bad system update

#2 2023-11-13 09:26:40

Re: [SOLVED] Keyboard failure after bad system update

#3 2023-11-13 14:41:48

Re: [SOLVED] Keyboard failure after bad system update

#4 2023-11-13 14:48:11

Re: [SOLVED] Keyboard failure after bad system update

#5 2023-11-16 03:10:50

Re: [SOLVED] Keyboard failure after bad system update

#6 2023-11-16 10:04:38

Re: [SOLVED] Keyboard failure after bad system update

#7 2023-11-17 00:39:26

Re: [SOLVED] Keyboard failure after bad system update

#8 2023-11-17 07:59:56

Re: [SOLVED] Keyboard failure after bad system update

#9 2023-11-17 12:43:46

Re: [SOLVED] Keyboard failure after bad system update

#10 2023-11-17 14:47:50

Re: [SOLVED] Keyboard failure after bad system update

#11 2023-11-18 16:57:53

Re: [SOLVED] Keyboard failure after bad system update

#12 2023-11-18 20:25:34

Re: [SOLVED] Keyboard failure after bad system update

#13 2023-11-18 21:18:45

Re: [SOLVED] Keyboard failure after bad system update

#14 2023-11-18 21:25:55

Re: [SOLVED] Keyboard failure after bad system update

#15 2023-11-19 23:50:41

Re: [SOLVED] Keyboard failure after bad system update

#16 2023-11-25 11:19:17

Re: [SOLVED] Keyboard failure after bad system update

#17 2023-11-28 18:51:48

Re: [SOLVED] Keyboard failure after bad system update

#18 2023-11-29 08:19:58

Re: [SOLVED] Keyboard failure after bad system update

#19 2023-11-29 23:08:49

Re: [SOLVED] Keyboard failure after bad system update

#20 2023-11-30 07:18:19

Re: [SOLVED] Keyboard failure after bad system update

Board footer