vmlinuz-linux not found after upgrade

sulejman · 2023-07-26 20:46:04

A short introduction first:

I did a general upgrade today with pacman -Syu, and left the computer for a few minutes. After I came back, I noticed it had rebooted and opened user login. Not bothered by this, I logged in and used it for some time. After another pacman command, it returned that database could not be locked. This time, I restarted computer myself, expecting that to close any hidden processes.

After booting and selecting Arch from grub, it gives me this:

Loading Linux linux ...
error: file 'vmlinuz-linux' not found.
Loading initial ramdisk ...
error: you need to loead the kernel first.

I started looking up solutions, and found that by chrooting to the system partition I should reinstall linux and base packages, and check if fstab has my uefi and system partitions in it. So I chrooted using

mount /dev/nvme0n1p1 /mnt/
mount /dev/nvme0n1p2 /mnt/
arch-chroot /mnt/
pacman -S linux base
pacman -Syu

I also checked if partitions are listed in /etc/fstab, and tried to boot the system again. It returned the same message.

Unfortunately, the system is installed on a nvme drive, which makes it practically impossible for me to mount it on another machine, so I have to work on it from a liveusb. I have no idea how to take screenshots in this situation.

Last edited by sulejman (2023-07-26 21:07:42)

seth · 2023-07-26 21:59:28

I noticed it had rebooted and opened user login

"pacman -Syu" will NOT reboot your computer.
Either you used some stupid wrapper/GUI tool or the system didn't reboot at all, but the GUI session crashed (perhaps because of https://bugs.archlinux.org/task/77789 ) or the system rebooted completely unrelated (Ryzen systems do that, but that'd be a frequent issue)

After another pacman command

Pretty sure that "another" is the key to the problem cause, here. What "another"?

mount /dev/nvme0n1p1 /mnt/
mount /dev/nvme0n1p2 /mnt/
arch-chroot /mnt/

One of the mounts is wrong for sure.
Did you mean

mount /dev/nvme0n1p2 /mnt/
mount /dev/nvme0n1p1 /mnt/boot

or so?

no idea how to take screenshots in this situation

You don't.
You can post file contents and command outputs from the console, see the 1st link below or just https://0x0.st or https://ix.io
Start by "lsblk -f" when NOT chrooted (the chroot will kill relevant information here) and the fstab of the installed system.
But also just have a look around the partitions you're booting.

A journal of a previous boot might be handy, https://wiki.archlinux.org/title/System … al_to_view

sulejman · 2023-07-27 00:04:33

After another pacman command
Pretty sure that "another" is the key to the problem cause, here. What "another"?

After another "pacman -Syu", as I wanted to see if the update finished.

(As for the screenshots, the issue is it's hard to copy output (or screenshots) from the broken machine to show my issue. But I found I can use another usb drive to write it and copy it to the one I am using now.)

Here is what I found:
fstab
journalctl
lsblk -f

seth · 2023-07-27 06:23:43

The journal part you posted lacks the entire boot sequence, so that's not useful at all.

fstab and lsblk suggest that you *did* mean

mount /dev/nvme0n1p2 /mnt/
mount /dev/nvme0n1p1 /mnt/boot

(but we'll have to check that against the journal)

There're also sda and sdd, onle is likely the usb key you dumped the output on, but the other might be the accidental boot device.
Did you check for existence, timestamps and sizes of vmlinuz* on the partitions?

sulejman · 2023-07-28 04:05:33

The issue turned out to be proper mounting, I had to include the boot partition

mount /dev/nvme0n1p2 /mnt/
mount /dev/nvme0n1p1 /mnt/boot

After that redownloading linux and base packages solved the issue. So does that mean last upgrade messed boot partition?

Answering your question, sdd was the liveusb from which I operated, and sda is my storage drive.

dakota · 2023-07-28 05:03:32

You might want to explore why there's a recurring problem with mounting your boot partition. A complete journal might help.

V1del wrote:

You should fix your bootloader setup if regenerating your initramfs leaves you with a non booting system. That means you either haven't mounted /boot and should or you did mount /boot and shouldn't.
boot.mount: Directory /boot to mount over is not empty, mounting anyway.
is a good indicator of that, from the logistics of that line you mounted /boot in your live system but haven't done so during initial install so your first initramfs will have landed on /boot directory of your root partition and that's what the bootloader tries to load).

Cheers,

dakota · 2023-07-29 22:05:13

seth wrote:

I noticed it had rebooted and opened user login
"pacman -Syu" will NOT reboot your computer.
Either you used some stupid wrapper/GUI tool or the system didn't reboot at all, but the GUI session crashed (perhaps because of https://bugs.archlinux.org/task/77789 ) or the system rebooted completely unrelated (Ryzen systems do that, but that'd be a frequent issue)

@seth -- this might be Off-Topic, since the solution has already been found...

... but is this a command to update the system, followed by an automatic reboot?

Jul 26 17:18:15 hakurei dbus-daemon[581]: [system] Activating via systemd: service name='org.freedesktop.home1' unit='dbus-org.freedesktop.home1.service' requested by ':1.759' (uid=0 pid=39777 comm="sudo pacman -Syu")
Jul 26 17:18:15 hakurei dbus-daemon[581]: [system] Activation via systemd failed for unit 'dbus-org.freedesktop.home1.service': Unit dbus-org.freedesktop.home1.service not found.
Jul 26 17:18:15 hakurei sudo[39777]: pam_systemd_home(sudo:auth): systemd-homed is not available: Unit dbus-org.freedesktop.home1.service not found.
Jul 26 17:18:18 hakurei sudo[39777]:     miko : TTY=pts/0 ; PWD=/home/miko ; USER=root ; COMMAND=/usr/bin/pacman -Syu
Jul 26 17:18:18 hakurei sudo[39777]: pam_unix(sudo:session): session opened for user root(uid=0) by miko(uid=1000)
Jul 26 17:18:18 hakurei sudo[39777]: pam_unix(sudo:session): session closed for user root
Jul 26 17:18:24 hakurei systemd-logind[585]: The system will reboot now!
Jul 26 17:18:24 hakurei systemd-logind[585]: System is rebooting.

... and would this fstab cause the /boot drive to be remounted RO, if an error were found?

# /dev/nvme0n1p1
UUID=DAE0-582F      	/boot     	vfat      	rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro	0 2

Cheers,

seth · 2023-07-30 07:20:26

Jul 26 17:18:15 hakurei dbus-daemon[581]: [system] Activating via systemd: service name='org.freedesktop.home1' unit='dbus-org.freedesktop.home1.service' requested by ':1.759' (uid=0 pid=39777 comm="sudo pacman -Syu")
Jul 26 17:18:15 hakurei dbus-daemon[581]: [system] Activation via systemd failed for unit 'dbus-org.freedesktop.home1.service': Unit dbus-org.freedesktop.home1.service not found.
Jul 26 17:18:15 hakurei sudo[39777]: pam_systemd_home(sudo:auth): systemd-homed is not available: Unit dbus-org.freedesktop.home1.service not found.
Jul 26 17:18:18 hakurei sudo[39777]:     miko : TTY=pts/0 ; PWD=/home/miko ; USER=root ; COMMAND=/usr/bin/pacman -Syu
Jul 26 17:18:18 hakurei sudo[39777]: pam_unix(sudo:session): session opened for user root(uid=0) by miko(uid=1000)
Jul 26 17:18:18 hakurei sudo[39777]: pam_unix(sudo:session): session closed for user root

1st three lines are your regular "can you please check out my last great idea it's super awesome you know PLEEASE" homed spam, followed by a root "session" (sudo pacman -Syu)

Jul 26 17:18:24 hakurei systemd-logind[585]: The system will reboot now!
Jul 26 17:18:24 hakurei systemd-logind[585]: System is rebooting.

6 seconds later the system was rebooted.
There's no contention that the system was rebooted, just "pacman -Syu" will not do that in and by itself

Looking at the journal

Jul 26 14:57:23 hakurei sudo[33396]:     miko : TTY=pts/0 ; PWD=/home/miko ; USER=root ; COMMAND=/usr/bin/pacman -Syu
Jul 26 14:57:27 hakurei sudo[33413]:     miko : TTY=pts/0 ; PWD=/home/miko ; USER=root ; COMMAND=/usr/bin/pacman -Syu
Jul 26 14:57:28 hakurei sudo[33416]:     miko : TTY=pts/0 ; PWD=/home/miko ; USER=root ; COMMAND=/usr/bin/pacman -Syu
Jul 26 14:57:31 hakurei sudo[33419]:     miko : TTY=pts/0 ; PWD=/home/miko ; USER=root ; COMMAND=/usr/bin/pacman -Syu
Jul 26 14:57:36 hakurei sudo[33422]:     miko : TTY=pts/0 ; PWD=/home/miko ; USER=root ; COMMAND=/usr/bin/pacman -Syu
Jul 26 14:58:33 hakurei sudo[34294]:     miko : TTY=pts/0 ; PWD=/home/miko ; USER=root ; COMMAND=/usr/bin/pacman -Syu
Jul 26 14:58:58 hakurei sudo[34315]:     miko : TTY=pts/1 ; PWD=/var/lib/pacman ; USER=root ; COMMAND=/usr/bin/cat db.lck
Jul 26 14:59:12 hakurei sudo[34321]:     miko : TTY=pts/0 ; PWD=/home/miko ; USER=root ; COMMAND=/usr/bin/pacman -Syu
Jul 26 15:00:40 hakurei sudo[34368]:     miko : TTY=pts/0 ; PWD=/home/miko ; USER=root ; COMMAND=/usr/bin/pacman -Syu
Jul 26 15:04:58 hakurei sudo[34653]:     miko : TTY=pts/0 ; PWD=/home/miko ; USER=root ; COMMAND=/usr/bin/pacman -Syu
Jul 26 15:04:59 hakurei sudo[34664]:     miko : TTY=pts/0 ; PWD=/home/miko ; USER=root ; COMMAND=/usr/bin/pacman -Syu
Jul 26 15:05:00 hakurei sudo[34667]:     miko : TTY=pts/0 ; PWD=/home/miko ; USER=root ; COMMAND=/usr/bin/pacman -Syu
Jul 26 17:18:18 hakurei sudo[39777]:     miko : TTY=pts/0 ; PWD=/home/miko ; USER=root ; COMMAND=/usr/bin/pacman -Syu
Jul 26 17:18:24 hakurei systemd-logind[585]: The system will reboot now!
Jul 26 17:18:24 hakurei systemd-logind[585]: System is rebooting.

it seems to reflect

After another pacman command, it returned that database could not be locked. This time, I restarted computer myself

The original incident might not be covered at all - most likely the session crashed while pacman was running (hence the locked database)

The fstab entry will re-mount the filesystem if there're errors during its use, there would ("should"…) have been errors from pacman being unable to write into that path.
When booting the tailing "2" makes fsck check the device and in doubt complain that there're errors to be please fixed.

Looking at the fstab:
the order is wrong, @sulejman please fix that - the root partition should be before the /boot partition - you're likely writing into the /boot path of the root partition with this.

dakota · 2023-07-30 15:02:57

seth wrote:

There's no contention that the system was rebooted, just "pacman -Syu" will not do that in and by itself

Understood. .. and thanks for the detailed explanation!

(This forum is way more fun than reading the news. )

Cheers,

seth · 2023-07-30 15:49:39

The first "not-reboot" might have been https://bugs.archlinux.org/task/77789 - notably if there's a stray evdev configlet around.

Arch Linux

#1 2023-07-26 20:46:04

vmlinuz-linux not found after upgrade

#2 2023-07-26 21:59:28

Re: vmlinuz-linux not found after upgrade

#3 2023-07-27 00:04:33

Re: vmlinuz-linux not found after upgrade

#4 2023-07-27 06:23:43

Re: vmlinuz-linux not found after upgrade

#5 2023-07-28 04:05:33

Re: vmlinuz-linux not found after upgrade

#6 2023-07-28 05:03:32

Re: vmlinuz-linux not found after upgrade

#7 2023-07-29 22:05:13

Re: vmlinuz-linux not found after upgrade

#8 2023-07-30 07:20:26

Re: vmlinuz-linux not found after upgrade

#9 2023-07-30 15:02:57

Re: vmlinuz-linux not found after upgrade

#10 2023-07-30 15:49:39

Re: vmlinuz-linux not found after upgrade

Board footer