[SOLVED] failed upgrade, cannot boot due to unknown filetype 'vfat'

segment289 · 2024-03-17 12:29:04

I ran my usual pacman -Syuv upgrade this morning, and my laptop froze during mkinitcpio (as in the mouse pointer stopped moving and the seconds on my clock stopped updating). Now it won't boot; the error message is "Failed to mount /boot" error (unknown filesystem type 'vfat')."

I found https://bbs.archlinux.org/viewtopic.php?id=285986 with a similar symptom, but my situation isn't quite the same because (a) I use rEFInd, and (b) I have two directories under /lib/modules: 6.7.9-arch1-1 and 6.8.1-arch1-1.

Based on that topic, I can provide the following information (apologies in advance for not copying/pasting from the emergency shell). I am somewhat hesitant to start trying things until I have specific things to try, but mostly because of the extra layer of indirection on the boot partition due to rEFInd.

"uname -rv" indicates I'm running 6.7.9-arch1-1.

"modprobe vfat"says "/lib/modules/6.7.9-arch-1/kernel/fs/fat/fat.ko.zst error=No such file or directory"

/lib is a symlink to /usr/lib.

/lib/modules/6.7.9-arch1-1 contains a collection of modules.* files, but no kernel directory.

/lib/modules/6.8.1-arch1-1 contains the directories kernel and extramodules, and the files modules.builtin, modules.builtin.modinfo, modules.order, pkgbase, and vmlinuz. Also, the kernel directory does contain the vfat module fs/fat/vfat.ko.zst.

I tried running depmod, but it can't open modules.order, modules.builtin, or modules.builtin.modinfo under /lib/modules/6.7.9-arch1-1 (I double checked; those particular files do *not* exist).

Unsurprisingly, "modprobe vfat" still fails with the same error.

How else can I help you help me?

Last edited by segment289 (2024-03-17 14:47:15)

V1del · 2024-03-17 13:21:31

Boot a live disk, chroot in and reinstall all packages that were part of that update (check /var/log/pacman.log)

As for why you crashed in the first place, possibly https://bugs.archlinux.org/task/77789 try creating the file /etc/systemd/do-not-udevadm-trigger-on-update

segment289 · 2024-03-17 13:49:29

Thanks for your reply. I think I've seen that bug before; it seems that I should reboot after certain upgrades more often.

Okay, I've booted from a live USB stick and connected to the internet.

I use rEFInd. I mounted /dev/nvme0n1p1 on /mnt. I can see EFI, initramfs-linux.img, refind_linux.conf, and vmlinuz-linux under /mnt.

Is it safe to make a /mnt/boot directory and mount /dev/nvme0np4 (which contains my linux boot filesystem) on it?

Would I then chroot into /mnt to do the update, unmount /dev/nvme0np4 from /mnt/boot, and remove /mnt/boot before rebooting?

Is that a regular chroot, or an arch-chroot?

Thanks,
Dan

V1del · 2024-03-17 14:02:27

your order looks wrong, you need to mount your root filesystem to /mnt, i.e. from what you mentioned unmount /mnt, then mount /dev/nvme0n1p4 to /mnt -- you should see /mnt/boot but that's likely empty, as well as all other linux dirs like /etc/ and /lib, then mount /dev/nvme0n1p1 to /mnt/boot, then do an arch-chroot /mnt

segment289 · 2024-03-17 14:06:16

Correction: I should mount nvme0n1p4 on /mnt, and then nvme0np1 on /mnt/boot.

However, it looks like /var/log/pacman.log is incomplete. I see yesterday's update, and today's "starting full system updrade" message, and then a bunch of NUL characters.

Can I get away with pacman -Syuv linux? I don't remember exactly what this morning's full list was.

Thanks,
Dan

Head_on_a_Stick · 2024-03-17 14:17:48

segment289 wrote:

Can I get away with pacman -Syuv linux?

Yes, that should rebuild the initramfs and get your system booting.

segment289 · 2024-03-17 14:25:08

Head_on_a_Stick wrote:

segment289 wrote:
Can I get away with pacman -Syuv linux?
Yes, that should rebuild the initramfs and get your system booting.

When I run that command, pacman downloads the file and starts the upgrade, but gives me pages of errors about files in /usr/lib/modules/6.8.1-arch1 (including pkgbase, vmlinuz, and piles of modules) already existing, and finally "Errors occured, no packages were upgraded."

I can tell I'm close. :-) Thanks for all of your help.

Head_on_a_Stick · 2024-03-17 14:44:23

https://wiki.archlinux.org/title/Pacman … )%22_error

segment289 · 2024-03-17 14:45:49

I moved /usr/lib/modules/6.8.1-arch1-1 out of the way.
I ran pacman -U linux-6.8.1.arch1-1-x86_64.pkg.tar.zst.

And now I can boot!

Thanks again for all of your help.

In light of https://bugs.archlinux.org/task/77789, I will reboot more often.

seth · 2024-04-05 21:42:49

Hi, do you still recall the nature of the update crash?
Did the system suddenly reboot or did it stall and you then rebooted it with the power button?

---
This is a mass-inquiry, so please excuse if your thread actually detailed that.
We're trying to get some data on the situation, so it would be very helpful if you can just briefly respond.
Thanks a lot.

Arch Linux

#1 2024-03-17 12:29:04

[SOLVED] failed upgrade, cannot boot due to unknown filetype 'vfat'

#2 2024-03-17 13:21:31

Re: [SOLVED] failed upgrade, cannot boot due to unknown filetype 'vfat'

#3 2024-03-17 13:49:29

Re: [SOLVED] failed upgrade, cannot boot due to unknown filetype 'vfat'

#4 2024-03-17 14:02:27

Re: [SOLVED] failed upgrade, cannot boot due to unknown filetype 'vfat'

#5 2024-03-17 14:06:16

Re: [SOLVED] failed upgrade, cannot boot due to unknown filetype 'vfat'

#6 2024-03-17 14:17:48

Re: [SOLVED] failed upgrade, cannot boot due to unknown filetype 'vfat'

#7 2024-03-17 14:25:08

Re: [SOLVED] failed upgrade, cannot boot due to unknown filetype 'vfat'

#8 2024-03-17 14:44:23

Re: [SOLVED] failed upgrade, cannot boot due to unknown filetype 'vfat'

#9 2024-03-17 14:45:49

Re: [SOLVED] failed upgrade, cannot boot due to unknown filetype 'vfat'

#10 2024-04-05 21:42:49

Re: [SOLVED] failed upgrade, cannot boot due to unknown filetype 'vfat'

Board footer