Failed to find /boot after update [pacman -Syu]

Kloxar · 2020-03-04 07:49:39

I installed arch around December and only updated it once around January. It broke my installation (failed to find /boot), but I was able to fix it by arch-chroot then pacman -Syu. I may have done other things too, but I don't remember. I rebooted and was able to use my computer again.
I decided to update arch again 2 days ago, and I ran into the same problem again, except doing what I did last time didn't work

I tried the following : boot into USB; mount devices where they belong (/boot, /, and swap); arch-chroot to /; pacman -Syu; mkinitcpio -p linux; grub-mkconfig /boot/grub/grub.cfg
I rebooted, but it was still broken. I booted with emergency mode and decided to look around. I should point out, when I used mkinitcpio -p linux in the previous mention (while arch-chrooted), it ran fine. I decided to run it again in the current emergency mode and received an error.

Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
-k /boot/vmlinux-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
ERROR: '/lib/modules/5.4.10-arch1-1' is not a valid kernel module directory
Building image from preset: /etc/mkinitcpio.d/linux.preset: 'fallback'
-k /boot/vmlinux-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-fallback.img -S autodetect
ERROR: '/lib/modules/5.4.10-arch1-1' is not a valid kernel module directory

I looked into

/lib/modules

and found that I have 5.5.7-arch1-1.
I'm still new to Arch, and linux in general, so I'm not sure what exactly is going on. My best guess is that mkinitcpio is still configured to look for the old kernel, but can't find it and so fails. OK, that makes sense. But then why did it run fine when I booted from the USB and arch-chrooted into root? Also, why wasn't its configuration updated when I updated the first time two days ago?
My fstab is fine (uuid's and paths are correct) and grub seems fine too, so I assume this mkinitcpio (I have no idea what it is/does btw) is the main culprit or is an indicator of a bigger problem.

Wild Penguin · 2020-03-04 08:33:08

Looks like something went wrong while installing the kernel. There should have been a pacman error message, or the upgrade got interrupted for whatever reason (power outage, /boot full etc.).

What I would do, is the following:

boot from a live CD, chroot
check that /boot mountpoint is empty (before mounting it!). Delete (or move away if feeling paranoid) files in there, if they exist
mount /boot (in the chroot!)
pacman -S linux -> observe linux kernel being reinstalled -> check these:
you should now have a matching dir in /lib/modules, and
there should be a working, new (timestamped) /boot/initramfs (and fallback) (as mkinitcpio should have been run automatically by pacman)

I'm guessing you get the error since you do not have a working initramfs.

As for why it failed in the first place: difficult to tell. AFAIK mkinitcpio should make the initramfs for all installed kernels. I've had a few hiccups myself, too, but they've always been down to user error (mine!). I once uninstalled all kernels by accident!

Last edited by Wild Penguin (2020-03-04 08:52:51)

skunktrader · 2020-03-04 08:38:17

The usual cause of this error is your boot partition was not mounted when you updated the kernel.

Wild Penguin · 2020-03-04 08:49:18

skunktrader wrote:

The usual cause of this error is your boot partition was not mounted when you updated the kernel.

Yeah, certainly one possibility; that's what I had in mind with step 2. above. Why it happened in the first place during upgrade, remains a mystery. Done that in a chroot (forgot to mount /boot)...

Kloxar · 2020-03-04 17:56:10

Wild Penguin wrote:

Looks like something went wrong while installing the kernel. There should have been a pacman error message, or the upgrade got interrupted for whatever reason (power outage, /boot full etc.).
What I would do, is the following:
boot from a live CD, chroot
check that /boot mountpoint is empty (before mounting it!). Delete (or move away if feeling paranoid) files in there, if they exist
mount /boot (in the chroot!)
pacman -S linux -> observe linux kernel being reinstalled -> check these:
you should now have a matching dir in /lib/modules, and
there should be a working, new (timestamped) /boot/initramfs (and fallback) (as mkinitcpio should have been run automatically by pacman)
I'm guessing you get the error since you do not have a working initramfs.
As for why it failed in the first place: difficult to tell. AFAIK mkinitcpio should make the initramfs for all installed kernels. I've had a few hiccups myself, too, but they've always been down to user error (mine!). I once uninstalled all kernels by accident!

Thanks for the response. It didn't work. I did : boot from USB: checked /boot is empty: delete files in that partition (rm -r /dev/nvme0n1p1):

mount /dev/nvme0n1p3 /mnt  //root partition
mount /dev/nvme0n1p1 /mnt/boot    //boot partition
swapon /dev/nvme0n1p2

//swap partition
: arch-chrooted to / : "pacman -S linux": looked at messages:

lib/modules

contained the right file (arch-5.5.57.arch1-1)

These are the install messages

warning: linux-5.5.57.arch1-1 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...

Packages (1) linux-5.5.7.arch1-1

Total Installed Size: 74:31 MiB
Net Upgrade Size:   0.00 MiB

:: Proceed with installation? [Y/n] Y
(1/1) checking keys in keyring
(1/1) checking package integrity
(1/1) loading packages files
(1/1) checking for file conflicts
(1/1) checking available disk space
:: Processing package changes...
(1/1)  reinstalling linux
:: Running post-transaction hooks...
(1/3) Arming ConditionNeedsUpdate...
(2/3) Updating module dependencies...
(3/3) Updating linux initcpios...
==> Building image from preset: /etc/mkinitcpio./linux.preset: 'default'
  -> -k /boot/vlinuz-linux -c /etc/mkinitcpio.conf -g /boot /initramfs-linux.img
==> Starting build: 5.5.7-arch1-1
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autocorrect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
  -> Running build hook: [fsck]
==> Generating module dpendencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-linux.img
==> Image generation successful 
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'fallback'
  -> -k /boot/vmlinuz-linux -c /etc/mkinitcpioconf -g /boot/initramfs-linux-fallback.img -S autodetect
==> Starting build: 5.5.7-arch1-1
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: aic94xx
==> WARNING: Possibly missing firmware for module: wd719x
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gziip-compressed initcpio image: /boot/initramfs-linux-fallback.img
==> Image generation successful

Booted up, the same failed message of "failed to load boot" becuase vfat is not a recognized filesystem
I tried mkinitcpio -o linux and recieved the exact same message as last time.

Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
-k /boot/vmlinux-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
ERROR: '/lib/modules/5.4.10-arch1-1' is not a valid kernel module directory
Building image from preset: /etc/mkinitcpio.d/linux.preset: 'fallback'
-k /boot/vmlinux-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-fallback.img -S autodetect
ERROR: '/lib/modules/5.4.10-arch1-1' is not a valid kernel module directory

Could this be a bug, or is there something else I could be overlooking? It looked like the initramfs were generated correctly.

Head_on_a_Stick · 2020-03-04 20:03:26

Kloxar wrote:

arch-chrooted to /

That could mean either

# arch-chroot /

Or

# arch-chroot /mnt

So what was the exact command that you used? Apologies if this is a silly question but the first command wouldn't work, for obvious reasons.

And have you tried checking the contents of the /boot partition itself (as Wild Penguin suggested)? Check the filesystem integrity as well, if there are any errors it will be mounted read-only and that can cause the problems you describe in the OP.

Kloxar · 2020-03-05 06:15:42

Head_on_a_Stick wrote:

Kloxar wrote:
arch-chrooted to /
That could mean either
# arch-chroot /
Or
# arch-chroot /mnt
So what was the exact command that you used? Apologies if this is a silly question but the first command wouldn't work, for obvious reasons.
And have you tried checking the contents of the /boot partition itself (as Wild Penguin suggested)? Check the filesystem integrity as well, if there are any errors it will be mounted read-only and that can cause the problems you describe in the OP.

Yes, sorry, I should be more specific. I typed the command

arch-chroot /mnt

as for boot, I ran ls -l and got this

total 45596
-rwxr-xr-x 1 root root 40960 Mar 2 06:25 amd-ucode.img
drwxr-xr-x 3 root root 4096 Mar 3 02:06 boot
drwxr-xr-x 3 root root 4096 Jan 6 15:45 EFI
drwxr-xr-x 2 root root 4096 Mar 4 01:00 grub
-rwxr-xr-x 1 root root 30760981 Mar 4 11:17 initramfs-linux-fallback.img
-rwxr-xr-x 1 root root 9361159 Mar 4 11:17 initramfs-linux.img
-rwxr-xr-x 1 root root  6506880 Mar 4 11:17 vmlinuz-linux

Is there a way to output this to a file? I've been taking pictures (as this is my main machine) and writing everything manually up to now. Also, I don't know how to check the file system integrity. I read about something called fsck, but I saw some warnings about it possibly breaking files and decided not to touch it.

I also ran

mkinitcpio -p linux

while arch-chrooted in /mnt to make sure what I mentioned in the OP was true. Somehow, it was able to run perfectly there

[root@archiso /]# mkinitcpio -p linux
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
 -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
==> Starting build: 5.5.7-arch1-1
 -> Running build hook: [base]
 -> Running build hook: [udev]
 -> Running build hook: [autodetect]
 -> Running build hook: [modconf]
 -> Running build hook: [block]
 -> Running build hook: [filesystems]
 -> Running build hook: [keyboard]
 -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip=compressed initcpio image: /boot/initramfs-linux.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'fallback'
 -> -k /boot/vmlinuz-linus -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-fallback.img -S autodetect
==> Starting build 5.5.57-archh1-1
 -> Running build hook: [base]
 -> Running build hook: [udev]
 -> Running build hook: [modconf]
 -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: aic94x
==> WARNING: Possibly missing firmware for module: wd719x
 -> Running build hook: [filesystems]
 -> Running build hook: [keyboard]
 -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-linux-fallback.img
==> image generation succesful
[root@archiso /]# _

I shut it down, removed the iso USB, booted into the normal install, and landed in the emergency screen (as expected, unfortunately). But i tried running

mkinitcpio -p linux

to compare and got the same error as last time. It's still trying to look for the old kernel.

I feel like I'm being too stubborn in mentioning again how the mkinitcpio works only while mounted with the USB, but it just doesn't seem right that it would only work in that instance. Why does it know to look for 5.5.7 while arch-chrooted, but not while in the system itself? Isn't the mkinitcpio that I run arch-chrooted the same I run in the normal system?

I read in the wiki that you can force mkinitcpio to work on the files you want by giving it the name, but it sounds like a workaround rather than an actual fix.

skunktrader · 2020-03-05 06:55:58

Any particular reason for the boot directory in your boot directory? Is that empty?

Kloxar · 2020-03-05 07:12:48

skunktrader wrote:

Any particular reason for the boot directory in your boot directory? Is that empty?

I'm about to get busy, but I'll answer your post before leaving
I saw that and thought it was weird too. I'm not sure if it's supposed to be there, but I remember seeing it while installing Arch back in December (I remember being perplexed by it, looking it up, but then left it alone as I assumed it was an Arch thing). I tried

cd /boot

while in /boot just now. I was expecting to be taken to /boot/boot, but when I typed pwd, It showed I was still in /boot. No matter how many times I type

cd /boot

while in /boot, I still stay in /boot. I'll try to cd into /boot/boot while in /root and see what happens when I get the chance.

skunktrader · 2020-03-05 07:23:25

Kloxar wrote:

I tried
cd /boot
while in /boot just now. I was expecting to be taken to /boot/boot, but when I typed pwd, It showed I was still in /boot. No matter how many times I type
cd /boot
while in /boot, I still stay in /boot.

To navigate to the subdirectory, you would type

cd boot

or

cd /boot/boot

Wild Penguin · 2020-03-05 12:11:27

Hi Kloxar,

I'm having a bit trouble following what you have exactly done. I still believe it is possible the right Kernel image is not installed in you EFI partition, which should be mounted in /boot on your booted up machine and in the *chrooted* environment, before running mkinitcpio (or pacman -S linux).
Also:

rm -r /dev/nvme0n1p1

is a nonsensical command; you do not want to remove the device node!

You want to list the contents of the mount point (directory) before mounting it. I may have been a bit unclear about this before: If there are files there, it just tells an error was made at some point (and just for tidiness sake the files could be removed). Removing the files will not solve your problem, as everything should just work despite the mount point not being empty.

What I would do, chroot once again to your Arch installation. Run these command, and post the contents here:

mount

lsblk

blkid

I find the error message you've posted now:

Kloxar wrote:

Booted up, the same failed message of "failed to load boot" becuase vfat is not a recognized filesystem

to be quite weird, if that is indeed what you get. It seems to imply your root partition is on a vfat partition, which is not possible (might possible be with some hackery, and if you do this on purpose and know what you are doing, but definitely not recommended).

EDIT: Unless, that error message if from grub. That could hint grub modules are not installed correctly and it can not even load the Kernel image.

Last edited by Wild Penguin (2020-03-05 12:28:24)

Wild Penguin · 2020-03-05 12:26:19

Also, at least in your posts there seems to be quite a lot of confusion about mountpoints (and directories in general).

"/" is the root of the currently running environment. If you prepend a directory name with "/", it will always look at a directory name there.

After running chroot (or arch-chroot; you may replace chroot with arch-chroot in the examples below), your new "/" will come whatever parameter you gave to chroot (usually a mount point of another root FS of a GNU operating system). For example, "chroot /mnt/repair" or "chroot /mnt/" or "chroot /mnt/newinstall". You do not ever want to run "chroot /", as that would be nonsensical!

When you mount your EFI partition (and perhaps other partitions), it is important to mount them correctly reflecting if you are mounting before chroot or after chrooting. I.e., before chroot, to mount partitions to be usable inside chroot, you would run:

mount /dev/dev_node_of_chroot_fs /mountpoint_of_newroot
mount /dev/dev_node_of_partition /mountpoint_of_newroot/mountpoint_to_appear_inside_chroot

Instead, if you mount after chrooting (or, more generally: you intend to use the EFI partition or any partition inside the currently active environment) you would run instead:

mount /dev/devnode /mountpoint

I can see this can become confusing, especially if you are not accustomed to these but even experienced users just make errors. That's why it makes sense to familiarize with "mount", "lsblk" and "blkid" and their outputs and always check everything is up and in order before attempting repairs (otherwise, things might just end up installed in wrong places again!).

Also, post your fstab just for checking.

Last edited by Wild Penguin (2020-03-05 12:40:35)

Head_on_a_Stick · 2020-03-05 19:20:57

Kloxar wrote:

Is there a way to output this to a file?

https://wiki.archlinux.org/index.php/Li … in_clients

I think we also need to see your bootloader configuration.

Kloxar · 2020-03-06 04:53:47

Wild Penguin wrote:

What I would do, chroot once again to your Arch installation. Run these command, and post the contents here:
mount
lsblk
blkid

mount

/dev/nvme0n1p3 on / type ext4 (rw,relatime)
/dev/nvme0n1p1 on /boot type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sys on /sys type sysfs (ro,nosuid,nodev,noexec,relatime)
efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=3868616k,nr_inodes=967154,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
shm on /dev/shm type tmpfs (rw,nosuid,nodev,relatime)
run on /run type tmpfs (rw,nosuid,nodev,relatime,mode=755)
tmp on /tmp type tmpfs (rw,nosuid,nodev)
airootfs on /etc/resolv.conf type overlay (rw,relatime,lowerdir=/run/archiso/sfs/airootfs,upperdir=/run/archiso/cowspace/persistent_ARCH_201911/x86_64/upperdir,workdir=/run/archiso/cowspace/persistent_ARCH_201911/x86_64/workdir)

lsblk

NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
loop0         7:0    0 520.9M  1 loop 
sda           8:0    0 931.5G  0 disk 
├─sda1        8:1    0   650M  0 part 
├─sda2        8:2    0   128M  0 part 
├─sda3        8:3    0 917.7G  0 part 
├─sda4        8:4    0   990M  0 part 
├─sda5        8:5    0    11G  0 part 
└─sda6        8:6    0   1.1G  0 part 
sdb           8:16   1   7.3G  0 disk 
└─sdb1        8:17   1   7.3G  0 part 
nvme0n1     259:0    0 465.8G  0 disk 
├─nvme0n1p1 259:1    0   300M  0 part /boot
├─nvme0n1p2 259:2    0   512M  0 part [SWAP]
└─nvme0n1p3 259:3    0   465G  0 part /

blkid

/dev/nvme0n1p1: UUID="8901-FDFB" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="primary" PARTUUID="8f55b884-ba57-42df-911f-12a4849963fd"
/dev/nvme0n1p2: UUID="a08b3af5-2cf5-48ce-93f4-e217f5a5072e" TYPE="swap" PARTUUID="d6390252-ef8f-5544-b741-ac1d23ca0db1"
/dev/nvme0n1p3: UUID="b391d27b-7839-4010-96e7-2ca0643104f3" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="51c68fcc-17f6-0344-a40e-8c59946b1e51"
/dev/sda1: LABEL="ESP" UUID="58DC-6038" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="a4de49cc-18cf-4c36-9de8-ec34b9f87e8b"
/dev/sda3: LABEL="OS" BLOCK_SIZE="512" UUID="10EC4CEFEC4CD0A0" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="6d6b0365-2eda-47d9-a908-8de71566b193"
/dev/sda4: LABEL="WINRETOOLS" BLOCK_SIZE="512" UUID="C8FC3F9FFC3F86AA" TYPE="ntfs" PARTUUID="44e5a896-c188-44ee-b48a-23e0084e81c7"
/dev/sda5: LABEL="Image" BLOCK_SIZE="512" UUID="F2B04063B0403081" TYPE="ntfs" PARTUUID="6cb052bd-e66e-4c1b-bc68-ed632c6862c8"
/dev/sda6: LABEL="DELLSUPPORT" BLOCK_SIZE="512" UUID="2EF43EBDF43E86DD" TYPE="ntfs" PARTUUID="7e3ebfc7-2ffa-4ea0-8f4c-44599688b264"
/dev/sdb1: LABEL="ARCH_201911" UUID="BC20-2F3C" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="Main Data Partition" PARTUUID="1f3c7cf2-96ab-4090-9aa4-f244d66b4340"
/dev/loop0: TYPE="squashfs"
/dev/sda2: PARTLABEL="Microsoft reserved partition" PARTUUID="5e7f70cb-1f70-4bb1-9937-3736315741e5"

/etc/fstab

# /dev/nvme0n1p3
UUID=b391d27b-7839-4010-96e7-2ca0643104f3	/         	ext4      	rw,relatime	0 1

# /dev/nvme0n1p1
UUID=8901-FDFB      	/boot     	vfat      	rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro	0 2

# /dev/nvme0n1p2
UUID=a08b3af5-2cf5-48ce-93f4-e217f5a5072e	none      	swap      	defaults  	0 0

I think we also need to see your bootloader configuration.

/boot/grub/grub.cfg

#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
insmod part_gpt
insmod part_msdos
if [ -s $prefix/grubenv ]; then
  load_env
fi
if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   set boot_once=true
else
   set default="0"
fi

if [ x"${feature_menuentry_id}" = xy ]; then
  menuentry_id_option="--id"
else
  menuentry_id_option=""
fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
  save_env saved_entry
  set prev_saved_entry=
  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
    save_env saved_entry
  fi
}

function load_video {
  if [ x$feature_all_video_module = xy ]; then
    insmod all_video
  else
    insmod efi_gop
    insmod efi_uga
    insmod ieee1275_fb
    insmod vbe
    insmod vga
    insmod video_bochs
    insmod video_cirrus
  fi
}

if [ x$feature_default_font_path = xy ] ; then
   font=unicode
else
insmod part_gpt
insmod ext2
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=root  b391d27b-7839-4010-96e7-2ca0643104f3
else
  search --no-floppy --fs-uuid --set=root b391d27b-7839-4010-96e7-2ca0643104f3
fi
    font="/usr/share/grub/unicode.pf2"
fi

if loadfont $font ; then
  set gfxmode=auto
  load_video
  insmod gfxterm
  set locale_dir=$prefix/locale
  set lang=en_US
  insmod gettext
fi
terminal_input console
terminal_output gfxterm
if [ x$feature_timeout_style = xy ] ; then
  set timeout_style=menu
  set timeout=5
# Fallback normal timeout code in case the timeout_style feature is
# unavailable.
else
  set timeout=5
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/10_linux ###
menuentry 'Arch Linux' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-b391d27b-7839-4010-96e7-2ca0643104f3' {
	load_video
	set gfxpayload=keep
	insmod gzio
	insmod part_gpt
	insmod fat
	if [ x$feature_platform_search_hint = xy ]; then
	  search --no-floppy --fs-uuid --set=root  8901-FDFB
	else
	  search --no-floppy --fs-uuid --set=root 8901-FDFB
	fi
	echo	'Loading Linux linux ...'
	linux	/vmlinuz-linux root=UUID=b391d27b-7839-4010-96e7-2ca0643104f3 rw  loglevel=3 quiet
	echo	'Loading initial ramdisk ...'
	initrd	/amd-ucode.img /initramfs-linux.img
}
submenu 'Advanced options for Arch Linux' $menuentry_id_option 'gnulinux-advanced-b391d27b-7839-4010-96e7-2ca0643104f3' {
	menuentry 'Arch Linux, with Linux linux' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-linux-advanced-b391d27b-7839-4010-96e7-2ca0643104f3' {
		load_video
		set gfxpayload=keep
		insmod gzio
		insmod part_gpt
		insmod fat
		if [ x$feature_platform_search_hint = xy ]; then
		  search --no-floppy --fs-uuid --set=root  8901-FDFB
		else
		  search --no-floppy --fs-uuid --set=root 8901-FDFB
		fi
		echo	'Loading Linux linux ...'
		linux	/vmlinuz-linux root=UUID=b391d27b-7839-4010-96e7-2ca0643104f3 rw  loglevel=3 quiet
		echo	'Loading initial ramdisk ...'
		initrd	/amd-ucode.img /initramfs-linux.img
	}
	menuentry 'Arch Linux, with Linux linux (fallback initramfs)' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-linux-fallback-b391d27b-7839-4010-96e7-2ca0643104f3' {
		load_video
		set gfxpayload=keep
		insmod gzio
		insmod part_gpt
		insmod fat
		if [ x$feature_platform_search_hint = xy ]; then
		  search --no-floppy --fs-uuid --set=root  8901-FDFB
		else
		  search --no-floppy --fs-uuid --set=root 8901-FDFB
		fi
		echo	'Loading Linux linux ...'
		linux	/vmlinuz-linux root=UUID=b391d27b-7839-4010-96e7-2ca0643104f3 rw  loglevel=3 quiet
		echo	'Loading initial ramdisk ...'
		initrd	/initramfs-linux-fallback.img
	}
}

### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/20_linux_xen ###
### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/30_os-prober ###
### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
menuentry 'Windows 10' {
	insmod part_gpt
	insmod fat
	insmod chain
	search --no-floppy --fs-uuid --set=root 58DC-6038
	chainloader /EFI/Microsoft/Boot/bootmgfw.efi
}
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  ${config_directory}/custom.cfg ]; then
  source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg;
fi
### END /etc/grub.d/41_custom ###

Wild Penguin · 2020-03-06 07:57:51

To me everything seems to be order in those.

What about:

ls -l /boot/grub/x86_64-efi/

and

ls -l /boot/grub/

what are the listings?

You may want to:

ls -l /boot/boot/

to see what is inside that extra directory, too.

The error about vfat being a not recognized file system, could be because grub modules are not installed correctly. Especially, if /boot/grub/x86_64-efi/ is empty, then there's the problem - something went wrong when installing grub (why upgrade broke it... that's a mystery!). See here, more specifically the part with:

# grub-install --target=x86_64-efi --efi-directory=esp --bootloader-id=GRUB

Also, if you want you might take a picture of the error message (a real one, since it's a bit difficult to get the output from the bootloader to be pasted in text form here in this case) and link the picture here. Do you have a guess if it is a grub or a Kernel error message (I believe most have assumed the latter here in this thread - I also did in the beginning)?

Another thing to try:

umount /boot

fsck.vfat /dev/nvme0n1p1

If there are no errors, fsck will tell you so; or more correctly, will not print any errors. It will not change anything without prompting.

Last edited by Wild Penguin (2020-03-06 08:10:47)

Kloxar · 2020-03-06 19:58:00

ls -l /boot/grub/x86_64-efi/

ls: cannot access '/boot/grub/x86_64-efi': No such file or directory

That doesn't look good

ls -l /boot/grub/

total 8
-rwxr-xr-x 1 root root 4791 Mar  4 01:00 grub.cfg

This is weird. I *clearly* remember seeing more files in here before. I had trouble adding windows to grub, so I often went in this directory. There were definitely more files there before.

ls -l /boot/boot/

total 4
drwxr-xr-x 2 root root 4096 Mar  3 02:06 efi

Now that I think about it, I think I made this directory in my first attempt at installing arch. I checked the 'efi' directory, and it's completely empty. Should I delete it?

# grub-install --target=x86_64-efi --efi-directory=esp --bootloader-id=GRUB

I didn't keep an exact copy of the error, it basically said "could not resolve canonical path of 'esp'" I just remembered linux is case-sensitive, so maybe it could be as 'ESP'? Unless it is specifically installed in lowercase.

fsck.vfat /dev/nvme0n1p1

https://imgur.com/a/3FgtcCN

fsck.fat 4.1 (2017-01-24)
/dev/nvme0n1p1: 14 files, 22811/76646 clusters

Head_on_a_Stick · 2020-03-07 00:03:22

Kloxar wrote:

# grub-install --target=x86_64-efi --efi-directory=esp --bootloader-id=GRUB
I didn't keep an exact copy of the error, it basically said "could not resolve canonical path of 'esp'" I just remembered linux is case-sensitive, so maybe it could be as 'ESP'? Unless it is specifically installed in lowercase.

You should replace "esp" with the full path to whichever directory is mounted to the ESP.

Kloxar wrote:

https://imgur.com/a/3FgtcCN

The kernel modules can't be loaded because the initramfs doesn't match the installed kernel. This would suggest that you didn't have your /boot partition mounted correctly when you were in the chroot running mkinitcpio. Use the findmnt command from the chroot to check.

Please list the exact, verbatim commands you used to attempt your fix.

Kloxar · 2020-03-09 06:05:09

Sorry for ghosting you over the weekend, but i had an issue come up that required my full and immediate attention

I tried

grub-install --target=x86_64-efi --efi-directory=/boot --bootloader-id=GRUB

then

pacman -S linux

I was simply going to run mkinitcpio -p linux but I decided to go install the kernel again just to be sure.

I rebooted, and, now, everything is working!

I'm very thankful for the help. I would have never figured it out on my own, but now I'm concerned it could happen in the future. Even though I now know how to fix it, I (of course) don't want to go through this every time I update my system.
Is there something I should check before closing this issue (or should I open a new one asking how to prevent this)?

V1del · 2020-03-09 09:07:23

Ensure that that /boot partition you've mounted from the chroot stays mounted on the real system via persistent entry in your fstab.

Wild Penguin · 2020-03-09 13:22:31

Also, you may want to make the mount point of the EFI partition read-only (has been suggested elsewhere). That way, an upgrade attempt will spit out an error message if the EFI partition is not mounted. This can be a rough protection / reminder for the absent-minded (like me).

Pacman (or mkinitcpio) does no check whatsoever whether things are mounted properly (it would be difficult to implement, because there are many valid configurations) but will just write the files, if they can.

Last edited by Wild Penguin (2020-03-09 13:23:10)

Arch Linux

#1 2020-03-04 07:49:39

Failed to find /boot after update [pacman -Syu]

#2 2020-03-04 08:33:08

Re: Failed to find /boot after update [pacman -Syu]

#3 2020-03-04 08:38:17

Re: Failed to find /boot after update [pacman -Syu]

#4 2020-03-04 08:49:18

Re: Failed to find /boot after update [pacman -Syu]

#5 2020-03-04 17:56:10

Re: Failed to find /boot after update [pacman -Syu]

#6 2020-03-04 20:03:26

Re: Failed to find /boot after update [pacman -Syu]

#7 2020-03-05 06:15:42

Re: Failed to find /boot after update [pacman -Syu]

#8 2020-03-05 06:55:58

Re: Failed to find /boot after update [pacman -Syu]

#9 2020-03-05 07:12:48

Re: Failed to find /boot after update [pacman -Syu]

#10 2020-03-05 07:23:25

Re: Failed to find /boot after update [pacman -Syu]

#11 2020-03-05 12:11:27

Re: Failed to find /boot after update [pacman -Syu]

#12 2020-03-05 12:26:19

Re: Failed to find /boot after update [pacman -Syu]

#13 2020-03-05 19:20:57

Re: Failed to find /boot after update [pacman -Syu]

#14 2020-03-06 04:53:47

Re: Failed to find /boot after update [pacman -Syu]

#15 2020-03-06 07:57:51

Re: Failed to find /boot after update [pacman -Syu]

#16 2020-03-06 19:58:00

Re: Failed to find /boot after update [pacman -Syu]

#17 2020-03-07 00:03:22

Re: Failed to find /boot after update [pacman -Syu]

#18 2020-03-09 06:05:09

Re: Failed to find /boot after update [pacman -Syu]

#19 2020-03-09 09:07:23

Re: Failed to find /boot after update [pacman -Syu]

#20 2020-03-09 13:22:31

Re: Failed to find /boot after update [pacman -Syu]

Board footer