EFI/refind boot- how to get kernel vmlinuz in /boot/efi/EFI/arch?

jantman · 2013-09-15 15:02:05

I have a MacBook Pro Retina (10,2) running arch. It EFI boots using rEFInd, booting from /boot/efi/EFI/arch. This problem has been happening since I first installed the system in July - when I use pacman to update to the latest kernel, the mkinitcpio hooks generate for the current kernel, not the one that was just installed. If I don't manually run mkinitcpio after the update, I'm left with an unbootable system (the ususal "Unable to find root device" error, and dumped to recovery shell) because of a kernel/initrd mismatch (I think).

So I think the problem isn't mkinitcpio, but rather that the "linux" package is installing the kernel at /boot/vmlinuz-linux (which I later manually copy to /boot/efi/EFI/arch/vmlinuz-linux), but mkinitcpio is looking in the correct path (/boot/efi/EFI/arch) for the kernel... which is the last kernel, since I haven't copied it there yet.

As an example (yeah, I know I went a LONG time between updates, mainly due to nvidia issues), yesterday I did a full update that bumped me from 3.9.8-1 to 3.10.10-1. But here's the mkinitcpio output during the pacman run:

upgrading linux...
>>> Updating module dependencies. Please wait ...
>>> Generating initial ramdisk, using mkinitcpio.  Please wait...
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
  -> -k /boot/efi/EFI/arch/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/efi/EFI/arch/initramfs-linux.img
==> Starting build: 3.9.8-1-ARCH
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
==> ERROR: module not found: `usbhid'
  -> Running build hook: [fsck]
  -> Running build hook: [resume]
==> WARNING: No modules were added to the image. This is probably not what you want.
==> Creating gzip initcpio image: /boot/efi/EFI/arch/initramfs-linux.img
==> WARNING: errors were encountered during the build. The image may not be complete.
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'fallback'
  -> -k /boot/efi/EFI/arch/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/efi/EFI/arch/initramfs-linux-fallback.img -S autodetect
==> Starting build: 3.9.8-1-ARCH
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
==> ERROR: module not found: `usbhid'
  -> Running build hook: [fsck]
  -> Running build hook: [resume]
==> WARNING: No modules were added to the image. This is probably not what you want.
==> Creating gzip initcpio image: /boot/efi/EFI/arch/initramfs-linux-fallback.img
==> WARNING: errors were encountered during the build. The image may not be complete.
error: command failed to execute correctly

Is there any solution to this problem? Because of the nature of rEFInd, I can't (don't think I can? I remember trying it and having problems) just symlink /boot/efi/EFI/arch/vmlinuz-linux to /boot/vmlinuz-linux.

Any ideas/help would be greatly appreciated.

Here's the uncommented lines in /etc/mkinitcpio.conf:

MODULES=""

BINARIES=""

FILES=""

HOOKS="base udev autodetect modconf block filesystems keyboard fsck resume"

And from /etc/mkinitcpio.d/linux.preset, which I manually specify on the command line (-p linux) when I re-run mkinitcpio after copying vmlinuz into place:

ALL_config="/etc/mkinitcpio.conf"
ALL_kver="/boot/efi/EFI/arch/vmlinuz-linux"

PRESETS=('default' 'fallback')

default_image="/boot/efi/EFI/arch/initramfs-linux.img"

fallback_image="/boot/efi/EFI/arch/initramfs-linux-fallback.img"
fallback_options="-S autodetect"

Is the right solution just changing ALL_kver to /boot/vmlinuz-linux and then copying that into /boot/efi/EFI/arch after the install, or is there some nicer solution so that this can actually be an automatic process?

Thanks for any advice/tips/suggestions.

PS - I'm running arch on the Retina with KDE. All of my configuration after the initial working install is handled by Puppet, and the module I use (along with the initial instructions and a list of what is and isn't working so far) is available at https://github.com/jantman/puppet-archl … bookretina

srs5694 · 2013-09-16 17:20:46

Depending on your partition setup, the simplest solution is likely to be to install an EFI driver for whatever filesystem holds your kernel. For instance, if you're not using LVM, and if you don't have a separate /boot partition, and if you're using ext4fs on your root (/) filesystem, then you should install the EFI ext4fs driver that comes with rEFInd. See the rEFInd documentation page on drivers for more information on their installation and use. Once the driver is installed, rEFInd should be able to see and use the kernel directly from where it lives. You'll have to either copy the refind_linux.conf file you're using on the ESP to where the kernel lives or adjust your manual boot stanza to point to the volume and directory where the kernel is rather than where it is on the ESP. This solution means that you won't have to copy the kernel any more, which should bypass the problem you're experiencing.

WonderWoofy · 2013-09-16 17:36:00

I'll just throw in here that I have been using srs5694's btrfs driver for his rEFInd, and it works fabulously. I have started to mount my ESP at /boot/efi so that I can include the kernel and initramfs in the snapshots of my system. The btrfs driver allows me to roll back to a previous rootfs state with a matching kernel and initramfs, neat!

But I do also have a number of other bootloaders and boot managers set up on my system. So for those I still do indeed have the kernel and initramfs copied to the ESP. For this I use a systemd.path that launches a oneshot systemd.service. I have the systemd.path use a "PathChanged=/boot/initramfs-linux-fallback.img" as the order of events when a new kernel is installed is that the kernel is written, then mkinitpcio uses the "file" command to determine the necessary version info. Then the initramfs-linux.img is created and written, then the initramfs-linux-fallback.img is created and written.

I use the PathChanged option because I do not mv the files, but rather cp them. So if I were to use a PathExists or PathExistsGlob, then it would try to run all the time. It is only when the actual contents of the file are modified that I want this event to occur.

So far, I have had no trouble with this setup. In the past I used to mv instead of cp, and I did run into trouble once or twice. But now with this method, I am able to actually fall back to the kernel and initramfs on the btrfs rootfs with rEFInd, since those will definitely be the correct kernel and initramfs version to match what is in /usr/lib/modules.

mikeserv · 2013-09-29 00:11:18

I don't like constant mv or cp solutions. I also don't like the special systemd services available. I spent a lot of time at rodsbooks last year when trying to figure out the UEFI thing and I eventually came up with a really easy solution: no real /boot folder at all.

First I create an exceptionally large ef00 type EFI system partition named esp of about 5gbs or so formatted as FAT32. It doesn't have to be that big - I've never used anywhere near the whole disk - but I've got space to spare and it might make more sense in a moment.

Then all kernel configs/OSs get their own folder on the esp at /EFI/boot/<system name> - like /EFI/boot/arch64-laptop for instance. Refind will automatically detect any new EFI bootable kernel installation there by default, though, as you'll see below, the kernels will need at least one parameter passed to them.

Anything that might go in a distro's /boot folder is moved into its respective /EFI/boot/<system name> folder instead.

On the root partition I create two empty folders: /esp and /boot one time.

I mount the EFI system partition to /esp and then bind mount the target system's /esp/EFI/boot/<system name> to /boot. This gets around the FAT32 partition's refusal to handle symlinks neatly.

Like this:

mount -L esp /esp
mount --bind /esp/EFI/boot/arch64-laptop /boot

The great thing about this is once you incorporate it into your /etc/fstab you never have to think about it again. It requires no special tools or scripts other than those already provided by every distro I've tried. It can be setup just once at system install.

## /etc/fstab

LABEL=arch64-laptop_rootfs / ext4 defaults 0 0 
LABEL=esp /esp vfat defaults 0 0 
/esp/EFI/boot/arch64-laptop /boot none defaults,bind 0 0

All kernel parameters are configured in /EFI/boot/<system name>/refind.conf - like the now necessary root=LABEL=arch64-laptop_rootfs for instance. All /boot folders are centrally located on the /esp/EFI/boot and boot the same with either qemu or on baremetal.

In fact, with the addition of an initially configured grub2 chainloaded from refind (or just grub2, I suppose, but I like refind) and a single BIOS boot partition I see no reason why any capable OS couldn't transparently be booted in either EFI or BIOS mode with no further configuration, though I've never had occasion to try.

The end result is a system that works exactly as any other would with no special consideration given to updates or kernel compilation with the exception of an extra /esp folder.

-Mike

srs5694 · 2013-09-30 17:20:07

If you're booting just one Linux distribution (especially Arch), mounting the ESP at /boot will work about as well as what you suggest, and will be simpler. The big problem with this is that some distributions refuse to let you use a FAT partition on /boot, and a subset of those actually use symbolic links within /boot. Arch isn't so fussy, though.

If you're using multiple distributions, using either GRUB or the combination of rEFInd and its EFI drivers can work well. With this configuration, you can put your kernel in a Linux filesystem and GRUB or rEFInd will be able to load it from there. This solution won't work with gummiboot, though, and it will work with ELILO or SYSLINUX only with some additional hoop-jumping.

mikeserv · 2013-09-30 22:10:17

Thanks, Rod! I agree, a simple mount -L esp /boot configuration is definitely ideal for a single installation system. That was what I first tried when I first installed my UEFI board. But I keep multiple installations on disk, and multiple configs per install, so I would often run into issues with installations interfering with one another's /boot files.

I also tried unique file names per installation - like prepending installation identifiers to filenames - but that required changing defaults at install, /boot monitor scripts and services and of course I would occasionally forget to implement it for an installation and be right back at square one. It was a headache.

Eventually I settled on the solution I described above because it was simply implemented, portable without modification, provided me a sane /boot tree, seemed to play nicely with default options across the board, and, well, because it was the best I could come up with. I figured that UEFI had already imposed the mandatory FAT32 partition on me so I might as well use it for all it was worth. I mean, even Windows fits in right along with the rest. This way I could interface with UEFI and linux bootloaders in the pre-UEFI customary way with simple configuration edits in /etc/fstab and in /boot, thanks mostly, of course, to rEFInd and what I learned reading rodsbooks.

I realize that I'm probably in violation of the GNU filesystem standard with the root-level /esp mount, but, to me, it seemed the intuitive place to put it. And frankly, I really don't know any better. Perhaps it belongs in /var/run or something? I'm fully confident GNU will have resolved a standard mount point for it once and for all sometime in the next decade or so.

Still, as you say, esp as /boot is ideal. Creshal very elegantly described it on the UEFI Bootloaders wiki Talk page like this:

efibootmgr -c -l /vmlinuz-linux -u "$(cat /proc/cmdline)"

Super simple, assuming an esp at /dev/sda1. He doesn't mention the necessary /boot entry in /etc/fstab or the various possible issues with efibootmgr/UEFI NVRAM though, which is of course perfectly acceptable in Talk. Still, I mentioned these things in reply to him there and described my solution like this:

$ mkdir -p /esp; mount -L {,/}esp 
$ mkdir -p /esp/EFI/boot/<NEW_SYSTEM>/; mv /boot/* $_; mount --bind $_ /boot 
$ cat <<'EOF' >/etc/fstab
LABEL=<NEW_ROOTFS> / <FSTYPE> defaults 0 0
LABEL=esp /esp vfat defaults 0 0
/esp/EFI/boot/<NEW_SYSTEM> /boot none defaults,bind 0 0
EOF 
$ sed -ri 's/root=[^ ]*/root=LABEL=<NEW_ROOTFS>/g' /boot/refind_linux.conf

Probably that code works, though it's entirely possible I've stumbled on syntax somewhere as I haven't tested it as written. It is admittedly less elegant than Creshal's proffered solution, but it includes an operable /etc/fstab, skeleton that it is, and edits your refind_linux.conf to boot. Assuming gpt partition labels and a rEFInd install it should do the job with very few required modifications if any aside from replacing the <VAR> strings and adding /etc/fstab entries as appropriate.

I am troubled, though, by the possible incompatibilities you mention with a vfat filesystem mounted at /boot and default symlinking behavior there. Which distributions behave this way? I don't think I've encountered it, though my default install is arch so I suppose it's possible I've just overlooked it.

I ask mainly because I've also added this to the wiki as a Sync EFIStub Kernel option and would like to highlight any inherent incompatibilities there or remove it if necessary, especially considering that I've already so boldly stated: "The following method should work similarly with any distribution or any combination thereof." Guess I jumped the gun on that one, huh?

Anyway, thanks again for rEFInd and your considerable work on rodsbooks.

-Mike

Last edited by mikeserv (2013-09-30 23:04:51)

srs5694 · 2013-10-01 13:49:55

mikeserv wrote:

I am troubled, though, by the possible incompatibilities you mention with a vfat filesystem mounted at /boot and default symlinking behavior there. Which distributions behave this way? I don't think I've encountered it, though my default install is arch so I suppose it's possible I've just overlooked it.

I'm not positive, but I believe that Ubuntu complained about being unable to upgrade kernels when I changed /boot to be the ESP's mount point. (Certainly I've seen such complaints in something. If it wasn't Ubuntu, it was probably either Debian or Fedora.) OpenSUSE definitely uses symbolic links in /boot (/boot/vmlinuz points to the current kernel, and there are also links for the initrd and GRUB 2 directory). Fedora, Ubuntu, and I believe OpenSUSE all refuse to accept a FAT partition as /boot in their setup programs, but of course this can be worked around after the fact by anybody who knows anything at all about /etc/fstab.

Overall, I'd say that using the ESP for a shared (cross-distribution) /boot partition, even with bind-mounting as you describe, is not a good solution -- at least, not unless or until other distributions begin officially supporting FAT for /boot. This might happen if the Freedesktop.org boot loader specification gains traction, but that specification includes file- and directory-naming features that would obviate the need for the bind mounting you're suggesting. For Arch specifically, mounting the ESP at /boot (with or without bind-mounting) is fine, though.

mikeserv · 2013-10-02 19:14:53

Thanks, again. I updated the Notes and Warnings section in my wiki entry accordingly and included an abbreviated quote attributed to rEFInd's author linked through to your post above. If you believe it warrants further editing only say so and I'm happy to do it, and you (and anyone else with a mind to, for that matter) should of course feel free to edit it as you like.

I'm curious, though, have you ever attempted to load, say, Windows' own bootmgfw.efi from a partition formatted with a Linux filesystem after loading one of your drivers? Recent research has led me to believe that incorporating such drivers directly in system flash is a very obtainable goal. Because I have a Gigabyte board which incorporates their secondary DualBIOS BIOS backup flash module, I believe experimenting should pose little risk. I think I'll try marking a partition as ef00, formatting as... probably ext2 would avoid possible complications with filesystem journaling, then rolling the appropriate driver into my ROM image and see if I can boot to it.

-Mike

srs5694 · 2013-10-02 19:26:37

No, I've never tried loading bootmgfw.efi from anything but a FAT filesystem.

mikeserv · 2013-10-02 19:30:20

Well, I'll be sure to let you know how it goes as soon as I muster the courage to see it through.

Hydranix · 2013-10-03 18:21:30

The easiest way for me was to make a script that wraps mkinitcpio

This is was I use... I'm sure you can build one that suits your purposes though.

This script also DOES work when pacman upgrades the kernel and runs mkinitcpio.

/usr/local/bin/mkinitcpio

#!/bin/bash
# Hnx
# Mount EFI partition first as kernel modules can fuck up after an upgrade
sudo mount /dev/dm-4 /boot/EFI

# Absolute path needed
sudo /usr/bin/mkinitcpio -p linux

# If the last command returns anything but 0 don't continue
if [ "$?" = "0" ]; then

	echo 'Copying initramfs and kernel to UEFI partition...'
	if [ -f /boot/EFI/linux.efi ]; then
		sudo cp /boot/vmlinuz-linux /boot/EFI/linux.efi
		return 0
	else
		cat << "EOF"
The kernel isn't on the EFI partition!! Is it mounted???
Copy the kernel over manually then try again to assure
that this script works correctly...
EOF
		return 1
	fi
# Likewise, stop the script if something went wrong...
	if [ "$?" = "0" ]; then
		echo 'Kernel copied succesfully...'
		sudo cp /boot/initramfs-linux.img /boot/EFI/initramfs.img
		echo 'initramfs copied successfully...'
	else
		echo 'Kernel did not copy correctly...'
		return 1
	fi
	beep -f 475
	return 0
fi
# Sleep for a second to lets filesystems sync or whatever (you might need more time, I use SSD RAID-0)
sleep 1
sudo umount /boot/EFI
beep -f 478 -r 2 -l 45

mikeserv · 2013-10-03 20:21:29

Well, yes, the program specific scripts are certainly one way to go with it, but I've always tried to avoid them because of how different they can be across installations. They're also prone to failure if the program they support should change in a crucial way at update. The little bind-mount hack I mentioned is much less likely to do so because that would pretty much require a breaking update in /etc/fstab and/or mount.

It's also only a few lines of code, needs implementing only once at install, and I really never have to think about it again. For instance, putting aside the multi-install single-system argument for a moment, I also work on more than computer. I have a desktop, also incorporating a sort of SSD-RAID configuration, if a little unusual, and I have a laptop, which does not. Boot config is nearly identical with one considerable caveat:

When I read bcache was merged in the kernel a few months ago I decided to pair two 3TB Western Digital Greens with a 128GB Kingston SSD. Because the bulk storage is largely used for video of which the best has a bitrate that easily fits within mechanical disks' limits, I convinced myself that I should be able to achieve all the gains of SSD performance without its size/cost constraints with bcache. I hadn't jumped for SSD before because it was so expensive, and keeping several different disks/partitions in a system often translates to frequent menial file-system restructuring to allow myself space because I'm a sloppy rule-keeper.

Anyway, though it did eventually result in considerable performance gains, it was not an easy endeavor. The developers hadn't considered Arch Linux at that point and the tested init systems for assembling the bcache RAID array did not play nicely with Arch. Still, a couple weary days in IRC later (probably it only took so long because I'm pretty dense), a set of udev rules and their install hook (barely) adapted from mdadm_udev, and I had a ~6TB btrfs software RAID running RAID-0 for data and RAID-1 for meta-data transparently and algorithmically cached to 128GBs of bcache managed SSD flash. It was cool.

Apologies - I so rarely get to toot my own horn like that - but my point is that the process involved many aborted attempts, which means many reinstalls, repartitions, and reformats because this bcache array is my boot device and my esp is located on the first ~5GBs of SSD before the rest is partitioned off and handed over to bcache; I boot from and to these disks before and after the RAID is assembled. Throughout all of my (dense) troubleshooting, I never once had an issue with UEFI or the esp's location or file-format, neither did it ever require any other special configuration or consideration to adapt it to the circumstances.

Despite their different boot device arrangements, thanks to rEFInd and Linux conventions such as /etc/fstab both my laptop and desktop require only the same, one-time, UEFI-specifc installation setup.

-Mike

P.S. You might like to look at the bcache wiki page too. When I was trying to put it all together most of the instructions I could find involved enumerating the disks with a scripted for loop at boot, some echo 1 > /proc/... calls, and random advice on how long to sleep to avoid stepping on/getting stepped on by udev as it finds and adds the disks to the device tree. It was much faster and much more reliable just to have udev add each disk to the array as soon as it found them, as I found was done by the mdadm_udev hook. Instructions for including the 61-bcache.rules automatically via the adapted mdadm_udev install hook are in installation step 7b of the bcache wiki page and the original hook is likely already among your mkinitcpio modules on your /.

Last edited by mikeserv (2013-10-03 21:56:34)

Arch Linux

#1 2013-09-15 15:02:05

EFI/refind boot- how to get kernel vmlinuz in /boot/efi/EFI/arch?

#2 2013-09-16 17:20:46

Re: EFI/refind boot- how to get kernel vmlinuz in /boot/efi/EFI/arch?

#3 2013-09-16 17:36:00

Re: EFI/refind boot- how to get kernel vmlinuz in /boot/efi/EFI/arch?

#4 2013-09-29 00:11:18

Re: EFI/refind boot- how to get kernel vmlinuz in /boot/efi/EFI/arch?

#5 2013-09-30 17:20:07

Re: EFI/refind boot- how to get kernel vmlinuz in /boot/efi/EFI/arch?

#6 2013-09-30 22:10:17

Re: EFI/refind boot- how to get kernel vmlinuz in /boot/efi/EFI/arch?

#7 2013-10-01 13:49:55

Re: EFI/refind boot- how to get kernel vmlinuz in /boot/efi/EFI/arch?

#8 2013-10-02 19:14:53

Re: EFI/refind boot- how to get kernel vmlinuz in /boot/efi/EFI/arch?

#9 2013-10-02 19:26:37

Re: EFI/refind boot- how to get kernel vmlinuz in /boot/efi/EFI/arch?

#10 2013-10-02 19:30:20

Re: EFI/refind boot- how to get kernel vmlinuz in /boot/efi/EFI/arch?

#11 2013-10-03 18:21:30

Re: EFI/refind boot- how to get kernel vmlinuz in /boot/efi/EFI/arch?

#12 2013-10-03 20:21:29

Re: EFI/refind boot- how to get kernel vmlinuz in /boot/efi/EFI/arch?

Board footer