You are not logged in.

#1 2013-03-05 18:59:40

tobsen
Member
Registered: 2011-10-13
Posts: 37

Pacman complains after Gummiboot upgrade

Hi,

just updated to gummiboot 24-1 which replaced gummiboot-efi. After installation pacman complained:

File system /boot is not a FAT EFI System Partition (ESP) file system.
Fehler: Befehl konnte nicht korrekt ausgeführt werden (Error: Command not executed)

I just wonder why gummiboot assumes /boot to be a FAT ESP? I mount my ESP at /boot/efi, because I thought this is common practice. Furthermore which command was not executed? Anyhow system is booting normal, so not a big deal.

Last edited by tobsen (2013-03-05 19:00:27)

Offline

#2 2013-03-05 19:15:09

65kid
Member
From: Germany
Registered: 2011-01-26
Posts: 663

Re: Pacman complains after Gummiboot upgrade

The new gummiboot version includes an installer which tries to install gummiboot on your ESP and into the EFI bootloader. And yes, gummiboot assumes the ESP to be mounted on /boot, so the install command will obviously fail in your case. You would have to manually call "gummiboot --path=/boot/efi/ install". Or do everything by hand as before...

IMHO mounting the ESP on /boot makes perfect sense and I never understood why some people mount in on /boot/efi, I actually opened an discussion about this particular issue a few days ago on the UEFI wiki talk page.

Offline

#3 2013-03-05 20:56:12

srs5694
Member
From: Woonsocket, RI
Registered: 2012-11-06
Posts: 719
Website

Re: Pacman complains after Gummiboot upgrade

65kid wrote:

IMHO mounting the ESP on /boot makes perfect sense and I never understood why some people mount in on /boot/efi, I actually opened an discussion about this particular issue a few days ago on the UEFI wiki talk page.

I don't claim to be "in" on the history of it, but my suspicion is that it has to do with certain common practices with respect to /boot. Most notably, /boot has traditionally been either an ordinary directory on the Linux root (/) filesystem or a Linux-only partition that uses a Linux-specific filesystem. In fact, many distributions' installers won't let you use FAT on a /boot partition. (Other distributions are relevant because use of /boot/efi as a mount point for the ESP is common across multiple distributions.) In fact, it's not uncommon to find symbolic links on /boot, which you can't use if that partition is FAT. Thus, using the ESP (which must use FAT) for /boot is not a perfect fit with the way /boot is often used.

Another concern is with integrating everything with both the EFI and with other OSes. Cluttering the cross-OS ESP with Linux-specific files is something that really should give a distribution maintainer pause. What happens when somebody installs an OS that expects the ESP's root directory to be empty? (AFAIK, no such OS exists, but you'd have to be crazy to assume no such OS will ever exist.) What about a computer that holds multiple Linux distributions? If they all mount a single ESP at /boot, it will become harder to distinguish one distribution's kernels from another's, and in fact one distribution might overwrite another's kernels. You and I as individuals may be able to handle this, especially if we know what we're doing, but a stupid installer program might easily trip up in such cases, and a newbie might be unable to recover. It's also conceivable that another OS might trash the ESP, either because of a filesystem driver bug or because of a poorly-designed installer or other system tool. Until version 12.04, Ubuntu's installer did precisely this, so if you were using the ESP as /boot in Arch and then decided to dual-boot with Ubuntu 11.10, you'd have ended up with no kernel for Arch, which would complicate recovery. (It'd be bad enough even with /boot elsewhere, but worse with the ESP as /boot.)

A related concern is disk space. The Windows installer creates a 100MiB ESP. That's big enough to hold several boot loaders, but it could easily fill up if a distribution like Fedora were to mount it at /boot, since Fedora's kernels are about 5MiB and its initrd files are about 20MiB. Thus, four Fedora kernels and initrd files will completely fill the ESP. I've seen Linux distributions create even smaller ESPs than what Windows creates, which would make this problem even worse.

For all of these reasons, mounting the ESP at /boot is a bit risky as a matter of distribution policy. It can work fine if you know what you're doing, and I even agree that it has advantages in this case. If I were maintaining the installers for Fedora or Ubuntu, though, I wouldn't change them to mount the ESP at /boot. Arch doesn't have such a point-and-click installer, though; for it, the issue is more one of documentation. Suggesting use of /boot as a mount point for the ESP in the documentation is OK, IMHO, if the documentation goes into sufficient detail about the advantages and disadvantages of doing that vs. mounting the ESP elsewhere (/boot/efi being the most common alternative).

Offline

#4 2013-03-05 21:38:19

65kid
Member
From: Germany
Registered: 2011-01-26
Posts: 663

Re: Pacman complains after Gummiboot upgrade

srs5694 wrote:

Another concern is with integrating everything with both the EFI and with other OSes. Cluttering the cross-OS ESP with Linux-specific files is something that really should give a distribution maintainer pause. What happens when somebody installs an OS that expects the ESP's root directory to be empty? (AFAIK, no such OS exists, but you'd have to be crazy to assume no such OS will ever exist.) What about a computer that holds multiple Linux distributions? If they all mount a single ESP at /boot, it will become harder to distinguish one distribution's kernels from another's, and in fact one distribution might overwrite another's kernels. You and I as individuals may be able to handle this, especially if we know what we're doing, but a stupid installer program might easily trip up in such cases, and a newbie might be unable to recover. It's also conceivable that another OS might trash the ESP, either because of a filesystem driver bug or because of a poorly-designed installer or other system tool. Until version 12.04, Ubuntu's installer did precisely this, so if you were using the ESP as /boot in Arch and then decided to dual-boot with Ubuntu 11.10, you'd have ended up with no kernel for Arch, which would complicate recovery. (It'd be bad enough even with /boot elsewhere, but worse with the ESP as /boot.)

Yeah, the situation seems to be kind of a mess at the moment, but from my understanding, using a single ESP across several linux distributions and even operating systems is supposed to be one of the advantages of EFI. Yes, there may be buggy installers at the moment, but this will hopefully be sorted out in time. And the fact that multiple distributions may clutter the ESP root and overide each others kernels is actually something that is being worked on with the kernel-install tool that wil be introduced with the next systemd version. I haven't looked at it in detail, but the idea seems to be that kernel is actually installed somewhere in /usr/ by the package manager and the kernel-install tool is then called and copies the kernel into an distribution-specific directory on the ESP, therefore not cluttering the ESP root. And according to tomgun, Arch will hopefully follow this standard.

srs5694 wrote:

A related concern is disk space. The Windows installer creates a 100MiB ESP. That's big enough to hold several boot loaders, but it could easily fill up if a distribution like Fedora were to mount it at /boot, since Fedora's kernels are about 5MiB and its initrd files are about 20MiB. Thus, four Fedora kernels and initrd files will completely fill the ESP. I've seen Linux distributions create even smaller ESPs than what Windows creates, which would make this problem even worse.

Well, if you want to use multiple operating systems, you should keep something like this in mind, I'm sure Fedora allows you to make the partition bigger (and if I remember correctly, at least Windows 7 allowed this as well). Using separate ESPs instead is not the solution, it just makes stuff more messy and complicated. If you currently mount the ESP to /boot/efi you have to manually copy the kernel and initramfs back to the ESP after every kernel upgrade. Yes, you can automate this with inotify, systemd path units etc., but these are just ugly hackish workarounds.

Offline

#5 2013-03-06 15:15:57

srs5694
Member
From: Woonsocket, RI
Registered: 2012-11-06
Posts: 719
Website

Re: Pacman complains after Gummiboot upgrade

65kid wrote:

If you currently mount the ESP to /boot/efi you have to manually copy the kernel and initramfs back to the ESP after every kernel upgrade. Yes, you can automate this with inotify, systemd path units etc., but these are just ugly hackish workarounds.

No, you can load the kernel from somewhere other than the ESP, even if you're using the EFI stub loader as the boot loader. You must have either a non-ESP FAT (or HFS+ on Macs) /boot partition or an EFI driver for the filesystem you use on /boot. Thereafter, you can load the kernel directly from /boot. One important caveat, though, is that the last time I checked, gummiboot required loading the kernel from its own partition, which complicates such configurations if you want to use gummiboot. Also, gummiboot provides no means to load filesystem drivers. (I haven't checked the latest versions of gummiboot, though; one or both of these limitations may have been rectified.) rEFInd is much more flexible in this respect; it can automatically load filesystem drivers and it can launch boot loaders from any partition. Thus, with rEFInd rather than gummiboot, you can easily mount the ESP at /boot/efi (or anywhere else, or not at all except for ESP maintenance), put your kernel in /boot, and not bother copying it. Similar comments apply to GRUB, as well.

Offline

#6 2013-03-06 15:33:28

65kid
Member
From: Germany
Registered: 2011-01-26
Posts: 663

Re: Pacman complains after Gummiboot upgrade

my bad, didn't know about this. In this case mounting on /boot/efi makes sense. But when at some point the kernel-install tool allows you to specify where the kernel is installed, putting the kernel on the ESP is imho much simpler than letting the bootloader load a filesystem driver - this is something that the kernel itself should do. This is also probably why gummiboot doesn't support this. gummiboot is supposed to be as simple as possible - loading a filesystem driver certainly isn't and the kernel already does it and knows how to do it best.

Offline

#7 2013-03-06 19:00:31

tobsen
Member
Registered: 2011-10-13
Posts: 37

Re: Pacman complains after Gummiboot upgrade

I totally agree with srs5694 on this point. With respect to different corner cases ESP should be on /boot/efi. Furthermore this package is in conflict with the information I got from the wiki.

Offline

#8 2013-03-06 21:00:04

srs5694
Member
From: Woonsocket, RI
Registered: 2012-11-06
Posts: 719
Website

Re: Pacman complains after Gummiboot upgrade

65kid wrote:

putting the kernel on the ESP is imho much simpler than letting the bootloader load a filesystem driver - this is something that the kernel itself should do.

The filesystem driver to which I referred is an EFI filesystem driver, not a Linux filesystem driver. This enables the EFI to read the kernel, and therefore to run its EFI stub loader. I agree that it is simpler to put the kernel on a FAT filesystem. The point, though, is that the Linux kernel has historically resided on Linux-specific filesystems. Although this isn't absolutely required, there are valid reasons to keep it on Linux-specific filesystems, such as providing a layer of protection against accidental damage from a non-Linux OS and enabling use of symbolic links to help manage the kernel, its initrd file, and other related files. IMHO, the desirability of placing the kernel on a Linux-specific filesystem vs. a non-ESP FAT filesystem vs. the ESP varies from one installation to another, so it's impossible to make a statement about what's best in an absolute sense; however, the method that's likely to cause the most problems for distribution maintainers is putting the kernel on the ESP. That said, Arch users tend to be more capable system administrators than, say, Ubuntu users, and Arch uses less in the way of do-it-all configuration tools. This makes mounting the ESP at /boot a more viable option for Arch than for most distributions -- but even for Arch, I wouldn't recommend it in every case.

Offline

#9 2013-03-07 09:06:54

tomegun
Developer
From: France
Registered: 2010-05-28
Posts: 661

Re: Pacman complains after Gummiboot upgrade

65kid wrote:

And according to tomgun, Arch will hopefully follow this standard.

Just to clarify, this should have read: "tomegun hopes Arch will follow this standard". It has not been discussed much and I have no idea if anyone will object to us adopting the scheme. I plan to suggest it though.

Offline

#10 2013-03-07 09:14:47

tomegun
Developer
From: France
Registered: 2010-05-28
Posts: 661

Re: Pacman complains after Gummiboot upgrade

srs5694 wrote:
65kid wrote:

If you currently mount the ESP to /boot/efi you have to manually copy the kernel and initramfs back to the ESP after every kernel upgrade. Yes, you can automate this with inotify, systemd path units etc., but these are just ugly hackish workarounds.

No, you can load the kernel from somewhere other than the ESP, even if you're using the EFI stub loader as the boot loader. You must have either a non-ESP FAT (or HFS+ on Macs) /boot partition or an EFI driver for the filesystem you use on /boot. Thereafter, you can load the kernel directly from /boot. One important caveat, though, is that the last time I checked, gummiboot required loading the kernel from its own partition, which complicates such configurations if you want to use gummiboot. Also, gummiboot provides no means to load filesystem drivers. (I haven't checked the latest versions of gummiboot, though; one or both of these limitations may have been rectified.) rEFInd is much more flexible in this respect; it can automatically load filesystem drivers and it can launch boot loaders from any partition. Thus, with rEFInd rather than gummiboot, you can easily mount the ESP at /boot/efi (or anywhere else, or not at all except for ESP maintenance), put your kernel in /boot, and not bother copying it. Similar comments apply to GRUB, as well.

gummiboot explicitly does not want to support reading from other partitions than the ESP. To me this makes a lot of sense as it means gummiboot can be extremely small and simple and does not need to contain a "mini OS", and I have so far not seen a compelling use-case for not installing kernel+initrd to the ESP.

Offline

#11 2013-03-07 10:08:47

tomegun
Developer
From: France
Registered: 2010-05-28
Posts: 661

Re: Pacman complains after Gummiboot upgrade

@srs5694: as you can probably tell I'd really like us to "mandate" that the ESP should be mounted at /boot (of course people can do whatever they want, even with the current gummiboot package, they just have to call the gummiboot installer manually with the correct parameter). That said, I'm very open to input if anyone has any use-cases where this is a problem.

There are a few issues we need to avoid (as you have touched upon). However, I think this is very possible. There is a proposed spec floating about which addresses most if not all of your concerns. To summarize:

* The ESP must be "big" (512MB might be a reasonable default). I don't see this as a problem at all, it is bigger than perhaps people typically use for /boot, but it is tiny in an absolute sense so resizing the ESP as part of distro installation should not be an issue.
* We need to avoid namespace clashes. We already have to deal with this on the ESP, so I don't see a problem with also doing proper namespacing of the rest of the files we install under /boot (see the spec I linked above for one way of doing it).
* Other OS's might create problems if we try to share /boot. That would be bad, and of course contrary to the intention of the UEFI spec (it allows storing things outside of /EFI, so OSs that can't deal with this should be fixed). That said, the problem of "installing a buggy second OS might wreck havoc" is not new, and even if we have separate /boot partitions a buggy installer can still mess things up... I guess my point is: bugs should be fixed, and if we need to work around them we will do that once they appear.
* FAT can't do symlinks. True, but not really a problem. With the right configuration file format and management of configuration files/kernel installation the use-cases for symlinks as a way to manage multiple kernels should be taken care of (see the above spec and how it allows wildcards for specifying the default entry).

Anything I left out? Any other concerns we should take into account?

Edit: I have been convinced that there is no generally safe way for resizing the ESP, and that if the ESP is too small to host kernels one would want to use a different boot loader from gummiboot.

Last edited by tomegun (2013-03-13 06:29:03)

Offline

#12 2013-03-07 18:43:14

srs5694
Member
From: Woonsocket, RI
Registered: 2012-11-06
Posts: 719
Website

Re: Pacman complains after Gummiboot upgrade

tomegun wrote:

* The ESP must be "big" (512MB might be a reasonable default). I don't see this as a problem at all, it is bigger than perhaps people typically use for /boot, but it is tiny in an absolute sense so resizing the ESP as part of distro installation should not be an issue.

I think you're brushing off the problems associated with a filesystem resize much too casually. This is especially true given the usual arrangement of partitions on a system with another OS already installed, in which a dual-boot configuration is planned. Such systems typically put the ESP first, or at least earlier than the OS partition. Thus, if you need to resize the ESP, that means resizing the existing OS partition from the start. This is the riskiest and most time-consuming type of partition resizing, and because of the risk factor, if it's done on a regular basis, it will result in trashed installations. IMHO, having users do this as the default or primary installation method is irresponsible.

There are at least two alternatives while still mounting the ESP at /boot:

  • Create a second ESP for Arch (and optionally shared with other OSes). The EFI spec explicitly allows this, so in terms of standards, this is perfectly viable. The trouble is that the Windows installer tends to flake out when it sees two ESPs, and Microsoft explicitly states that they don't support this configuration. Thus, IMO this is a poor option, at least as the primary supported procedure.

  • Create a larger ESP elsewhere on the disk, copy the original ESP's files to it, and delete or re-purpose the original ESP. This should work, but it wastes a bit more disk space and it makes the installation process vulnerable to problems should something go wrong with the file copying. You'll also need to either duplicate the original partition's GUID onto its replacement (and delete the original or change its GUID to avoid a conflict) or re-do all the existing NVRAM entries as part of the copying process.

If you insist on mounting the ESP at /boot by default, I'd suggest going with the second alternative if the original ESP isn't big enough; however, it's a big, ugly process. IMO, as a default, it's cleaner to use a Linux-specific /boot partition. It could be FAT, if you like. It could even host gummiboot, if the user chooses to use it, although I'm not sure how many ESPs support booting from a non-ESP FAT partition. (Some certainly do.) Mounting the ESP at /boot works well under certain conditions, but when designing a default OS installation procedure, you can't really assume a certain set of pre-existing conditions that benefit your desired configuration.

Offline

#13 2013-03-07 22:58:23

tomegun
Developer
From: France
Registered: 2010-05-28
Posts: 661

Re: Pacman complains after Gummiboot upgrade

@srs5694: Thanks for your input, much appreciated.

So there are three options to "resizing" the ESP: add a second one, add a second and delete the old or actually resize the old. The first option seems like the sanest default to me, but it is useful to know that there is one case where this would be problematic (and where perhaps one of the other options should be used instead). The conditions needed to trigger this are (as far as I can tell):

* The user can not control the creation of the ESP to set the desired size (i.e., it is from the factory).
* The ESP was created too small (where 'too small' depends on how many OS's and kernels you wish to install. The smallest ESP I have seen is 100MB, which would allow you to install at least four different kernels, but typically many more).
* The user can not simply delete the original ESP (i.e., you are not on a Mac).
* The user can not reinstall the original OS on the machine to set the correct ESP size.
* You want the ability to install Windows on the machine _after_ you install Linux.

So I suppose you are on a Windows machine, the default Windows ESP size of 100MB is not big enough, and you intend to install a future Windows version in parallel with the old one (which you cannot reinstall or remove).

This is definitely something we should make sure is possible, but I think it is reasonable that we don't optimize for this case, but ask the user to deal with it specially. Either by recreating or resizing the ESP, both of which are possible but cumbersome; or by mounting the ESP on /boot/efi and deal with this in the same way as they currently do (i.e., they won't get the benefits of future simplifications we make and might get the occasional warning).

Last edited by tomegun (2013-03-07 22:59:05)

Offline

#14 2013-03-08 01:27:30

srs5694
Member
From: Woonsocket, RI
Registered: 2012-11-06
Posts: 719
Website

Re: Pacman complains after Gummiboot upgrade

FWIW, I've now read the Freedesktop.org proposed spec that was referred to earlier. Under "Technical Details," that page lays out a set of if-then scenarios that's informed by many of the issues I've pointed out -- you can't rely on an existing ESP being big enough to hold lots of kernels, etc. Some of what I've stated explicitly in this thread is implicit in their rules, though. They've also made one important addition: In case there's an existing ESP that isn't big enough, what they call the $BOOT partition will have a new type code (bc13c2ff-59e6-4262-a352-b275fd6f7172). The end result is a notable lack of uniformity under this proposal. Most importantly:

  • Kernel files may be stored on the ESP or on this new type-bc13c2ff-59e6-4262-a352-b275fd6f7172 partition (let's call it the "Freedesktop partition" for simplicity's sake).

  • The Freedesktop partition may be FAT, ext2fs, ext3fs, or ext4fs. Anything but FAT would presumably cause serious problems for gummiboot.

  • No mount point for the Freedesktop partition is specified, although /boot is implied, particularly if you look at the comment in the example .conf file.

  • If the kernel goes on the ESP, then presumably it would get mounted at /boot.

It seems to me that this could result in more than a little confusion, particularly if different distributions create subtly different rules for when to create a new partition vs. use an existing one. You could end up with loader definitions split across two or more different partitions, for instance. This will be particularly problematic with boot loaders like gummiboot, which are tied to a single partition. Imagine, for instance, one distribution setting up gummiboot on the ESP; then another one comes along and its installer decides that the ESP is too crowded, so it sets up gummiboot on a separate Freedesktop partition. Now the user needs to use the firmware's boot manager to choose which version of gummiboot to run, which is exactly the sort of problem that the Freedesktop proposal is intended to avoid.

Now suppose that the existing ESP is too small so the installer creates a Freedesktop partition, puts the kernel there, and puts gummiboot there. Maybe this boots Linux fine -- but because gummiboot can't chainload to an OS with a loader on another partition, you can no longer boot anything with a boot loader on the ESP. Also, if there are EFI implementations that insist on booting from an ESP, such a configuration won't work for them, even if Linux (via gummiboot) is the only OS.

All in all, it looks like the Freedesktop proposal is full of holes. Maybe they'll be able to plug them, but for the moment I'd urge caution in jumping in that particular boat, since it doesn't look very seaworthy to me. Instead, it looks like a design intended to push gummiboot as a standard, but gummiboot's design limitations will come back to bite any serious attempt to use that proposal.

The simplest solution, it seems to me, is to go back to what the EFI designers seem to have intended: Put boot loaders and boot managers on the ESP and OS kernels elsewhere. (For this purpose, the Linux kernel with EFI stub doesn't count as a boot loader.) The boot loaders must be able to load kernels from non-ESP partitions -- but those partitions could certainly be FAT partitions, so as to be easily read by the EFI, if the OS wants to set things up that way. A standardized configuration file, placed on the ESP, could help Linux distributions cooperate, and even a shared (or non-shared) Freedesktop partition type code has merit.

Offline

#15 2013-03-08 02:43:16

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: Pacman complains after Gummiboot upgrade

tomegun wrote:

@srs5694: Thanks for your input, much appreciated.

So there are three options to "resizing" the ESP: add a second one, add a second and delete the old or actually resize the old. The first option seems like the sanest default to me, but it is useful to know that there is one case where this would be problematic (and where perhaps one of the other options should be used instead). The conditions needed to trigger this are (as far as I can tell):

* The user can not control the creation of the ESP to set the desired size (i.e., it is from the factory).
* The ESP was created too small (where 'too small' depends on how many OS's and kernels you wish to install. The smallest ESP I have seen is 100MB, which would allow you to install at least four different kernels, but typically many more).
* The user can not simply delete the original ESP (i.e., you are not on a Mac).
* The user can not reinstall the original OS on the machine to set the correct ESP size.
* You want the ability to install Windows on the machine _after_ you install Linux.

So I suppose you are on a Windows machine, the default Windows ESP size of 100MB is not big enough, and you intend to install a future Windows version in parallel with the old one (which you cannot reinstall or remove).

This is definitely something we should make sure is possible, but I think it is reasonable that we don't optimize for this case, but ask the user to deal with it specially. Either by recreating or resizing the ESP, both of which are possible but cumbersome; or by mounting the ESP on /boot/efi and deal with this in the same way as they currently do (i.e., they won't get the benefits of future simplifications we make and might get the occasional warning).

I am confused by this.

I should perhaps say that my working knowledge of Windows currently consists of an ability to find a file on a USB stick and open it. (Even that I had to have help with a couple of weeks ago because I didn't have a desktop shortcut to the file browser.) Oh, I also know how to log out or shut down.

But if the Windows installer will flake out if there is more than one ESP and if MS does not support that configuration, won't this also be a problem for anybody who needs to re-install Windows after installing an OS which decides to create a second ESP? As far as I can tell, Windows still requires re-installation at regular intervals - much more frequently than OS X or Linux. (My knowledge of this is somewhat current as the IT department have told me that Windows boxes slow down and need to be reinstalled at least annually. Weirdly they use Linux to install Windows which is the only time I understand what is going on.)

Or is this what the new type code would avoid? (Presumably firmware doesn't care about type codes but the Windows installer does?)


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#16 2013-03-08 18:09:04

srs5694
Member
From: Woonsocket, RI
Registered: 2012-11-06
Posts: 719
Website

Re: Pacman complains after Gummiboot upgrade

cfr wrote:

But if the Windows installer will flake out if there is more than one ESP and if MS does not support that configuration, won't this also be a problem for anybody who needs to re-install Windows after installing an OS which decides to create a second ESP?

Yes. That's why I recommend against creating two ESPs, at least on a computer that dual-boots with Windows.

Or is this what the new type code would avoid? (Presumably firmware doesn't care about type codes but the Windows installer does?)

A new type code should fix the problem with the Windows installer -- a FAT32 partition with anything but the ESP's type code (C12A7328-F81F-11D2-BA4B-00A0C93EC93B; identified as EF00 in gdisk or by a "boot flag" in parted) is not an ESP, and Windows should be fine with that. (I've not tried this configuration, so I'm not willing to promise that the Windows installer will work correctly in this case, though.) Unfortunately, I also can't be sure that all EFIs would boot from a boot loader on such a partition; some might restrict their boot loader choices to the ESP, since the EFI spec says that boot loaders should be on an ESP. That's the rub, really: Creating a second ESP works around the problem of an undersized existing ESP but causes problems with Windows; and putting a boot loader on a non-ESP partition is not guaranteed to work with all EFIs. Thus, the safest choice -- the one that's most likely to work with the greatest number of computers -- is to put the Linux boot loader (or a boot manager, at least) on the ESP, even if it's small, and put the kernel elsewhere. This won't work with gummiboot as currently written, though, and this limitation is part of the reason for the complex set of rules in the Freedesktop proposed boot loader spec. If gummiboot, like rEFInd, GRUB, and even (theoretically) ELILO, supported multiple partitions, the rules in Freedesktop spec could be simplified and would be more reliable. My concern is that if the Freedesktop proposal were to gain traction as currently written, it would create more problems than it solves, at least on EFI systems. (I've not given the BIOS side much thought.)

Offline

#17 2013-03-08 18:54:56

65kid
Member
From: Germany
Registered: 2011-01-26
Posts: 663

Re: Pacman complains after Gummiboot upgrade

srs5694 wrote:

Imagine, for instance, one distribution setting up gummiboot on the ESP; then another one comes along and its installer decides that the ESP is too crowded, so it sets up gummiboot on a separate Freedesktop partition.

This is something that I would consider a buggy installer. If the ESP is too crowded, an installer should tell the user that there is no space left on the ESP to install the distribution - end of story. Even if creating a separate ESP was a good idea, how would the installer even do that? It would have to shrink another partition first which is way too risky.

srs5694 wrote:

Thus, the safest choice -- the one that's most likely to work with the greatest number of computers -- is to put the Linux boot loader (or a boot manager, at least) on the ESP, even if it's small, and put the kernel elsewhere.

But putting the kernel elsewhere is crazy for a simple reason: what if my root including the kernel is on an encrypted drive, RAID, LVM etc.? Do you want to duplicate all that logic in the boot loader? No, that would be stupid, because the kernel already knows how to do all that stuff. That's exactly what gummiboot is about - do only what the boot loader needs to do, let the kernel deal with the rest.
And "the greatest number of computers" won't even care if their ESP is only 100MB, because they only use one OS. And whoever needs more should simply make the ESP big enough in the first place. Yes, I know Windows makes it a 100MB by default, but even that should be enough to dual boot for most people (considering that my /boot currently uses 8.4MB, including the kernel, initramfs and bootloader).

Offline

#18 2013-03-08 19:36:10

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,414

Re: Pacman complains after Gummiboot upgrade

65kid wrote:

<snip>
(considering that my /boot currently uses 8.4MB, including the kernel, initramfs and bootloader).

I take it this means that you have only the kernel and a single initramfs?  I have the Arch kernel, graysky's CK kernel and two initramfs' for each (the primary and fallback), and it takes ~45M. 

I don understand why srs5694 includes the filesystem drivers in his rEFInd, but also understand your point about having the boot manager be as simple as possible like gummiboot.  Personally I go for the simplicity of gummiboot, but from reading other threads where Rod discusses the ext4 support, it seems that the filesystem driver is optional and is only available if the necessary drivers are placed on the ESP.  So in that sense, that functionality is somewhat modular, unline grub2 where it just includes all the functionalities you listed above and more, whether the user likes/uses them or not.

Offline

#19 2013-03-08 19:59:14

65kid
Member
From: Germany
Registered: 2011-01-26
Posts: 663

Re: Pacman complains after Gummiboot upgrade

WonderWoofy wrote:
65kid wrote:

<snip>
(considering that my /boot currently uses 8.4MB, including the kernel, initramfs and bootloader).

I take it this means that you have only the kernel and a single initramfs?  I have the Arch kernel, graysky's CK kernel and two initramfs' for each (the primary and fallback), and it takes ~45M.

yes, I should have mentioned that I don't have a fallback image. I disabled it because I haven't used it a single time in using Arch for over 2 years. Well, add another 6MB if you want, doesn't really change my point. wink The initramfs would even be a little smaller if my root wasn't encrypted.

WonderWoofy wrote:

I don understand why srs5694 includes the filesystem drivers in his rEFInd, but also understand your point about having the boot manager be as simple as possible like gummiboot.  Personally I go for the simplicity of gummiboot, but from reading other threads where Rod discusses the ext4 support, it seems that the filesystem driver is optional and is only available if the necessary drivers are placed on the ESP.  So in that sense, that functionality is somewhat modular, unline grub2 where it just includes all the functionalities you listed above and more, whether the user likes/uses them or not.

Modularity may be a good thing, duplicating logic still isn't, even if it's "just" a filesystem driver. The kernel already knows how to do it and it knows how to do it best, why waste effort duplicating the same logic into the bootloader? I admit that I don't have a good understanding of how these EFI filesystem drivers work, but wouldn't you have also have to keep the EFI driver up-to-date separately?

Last edited by 65kid (2013-03-08 20:01:39)

Offline

#20 2013-03-08 20:11:06

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,414

Re: Pacman complains after Gummiboot upgrade

65kid wrote:

yes, I should have mentioned that I don't have a fallback image. I disabled it because I haven't used it a single time in using Arch for over 2 years. Well, add another 6MB if you want, doesn't really change my point.  The initramfs would even be a little smaller if my root wasn't encrypted.

My fallback is ~15MB, and I have used it once when I moved my drive to a brand new sparkly computer (maybe not sparkly since it is a thinkpad... more like blacky).  But I am using LVM2, so like your larger encryption capable initramfs, mine is probably a bit larger than a non-LVM2 ready one.

65kid wrote:

Modularity may be a good thing, duplicating logic still isn't, even if it's "just" a filesystem driver. The kernel already knows how to do it and it knows how to do it best, why waste effort duplicating the same logic into the bootloader? I admit that don't have a good understanding of how these EFI filesystem drivers work, but wouldn't you have also have to keep the EFI driver up-to-date separately?

Yeah, I don't necessarily disagree with you here.  I just understand why that effort was put forth.  In our modern day grub2 dominated Linux world, this seems to be functionality that most users of other distributions have come to expect.

As far as updating the driver, I would imagine that this would be inclusive in the rEFInd updates.  But moving the necessary files to the proper places might indeed be manual (rEFInd is not my primary boot manager, so I am not sure if there is automagic function to be included here).  But considering that it is used only to read (not r/w) the initramfs and kernel, at which point the functionality is handed back to the kernel, keeping it totally up to date with all the security updates and whatnot are probably not quite as critical. 

I am just saying that if this is functionality that srs5694 feels is beneficial to his boot manager, I really do appreciate the fact that I, the end user, and given the option to exclude that functionality in my personal setup.

Last edited by WonderWoofy (2013-03-08 20:11:40)

Offline

#21 2013-03-08 20:34:29

65kid
Member
From: Germany
Registered: 2011-01-26
Posts: 663

Re: Pacman complains after Gummiboot upgrade

Just to clarify, I didn't mean to say that these EFI drivers are complete non-sense, whoever thinks this is the better approach or has a special usecase for it has the option and may use it. I personally just don't see the usecase and think that using the kernel is the simpler and better approach (and that is exactly what gummiboot is going for).

Offline

#22 2013-03-08 20:51:00

progandy
Member
Registered: 2012-05-17
Posts: 5,180

Re: Pacman complains after Gummiboot upgrade

Why does it have to be so difficult?
Simply support kernel on ESP, if Windows cannot deal with more than one ESP that is not our problem to solve. The user will have to resize the partition in this case.
Each distribution following some of the systemd-specifications already has a distribution-id in /etc/os-release. Why not use this and put the kernel in /boot/$ID/, e.g. /boot/arch/vmlinuz-linux by default. If someone wants to install arch more than once, you can always modify the paths for mkinitcpio in e.g. /etc/mkinitcpio.d/linux.preset for the default kernel. Or add a new global variable for mkinitcpio to specify the root directory for all kernels of the currently booted system.


| alias CUTF='LANG=en_XX.UTF-8@POSIX ' |

Offline

#23 2013-03-09 00:13:37

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: Pacman complains after Gummiboot upgrade

FWIW: I'm using 486M on /boot and 352M on my ESP. The only OS installed to disk is Arch.

I assume, by the way, that nobody is suggesting making a certain way be the only way to boot Linux on EFI machines. It is rather a question of what installers do as a default. (So it presumably should not affect Arch except in terms of the documentation's recommendations? That is, since you set this up manually anyway, presumably any default for Arch would just be a recommendation in the Beginners' Guide/Installation Guide?)


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#24 2013-03-09 00:21:52

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: Pacman complains after Gummiboot upgrade

progandy wrote:

Why does it have to be so difficult?
Simply support kernel on ESP, if Windows cannot deal with more than one ESP that is not our problem to solve. The user will have to resize the partition in this case.

Aside from the fact that I very much doubt that all distributions would be happy to adopt this approach --- and if some do not, it will undermine the point of having a standard approach --- does anybody actually know that all EFI *firmware* will deal fine with more than one ESP?

Of course, one can say, "buggy firmware is not our problem" but so much EFI firmware is buggy that the effective number of people for whom the default will actually work is shrinking rapidly. (We've already lost people who want to dual boot windows - at least if they might ever need to reinstall which is to say, all who plan to keep their machine for more than a year or so - and anybody who has firmware which won't like more than one ESP - no idea how common this is likely to be or if it will happen at all.) Of course, you can at that point simply point to the "standard" but if only a tiny fraction of users can use that standard, it is not really worth the electronic paper it is written on. And again, there are certainly distros which will --- not unreasonably --- want their installers to result in working systems for at least the vast majority of users. So they will be unlikely to adopt such a standard and, once again, the whole point of having one will be lost.


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#25 2013-03-09 01:33:48

srs5694
Member
From: Woonsocket, RI
Registered: 2012-11-06
Posts: 719
Website

Re: Pacman complains after Gummiboot upgrade

65kid wrote:
srs5694 wrote:

Imagine, for instance, one distribution setting up gummiboot on the ESP; then another one comes along and its installer decides that the ESP is too crowded, so it sets up gummiboot on a separate Freedesktop partition.

This is something that I would consider a buggy installer.

Perhaps -- but the Freedesktop spec creates more room for such bugs to be created than would other alternatives. All other things being equal (which I realize they aren't), it's better to have a spec that gives fewer opportunities for bugs in implementations than a spec that gives more opportunities for such bugs to breed.

Even if creating a separate ESP was a good idea, how would the installer even do that? It would have to shrink another partition first which is way too risky.

No, it wouldn't. An OS installer is presumably creating partition(s) for the OS, so that means that either the disk already has free space or an existing partition must be shrunk to make free space to install the OS. The second ESP or Freedesktop partition would just go in that free space. There's no rule that says that either the ESP or the Freedesktop partition must go before other partitions on the disk, although doing so is advisable because putting an ESP in the middle of the disk increases the odds that it will have to be moved in the future. Also, note that one of my points is that having two ESPs on a disk is inadvisable because Windows doesn't like such configurations; thus, I'd say that it's better to create a Freedesktop boot partition than a second ESP.

srs5694 wrote:

Thus, the safest choice -- the one that's most likely to work with the greatest number of computers -- is to put the Linux boot loader (or a boot manager, at least) on the ESP, even if it's small, and put the kernel elsewhere.

But putting the kernel elsewhere is crazy for a simple reason: what if my root including the kernel is on an encrypted drive, RAID, LVM etc.?

I said "elsewhere," not "the Linux root (/) partition." In this context, "elsewhere" could easily be a separate Linux /boot partition, which can reside outside of the LVM or RAID, be unencrypted, be FAT, etc.

That's exactly what gummiboot is about - do only what the boot loader needs to do, let the kernel deal with the rest.

I would argue that "what the boot loader needs to do" should include "read partitions other than the one from which it was loaded." In the BIOS world, this pretty much goes without saying. In the EFI world, it's easier to create a boot loader that can read only from its own partition, but that will inevitably lead to problems on some systems and configurations.

And "the greatest number of computers" won't even care if their ESP is only 100MB, because they only use one OS. And whoever needs more should simply make the ESP big enough in the first place.

By "greatest number of computers" I meant the largest possible number, not the majority. Yes, the majority can probably make do with a 100MiB ESP, but to maximize usability, other configurations are more flexible and therefore preferable.

WonderWoofy wrote:

I don understand why srs5694 includes the filesystem drivers in his rEFInd, but also understand your point about having the boot manager be as simple as possible like gummiboot.

Clearly there's a typo; I'm not sure if you mean you do understand or that you don't understand. In case it's the latter, it's because (a) they were available with the rEFIt source code and so could be (relatively) easily included with rEFInd; and (b) because including them in rEFInd greatly simplifies installing and using rEFInd on typical Fedora, Ubuntu, SUSE, and other distributions. These distributions typically either don't use LVM or use a separate ext2/3/4fs /boot partition. It's currently possible to download a Debian package or RPM of rEFInd, install it on one of these distributions, and have it all work without further tweaking, at least in most cases. (If the user uses XFS, JFS, Btrfs, or something else unusual to hold the kernel, this won't work without additional reconfiguration; but the default settings for these distributions put the kernel outside of LVM/RAID setups in ext4fs partitions.) In the end, this is partially labor saving for me, too; users tend to want things to Just Work, and when they don't, they e-mail me for help. Without those drivers, rEFInd does not Just Work on the more popular distributions, but with them, it does, so I spend less time answering e-mail.

It seems that the filesystem driver is optional and is only available if the necessary drivers are placed on the ESP.

rEFInd's filesystem drivers are implemented as separate binaries from rEFInd itself, so you can run rEFInd without using the drivers at all. Currently, the install.sh script (which I believe the Arch package omits) attempts to determine what filesystem is used on /boot and installs it, if necessary. If it's not necessary (say, if the filesystem is FAT), no filesystem driver gets installed unless the --alldrivers option is passed to install.sh. It's possible to use rEFInd's filesystem drivers without using rEFInd, but most EFIs lack support for loading filesystem drivers automatically, so the trick is to get them loaded.

65kid wrote:

Modularity may be a good thing, duplicating logic still isn't, even if it's "just" a filesystem driver. The kernel already knows how to do it and it knows how to do it best, why waste effort duplicating the same logic into the bootloader?

Rudictio ad absurdum: The kernel already knows how to accept keyboard input, so why duplicate that functionality in a boot loader?

The answer is the same in both cases: Because it's necessary, or at least helpful, before the kernel loads.

I'm more than happy to admit that it's not necessary for the EFI or a boot loader to read Linux-specific filesystems if the user/installer sets up the computer with the kernel on a FAT filesystem. The problem is that in the real world, that conditional clause is often false. Note my earlier comments about Ubuntu, Fedora, etc. -- they all place their kernels on ext2/3/4 filesystems by default. Even many Arch users today have scripts and whatnot to copy their kernels to the ESP from /boot on some other filesystem. Yes, this becomes simpler if the ESP is mounted at /boot -- but as I've pointed out in this thread, doing this has its drawbacks, too, particularly when the user is a novice who hasn't thought things through or when using software (including third-party OSes) that set things up in a way that's not optimal for this type of configuration. I chose to fork rEFIt to rEFInd in order to glue together all the pieces I saw lying around that could almost handle the real-world situation. Now (as rEFInd) it can.

65kid wrote:

I admit that I don't have a good understanding of how these EFI filesystem drivers work, but wouldn't you have also have to keep the EFI driver up-to-date separately?

Yes, but read-only filesystem drivers need little maintenance once they're stabilized. All of rEFInd's filesystem drivers are read-only, so they pose no real threats to your data.

WonderWoofy wrote:

As far as updating the driver, I would imagine that this would be inclusive in the rEFInd updates.  But moving the necessary files to the proper places might indeed be manual (rEFInd is not my primary boot manager, so I am not sure if there is automagic function to be included here).

The install.sh script included with rEFInd will update the driver(s) along with the rEFInd binary and icons. Unfortunately, the Arch packagers have chosen to omit this script from the Arch packages (or they had the last time I checked).

65kid wrote:

I didn't mean to say that these EFI drivers are complete non-sense, whoever thinks this is the better approach or has a special usecase for it has the option and may use it. I personally just don't see the usecase

Given the relative market shares of Arch vs. Fedora or Arch vs. Ubuntu, it's really the Arch way of doing things that qualifies as the "special usecase." The drivers -- and especially the ext4fs driver -- help a lot with installing rEFInd on these distributions and getting it working painlessly.

progandy wrote:

Simply support kernel on ESP, if Windows cannot deal with more than one ESP that is not our problem to solve. The user will have to resize the partition in this case.

Philosophically, you're right -- Windows' bugs and limitations belong to Microsoft. Practically, I'm afraid it's not that simple -- if John Doe installs Linux and then tries to re-install Windows and discovers that it flakes out because of something that Linux put on the hard disk, who will John blame? Maybe Microsoft, but maybe Linux. Furthermore, if a Linux developer knows that the Windows bug exists, is the developer justified in ignoring that bug, knowing that doing so will cause John Doe problems in the future? IMHO, the answer is "no" -- although with the caveat that the difficulty of finding a solution is a factor, too.

As to resizing the ESP, as I wrote in post #12 in this thread, you shouldn't do that unnecessarily, since it will almost certainly require deleting or resizing the first partition that follows the ESP, and in some cases a partition or two after that, as well. Resizing partitions from their start points is inherently dangerous, so any policy that requires doing this will damage somebody's filesystem. This is an even clearer case where developers' decisions can have negative consequences, and developers have a moral obligation to avoid causing such problems (such as data loss), even indirectly.

I'd just like to take a step back from all this to clarify my main point: The Freedesktop proposed loader spec is a worthwhile attempt to address certain problems with boot loaders as they're currently implemented in Linux. I agree with most of their goals and some of the details of that proposal. If it were implemented universally tomorrow, though, it would cause problems. The main reason is because of gummiboot's limitations, which seem to have motivated many of the proposal's details. (Let me say that I'm not trying to trash gummiboot here; I do respect it, but it also has limitations that have important implications when it's used as the core of a proposal like this one.) Consider:

An existing installation has Windows and an ESP that's too small. Under the Freedesktop guidelines, a new Freedesktop partition will be created and used to hold kernels. I don't believe the proposal is specific about where the boot loader must reside, but there are two choices: Put gummiboot on the ESP, in which case it can launch Windows but not Linux; or put gummiboot on the Freedesktop boot partition, where it can launch Linux but not Windows. In either case it's not very useful by itself. In the latter case, the EFI might not even launch it, since the Freedesktop partition isn't an ESP, and IIRC, the EFI spec doesn't require the EFI to be able to launch boot programs on a non-ESP.

Now, consider if gummiboot were modified to support booting from other partitions. The six Freedesktop rules could be cut by two (removing #4 and #6) and slightly amended (to make $BOOT always be a Freedesktop partition and never an ESP, and to permit multiple Freedesktop partitions per disk should the need arise). This will eliminate the problem I've just identified and simplify the rules that OS installers or administrators should follow, thus reducing the odds of bugs or different interpretations causing problems. Yes, it means that the Freedesktop developers will need to add some code to their boot loader. OTOH, this also means there will be:

  • Less ambiguity about what /boot is (the ESP vs. the Freedesktop boot partition) on any given system; this will lead to simpler code in support scripts and less chance of user error because of confusion or help from others based on false assumptions

  • Less chance of another OS altering or damaging the Linux boot loader files, thus improving overall system reliability

  • Less chance of creating a configuration in which gummiboot is incapable of booting some OSes that are legally installed

  • A need for fewer rules in the proposal, thereby reducing the odds of differing interpretations causing compatibility problems

Overall, I'd say these benefits are worth a few extra lines of code.

Offline

Board footer

Powered by FluxBB