You are not logged in.

#26 2013-03-09 05:15:21

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,414

Re: Pacman complains after Gummiboot upgrade

Sorry, srs5694, that was indeed a typo.  I do understand why you would want to do that, and now your explanation regarding its availability from refit makes it even more understandable.

Offline

#27 2013-03-09 08:18:52

tomegun
Developer
From: France
Registered: 2010-05-28
Posts: 661

Re: Pacman complains after Gummiboot upgrade

cfr wrote:
tomegun wrote:

@srs5694: Thanks for your input, much appreciated.

So there are three options to "resizing" the ESP: add a second one, add a second and delete the old or actually resize the old. The first option seems like the sanest default to me, but it is useful to know that there is one case where this would be problematic (and where perhaps one of the other options should be used instead). The conditions needed to trigger this are (as far as I can tell):

* The user can not control the creation of the ESP to set the desired size (i.e., it is from the factory).
* The ESP was created too small (where 'too small' depends on how many OS's and kernels you wish to install. The smallest ESP I have seen is 100MB, which would allow you to install at least four different kernels, but typically many more).
* The user can not simply delete the original ESP (i.e., you are not on a Mac).
* The user can not reinstall the original OS on the machine to set the correct ESP size.
* You want the ability to install Windows on the machine _after_ you install Linux.

So I suppose you are on a Windows machine, the default Windows ESP size of 100MB is not big enough, and you intend to install a future Windows version in parallel with the old one (which you cannot reinstall or remove).

This is definitely something we should make sure is possible, but I think it is reasonable that we don't optimize for this case, but ask the user to deal with it specially. Either by recreating or resizing the ESP, both of which are possible but cumbersome; or by mounting the ESP on /boot/efi and deal with this in the same way as they currently do (i.e., they won't get the benefits of future simplifications we make and might get the occasional warning).

I am confused by this.

I should perhaps say that my working knowledge of Windows currently consists of an ability to find a file on a USB stick and open it. (Even that I had to have help with a couple of weeks ago because I didn't have a desktop shortcut to the file browser.) Oh, I also know how to log out or shut down.

But if the Windows installer will flake out if there is more than one ESP and if MS does not support that configuration, won't this also be a problem for anybody who needs to re-install Windows after installing an OS which decides to create a second ESP? As far as I can tell, Windows still requires re-installation at regular intervals - much more frequently than OS X or Linux. (My knowledge of this is somewhat current as the IT department have told me that Windows boxes slow down and need to be reinstalled at least annually. Weirdly they use Linux to install Windows which is the only time I understand what is going on.)

My point was that if you can reinstall Windows in the future, then surely you can do it also now (and in the process resize your ESP). Anyway, I don't think this is necessary and that a better solution can be found.

Offline

#28 2013-03-09 08:29:13

tomegun
Developer
From: France
Registered: 2010-05-28
Posts: 661

Re: Pacman complains after Gummiboot upgrade

@srs5694: I'm sorry that I can't reply in detail (I'm travelling at the moment). Just a few comments:

* I don't think the specification is meant to be limited by what gummiboot can do (extending gummiboot would not be a problem). Rather, both gummiboot and the spec are written with the same purpose: to be as simple as possible to cover all use cases.
* If your bootloader allow more flexibility than the spec, that's just fine and there should be no contradiction there. Would be cool if as many boot loaders as possible implemented this spec though (and whatever else they want on top). Obviously, it is still in draft stage, so I'm sure any input would be appreciated so any issues can be resolved ASAP.
* I agree that this "fake" ESP (or the Freedesktop partition as you say) is probably not a good idea. I had not given that part much thought, as I think the correct solution would be to resize the ESP, so this part we could ignore in Arch.

Lastly, it seems that the main (only?) concern here is about the safety of resizing partitions. I agree that moving/resizing some unknown and potentially huge partition is scary. I have not looked much into it, but should it not be possible to simply move the ESP to a large free area of the disk before resizing it there?

Offline

#29 2013-03-09 17:53:17

srs5694
Member
From: Woonsocket, RI
Registered: 2012-11-06
Posts: 719
Website

Re: Pacman complains after Gummiboot upgrade

tomegun wrote:

* I don't think the specification is meant to be limited by what gummiboot can do (extending gummiboot would not be a problem). Rather, both gummiboot and the spec are written with the same purpose: to be as simple as possible to cover all use cases.

I think it goes beyond that. The only name on the spec is Harald Hoyer, and he's one of two authors credited in gummiboot's main file. The spec seems to bend over backwards to accommodate some of gummiboot's features (it specifies the same configuration file format, for instance). gummiboot's been out for several months, but the proposed spec is new. Add it all up, and it's clear that the proposed spec is written around gummiboot -- or at least, they both come from the same mind and so reflect the same biases. (I'm not saying this in a critical way; we've all got our biases.)

* If your bootloader allow more flexibility than the spec, that's just fine and there should be no contradiction there. Would be cool if as many boot loaders as possible implemented this spec though (and whatever else they want on top). Obviously, it is still in draft stage, so I'm sure any input would be appreciated so any issues can be resolved ASAP.

There's no link on the page for readers to post comments. There is a link to the name "HaraldHoyer," but the link is broken. I've just perused the gummiboot source code and found an e-mail address for this person, so I'll probably fire off an e-mail; but they aren't exactly making it easy to provide input on the proposal!

* I agree that this "fake" ESP (or the Freedesktop partition as you say) is probably not a good idea. I had not given that part much thought, as I think the correct solution would be to resize the ESP, so this part we could ignore in Arch.

That's not my point -- in fact, it's the opposite of my position, in at least some respects! My suggestion is that the boot loader/manager be stored on the ESP, without resizing it if at all possible, and kernels be stored on something that could be described as either "a Linux /boot partition" or "the Freedesktop boot partition." Giving this partition a unique GUID and perhaps specifying that it be FAT would help make the standard implementable by a wide variety of boot loaders. Requiring that an undersized ESP be resized is a bad idea, since that will lead to either the need to resize other partitions from their start points, which in turn is dangerous, or to the need to copy the ESP to a larger partition, which poses a host of other problems (admittedly lesser in danger, but greater in number).

Lastly, it seems that the main (only?) concern here is about the safety of resizing partitions.

That's a major concern, but far from the only one. For instance, see my comments about scenarios that would lead to an inability to boot all the OSes on the computer (at least when using gummiboot as it's currently written). There are also risks involved in exposing the Linux kernel on the ESP, which is intended as a cross-OS storage space. Putting the boot loader/manager there is architecturally necessary, but putting the kernel there is not.

I agree that moving/resizing some unknown and potentially huge partition is scary. I have not looked much into it, but should it not be possible to simply move the ESP to a large free area of the disk before resizing it there?

In theory, yes; but that creates its own problems, such as wasted disk space, the possibility of problems during the move that could render the system unbootable and/or destroy data, and the possibility of an OS becoming confused over the move and therefore causing unknown problems. Also, consider the case of a two-disk installation, with the ESP and an existing OS (let's say Windows, but it could be OS X, FreeBSD, or whatever) on one disk and an empty disk intended for Linux. In this case, you could legitimately create an ESP on the new disk independent of the first ESP, but copying the boot loader files from the first ESP to the new one would just add to the potential problems since the NVRAM references would no longer be valid and so would need updating. In theory, it could be done, but it would be a lot of extra code for an installer script or a lot of extra steps for a more manual (Arch-style) installation procedure.

In sum, there are a host of problems with the proposal as currently written. They vary in severity from minor to huge, but most of them go away if you add the requirement that boot loaders be able to boot from partitions other than the ones from which they were launched and tweak the proposal to store Linux kernels on Linux-specific partitions rather than on the ESP.

Offline

#30 2013-03-11 08:15:23

sidneyk
Member
From: Bonner Springs, KS. USA
Registered: 2011-04-22
Posts: 129

Re: Pacman complains after Gummiboot upgrade

I just read the freedesktop boot loader spec. proposal and then re-read this thread. It seems to me like the new proposal just complicates things, at least from the UEFI perspective. If most distros are already mounting the ESP at /boot/efi on boot, then that would seem to already be a defacto standard. I could understand trying to change that to maybe a separate /boot or freedesktop partition, that would be shared by all Linux distros to store kernels and initrds, with requirements that each have a separate sub directory there, along with a minimum size standard and such, but I agree with the idea (and possibly the current requirement) that any boot loaders or boot managers and other EFI files be placed on the ESP. I also like the idea of just being able to put my kernel and initrd on the ESP and let it be it's own boot loader, but I prepared and needed to reinstall Windows anyway so I was able to make a bigger ESP to start with.

I was lucky and the instructions in the Arch Beginner's Guide for rEFInd worked quite well for me and systemd is now happily copying any updated kernel and initrd files for me. I haven't tried gummiboot, but rEFInd seems simple enough for me and comes with a nice graphical OS chooser which doesn't confuse my Windows using wife. I tried out the ext4fs driver to see how that worked and could see that, while it did detect kernels on any of ext4 file systems, it only gave them generic icons and found even old versions that I may not necessarily want to boot with anymore and would require further configuration to make the graphical display as simple and meaningful as it is when the kernel and initrds are on the ESP and renamed appropriately. I fully agree with the idea that a boot loader or boot manager should have the capability to load a kernel and initrd from any drive or partition. I don't see what the problem is with a program (or it's dev) deciding to include these for my optional use and with it's own install program being able to at least give me the option to use them or tell me if other choices I have made make it necessary.

I like the idea of keeping boot loaders / managers on the ESP and distros keeping their kernels wherever they want them and the bootloader supporting a filesystem driver or being able to utilize one to find Linux kernels on non-FAT32 locations. In that scenario, then the ESP wouldn't necessarily have to be very big because it could get by with only one or two boot loaders in there, one for Windows or MAC and one that could handle all the Linux'. Maybe it's even possible for just one boot loader to handle it all, but there would have to be the proper EFI file system driver support. I'm no expert, just sharing my thoughts after reading the spec proposal and in light of my own experience with UEFI on a triple booting setup. I'm curious how Windows 7 or Windows 8 handles things if an install or reinstall of either is attempted on a UEFI / GPT system with other existing OS's in place? Will it just rewrite it's own directories in the ESP and make it's own NVRAM entries or will it reformat any ESP it finds first like it used to do with the MBR on a BIOS based system, requiring at minimum that grub or whatever be reinstalled? I'm not curious enough to try, just wondering. I installed Win 7 first and it had no problem with my EFI and MSR partitions created with gdisk, but they were both empty at this point.

Last edited by sidneyk (2013-03-11 08:17:23)

Offline

#31 2013-03-11 17:06:22

srs5694
Member
From: Woonsocket, RI
Registered: 2012-11-06
Posts: 719
Website

Re: Pacman complains after Gummiboot upgrade

sidneyk wrote:

rEFInd seems simple enough for me and comes with a nice graphical OS chooser which doesn't confuse my Windows using wife. I tried out the ext4fs driver to see how that worked and could see that, while it did detect kernels on any of ext4 file systems, it only gave them generic icons

You can customize the icons in any number of ways, as described here. The simplest way is usually to give a suitable name to the filesystem that holds the kernels. For instance, a name of "Arch" or "Arch boot" will bring up the Arch icon.

and found even old versions that I may not necessarily want to boot with anymore and would require further configuration

This will admittedly require either deleting the now-unwanted files or adjusting options in refind.conf.

I like the idea of keeping boot loaders / managers on the ESP and distros keeping their kernels wherever they want them

One of the key points of the proposal, which I agree with, is to limit and standardize where a conformant OS places its kernels. If this is widely adopted, it will enable simplification of boot loaders and associated system support scripts, such as by removing the need for boot loaders to support LVM, RAID, and exotic filesystems.

I'm curious how Windows 7 or Windows 8 handles things if an install or reinstall of either is attempted on a UEFI / GPT system with other existing OS's in place? Will it just rewrite it's own directories in the ESP and make it's own NVRAM entries or will it reformat any ESP it finds first like it used to do with the MBR on a BIOS based system

I'm pretty sure I checked this myself, and Windows 7 is pretty well-behaved -- it rewrites its own ESP files but doesn't erase the entire ESP. I'm not 100% positive that I've checked this, though, so I could be wrong.

Offline

#32 2013-03-12 02:48:20

sidneyk
Member
From: Bonner Springs, KS. USA
Registered: 2011-04-22
Posts: 129

Re: Pacman complains after Gummiboot upgrade

srs5694 wrote:
sidneyk wrote:

rEFInd seems simple enough for me and comes with a nice graphical OS chooser which doesn't confuse my Windows using wife. I tried out the ext4fs driver to see how that worked and could see that, while it did detect kernels on any of ext4 file systems, it only gave them generic icons

You can customize the icons in any number of ways, as described here. The simplest way is usually to give a suitable name to the filesystem that holds the kernels. For instance, a name of "Arch" or "Arch boot" will bring up the Arch icon.

and found even old versions that I may not necessarily want to boot with anymore and would require further configuration

This will admittedly require either deleting the now-unwanted files or adjusting options in refind.conf.

My main point here was, that with the appropriate file system driver, that rEFInd does what it was designed to do. I know that Arch has decided to use a simple nomenclature for their kernels to simplify the kernel upgrade process, but the fact that their naming is generic pulls in the generic icons. I was aware that there is a configuration mechanism in rEFInd to deal with this, but if each distro had distro specific naming of their kernels then no further configuration of rEFInd would be required in this respect, if it's got a distro specific icon in there then the recognized distro name would pull in the right icon, no matter how many kernels were there.

While configuration options are nice to have when needed, it's even nicer when a program can automatically figure out proper configuration on it's own, something that I believe you have made a great attempt to accomplish, but the actual implementation sometimes lacks because of the fact that there are still areas in the different Linux' without standardization between distros. It would seem to be a pretty logical step to standardize things like kernel naming conventions and even where kernels are placed in the filesystem so that boot loaders and boot managers and such would always know where to find these regardless of distro. I'm not knocking Arch for trying to keep the kernel update mechanism simple and clean, but the generic naming can present problems in this scenario.

Offline

#33 2013-03-13 06:26:35

tomegun
Developer
From: France
Registered: 2010-05-28
Posts: 661

Re: Pacman complains after Gummiboot upgrade

It seems I have misread the spec, so need to clarify (I expect the spec will be updated soon-ish to make this abundantly clear):

* The spec is meant to allow $BOOT (where you install your kernels + boot loader config, and which is mounted at /boot) to be different from the ESP (where you install the actual boot loaders). Which is exactly what srs5694 was arguing for. So my bad.
* The spec is meant to make $BOOT equal to the ESP when this makes sense (i.e., the ESP is big enough), the reason being that having two levels of boot partitions is redundant and doesn't give you anything (so let's KISS smile )
* gummiboot does not fully conform to the spec as it does not support $BOOT != ESP. This is by design as gummiboot is meant to be minimal, whereas the spec is meant to be universal.

It seems everyone agrees that in general we cannot modify the ESP, but have to live with whatever we got. In the particular case of a competent user (also known as an "Arch user") I still think making sure the ESP is big enough is the only sensible thing to do though.

@srs5694: as to the difficulty of providing feedback, the spec is not really public yet (has not been announced), so I expect that it will be easier to provide feedback once it is. I made some requests for clarification already, so hopefully it will be amended soon.

Offline

#34 2013-03-13 07:42:13

sidneyk
Member
From: Bonner Springs, KS. USA
Registered: 2011-04-22
Posts: 129

Re: Pacman complains after Gummiboot upgrade

I'm not an expert, by any means, but I'm learning alot in these past few days about UEFI systems. I can see where it might be nice to have a standard /boot partition, shared by all Linux distros, with the first Linux installed ( or the user ) making sure that the /boot partition was a certain minimal size, big enough to hold all the kernels and initramfs and whatever else usually goes in there for, at least, a reasonable expectation of what most users might want. I mean, is anyone really going to install 50 separate distros on one desktop system? Then, any subsequent distros would search for an appropriate /boot partition already existing and maybe even refuse to create a separate one if it already exists, or at least balk at the idea. Maybe each distro could also scan the system to see if existing Linux installs are in place ( or at least Linux file systems ).


Again, if all that HAD to go into the ESP was a boot loader or boot manager efi program which should be able to handle all the Linux systems ( and even the Windows systems ) then a Windows 100MB ESP partition probably wouldn't pose a problem or require reinstallation to make it bigger. I know there have been some users who have said that anything smaller than 512MB just didn't work for them, but if an existing Windows 8 or WIndows 7 user is already booting off of a 100MB EFI into EFI mode, then it probably works OK on that particular system. Anyone desiring to keep the kernels and initramfs on the ESP probably already knows that he or she is going to have to do some more drastic steps to prepare for it. But it sounds like the spec is moving toward keeping only boot loaders in the ESP, as the standard way of doing things. If I am understanding things properly, it sounds like I could probably only have rEFInd or gummiboot, if I choose, as the only boot manager and could go as far as deleting Windows boot manager from NVRAM settings, that's what I believe is happening already - I'm using rEFInd as the default boot manager choice and I think if I choose Windows then it just starts up Windows boot loader. I don't think it is reverting control back to Windows boot manager. I know from the firmware that Windows boot manager is a choice, it's just not my default choice at the present.


Not meaning to ramble on here, just sharing thoughts and observations.

Offline

#35 2013-03-13 08:10:00

tomegun
Developer
From: France
Registered: 2010-05-28
Posts: 661

Re: Pacman complains after Gummiboot upgrade

sidneyk wrote:

Anyone desiring to keep the kernels and initramfs on the ESP probably already knows that he or she is going to have to do some more drastic steps to prepare for it. But it sounds like the spec is moving toward keeping only boot loaders in the ESP, as the standard way of doing things

The aim is to keep both kernels and boot loaders on the ESP if that is possible. If it is not, then the spec specifies how to deal with that in a uniform way so that different distros and boot loaders can work together, namely to create a special boot partition to store your kernels in.

Offline

#36 2013-03-13 08:34:34

sidneyk
Member
From: Bonner Springs, KS. USA
Registered: 2011-04-22
Posts: 129

Re: Pacman complains after Gummiboot upgrade

tomegun wrote:
sidneyk wrote:

Anyone desiring to keep the kernels and initramfs on the ESP probably already knows that he or she is going to have to do some more drastic steps to prepare for it. But it sounds like the spec is moving toward keeping only boot loaders in the ESP, as the standard way of doing things

The aim is to keep both kernels and boot loaders on the ESP if that is possible. If it is not, then the spec specifies how to deal with that in a uniform way so that different distros and boot loaders can work together, namely to create a special boot partition to store your kernels in.

Since the kernel now can be it's own boot loader, I kind of like it that way, in it's own directory on the ESP. I used grub and grub2 for a long time on MBR systems, and no offense to grub, but I'm happy to lose that layer. I can't see a need for the grub boot loader now when the kernel can do the job just fine. I do like the rEFInd boot manager though because it presents a little more elegant approach than the EFI firmware boot manager and I don't have to hit F8 at boot to get to it. I made a 300MB ESP prior to a Windows reinstall and it's about half full with Windows stuff, Arch kernel and initramfs, Ubuntu kernel and initramfs and the rEFInd boot manager files. Now that I've learned to build Android successfully on Arch I may just lose the Ubuntu altogether.

Offline

#37 2013-03-13 17:05:50

srs5694
Member
From: Woonsocket, RI
Registered: 2012-11-06
Posts: 719
Website

Re: Pacman complains after Gummiboot upgrade

tomegun wrote:

* The spec is meant to make $BOOT equal to the ESP when this makes sense (i.e., the ESP is big enough), the reason being that having two levels of boot partitions is redundant and doesn't give you anything (so let's KISS smile )

For an individual installation that's properly managed, that may be true; but from the point of view of a distribution maintainer, having $BOOT be either the ESP or a Freedesktop boot partition adds complexity, since documentation and/or system installation and configuration tools need to deal with two possibilities:

  • The ESP mounted at $BOOT

  • The Freedesktop boot partition mounted at $BOOT and the ESP mounted at /boot/efi (or wherever)

The proposed spec masks this by using the "$BOOT" nomenclature, which obscures the fact that $BOOT could be very different things and the fact that another partition (the ESP) may need to be managed in some cases.

What's more, boot loader configuration files may have to be different for the two different possibilities, which adds another layer of differences.

Doing a flat-out requirement that there be two different partitions actually simplifies the documentation and logic in support scripts. Granted, this does mean an extra command or two for users who might be able to use just one and who use Arch, Gentoo, or other more hands-on distributions, but IMHO this cost is more than outweighed by the savings in confusion when people go looking for help and in the cost of extra bugs in automated tools.

* gummiboot does not fully conform to the spec as it does not support $BOOT != ESP. This is by design as gummiboot is meant to be minimal, whereas the spec is meant to be universal.

The problem is that "universal" implies that it works with anything, including with gummiboot. My contention is that gummiboot ought to change to be more flexible on this score. That's not really an issue with the spec per se, but with gummiboot, especially if the spec is widely adopted.

@srs5694: as to the difficulty of providing feedback, the spec is not really public yet (has not been announced), so I expect that it will be easier to provide feedback once it is. I made some requests for clarification already, so hopefully it will be amended soon.

FWIW, I've gotten back some replies to my private queries. It seems that Lennart Poettering is actually handling this, although his name doesn't appear on the page.

sidneyk wrote:

If I am understanding things properly, it sounds like I could probably only have rEFInd or gummiboot, if I choose, as the only boot manager and could go as far as deleting Windows boot manager from NVRAM settings, that's what I believe is happening already - I'm using rEFInd as the default boot manager choice and I think if I choose Windows then it just starts up Windows boot loader. I don't think it is reverting control back to Windows boot manager.

That's mostly correct -- rEFInd doesn't look at NVRAM entries; it just generates a menu and passes control to other EFI program files. In this sense, though, it does "revert... control back to Windows boot manager" -- it just does so directly rather than by using an NVRAM entry.

Offline

Board footer

Powered by FluxBB