Kernel 3.7 doesn't boot using EFI Stub

srs5694 · 2013-03-22 17:23:24

blackout23 wrote:

I wonder what triggers this. I can boot 3.7.10-1-ARCH with rEFInd while others can't. I have add_efimemap as Kernel attribute whatever that does. Almost looks hardware specific.

The evidence to date points to a three-way bug between the kernel (probably its EFI stub code), the boot manager (which would include the EFI shell if that's used to launch the kernel), and the firmware. My suspicion is that there's a firmware bug that's triggered by certain combinations of compilation options in the boot manager and kernel.

dhave · 2013-03-22 17:26:00

srs5694 wrote:

blackout23 wrote:
I wonder what triggers this. I can boot 3.7.10-1-ARCH with rEFInd while others can't. I have add_efimemap as Kernel attribute whatever that does. Almost looks hardware specific.
The evidence to date points to a three-way bug between the kernel (probably its EFI stub code), the boot manager (which would include the EFI shell if that's used to launch the kernel), and the firmware. My suspicion is that there's a firmware bug that's triggered by certain combinations of compilation options in the boot manager and kernel.

Thanks for the diagnosis. If you're right, then I guess we need to wait for a firmware update for our machines, right?

srs5694 · 2013-03-22 17:29:05

dhave wrote:

srs5694 wrote:
The evidence to date points to a three-way bug between the kernel (probably its EFI stub code), the boot manager (which would include the EFI shell if that's used to launch the kernel), and the firmware. My suspicion is that there's a firmware bug that's triggered by certain combinations of compilation options in the boot manager and kernel.
Thanks for the diagnosis. If you're right, then I guess we need to wait for a firmware update for our machines, right?

Either that or employ a workaround like a known-good combination of boot manager and EFI stub loader. Since rEFInd compiled with GNU-EFI seems to be working for a lot of people, that seems like a reasonable workaround, at least for the moment.

cfr · 2013-03-22 21:57:15

Is that version of rEFInd going to make it into the official repos, do you know?

mrunion · 2013-03-22 22:02:07

Now 3.8.4-1 from pacman update today boots fine -- just like 3.7.9-2 did. Geesh!

srs5694 · 2013-03-23 01:03:07

cfr wrote:

Is that version of rEFInd going to make it into the official repos, do you know?

It's not so much a different version (although what I posted was the latest, and I'm not sure if Arch is up to that version yet); it's the build environment that's different. Unfortunately, using GNU-EFI has the drawback of disabling the ability to boot BIOS-mode boot loaders on UEFI-based PCs, which using TianoCore supports. Thus, switching to GNU-EFI would have a drawback for at least some users. Of course, the Arch developers could choose to provide binaries built with both development kits. What they'll do is their decision.

cfr · 2013-03-23 04:56:22

The 3.8.4-1 doesn't work any better for me, unfortnuately.

I guess I'll just switch back to grub. Right now I'm booting rEFInd -> grub -> arch which seems a bit of a round about way to go about things.

Directly booting grub should work, right/ It is only the stub kernel stuff which is screwy?

Wonder what's different in the 3.8.* kernels? (The 3.7.* booted OK for me.)

WonderWoofy · 2013-03-23 06:04:15

cfr wrote:

The 3.8.4-1 doesn't work any better for me, unfortnuately.
I guess I'll just switch back to grub. Right now I'm booting rEFInd -> grub -> arch which seems a bit of a round about way to go about things.
Directly booting grub should work, right/ It is only the stub kernel stuff which is screwy?
Wonder what's different in the 3.8.* kernels? (The 3.7.* booted OK for me.)

Yeah you should be able to have grub load automatically. Just create an efibootmgr entry for it, and you should be good to go since it puts new entries in the first position.

aphirst · 2013-03-23 13:22:43

srs5694 wrote:

I'm definitely interested to hear if the new rEFInd binary works for others, too. Here's the link:
http://www.rodsbooks.com/refind_x64_gnuefi_3.6.8.efi
That's a "bare" binary that needs to be installed over (or in parallel with) your existing rEFInd binary. It differs from my official 3.6.8 rEFInd binary only in that it was compiled with GNU-EFI rather than TianoCore and it was not cryptographically signed.
There is one loose end to the theory that there's a TianoCore bug that's the root cause of the problem: People have reported that kernels hang when launched from gummiboot, and it's built with GNU-EFI. Still, if this new binary works for others, it's an important clue about what's going on....

I can confirm that using this GNU-EFI binary works for me too. I initially made my own thread before I realised this was a more far-reaching problem. With the TianoCore binary, kernel version 3.7.x worked fine for me, but didn't with 3.8.4 when I finally risked the upgrade. Downgrading to 3.8.3 'fixed' the problem, but I'm now on 3.8.4 with this new binary.

I also noticed that there was a refind update in extra from today, though that still exhibited the same behaviour on my end. Updating my BIOS didn't seem to affect anything either, though I was surprised that there had been several (probably actually unrelated) updates to that since my last one a month ago.

This is an awfully strange state of affairs.

Last edited by aphirst (2013-03-23 13:27:33)

mcloaked · 2013-03-23 13:59:47

aphirst wrote:

srs5694 wrote:
I'm definitely interested to hear if the new rEFInd binary works for others, too. Here's the link:
http://www.rodsbooks.com/refind_x64_gnuefi_3.6.8.efi
That's a "bare" binary that needs to be installed over (or in parallel with) your existing rEFInd binary. It differs from my official 3.6.8 rEFInd binary only in that it was compiled with GNU-EFI rather than TianoCore and it was not cryptographically signed.
There is one loose end to the theory that there's a TianoCore bug that's the root cause of the problem: People have reported that kernels hang when launched from gummiboot, and it's built with GNU-EFI. Still, if this new binary works for others, it's an important clue about what's going on....
I can confirm that using this GNU-EFI binary works for me too. I initially made my own thread before I realised this was a more far-reaching problem. With the TianoCore binary, kernel version 3.7.x worked fine for me, but didn't with 3.8.4 when I finally risked the upgrade. Downgrading to 3.8.3 'fixed' the problem, but I'm now on 3.8.4 with this new binary.
I also noticed that there was a refind update in extra from today, though that still exhibited the same behaviour on my end. Updating my BIOS didn't seem to affect anything either, though I was surprised that there had been several (probably actually unrelated) updates to that since my last one a month ago.
This is an awfully strange state of affairs.

This may be a wrong presumption to ask, but I guess you copied out the files from /usr/lib/refind/ to your ESP after the update before testing the new version? My system running rEFInd worked fine before the update but I did copy out both the new refind_x64.efi from the above directory into my ESP, as well as the ext4 driver from /usr/lib/refind/drivers_x64/ext4_x64.efi into the relevant directory in the ESP as well since the new files are different to the previous ones.

I have two systems both with an Intel DQ77KB motherboard and they do work well with rEFInd once I had got the UEFI files in the proper places. Hopefully other systems that have problematic UEFI boot issues will get fixed especially with the good work of developers like Rod Smith who has been extremely helpful on this forum.

aphirst · 2013-03-23 14:07:36

mcloaked wrote:

This may be a wrong presumption to ask, but I guess you copied out the files from /usr/lib/refind/ to your ESP after the update before testing the new version? My system running rEFInd worked fine before the update but I did copy out both the new refind_x64.efi from the above directory into my ESP, as well as the ext4 driver from /usr/lib/refind/drivers_x64/ext4_x64.efi into the relevant directory in the ESP as well since the new files are different to the previous ones.

I did copy the drivers_x64 files as part of what I did after the update, yeah. On my end it seems purely to be the TC version having issue with some versions, where GNU-EFI works fine.

cfr · 2013-03-23 20:56:23

srs5694 wrote:

I'm definitely interested to hear if the new rEFInd binary works for others, too. Here's the link:
http://www.rodsbooks.com/refind_x64_gnuefi_3.6.8.efi
That's a "bare" binary that needs to be installed over (or in parallel with) your existing rEFInd binary. It differs from my official 3.6.8 rEFInd binary only in that it was compiled with GNU-EFI rather than TianoCore and it was not cryptographically signed.
There is one loose end to the theory that there's a TianoCore bug that's the root cause of the problem: People have reported that kernels hang when launched from gummiboot, and it's built with GNU-EFI. Still, if this new binary works for others, it's an important clue about what's going on....

Will this work OK with the fs drivers etc. from the version in the official repos or do I need to get those from the official 3.6.8 release?

cfr · 2013-03-23 22:13:37

WonderWoofy wrote:

cfr wrote:
The 3.8.4-1 doesn't work any better for me, unfortnuately.
I guess I'll just switch back to grub. Right now I'm booting rEFInd -> grub -> arch which seems a bit of a round about way to go about things.
Directly booting grub should work, right/ It is only the stub kernel stuff which is screwy?
Wonder what's different in the 3.8.* kernels? (The 3.7.* booted OK for me.)
Yeah you should be able to have grub load automatically. Just create an efibootmgr entry for it, and you should be good to go since it puts new entries in the first position.

Thanks, WonderWoofy. That's what I figured. (I already have an entry for it, though.)

I realised that I could actually do something different which is to set rEFInd to use grub as the default. This is a longer way around but will make it easier to test things so I may just stick to that for now (assuming it all works and I haven't messed up the config).

I noticed Arch's rEFInd just got updated but I'm guessing that won't make any difference since it is still built the same way.

I noticed that the 3.8.* kernels seem to load efivars by default or else have this built in rather than separated as a module. Is that right?

WonderWoofy · 2013-03-23 23:43:00

Actually, the last version of rEFInd wouldn't even start for me. It gave me a garbled screen similar to when some machines come out of hibernation and it is switching from the normal boot process to the thawing. So I copied over the new version, and just out of pure curiosity (I have it only installed as a backup... apparently a bad one with the last version) I loaded it with the same config and everything. It worked flawlessly... just as I expected. I have all manual boot stanzas and textonly, so it doesn't have to scan or do anything fancy.

I too noticed the recent change to not having to load efivars... so your question prompted me to do this:

% zgrep -i efi /proc/config.gz
...
CONFIG_EFI_VARS=y
...

So it is now compiled into the kernel. If it were still a module (loaded automagically), it would be an "m" instead of a "y".

cfr · 2013-03-24 00:02:34

Thanks. That's really useful re. finding out it is compiled in - I didn't realise you could do that.

When you say that the last version didn't work, do you mean that 0.6.8-1 from the repos didn't even launch so you copied the 0.6.8 version configured with GNU-EFI over the top? Or do you mean that you copied that over the top of the previous 0.6.7-1 version from the repos because that one wouldn't work? Sorry - "last version" is a bit ambiguous.

Has gummiboot been working OK for you or do you typically have the same issues with that as rEFInd? i.e. if the kernel won't boot from one, it won't from the other? I ask because I was thinking of trying it out but the wiki is a little sparse and I don't want to go through the hassle of rearranging things if it won't solve anything. I'm also not clear whether I can load grub from gummiboot and from there launch an .iso designed for bios booting and/or the LTS kernel. (This isn't absolutely critical for me but would be nice.)

Last edited by cfr (2013-03-24 00:04:06)

WonderWoofy · 2013-03-24 01:18:33

Yeah that probably wasn't entirely clear... I had 0.6.7-1 and simply copied 0.6.8-1 on top of it. I have not tried the GNU-EFI compiled version as it is not my primary method of booting.

Gummiboot has continued to work fine for me, so I see no reason to switch. However, I do see reason to have backup methods. It is far too often that I see in forums around the interwebs that say something like "I have broken my bootloader..." and when you use bios booting, that is your only bootloader.

Honestly, I have not used rEFInd for very long, nor long enough to really make any kind of statement as to how well it works for me. The only issue I have had is the last version not working at all (which I guess is kind of an issue). I did notice that the previous kernel in core (3.7.10) did not boot for me, but as I use graysky's CK kernel, I didn't really much care to debug it either.

I think it is well worth it to try gummiboot. It is super easy to set up. I think it was actually far easier than rEFInd. You just need <ESP>\loader\loader.conf and *.conf files in <ESP>\loader\entries. The loader only needs to contain the default (which is the name of the appropriate *.conf file w/o the .conf, and the time out if you want a timeout.

For the official kernel, you would have an \EFI\loader\entries\arch.conf and it would contain something like this:

cat /boot/loader/entries/arch.conf
title   Arch Linux (Official Kernel)
linux   /EFI/arch/vmlinuz-linux.efi
initrd  /EFI/arch/initramfs-linux.img
options root=/dev/volgrp0/root_arch add_efi_memmap libahci.ignore_sss=1 quiet acpi_osi="!Windows 2012"

cfr · 2013-03-24 02:57:12

Thanks. I think I may give it a whirl. It is only a pain because I currently have an ext4 boot. I think I will probably merge my /boot and ESP and redo the lot while I am at it. I'm increasingly of the view that having a separate /boot is a pain in my case. (I think I might see it differently if I dual booted Windows.) And we'll see.

Couldn't agree more about back up methods. Grub is essentially mine - partly because I can boot into stuff from grub which I can't get into from rEFInd (or gummiboot, I guess since that's also a boot manager rather tan loader as I understand it). People moan a lot about grub but I do find it works which ought to be worth something.

EDIT: Is it possible to configure gummiboot NOT to install itself to the default bootloader location? I don't mind undoing this once but the wiki suggests it will do this automatically whenever it is updated! For the record, I do not consider this good behaviour on my system.

Last edited by cfr (2013-03-24 03:19:57)

WonderWoofy · 2013-03-24 03:53:22

Honestly, I don't really like the automatic creation of a bootloader entry either by gummiboot. But the inconvenience for me is far less than that of someone who doens't use gummiboot as their main boot manager. But if you want to disable it, you could create your own package (or edit the existing one) to modify the gummiboot.install script. There is a post_upgrade that does this via "/usr/bin/gummiboot update". So if you were to simply append "--no-variables" to the end of that command it would only install gummiboot to the proper section.

I really think this kind of goes against the whole idea of allowing the Arch user to configure the machine as desired. Also it somehow pushed itself into Boot0000 which used the be Setup, and now Setup is Boot0011. I don't think that is good form either. I liked it much better when the configuration was totally up to me.

I have thought about installing grub, but since I mount the ESP to /boot directly, I am not sure where I would then put /boot/grub. I think that grub might actually be expecting this to be on a different partition, where it would be on the same partition as grub.efi with how I have it set up. So for a real boot loader, I use elilo. The AUR package is an orphan and doesn't build, but if you download the source, there is also a binary included (three actually), so you just have to move it in the right place and create a lilo.conf. Rod Smith has some great instructions on how to set this up on his site.

As far as your ext4 /boot, I don't think that it is necessary to have a separate /boot anymore anyway if you are using UEFI. I guess that is unless you are using a filesystem for your rootfs that is not compatible with grub or another boot manager. You aren't really asking for my opinion, but I think you should just integrate your /boot into / and simplify things. I think if you are comfortable mounting at /boot/efi, then continue to do that, but it seems useless to have a separate /boot (don't you use ext4 as your rootfs anyway?).

Edit: Maybe if you just do NoExtract = /usr/bin/gummiboot, there will be no exectuable to do that update and create the efibootmgr entry.

Last edited by WonderWoofy (2013-03-24 03:54:31)

65kid · 2013-03-24 09:46:26

cfr wrote:

EDIT: Is it possible to configure gummiboot NOT to install itself to the default bootloader location? I don't mind undoing this once but the wiki suggests it will do this automatically whenever it is updated! For the record, I do not consider this good behaviour on my system.

Does "gummiboot update" still create the bootloader entry if you initially installed it with "gummiboot --no-variables install" ? I agree that it shouldn't. I think this could be worth discussing with upstream, as far as I can see they have been pretty open to suggestions (a few bugs have already been fixed because of reports from Arch users).

edit: oh wait, did you mean the default boot location BOOTX64.EFI or the actual boot entry in the EFI firmware? Sorry if I misunderstood. Either way, talk to upstream.

Last edited by 65kid (2013-03-24 09:48:56)

hushkyotosleeps · 2013-03-24 17:16:42

I actually need BOOTX64.EFI in order for my thinkpad to automatically boot (for some reason gummiboot won't start with it's own entry). It would probably be best if "gummiboot update" and "gummiboot install" were left up to the user (and instructed to in the wiki).

WonderWoofy · 2013-03-24 20:03:05

65kid wrote:

cfr wrote:
EDIT: Is it possible to configure gummiboot NOT to install itself to the default bootloader location? I don't mind undoing this once but the wiki suggests it will do this automatically whenever it is updated! For the record, I do not consider this good behaviour on my system.
Does "gummiboot update" still create the bootloader entry if you initially installed it with "gummiboot --no-variables install" ? I agree that it shouldn't. I think this could be worth discussing with upstream, as far as I can see they have been pretty open to suggestions (a few bugs have already been fixed because of reports from Arch users).
edit: oh wait, did you mean the default boot location BOOTX64.EFI or the actual boot entry in the EFI firmware? Sorry if I misunderstood. Either way, talk to upstream.

I am fine with the "gummiboot update" command installing as \EFI\boot\bootx64.efi, but I do not like how the "install" command creates an actual firmware entry. This might be good for other distributions where the user can't be bothered to configure this themselves, but I don't think it is really appropriate here.

But what really bothers me about it is how it pushes "setup" (or the bios configuration menu) out of the 0000 spot. In my opinion, this is very very bad. I think it is actually you 65kid who mentioned that you had to restore the "setup" entry from a backup of efivars, yeah? Fortunately, on my machine, it created a new "setup" in the next available spot, which was 0011. But with all the different implementations of UEFI bioses, I can't imagine how this seems like a good design decision.

cfr · 2013-03-24 21:18:13

Am going rapidly off the idea of installing gummiboot. I actually meant that I objected to it installing to \EFI\boot\bootx64.efi. *I* should get to decide what my fall back boot loader is. I do, however, *also* object to it messing with the EFI boot menu - I just didn't realise it would do that.

I can't not have a separate /boot i.e. I can't have /boot integrated into /. I could merge /boot and /boot/efi but, as suggested, I'm not sure how grub would react to that. (/ is encrypted on my system. All that is outside the LUKS container is /boot and the ESP.)

EDIT: In fact, I think I'm going to uninstall the package. It just updated and complained because it could not find the ESP at /boot. Otherwise, it would quite happily have rendered my machine unbootable as far as I can tell.

Last edited by cfr (2013-03-24 21:23:03)

WonderWoofy · 2013-03-24 21:25:44

I was thinking I might uninstall it as well, and then simply download the package and pull the binary every update. The more we discuss this, the less and less I am okay with the idea of it updating itself.

65kid · 2013-03-24 21:56:32

WonderWoofy wrote:

I am fine with the "gummiboot update" command installing as \EFI\boot\bootx64.efi, but I do not like how the "install" command creates an actual firmware entry. This might be good for other distributions where the user can't be bothered to configure this themselves, but I don't think it is really appropriate here.

Well there is the --no-variables switch. What I do not like either is that "update" creates an boot loader entry even if it was initially installed with --no-variables.

WonderWoofy wrote:

But what really bothers me about it is how it pushes "setup" (or the bios configuration menu) out of the 0000 spot. In my opinion, this is very very bad. I think it is actually you 65kid who mentioned that you had to restore the "setup" entry from a backup of efivars, yeah? Fortunately, on my machine, it created a new "setup" in the next available spot, which was 0011. But with all the different implementations of UEFI bioses, I can't imagine how this seems like a good design decision.

As you might have seen in the "samsung bricking" thread, there actually was bug in gummiboot regarding the creation of the boot loader entry (was fixed in version 28). Maybe this changes the behaviour on your machine as well?
I actually think that the vendors are not supposed to use the EFI boot entries for stuff like the BIOS Setup. I mean, these entries are supposed to be editable by the user, so what the hell are entries for the BIOS Setup doing there in the first place? This is clearly a bad design choice from the EFI vendor and you can't really blame gummiboot/efibootmgr or whatever for this.

cfr wrote:

Am going rapidly off the idea of installing gummiboot. I actually meant that I objected to it installing to \EFI\boot\bootx64.efi. *I* should get to decide what my fall back boot loader is. I do, however, *also* object to it messing with the EFI boot menu - I just didn't realise it would do that.

I don't really see the big deal about the BOOTX64.EFI file. Yes, it would be nice to have a switch for this (I'm sure upstream would add if someone asked), but I doubt that gummiboot would overwrite another bootloader that already installed itself there (if it does this is most definitively a bug).

WonderWoofy · 2013-03-24 22:13:10

65kid wrote:

...
I actually think that the vendors are not supposed to use the EFI boot entries for stuff like the BIOS Setup. I mean, these entries are supposed to be editable by the user, so what the hell are entries for the BIOS Setup doing there in the first place? This is clearly a bad design choice from the EFI vendor and you can't really blame gummiboot/efibootmgr or whatever for this.

I agree that this is poor design by the manufacturers, but I just don't understand why gummiboot insists on being 0000. I think it should look at the entries and determine from the output that it should not use the boot entry identifiers that are already in use. So I guess it is primarily a poor design decision of the manufacturers, it is also a flaw of this updater.

65kid wrote:

As you might have seen in the "samsung bricking" thread, there actually was bug in gummiboot regarding the creation of the boot loader entry (was fixed in version 28). Maybe this changes the behaviour on your machine as well?

I just got my machine back with a new motherboard, so the entry it made was made on March 14th. I wonder if maybe it was created just before this fixed version was pushed, and maybe it no longer uses 0000? I don't know, but I do remember reading about the fix in that same thread you mention here.

65kid wrote:

I don't really see the big deal about the BOOTX64.EFI file. Yes, it would be nice to have a switch for this (I'm sure upstream would add if someone asked), but I doubt that gummiboot would overwrite another bootloader that already installed itself there (if it does this is most definitively a bug).

Normally this would not be a real problem, but cfr has a rather crazy/buggy firmware. So his options for booting seem to be rather limited. So what I think he is saying is that if it were to overwrite that file, he would not be booting into his only reliable method of boot.

For me, gummiboot is what I put there anyway, so it doesn't upset me at all.

Additionally, I have noticed (meaning I hadn't tried until just now) that I cannot boot 3.8.4-1-ARCH with gummiboot. Though I just made an efibootmgr entry to directly boot that kernel and it works fine from there. Also it works from elilo as well (though this is not a surprise since it is not using the efistub). I am going to investigate a bit further and see if I can get it to work with any other boot manager.

Arch Linux

#126 2013-03-22 17:23:24

Re: Kernel 3.7 doesn't boot using EFI Stub

#127 2013-03-22 17:26:00

Re: Kernel 3.7 doesn't boot using EFI Stub

#128 2013-03-22 17:29:05

Re: Kernel 3.7 doesn't boot using EFI Stub

#129 2013-03-22 21:57:15

Re: Kernel 3.7 doesn't boot using EFI Stub

#130 2013-03-22 22:02:07

Re: Kernel 3.7 doesn't boot using EFI Stub

#131 2013-03-23 01:03:07

Re: Kernel 3.7 doesn't boot using EFI Stub

#132 2013-03-23 04:56:22

Re: Kernel 3.7 doesn't boot using EFI Stub

#133 2013-03-23 06:04:15

Re: Kernel 3.7 doesn't boot using EFI Stub

#134 2013-03-23 13:22:43

Re: Kernel 3.7 doesn't boot using EFI Stub

#135 2013-03-23 13:59:47

Re: Kernel 3.7 doesn't boot using EFI Stub

#136 2013-03-23 14:07:36

Re: Kernel 3.7 doesn't boot using EFI Stub

#137 2013-03-23 20:56:23

Re: Kernel 3.7 doesn't boot using EFI Stub

#138 2013-03-23 22:13:37

Re: Kernel 3.7 doesn't boot using EFI Stub

#139 2013-03-23 23:43:00

Re: Kernel 3.7 doesn't boot using EFI Stub

#140 2013-03-24 00:02:34

Re: Kernel 3.7 doesn't boot using EFI Stub

#141 2013-03-24 01:18:33

Re: Kernel 3.7 doesn't boot using EFI Stub

#142 2013-03-24 02:57:12

Re: Kernel 3.7 doesn't boot using EFI Stub

#143 2013-03-24 03:53:22

Re: Kernel 3.7 doesn't boot using EFI Stub

#144 2013-03-24 09:46:26

Re: Kernel 3.7 doesn't boot using EFI Stub

#145 2013-03-24 17:16:42

Re: Kernel 3.7 doesn't boot using EFI Stub

#146 2013-03-24 20:03:05

Re: Kernel 3.7 doesn't boot using EFI Stub

#147 2013-03-24 21:18:13

Re: Kernel 3.7 doesn't boot using EFI Stub

#148 2013-03-24 21:25:44

Re: Kernel 3.7 doesn't boot using EFI Stub

#149 2013-03-24 21:56:32

Re: Kernel 3.7 doesn't boot using EFI Stub

#150 2013-03-24 22:13:10

Re: Kernel 3.7 doesn't boot using EFI Stub

Board footer