[Solved] Grub, LUKS2, and my installation script nightmare

NovaViper · 2022-01-25 03:06:45

Hey guys, so a little over a month ago I wrote up a bash script designed to install ArchLinux. I’m however having the strangest issues with GRUB and LUKS (the root partition is encrypted but the boot partition is not encrypted) whenever I use either the ESP locations /efi or /boot/efi for UEFI machines. I tested on my laptop and desktop a few times but mainly been doing my testing on my VM. Here’s some of my cases I’ve tried and their results:

GRUB with ESP locations either /efi or /boot/efi
1. LUKS disabled
  - BTRFS: Boots up perfectly fine, tested on my custom desktop, QEMU UEFI virtural machine with Virt-Manager, and my Thinkpad L390 Yoga.
  - EXT4: Boots up perfectly fine, tested on my custom desktop, QEMU UEFI virtural machine with Virt-Manager, and my Thinkpad L390 Yoga.
2. LUKS enabled
  - BTRFS: Immediately throws systems into grub, saying “Error No such device. unknown file system” followed by the UUID of the root directory. Tested on the laptop and VM.
  - EXT4: Immediately throws systems into grub, saying “Error No such device. unknown file system” followed by the UUID of the root directory. Tested on the laptop and VM.
GRUB with ESP location set to /boot
1. LUKS disabled
  - BTRFS: Boots up perfectly fine, tested on my custom desktop, QEMU UEFI virtural machine with Virt-Manager, and my Thinkpad L390 Yoga.
  - EXT4: Boots up perfectly fine, tested on my custom desktop, QEMU UEFI virtural machine with Virt-Manager, and my Thinkpad L390 Yoga.
2. LUKS enabled
  - BTRFS: Boots up perfectly fine, tested on my custom desktop, QEMU UEFI virtural machine with Virt-Manager, and my Thinkpad L390 Yoga.
  - EXT4: Boots up perfectly fine, tested on my custom desktop, QEMU UEFI virtural machine with Virt-Manager, and my Thinkpad L390 Yoga.

This is the sort of error it would show booth with an encrypted drive and esp is set to /efi
https://i.imgur.com/YOdmL6b.png

This here is the link to the particular commit that has the esp directory set to /efi: https://gitlab.com/NovaViper/aalis/-/co … 1a77a02996. I honestly have no idea what could be the issue, could someone lend me some insight on how I can fix this issue?

Last edited by NovaViper (2022-02-04 18:00:20)

V1del · 2022-01-25 07:40:44

Please don't embed huge images directly.

By default a GRUB install in the way you are doing it always looks for it's general configuration under /boot/grub. If you just change the mount point and the --efi-directory argument then you simply change the EFI portion of GRUB to that location, it will still look for it's configuration under "/boot/grub" (or rather, whatever partition/filesystem was mounted to /boot during grub-install invocation) if you want to change the config lookup path you need to explicitly and additionally pass the --boot-directory= option to grub-install which would need to be /efi or /boot/efi respectively.

Last edited by V1del (2022-01-25 07:45:12)

NovaViper · 2022-01-25 07:47:10

V1del wrote:

Please don't embed huge images directly.
By default a GRUB install in the way you are doing it always looks for it's general configuration under /boot/grub. If you just change the mount point then you simply change the EFI portion of GRUB to that location, it will still look for it's configuration under "/boot/grub" (or rather, whatever partition/filesystem was mounted to /boot during grub-install invocation) if you want to change the config lookup path you need to explicitly and additionally pass the --boot-directory= option to grub-install which would need to be /efi or /boot/efi respectively.

The main issue im having is grub only seems to run when I install it to /boot, but any other directory (even if it's under /boot/), it refuses to start, giving me the errors I've noted in my original post. And in particular, it throws the error when I have LUKS enabled, whenever I don't it boots up just fine.

I did actually figure out that I needed to separate the /boot path from the / folder since that gets encrypted by LUKS which did actually get it to where it would reboot but now I have another issue. After I boot it up and tell it to unlock my system, the computer seems to hang and never actually get into the operating system. I made the changes in this commit: https://gitlab.com/NovaViper/aalis/-/co … f0d32c32de

kousu · 2022-01-26 07:07:22

Hey @NovaViper, I think this is a cool project! I had the same idea about the same time you did it, because I also want to standardize my systems and, particularly, use disk encryption by default, but I always forget the details every time. I haven't worked that part into my script yet so if you don't mind I'll be studying yours to get some tips

I don't immediately see why this shouldn't be working for you. I tried reading grub-install and grub-mkconfig and found this:

#! /bin/sh
set -e

# Generate grub.cfg by inspecting /boot contents.
# Copyright (C) 2006,2007,2008,2009,2010 Free Software Foundation, Inc.
#

so it seems like grub-mkconfig probably wasn't designed to handle non-/boot setups, and is buggy when combined with LUKS. It doesn't have a --efi-directory option like grub-install does. It's probably doing some fiddly lsblk shennigans and those aren't able to cope with LUKS. https://wiki.archlinux.org/title/EFI_sy … _partition made it sound to me like /boot is the only well-tested EFI set up.

But I'll try to find some time after work tomorrow to run your script and see if I can see better what's going on, especially since it's something I've been procrastinating on dealing with for my own

V1del · 2022-01-26 08:02:14

General disclaimer, while it's fine to ask for help with a script for personal use, as soon as someone else just "runs" this and expects help here they should be referring to you/you should have relevant supporting infrastructure in place to help your users, as this will fall into unsupported third party guide territory.

@NovaViper I think you didn't quite understand where I went with this. In essence there are two things happening when you run a grub-install command.

GRUB copies it's main program (with themes and other static config) and configuration to the --boot-directory (which defaults to /boot/grub)
GRUB copies a smaller part to the appropriate space, in case of an UEFI install to the partition you pass with the --efi-directory line

Now what happens if you do not adjust that first flag, GRUB puts its main program onto an "inaccessible", encrypted drive (technically you can configure GRUB to support and decrypt a LUKS drive under certain limitations but I'd have to look that up as well) and correctly installs the secondary part on the ESP. When you boot GRUB does not find it's main program and leaves you on the rescue shell.

However if you want compatibility with systemd-boot as well it makes sense anyway to just make a big enough ESP and mount that to /boot, or alternatively, as you've opted now a secondary unencrypted FAT /boot partition, but that's not entirely necessary you could just make the ESP big enough in the first place (though it will help with an MBR boot in any case)

NovaViper · 2022-01-28 15:17:44

V1del wrote:

General disclaimer, while it's fine to ask for help with a script for personal use, as soon as someone else just "runs" this and expects help here they should be referring to you/you should have relevant supporting infrastructure in place to help your users, as this will fall into unsupported third party guide territory.
@NovaViper I think you didn't quite understand where I went with this. In essence there are two things happening when you run a grub-install command.
GRUB copies it's main program (with themes and other static config) and configuration to the --boot-directory (which defaults to /boot/grub)
GRUB copies a smaller part to the appropriate space, in case of an UEFI install to the partition you pass with the --efi-directory line
Now what happens if you do not adjust that first flag, GRUB puts its main program onto an "inaccessible", encrypted drive (technically you can configure GRUB to support and decrypt a LUKS drive under certain limitations but I'd have to look that up as well) and correctly installs the secondary part on the ESP. When you boot GRUB does not find it's main program and leaves you on the rescue shell.
However if you want compatibility with systemd-boot as well it makes sense anyway to just make a big enough ESP and mount that to /boot, or alternatively, as you've opted now a secondary unencrypted FAT /boot partition, but that's not entirely necessary you could just make the ESP big enough in the first place (though it will help with an MBR boot in any case)

Ah, that second method worked! Turns out I had to mount that /boot directory to the BIOSBOOT partition, once I did that it finally started booting properly. I created a ext2 filesystem for /boot too. Thanks for the help and explanation!

ALso, by any chance, is it possible to have the /boot partition set as a filesystem like ext4 or btrfs?

Last edited by NovaViper (2022-01-28 15:19:55)

V1del · 2022-01-28 15:52:20

With GRUB, yes, with systemd-boot, no. (of course going under the assumption you are using the default setup of generating your kernel images/initramfs into /boot)

GRUB has read support for most UNIX file systems, systemd-boot can only do what the EFI supports natively which needs to be a FAT variant for the large majority of usecases.

NovaViper · 2022-01-28 18:47:28

V1del wrote:

With GRUB, yes, with systemd-boot, no. (of course going under the assumption you are using the default setup of generating your kernel images/initramfs into /boot)
GRUB has read support for most UNIX file systems, systemd-boot can only do what the EFI supports natively which needs to be a FAT variant for the large majority of usecases.

Ah ok, thank you! I did run a test a bit ago with /boot as btrfs and saw that the system didn't boot, should the EFI partition also be btrfs since /boot is btrfs?. Also, should the BIOSBOOT partition be 100MB and EFIBOOT partition be 250mb? I saw another script that had that sort of configuration (and was actually the script that lead me to being able to fix the booting issues)

V1del · 2022-01-29 17:25:59

No, an EFI partition must be a FAT variant and unencrypted. The ESP primary purpose is so that your mainboard has something to boot from, so it needs to be a simple file system your mainboard can read before an actual operating system takes over. How big you want to make this and whether you want to combine that with a distinct, separate /boot partition or not depends on what you want to do and why exactly. For simplicity and especially if you intend to be able to install systemd-boot or GRUB interchangeably, I'd suggest you make the ESP big enough, e.g. 250MB and just make that your /boot. This will work properly without too much special handling for both GRUB and systemd-boot.

Your questions suggest to me that you have quite some misunderstandings of how UEFI works, which you really need to fill up if you intend to have a "general purpose" installation script that actually works properly. You should read

https://www.happyassassin.net/posts/201 … work-then/ - for an actual overview of what exactly needs to happen

and maybe for some general linux/arch specifics
https://wiki.archlinux.org/title/Unifie … _Interface
https://wiki.archlinux.org/title/EFI_system_partition

NovaViper · 2022-01-29 18:32:35

V1del wrote:

No, an EFI partition must be a FAT variant and unencrypted. The ESP primary purpose is so that your mainboard has something to boot from, so it needs to be a simple file system your mainboard can read before an actual operating system takes over. How big you want to make this and whether you want to combine that with a distinct, separate /boot partition or not depends on what you want to do and why exactly. For simplicity and especially if you intend to be able to install systemd-boot or GRUB interchangeably, I'd suggest you make the ESP big enough, e.g. 250MB and just make that your /boot. This will work properly without too much special handling for both GRUB and systemd-boot.
Your questions suggest to me that you have quite some misunderstandings of how UEFI works, which you really need to fill up if you intend to have a "general purpose" installation script that actually works properly. You should read
https://www.happyassassin.net/posts/201 … work-then/ - for an actual overview of what exactly needs to happen
and maybe for some general linux/arch specifics
https://wiki.archlinux.org/title/Unifie … _Interface
https://wiki.archlinux.org/title/EFI_system_partition

Originally, I wanted to make the esp partition use /boot/efi at the request of a user as they wanted to be able to dual boot systems and not have kernel updates mess with the boot partition, but setting the partition to point to /boot/efi ended up causing alot of issues with booting (especially with LUKS encryption enabled), and ended up having to create another partition specifically for /boot (which is what the BIOSBOOT partition is) and had the EFIBOOT partition set to /boot/efi.

Speaking of bootloaders.. I gave the script a spin with systemd-boot as the boot loader and now that doesn't show the OS as an option to boot to.
This the code used for creating the loader file for systemd-boot

V1del · 2022-01-29 18:51:58

Please don't fullquote if you directly reply to the post before yours.

Yes the users concern/request makes complete sense for GRUB and will break for systemd-boot unless you either set the two up differently from a partitions perspective, or make the partition setup in such a way that both loaders can handle it, whereas you will always need to keep systemd-boot in mind since it's the less featured one. That systemd-boot doesn't work anymore makes complete sense, as systemd-boot can only boot kernels from a FAT partition but you changed /boot to ext2.

For the current setup with a distinct /boot partition and a distinct ESP if you want to retain systemd-boot compatibility without changing anything else your /boot partition needs to meet the requirements outlined in: https://wiki.archlinux.org/title/System … g_XBOOTLDR

NovaViper · 2022-01-29 19:17:01

Oops, sorry for the quoting! I have been looking at the XBOOTLDR specification, but I mainly can't figured out what hex code for gdisk I need to select in order to get the "Extended Linux Boot" type of partition under the GPT partition type

Last edited by NovaViper (2022-01-29 19:17:49)

V1del · 2022-01-29 19:26:35

You can pass the full GUID as is if there's no direct hexcode... hexcodes are just premade abbreviations of a much bigger GUID, you can just pass that instead of the hexcode.

Last edited by V1del (2022-01-29 19:27:57)

NovaViper · 2022-01-29 20:52:34

Ah ok! So I just pass the id: bc13c2ff-59e6-4262-a352-b275fd6f7172 to gdisk (or in my case of using the script, gdisk)?

V1del · 2022-01-29 20:57:17

That should be the case yes

NovaViper · 2022-02-04 17:36:46

Sorry for the delay! So I went and did the changes, but sadly Systemd-boot still wasn't detecting the install <_>

I might just go and drop support for Systemd-boot since I can't figure out a better way to partition for both types in the beginning of the installer

V1del · 2022-02-04 17:45:05

Because your XBOOTLDR partition is still ext2, which systemd-boot can't read.

In any case this is going beyond the original scope here, if your original question is answered, please mark it as [SOLVED] by editing the title in your first post. And make new topics for new questions.

Last edited by V1del (2022-02-04 17:47:16)

NovaViper · 2022-02-04 18:02:52

I also tried it without formatting it at all aswell. But yeah, i'll go and close this topic and make another one in reguards to systemd-boot

V1del · 2022-02-04 18:14:40

Again: systemd-boot can only read from a FAT partition whether that's FAT32 or FAT16 shouldn't matter, but one of these two.

Arch Linux

#1 2022-01-25 03:06:45

[Solved] Grub, LUKS2, and my installation script nightmare

#2 2022-01-25 07:40:44

Re: [Solved] Grub, LUKS2, and my installation script nightmare

#3 2022-01-25 07:47:10

Re: [Solved] Grub, LUKS2, and my installation script nightmare

#4 2022-01-26 07:07:22

Re: [Solved] Grub, LUKS2, and my installation script nightmare

#5 2022-01-26 08:02:14

Re: [Solved] Grub, LUKS2, and my installation script nightmare

#6 2022-01-28 15:17:44

Re: [Solved] Grub, LUKS2, and my installation script nightmare

#7 2022-01-28 15:52:20

Re: [Solved] Grub, LUKS2, and my installation script nightmare

#8 2022-01-28 18:47:28

Re: [Solved] Grub, LUKS2, and my installation script nightmare

#9 2022-01-29 17:25:59

Re: [Solved] Grub, LUKS2, and my installation script nightmare

#10 2022-01-29 18:32:35

Re: [Solved] Grub, LUKS2, and my installation script nightmare

#11 2022-01-29 18:51:58

Re: [Solved] Grub, LUKS2, and my installation script nightmare

#12 2022-01-29 19:17:01

Re: [Solved] Grub, LUKS2, and my installation script nightmare

#13 2022-01-29 19:26:35

Re: [Solved] Grub, LUKS2, and my installation script nightmare

#14 2022-01-29 20:52:34

Re: [Solved] Grub, LUKS2, and my installation script nightmare

#15 2022-01-29 20:57:17

Re: [Solved] Grub, LUKS2, and my installation script nightmare

#16 2022-02-04 17:36:46

Re: [Solved] Grub, LUKS2, and my installation script nightmare

#17 2022-02-04 17:45:05

Re: [Solved] Grub, LUKS2, and my installation script nightmare

#18 2022-02-04 18:02:52

Re: [Solved] Grub, LUKS2, and my installation script nightmare

#19 2022-02-04 18:14:40

Re: [Solved] Grub, LUKS2, and my installation script nightmare

Board footer