You are not logged in.

Pages: 1

#1 2023-07-02 19:39:07

jabertak
Member
Registered: 2018-09-24
Posts: 9

Unable to boot with GRUB and encrypted root partition

Hello friends, I had my Arch working fine for a few years, and I upgraded the kernel last week and I careless saw it was telling me to run 

grub-mkconfig -o /boot/grub/grub.cfg

and 

grub-install

and I just did it without thinking through (I had never had any problems with Arch so I followed it, so obviously my OS didn't boot any more, it wasn't even getting the GRUB screen, so I tried booting from an installation USB stick, the same the same way as what I have done installing it a few years back and obviously not formatting the root partition and so on (I took it from my documentation and checking if there is anything new in Arch wiki):
1. mounted the root and boot partitions and ch-rooted to it

cryptsetup open /dev/sda2 encrypted
mount /dev/mapper/encrypted /mnt
mount /dev/sda1 /mnt/boot/
arch-chroot /mnt

2. checked if it had  "encrypt" inside mkinitcpio.conf and generated it

grep "^HOOKS" /etc/mkinitcpio.conf
HOOKS=(base udev autodetect modconf kms keyboard keymap consolefont block encrypt filesystems fsck)
mkinitcpio -P linux

3. reinstalled GRUB

grub-install --target=x86_64-efi --efi-directory=/boot --bootloader-id=grub
grub-mkconfig -o /boot/grub/grub.cfg

4. checked if the config was correct as it wasn't booting

 grep -v "^#" /etc/default/grub
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Arch"
GRUB_CMDLINE_LINUX_DEFAULT="loglevel=3 quiet"
GRUB_CMDLINE_LINUX="cryptdevice=UUID=15bb251f-e4a7-4b63-a622-50bd9f53386e:encrypted root=/dev/mapper/encrypted"
GRUB_PRELOAD_MODULES="part_gpt part_msdos"
GRUB_TIMEOUT_STYLE=menu
GRUB_TERMINAL_INPUT=console
GRUB_GFXMODE=auto
GRUB_GFXPAYLOAD_LINUX=keep
GRUB_DISABLE_RECOVERY=true

Then rebooted in as normal and it still doesn't work,I'm getting the following error when trying to boot now:

   ERROR: device '/dev/mapper/encrypted' not found. Skipping fsck.
mount: /new_root: not valid filesystem type specified.
You are now bing dropped into an emergency shell.
sh: can't acces tty; job control turned off
[rootfs ]#

Any help is very much appreciated

Last edited by jabertak (2023-07-11 20:53:27)

Offline

#2 2023-07-03 03:03:34

dimich
Member
From: Kharkiv, Ukraine
Registered: 2009-11-03
Posts: 326

Re: Unable to boot with GRUB and encrypted root partition

Did you try to use initramfs fallback image?

Do you have `dm_mod` in /etc/mkinitcpio.conf MODULES= list? Did you try to rebuild initramfs image?

Add `rd.log=file` to kernel parameters (also remove `quiet`) and check `/run/initramfs/init.log` from emergency console.

Offline

#3 2023-07-11 19:01:56

jabertak
Member
Registered: 2018-09-24
Posts: 9

Re: Unable to boot with GRUB and encrypted root partition

Thank you dimich for your answer, back into the issue.

Did you try to use initramfs fallback image?

No I didn't, can I still do it? I executed mkinitcpio many times, if so can you direct me to the right document please?

Do you have `dm_mod` in /etc/mkinitcpio.conf MODULES= list? Did you try to rebuild initramfs image?

I corrected my first post (added `mkinitcpio -P`), so yes, I did try to rebuild the initramfs image. Also I added `dm_mod` to  /etc/mkinitcpio.conf MODULES= list to try it but got the same result after rebuilding the initramfs image again

Add `rd.log=file` to kernel parameters (also remove `quiet`) and check `/run/initramfs/init.log` from emergency console.

Nice one!
I'm getting:

...
Starting systemd-udevd version 253.5-2-arch
:: running hook [udev]
:: Triggering uevents...
:: running hook [encrypt]
Waiting 10 seconds for device /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e ...
Waiting 10 seconds for device /dev/mapper/encrypted ...
Waiting 10 seconds for device /dev/mapper/encrypted ...
ERROR: device '/dev/mapper/encrypted' not found. Skipping fsck.
:: mounting '/dev/mapper/encrypted' on real root
mount: /new_root: no valid filesystem type specified.
ERROR: Failed to mount '/dev/mapper/encrypted' on real root
You are now being dropped into an emergency shell.
[rootfs ]#

Offline

#4 2023-07-13 17:21:49

dimich
Member
From: Kharkiv, Ukraine
Registered: 2009-11-03
Posts: 326

Re: Unable to boot with GRUB and encrypted root partition

Is /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e present when you get emergency shell?

What if run cryptsetup with UUID, not /dev/sda2 ?

# cryptsetup open  /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e encrypted

Does UUID 15bb251f-e4a7-4b63-a622-50bd9f53386e correspond to your /dev/sda2? What is output of

# lsblk -af

?

Offline

#5 2023-07-14 21:24:44

jabertak
Member
Registered: 2018-09-24
Posts: 9

Re: Unable to boot with GRUB and encrypted root partition

dimich wrote:

Is /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e present when you get emergency shell?

No, it's not present because `15bb251f-e4a7-4b63-a622-50bd9f53386e` is the UUID for `encrypted`, if I run `cryptsetup open` it is present , I can mount it and read it

dimich wrote:

What if run cryptsetup with UUID, not /dev/sda2 ?

# cryptsetup open  /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e encrypted

The same, it doesn't work because it's not been given an UUID yet, but if I run with /dev/disk/by-uuid/88bded0f-1804-46e9-bd77-8443e5543401 which is the UUID of /dev/sda2 and I think is what you meant, it hangs completely in boot time and doesn't give me the emergency console

cryptsetup open /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e encrypted
Device /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e is not a valid LUKS device.
dimich wrote:

Does UUID 15bb251f-e4a7-4b63-a622-50bd9f53386e correspond to your /dev/sda2? What is output of

# lsblk -af

here the output

130 root@archiso ~ # lsblk -af
NAME          FSTYPE      FSVER            LABEL       UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
loop0         squashfs    4.0                                                                     0   100% /run/archiso/airootfs
sda                                                                                                        
├─sda1           vfat        FAT32                        0FE2-1966                             427.7M    22% /mnt/boot
└─sda2           crypto_LUKS 2                            88bded0f-1804-46e9-bd77-8443e5543401                
  └─encrypted f2fs        1.12                         15bb251f-e4a7-4b63-a622-50bd9f53386e   60.9G    74% /mnt

Offline

#6 2023-07-15 03:47:51

kermit63
Member
Registered: 2018-07-04
Posts: 312

Re: Unable to boot with GRUB and encrypted root partition

I think you are passing the wrong UUID. Your cryptdevice-UUID should be 88bded0f-1804-46e9-bd77-8443e5543401 which corresponds to /dev/sda2.

Never argue with an idiot, they will drag you down to their level and then beat you with experience.
It is better to light a candle than curse the darkness.
A journey of a thousand miles begins with a single step.

Offline

#7 2023-07-15 08:33:49

dimich
Member
From: Kharkiv, Ukraine
Registered: 2009-11-03
Posts: 326

Re: Unable to boot with GRUB and encrypted root partition

jabertak wrote:
dimich wrote:

Is /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e present when you get emergency shell?

No, it's not present because `15bb251f-e4a7-4b63-a622-50bd9f53386e` is the UUID for `encrypted`, if I run `cryptsetup open` it is present , I can mount it and read it

UUID passed with cryptdevice must be a LUKS partition. The system doesn't know what is 15bb251f-e4a7-4b63-a622-50bd9f53386e until cryptsetup open 88bded0f-1804-46e9-bd77-8443e5543401.
@kermit63 is correct, you should pass

cryptdevice=UUID=88bded0f-1804-46e9-bd77-8443e5543401:encrypted
jabertak wrote:

but if I run with /dev/disk/by-uuid/88bded0f-1804-46e9-bd77-8443e5543401 which is the UUID of /dev/sda2 and I think is what you meant, it hangs completely in boot time and doesn't give me the emergency console

It looks like another next problem.

jabertak wrote:
  └─encrypted f2fs        1.12                         15bb251f-e4a7-4b63-a622-50bd9f53386e   60.9G    74% /mnt

Does initramfs know how to mount f2fs? Try to add f2fs module to MODULES list in /etc/mkinitcpio.conf (and rebuild initramfs, of course).

Offline

#8 2023-07-15 09:09:32

nl6720
The Evil Wiki Admin
Registered: 2016-07-02
Posts: 671

Re: Unable to boot with GRUB and encrypted root partition

dimich wrote:
jabertak wrote:
  └─encrypted f2fs        1.12                         15bb251f-e4a7-4b63-a622-50bd9f53386e   60.9G    74% /mnt

Does initramfs know how to mount f2fs? Try to add f2fs module to MODULES list in /etc/mkinitcpio.conf (and rebuild initramfs, of course).

The "filesystems" hook takes care of adding the kernel modules for the root file system.

Offline

#9 2023-07-15 09:31:08

dimich
Member
From: Kharkiv, Ukraine
Registered: 2009-11-03
Posts: 326

Re: Unable to boot with GRUB and encrypted root partition

nl6720 wrote:

The "filesystems" hook takes care of adding the kernel modules for the root file system.

Indeed. Then we need to see /etc/fstab and findmnt output.

Offline

#10 2023-07-18 08:39:06

jabertak
Member
Registered: 2018-09-24
Posts: 9

Re: Unable to boot with GRUB and encrypted root partition

Hi dudes, here it is what I have at the moment:

[root@archiso /]# grep -vE "^#|^$" /etc/default/grub
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Arch"
GRUB_CMDLINE_LINUX_DEFAULT="loglevel=5 rd.log=file"
GRUB_CMDLINE_LINUX="cryptdevice=UUID=88bded0f-1804-46e9-bd77-8443e5543401:encrypted root=/dev/mapper/encrypted"
GRUB_PRELOAD_MODULES="part_gpt part_msdos"
GRUB_TIMEOUT_STYLE=menu
GRUB_TERMINAL_INPUT=console
GRUB_GFXMODE=auto
GRUB_GFXPAYLOAD_LINUX=keep
GRUB_DISABLE_RECOVERY=true

It used to work fine without `dm_mod` in in the `mkinitcpio.conf` but I added it anyway

[root@archiso /]# grep -vE "^$|^#" /etc/mkinitcpio.conf
MODULES=(dm_mod)
BINARIES=()
FILES=()
HOOKS=(base udev autodetect modconf kms keyboard keymap consolefont block encrypt filesystems fsck)

I tried changing the UUID of the encrypted (commented line) with the device name but the same result, at the moment it hangs completely at emergency console

[root@archiso /]# cat /etc/fstab
# /dev/mapper/encrypted
#UUID=15bb251f-e4a7-4b63-a622-50bd9f53386e	/         	f2fs      	rw,relatime,lazytime,background_gc=on,nogc_merge,nodiscard,no_heap,inline_xattr,inline_data,inline_dentry,flush_merge,barrier,extent_cache,mode=adaptive,active_logs=6,alloc_mode=default,checkpoint_merge,fsync_mode=posix,memory=normal	0 1
/dev/mapper/encrypted   			/         	f2fs      	rw,relatime,lazytime,background_gc=on,nogc_merge,nodiscard,no_heap,inline_xattr,inline_data,inline_dentry,flush_merge,barrier,extent_cache,mode=adaptive,active_logs=6,alloc_mode=default,checkpoint_merge,fsync_mode=posix,memory=normal	0 1
# /dev/sda1
UUID=0FE2-1966      	/boot     	vfat      	rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro	0 2

Offline

#11 2023-07-19 22:24:05

dimich
Member
From: Kharkiv, Ukraine
Registered: 2009-11-03
Posts: 326

Re: Unable to boot with GRUB and encrypted root partition

jabertak wrote:

at the moment it hangs completely at emergency console

There are multiple places in initrd's /init where it can drop into emergency console . What exact error message is?
"It hangs completely at emergency console" means you can't run commands in the console? Or you can? Did you try to setup cryptdevice and mount rootfs manually step by step? On which step it fails? With what error message?

Offline

#12 2023-07-23 18:50:55

jabertak
Member
Registered: 2018-09-24
Posts: 9

Re: Unable to boot with GRUB and encrypted root partition

dimich wrote:

There are multiple places in initrd's /init where it can drop into emergency console . What exact error message is? "It hangs completely at emergency console" means you can't run commands in the console? Or you can?

- When I use the config prosted on 2023-07-18 (`/etc/default/grub` with `GRUB_CMDLINE_LINUX="cryptdevice=UUID=88bded0f-1804-46e9-bd77-8443e5543401:encrypted root=/dev/mapper/encrypted"`) it hangs completely and does not reach the emergency console, no key can be typed (sorry for the confusing message). When it hangs the message is:

[  0.140877] x86/cpu: SGX disabled by BIOS.
[  0.174624] MDS CPU bug present and SMT on, data leak possible, see https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more details
[  0.174634] MMIO Stale Data CPU bug present and SMT on, data leak possible, see https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/processor_mmio_stale_data.html for more details
[  0.174775] #3
[  0.181568] ENERGY_PREF_BIAS: Set to 'normal', was 'performance'
[  4.904657] usb: port power management may be unreliable

- When the `/etc/default/grub` has `GRUB_CMDLINE_LINUX="cryptdevice=UUID=15bb251f-e4a7-4b63-a622-50bd9f53386e:encrypted root=/dev/mapper/encrypted"` regardles if the `/etc/fstab` has `/dev/mapper/encrypted` or `UUID=15bb251f-e4a7-4b63-a622-50bd9f53386e`, it shows exactly the same message as above when it hangs but it adds the below just before reaching the emergency mode (rootfs)

sh: can't access tty; job control turned off
[rootfs ]# cat /run/initramfs/init.log
:: running early hook [udev]
Starting systemd-udevd version 253.5-2-arch
:: running hook [udev]
:: Triggering uevents...
:: running hook [encrypt]
Waiting 10 seconds for device /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e ...
Waiting 10 seconds for device /dev/mapper/encrypted ...
Waiting 10 seconds for device /dev/mapper/encrypted ...
ERROR: device '/dev/mapper/encrypted' not found. Skipping fsck.
:: mounting '/dev/mapper/encrypted' on real root
mount: /new_root: no valid filesystem type specified.
ERROR: Failed to mount '/dev/mapper/encrypted' on real root
You are now being dropped into an emergency shell.
[rootfs ]#

   but I can still cryptsetup open either using "/dev/disk/by-uuid/88bded0f-1804-46e9-bd77-8443e5543401" or "/dev/sda2" and then mount it
I hope this is the answer to the questions?

Last edited by jabertak (2023-07-23 21:34:53)

Offline

#13 2023-07-23 22:58:19

dimich
Member
From: Kharkiv, Ukraine
Registered: 2009-11-03
Posts: 326

Re: Unable to boot with GRUB and encrypted root partition

jabertak wrote:

- When I use the config prosted on 2023-07-18 (`/etc/default/grub` with `GRUB_CMDLINE_LINUX="cryptdevice=UUID=88bded0f-1804-46e9-bd77-8443e5543401:encrypted root=/dev/mapper/encrypted"`) it hangs completely and does not reach the emergency console, no key can be typed (sorry for the confusing message).

Maybe cryptsetup is waiting for password but the prompt isn't displayed for some reason.

jabertak wrote:

but I can still cryptsetup open either using "/dev/disk/by-uuid/88bded0f-1804-46e9-bd77-8443e5543401" or "/dev/sda2" and then mount it

What happens if you mount it to /new_root and then exit from emergency console (Ctrl+D or type `exit`)?

Offline

#14 2023-07-28 20:47:06

jabertak
Member
Registered: 2018-09-24
Posts: 9

Re: Unable to boot with GRUB and encrypted root partition

dimich wrote:

Maybe cryptsetup is waiting for password but the prompt isn't displayed for some reason.

A bit of thinking from my side might have helped LOL. Thank you! that worked!!!

Offline

Pages: 1

Board footer

Powered by FluxBB