You are not logged in.
Hello friends, I had my Arch working fine for a few years, and I upgraded the kernel last week and I careless saw it was telling me to run
grub-mkconfig -o /boot/grub/grub.cfg
and
grub-install
and I just did it without thinking through (I had never had any problems with Arch so I followed it, so obviously my OS didn't boot any more, it wasn't even getting the GRUB screen, so I tried booting from an installation USB stick, the same the same way as what I have done installing it a few years back and obviously not formatting the root partition and so on (I took it from my documentation and checking if there is anything new in Arch wiki):
1. mounted the root and boot partitions and ch-rooted to it
cryptsetup open /dev/sda2 encrypted
mount /dev/mapper/encrypted /mnt
mount /dev/sda1 /mnt/boot/
arch-chroot /mnt
2. checked if it had "encrypt" inside mkinitcpio.conf and generated it
grep "^HOOKS" /etc/mkinitcpio.conf
HOOKS=(base udev autodetect modconf kms keyboard keymap consolefont block encrypt filesystems fsck)
mkinitcpio -P linux
3. reinstalled GRUB
grub-install --target=x86_64-efi --efi-directory=/boot --bootloader-id=grub
grub-mkconfig -o /boot/grub/grub.cfg
4. checked if the config was correct as it wasn't booting
grep -v "^#" /etc/default/grub
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Arch"
GRUB_CMDLINE_LINUX_DEFAULT="loglevel=3 quiet"
GRUB_CMDLINE_LINUX="cryptdevice=UUID=15bb251f-e4a7-4b63-a622-50bd9f53386e:encrypted root=/dev/mapper/encrypted"
GRUB_PRELOAD_MODULES="part_gpt part_msdos"
GRUB_TIMEOUT_STYLE=menu
GRUB_TERMINAL_INPUT=console
GRUB_GFXMODE=auto
GRUB_GFXPAYLOAD_LINUX=keep
GRUB_DISABLE_RECOVERY=true
Then rebooted in as normal and it still doesn't work,I'm getting the following error when trying to boot now:
ERROR: device '/dev/mapper/encrypted' not found. Skipping fsck.
mount: /new_root: not valid filesystem type specified.
You are now bing dropped into an emergency shell.
sh: can't acces tty; job control turned off
[rootfs ]#
Any help is very much appreciated
Last edited by jabertak (2023-07-11 20:53:27)
Offline
Did you try to use initramfs fallback image?
Do you have `dm_mod` in /etc/mkinitcpio.conf MODULES= list? Did you try to rebuild initramfs image?
Add `rd.log=file` to kernel parameters (also remove `quiet`) and check `/run/initramfs/init.log` from emergency console.
Offline
Thank you dimich for your answer, back into the issue.
Did you try to use initramfs fallback image?
No I didn't, can I still do it? I executed mkinitcpio many times, if so can you direct me to the right document please?
Do you have `dm_mod` in /etc/mkinitcpio.conf MODULES= list? Did you try to rebuild initramfs image?
I corrected my first post (added `mkinitcpio -P`), so yes, I did try to rebuild the initramfs image. Also I added `dm_mod` to /etc/mkinitcpio.conf MODULES= list to try it but got the same result after rebuilding the initramfs image again
Add `rd.log=file` to kernel parameters (also remove `quiet`) and check `/run/initramfs/init.log` from emergency console.
Nice one!
I'm getting:
...
Starting systemd-udevd version 253.5-2-arch
:: running hook [udev]
:: Triggering uevents...
:: running hook [encrypt]
Waiting 10 seconds for device /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e ...
Waiting 10 seconds for device /dev/mapper/encrypted ...
Waiting 10 seconds for device /dev/mapper/encrypted ...
ERROR: device '/dev/mapper/encrypted' not found. Skipping fsck.
:: mounting '/dev/mapper/encrypted' on real root
mount: /new_root: no valid filesystem type specified.
ERROR: Failed to mount '/dev/mapper/encrypted' on real root
You are now being dropped into an emergency shell.
[rootfs ]#
Offline
Is /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e present when you get emergency shell?
What if run cryptsetup with UUID, not /dev/sda2 ?
# cryptsetup open /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e encrypted
Does UUID 15bb251f-e4a7-4b63-a622-50bd9f53386e correspond to your /dev/sda2? What is output of
# lsblk -af
?
Offline
Is /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e present when you get emergency shell?
No, it's not present because `15bb251f-e4a7-4b63-a622-50bd9f53386e` is the UUID for `encrypted`, if I run `cryptsetup open` it is present , I can mount it and read it
What if run cryptsetup with UUID, not /dev/sda2 ?
# cryptsetup open /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e encrypted
The same, it doesn't work because it's not been given an UUID yet, but if I run with /dev/disk/by-uuid/88bded0f-1804-46e9-bd77-8443e5543401 which is the UUID of /dev/sda2 and I think is what you meant, it hangs completely in boot time and doesn't give me the emergency console
cryptsetup open /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e encrypted
Device /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e is not a valid LUKS device.
Does UUID 15bb251f-e4a7-4b63-a622-50bd9f53386e correspond to your /dev/sda2? What is output of
# lsblk -af
here the output
130 root@archiso ~ # lsblk -af
NAME FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS
loop0 squashfs 4.0 0 100% /run/archiso/airootfs
sda
├─sda1 vfat FAT32 0FE2-1966 427.7M 22% /mnt/boot
└─sda2 crypto_LUKS 2 88bded0f-1804-46e9-bd77-8443e5543401
└─encrypted f2fs 1.12 15bb251f-e4a7-4b63-a622-50bd9f53386e 60.9G 74% /mnt
Offline
I think you are passing the wrong UUID. Your cryptdevice-UUID should be 88bded0f-1804-46e9-bd77-8443e5543401 which corresponds to /dev/sda2.
Never argue with an idiot, they will drag you down to their level and then beat you with experience.
It is better to light a candle than curse the darkness.
A journey of a thousand miles begins with a single step.
Offline
dimich wrote:Is /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e present when you get emergency shell?
No, it's not present because `15bb251f-e4a7-4b63-a622-50bd9f53386e` is the UUID for `encrypted`, if I run `cryptsetup open` it is present , I can mount it and read it
UUID passed with cryptdevice must be a LUKS partition. The system doesn't know what is 15bb251f-e4a7-4b63-a622-50bd9f53386e until cryptsetup open 88bded0f-1804-46e9-bd77-8443e5543401.
@kermit63 is correct, you should pass
cryptdevice=UUID=88bded0f-1804-46e9-bd77-8443e5543401:encrypted
but if I run with /dev/disk/by-uuid/88bded0f-1804-46e9-bd77-8443e5543401 which is the UUID of /dev/sda2 and I think is what you meant, it hangs completely in boot time and doesn't give me the emergency console
It looks like another next problem.
└─encrypted f2fs 1.12 15bb251f-e4a7-4b63-a622-50bd9f53386e 60.9G 74% /mnt
Does initramfs know how to mount f2fs? Try to add f2fs module to MODULES list in /etc/mkinitcpio.conf (and rebuild initramfs, of course).
Offline
jabertak wrote:└─encrypted f2fs 1.12 15bb251f-e4a7-4b63-a622-50bd9f53386e 60.9G 74% /mnt
Does initramfs know how to mount f2fs? Try to add f2fs module to MODULES list in /etc/mkinitcpio.conf (and rebuild initramfs, of course).
The "filesystems" hook takes care of adding the kernel modules for the root file system.
Offline
The "filesystems" hook takes care of adding the kernel modules for the root file system.
Indeed. Then we need to see /etc/fstab and findmnt output.
Offline
Hi dudes, here it is what I have at the moment:
[root@archiso /]# grep -vE "^#|^$" /etc/default/grub
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Arch"
GRUB_CMDLINE_LINUX_DEFAULT="loglevel=5 rd.log=file"
GRUB_CMDLINE_LINUX="cryptdevice=UUID=88bded0f-1804-46e9-bd77-8443e5543401:encrypted root=/dev/mapper/encrypted"
GRUB_PRELOAD_MODULES="part_gpt part_msdos"
GRUB_TIMEOUT_STYLE=menu
GRUB_TERMINAL_INPUT=console
GRUB_GFXMODE=auto
GRUB_GFXPAYLOAD_LINUX=keep
GRUB_DISABLE_RECOVERY=true
It used to work fine without `dm_mod` in in the `mkinitcpio.conf` but I added it anyway
[root@archiso /]# grep -vE "^$|^#" /etc/mkinitcpio.conf
MODULES=(dm_mod)
BINARIES=()
FILES=()
HOOKS=(base udev autodetect modconf kms keyboard keymap consolefont block encrypt filesystems fsck)
I tried changing the UUID of the encrypted (commented line) with the device name but the same result, at the moment it hangs completely at emergency console
[root@archiso /]# cat /etc/fstab
# /dev/mapper/encrypted
#UUID=15bb251f-e4a7-4b63-a622-50bd9f53386e / f2fs rw,relatime,lazytime,background_gc=on,nogc_merge,nodiscard,no_heap,inline_xattr,inline_data,inline_dentry,flush_merge,barrier,extent_cache,mode=adaptive,active_logs=6,alloc_mode=default,checkpoint_merge,fsync_mode=posix,memory=normal 0 1
/dev/mapper/encrypted / f2fs rw,relatime,lazytime,background_gc=on,nogc_merge,nodiscard,no_heap,inline_xattr,inline_data,inline_dentry,flush_merge,barrier,extent_cache,mode=adaptive,active_logs=6,alloc_mode=default,checkpoint_merge,fsync_mode=posix,memory=normal 0 1
# /dev/sda1
UUID=0FE2-1966 /boot vfat rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro 0 2
Offline
at the moment it hangs completely at emergency console
There are multiple places in initrd's /init where it can drop into emergency console . What exact error message is?
"It hangs completely at emergency console" means you can't run commands in the console? Or you can? Did you try to setup cryptdevice and mount rootfs manually step by step? On which step it fails? With what error message?
Offline
There are multiple places in initrd's /init where it can drop into emergency console . What exact error message is? "It hangs completely at emergency console" means you can't run commands in the console? Or you can?
- When I use the config prosted on 2023-07-18 (`/etc/default/grub` with `GRUB_CMDLINE_LINUX="cryptdevice=UUID=88bded0f-1804-46e9-bd77-8443e5543401:encrypted root=/dev/mapper/encrypted"`) it hangs completely and does not reach the emergency console, no key can be typed (sorry for the confusing message). When it hangs the message is:
[ 0.140877] x86/cpu: SGX disabled by BIOS.
[ 0.174624] MDS CPU bug present and SMT on, data leak possible, see https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more details
[ 0.174634] MMIO Stale Data CPU bug present and SMT on, data leak possible, see https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/processor_mmio_stale_data.html for more details
[ 0.174775] #3
[ 0.181568] ENERGY_PREF_BIAS: Set to 'normal', was 'performance'
[ 4.904657] usb: port power management may be unreliable
- When the `/etc/default/grub` has `GRUB_CMDLINE_LINUX="cryptdevice=UUID=15bb251f-e4a7-4b63-a622-50bd9f53386e:encrypted root=/dev/mapper/encrypted"` regardles if the `/etc/fstab` has `/dev/mapper/encrypted` or `UUID=15bb251f-e4a7-4b63-a622-50bd9f53386e`, it shows exactly the same message as above when it hangs but it adds the below just before reaching the emergency mode (rootfs)
sh: can't access tty; job control turned off
[rootfs ]# cat /run/initramfs/init.log
:: running early hook [udev]
Starting systemd-udevd version 253.5-2-arch
:: running hook [udev]
:: Triggering uevents...
:: running hook [encrypt]
Waiting 10 seconds for device /dev/disk/by-uuid/15bb251f-e4a7-4b63-a622-50bd9f53386e ...
Waiting 10 seconds for device /dev/mapper/encrypted ...
Waiting 10 seconds for device /dev/mapper/encrypted ...
ERROR: device '/dev/mapper/encrypted' not found. Skipping fsck.
:: mounting '/dev/mapper/encrypted' on real root
mount: /new_root: no valid filesystem type specified.
ERROR: Failed to mount '/dev/mapper/encrypted' on real root
You are now being dropped into an emergency shell.
[rootfs ]#
but I can still cryptsetup open either using "/dev/disk/by-uuid/88bded0f-1804-46e9-bd77-8443e5543401" or "/dev/sda2" and then mount it
I hope this is the answer to the questions?
Last edited by jabertak (2023-07-23 21:34:53)
Offline
- When I use the config prosted on 2023-07-18 (`/etc/default/grub` with `GRUB_CMDLINE_LINUX="cryptdevice=UUID=88bded0f-1804-46e9-bd77-8443e5543401:encrypted root=/dev/mapper/encrypted"`) it hangs completely and does not reach the emergency console, no key can be typed (sorry for the confusing message).
Maybe cryptsetup is waiting for password but the prompt isn't displayed for some reason.
but I can still cryptsetup open either using "/dev/disk/by-uuid/88bded0f-1804-46e9-bd77-8443e5543401" or "/dev/sda2" and then mount it
What happens if you mount it to /new_root and then exit from emergency console (Ctrl+D or type `exit`)?
Offline
Maybe cryptsetup is waiting for password but the prompt isn't displayed for some reason.
A bit of thinking from my side might have helped LOL. Thank you! that worked!!!
Offline