Arch Linux

stfischr

Member

Registered: 2020-04-29

Posts: 6

Timed out waiting - LVM on LUKS on mdadm with systemd and Grub

Hi, i just reinstalled my server but it won't boot.

It never asks for my password, so i imagine that somewhere i need to provide more/different kernel parameters, but witch ones?
Error message is:

[ TIME ] Timed out waiting for device /dev/archvg/root

followed by some [DEPEND] messages and an emergency mode where i can only hit Enter

Some details on my Layout:

 -> sde: USB-Stick with MBR and Grub
 |-> sde1: ext4 boot partition

 -> sda: |
 -> sdb: |-> md0: mdadm raid5 -> cryptdev: LUKS2 container -> archvg: LVM -> root: ext4 root partition
 -> sdc: |

Yes i know its al little special, but i had this exact layout running without systemd-hooks previously. Remember i had to fight hard to get that running but i managed it somehow.

/etc/default/grub:

# GRUB boot loader configuration

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Arch"
GRUB_CMDLINE_LINUX_DEFAULT="loglevel=3 quiet rd.luks.name=UUID=7fa28d88-133b-40f9-974c-5d0bec44c083=cryptdev rd.luks.options=discard root=/dev/archvg/root libata.force=noncq"
GRUB_CMDLINE_LINUX=""

# Preload both GPT and MBR modules so that they are not missed
#GRUB_PRELOAD_MODULES="part_gpt part_msdos"
GRUB_PRELOAD_MODULES="part_msdos"

# Uncomment to enable booting from LUKS encrypted devices
#GRUB_ENABLE_CRYPTODISK=y

# Set to 'countdown' or 'hidden' to change timeout behavior,
# press ESC key to display menu.
GRUB_TIMEOUT_STYLE=menu

# Uncomment to use basic console
GRUB_TERMINAL_INPUT=console

# Uncomment to disable graphical terminal
#GRUB_TERMINAL_OUTPUT=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
GRUB_GFXMODE=auto

# Uncomment to allow the kernel use the same resolution used by grub
GRUB_GFXPAYLOAD_LINUX=keep

# Uncomment if you want GRUB to pass to the Linux kernel the old parameter
# format "root=/dev/xxx" instead of "root=/dev/disk/by-uuid/xxx"
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
GRUB_DISABLE_RECOVERY=true

# Uncomment and set to the desired menu colors.  Used by normal and wallpaper
# modes only.  Entries specified as foreground/background.
#GRUB_COLOR_NORMAL="light-blue/black"
#GRUB_COLOR_HIGHLIGHT="light-cyan/blue"

# Uncomment one of them for the gfx desired, a image background or a gfxtheme
#GRUB_BACKGROUND="/path/to/wallpaper"
#GRUB_THEME="/path/to/gfxtheme"

# Uncomment to get a beep at GRUB start
#GRUB_INIT_TUNE="480 440 1"

# Uncomment to make GRUB remember the last selection. This requires
# setting 'GRUB_DEFAULT=saved' above.
#GRUB_SAVEDEFAULT="true"

/boot/grub/grub.cfg:

#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
insmod part_msdos
if [ -s $prefix/grubenv ]; then
  load_env
fi
if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   set boot_once=true
else
   set default="0"
fi

if [ x"${feature_menuentry_id}" = xy ]; then
  menuentry_id_option="--id"
else
  menuentry_id_option=""
fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
  save_env saved_entry
  set prev_saved_entry=
  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
    save_env saved_entry
  fi
}

function load_video {
  if [ x$feature_all_video_module = xy ]; then
    insmod all_video
  else
    insmod efi_gop
    insmod efi_uga
    insmod ieee1275_fb
    insmod vbe
    insmod vga
    insmod video_bochs
    insmod video_cirrus
  fi
}

if [ x$feature_default_font_path = xy ] ; then
   font=unicode
else
insmod lvm
insmod ext2
set root='lvmid/A8hrJ3-lAOz-waaD-1asx-FfHW-cMI3-srLIEY/NO4SF2-vGEc-BFCf-LTdp-DpWK-aFbq-K1fg1z'
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=root --hint='lvmid/A8hrJ3-lAOz-waaD-1asx-FfHW-cMI3-srLIEY/NO4SF2-vGEc-BFCf-LTdp-DpWK-aFbq-K1fg1z'  b947cbab-5bfa-4df3-886a-70ad148b036f
else
  search --no-floppy --fs-uuid --set=root b947cbab-5bfa-4df3-886a-70ad148b036f
fi
    font="/usr/share/grub/unicode.pf2"
fi

if loadfont $font ; then
  set gfxmode=auto
  load_video
  insmod gfxterm
  set locale_dir=$prefix/locale
  set lang=en_US
  insmod gettext
fi
terminal_input console
terminal_output gfxterm
if [ x$feature_timeout_style = xy ] ; then
  set timeout_style=menu
  set timeout=5
# Fallback normal timeout code in case the timeout_style feature is
# unavailable.
else
  set timeout=5
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/10_linux ###
menuentry 'Arch Linux' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-b947cbab-5bfa-4df3-886a-70ad148b036f' {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_msdos
        insmod ext2
        set root='hd4,msdos1'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint-ieee1275='ieee1275/(null)/sas/disk@0,msdos1' --hint-bios=hd4,msdos1 --hint-efi=hd4,msdos1 --hint-baremetal=ahci4,msdos1  0c1a6d11-1ecd-4696-b90b-7764f36bc1ab
        else
          search --no-floppy --fs-uuid --set=root 0c1a6d11-1ecd-4696-b90b-7764f36bc1ab
        fi
        echo    'Loading Linux linux ...'
        linux   /vmlinuz-linux root=/dev/mapper/archvg-root rw  loglevel=3 quiet rd.luks.name=UUID=7fa28d88-133b-40f9-974c-5d0bec44c083=cryptdev rd.luks.options=discard root=/dev/archvg/root libata.force=noncq
        echo    'Loading initial ramdisk ...'
        initrd  /initramfs-linux.img
}
submenu 'Advanced options for Arch Linux' $menuentry_id_option 'gnulinux-advanced-b947cbab-5bfa-4df3-886a-70ad148b036f' {
        menuentry 'Arch Linux, with Linux linux' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-linux-advanced-b947cbab-5bfa-4df3-886a-70ad148b036f' {
                load_video
                set gfxpayload=keep
                insmod gzio
                insmod part_msdos
                insmod ext2
                set root='hd4,msdos1'
                if [ x$feature_platform_search_hint = xy ]; then
                  search --no-floppy --fs-uuid --set=root --hint-ieee1275='ieee1275/(null)/sas/disk@0,msdos1' --hint-bios=hd4,msdos1 --hint-efi=hd4,msdos1 --hint-baremetal=ahci4,msdos1  0c1a6d11-1ecd-4696-b90b-7764f36bc1ab
                else
                  search --no-floppy --fs-uuid --set=root 0c1a6d11-1ecd-4696-b90b-7764f36bc1ab
                fi
                echo    'Loading Linux linux ...'
                linux   /vmlinuz-linux root=/dev/mapper/archvg-root rw  loglevel=3 quiet rd.luks.name=UUID=7fa28d88-133b-40f9-974c-5d0bec44c083=cryptdev rd.luks.options=discard root=/dev/archvg/root libata.force=noncq
                echo    'Loading initial ramdisk ...'
                initrd  /initramfs-linux.img
        }
        menuentry 'Arch Linux, with Linux linux (fallback initramfs)' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-linux-fallback-b947cbab-5bfa-4df3-886a-70ad148b036f' {
                load_video
                set gfxpayload=keep
                insmod gzio
                insmod part_msdos
                insmod ext2
                set root='hd4,msdos1'
                if [ x$feature_platform_search_hint = xy ]; then
                  search --no-floppy --fs-uuid --set=root --hint-ieee1275='ieee1275/(null)/sas/disk@0,msdos1' --hint-bios=hd4,msdos1 --hint-efi=hd4,msdos1 --hint-baremetal=ahci4,msdos1  0c1a6d11-1ecd-4696-b90b-7764f36bc1ab
                else
                  search --no-floppy --fs-uuid --set=root 0c1a6d11-1ecd-4696-b90b-7764f36bc1ab
                fi
                echo    'Loading Linux linux ...'
                linux   /vmlinuz-linux root=/dev/mapper/archvg-root rw  loglevel=3 quiet rd.luks.name=UUID=7fa28d88-133b-40f9-974c-5d0bec44c083=cryptdev rd.luks.options=discard root=/dev/archvg/root libata.force=noncq
                echo    'Loading initial ramdisk ...'
                initrd  /initramfs-linux-fallback.img
        }
}

### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/20_linux_xen ###
### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/30_os-prober ###
### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  ${config_directory}/custom.cfg ]; then
  source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg;
fi
### END /etc/grub.d/41_custom ###

/etc/mkinitcpio.conf:

# vim:set ft=sh
# MODULES
# The following modules are loaded before any boot hooks are
# run.  Advanced users may wish to specify all system modules
# in this array.  For instance:
#     MODULES=(piix ide_disk reiserfs)
MODULES=(raid456 dm_mod ext4)

# BINARIES
# This setting includes any additional binaries a given user may
# wish into the CPIO image.  This is run last, so it may be used to
# override the actual binaries included by a given hook
# BINARIES are dependency parsed, so you may safely ignore libraries
BINARIES=()

# FILES
# This setting is similar to BINARIES above, however, files are added
# as-is and are not parsed in any way.  This is useful for config files.
FILES=()

# HOOKS
# This is the most important setting in this file.  The HOOKS control the
# modules and scripts added to the image, and what happens at boot time.
# Order is important, and it is recommended that you do not change the
# order in which HOOKS are added.  Run 'mkinitcpio -H <hook name>' for
# help on a given hook.
# 'base' is _required_ unless you know precisely what you are doing.
# 'udev' is _required_ in order to automatically load modules
# 'filesystems' is _required_ unless you specify your fs modules in MODULES
# Examples:
##   This setup specifies all modules in the MODULES setting above.
##   No raid, lvm2, or encrypted root is needed.
#    HOOKS=(base)
#
##   This setup will autodetect all modules for your system and should
##   work as a sane default
#    HOOKS=(base udev autodetect block filesystems)
#
##   This setup will generate a 'full' image which supports most systems.
##   No autodetection is done.
#    HOOKS=(base udev block filesystems)
#
##   This setup assembles a pata mdadm array with an encrypted root FS.
##   Note: See 'mkinitcpio -H mdadm' for more information on raid devices.
#    HOOKS=(base udev block mdadm encrypt filesystems)
#
##   This setup loads an lvm2 volume group on a usb device.
#    HOOKS=(base udev block lvm2 filesystems)
#
##   NOTE: If you have /usr on a separate partition, you MUST include the
#    usr, fsck and shutdown hooks.
HOOKS=(base systemd autodetect modconf block mdadm_udev keyboard sd-vconsole sd-encrypt sd-lvm2 filesystems fsck)

# COMPRESSION
# Use this to compress the initramfs image. By default, gzip compression
# is used. Use 'cat' to create an uncompressed image.
#COMPRESSION="gzip"
#COMPRESSION="bzip2"
#COMPRESSION="lzma"
COMPRESSION="xz"
#COMPRESSION="lzop"
#COMPRESSION="lz4"
#COMPRESSION="zstd"

# COMPRESSION_OPTIONS
# Additional options for the compressor
COMPRESSION_OPTIONS=(-9)

blkid:

/dev/sda: UUID="aec34b01-6ab1-8c85-798f-193652cae6a9" UUID_SUB="b594f6d5-2352-431a-96b9-97bc1d61d01c" LABEL="archiso:0" TYPE="linux_raid_member"
/dev/sdb: UUID="aec34b01-6ab1-8c85-798f-193652cae6a9" UUID_SUB="53046cd5-dc0a-22ee-a305-9e80f1b8c489" LABEL="archiso:0" TYPE="linux_raid_member"
/dev/sdc: UUID="aec34b01-6ab1-8c85-798f-193652cae6a9" UUID_SUB="4ef4b078-b218-7025-eec9-a98605b622c3" LABEL="archiso:0" TYPE="linux_raid_member"
/dev/sde1: LABEL="boot" UUID="0c1a6d11-1ecd-4696-b90b-7764f36bc1ab" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="29a471c6-01"
/dev/md0: UUID="7fa28d88-133b-40f9-974c-5d0bec44c083" TYPE="crypto_LUKS"
/dev/mapper/cryptdev: UUID="EmIm3A-00G6-E5N0-6Jpc-MfOw-A1Z6-WvGJ52" TYPE="LVM2_member"
/dev/mapper/archvg-swap: LABEL="swap" UUID="0a9655cb-5759-4859-bb90-1042a1bf3ae8" TYPE="swap"
/dev/mapper/archvg-root: LABEL="root" UUID="b947cbab-5bfa-4df3-886a-70ad148b036f" BLOCK_SIZE="4096" TYPE="ext4"
/dev/mapper/archvg-data: LABEL="data" UUID="e3f20b9f-a870-4547-b7f2-eb85aed0970e" BLOCK_SIZE="4096" TYPE="ext4"

I have no crypttab or mdadmconf, did not need them before, it was all automagic.

Can anyone see my problem? Do you need more config files?
Thanks for any hint.

Last edited by stfischr (2020-11-09 19:11:48)

stfischr

Member

Registered: 2020-04-29

Posts: 6

Re: Timed out waiting - LVM on LUKS on mdadm with systemd and Grub

Finally, i got it running, with a small hitch of course: no systemd-hooks:

changes in /etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="loglevel=3 quiet cryptdevice=/dev/md127:cryptdev root=/dev/mapper/archvg-root libata.force=noncq"

/boot/grub/grub.cfg

set root='lvmid/A8hrJ3-lAOz-waaD-1asx-FfHW-cMI3-srLIEY/NO4SF2-vGEc-BFCf-LTdp-DpWK-aFbq-K1fg1z'
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=root --hint='lvmid/A8hrJ3-lAOz-waaD-1asx-FfHW-cMI3-srLIEY/NO4SF2-vGEc-BFCf-LTdp-DpWK-aFbq-K1fg1z'  b947cbab-5bfa-4df3-886a-70ad148b036f
else
  search --no-floppy --fs-uuid --set=root b947cbab-5bfa-4df3-886a-70ad148b036f
fi
    font="/usr/share/grub/unicode.pf2"
fi

if loadfont $font ; then
  set gfxmode=auto
  load_video
  insmod gfxterm
  set locale_dir=$prefix/locale
  set lang=en_US
  insmod gettext
fi
terminal_input console
terminal_output gfxterm
if [ x$feature_timeout_style = xy ] ; then
  set timeout_style=menu
  set timeout=5
# Fallback normal timeout code in case the timeout_style feature is
# unavailable.
else
  set timeout=5
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/10_linux ###
menuentry 'Arch Linux' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-b947cbab-5bfa-4df3-886a-70ad148b036f' {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_msdos
        insmod ext2
        set root='hd4,msdos1'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint-ieee1275='ieee1275/(null)/sas/disk@0,msdos1' --hint-bios=hd4,msdos1 --hint-efi=hd4,msdos1 --hint-baremetal=ahci4,msdos1  0c1a6d11-1ecd-4696-b90b-7764f36bc1ab
        else
          search --no-floppy --fs-uuid --set=root 0c1a6d11-1ecd-4696-b90b-7764f36bc1ab
        fi
        echo    'Loading Linux linux ...'
        linux   /vmlinuz-linux root=/dev/mapper/archvg-root rw  loglevel=3 quiet cryptdevice=/dev/md127:cryptdev root=/dev/mapper/archvg-root libata.force=noncq
        echo    'Loading initial ramdisk ...'
        initrd  /initramfs-linux.img
}
submenu 'Advanced options for Arch Linux' $menuentry_id_option 'gnulinux-advanced-b947cbab-5bfa-4df3-886a-70ad148b036f' {
        menuentry 'Arch Linux, with Linux linux' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-linux-advanced-b947cbab-5bfa-4df3-886a-70ad148b036f' {
                load_video
                set gfxpayload=keep
                insmod gzio
                insmod part_msdos
                insmod ext2
                set root='hd4,msdos1'
                if [ x$feature_platform_search_hint = xy ]; then
                  search --no-floppy --fs-uuid --set=root --hint-ieee1275='ieee1275/(null)/sas/disk@0,msdos1' --hint-bios=hd4,msdos1 --hint-efi=hd4,msdos1 --hint-baremetal=ahci4,msdos1  0c1a6d11-1ecd-4696-b90b-7764f36bc1ab
                else
                  search --no-floppy --fs-uuid --set=root 0c1a6d11-1ecd-4696-b90b-7764f36bc1ab
                fi
                echo    'Loading Linux linux ...'
                linux   /vmlinuz-linux root=/dev/mapper/archvg-root rw  loglevel=3 quiet cryptdevice=/dev/md127:cryptdev root=/dev/mapper/archvg-root libata.force=noncq
                echo    'Loading initial ramdisk ...'
                initrd  /initramfs-linux.img
        }
        menuentry 'Arch Linux, with Linux linux (fallback initramfs)' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-linux-fallback-b947cbab-5bfa-4df3-886a-70ad148b036f' {
                load_video
                set gfxpayload=keep
                insmod gzio
                insmod part_msdos
                insmod ext2
                set root='hd4,msdos1'
                if [ x$feature_platform_search_hint = xy ]; then
                  search --no-floppy --fs-uuid --set=root --hint-ieee1275='ieee1275/(null)/sas/disk@0,msdos1' --hint-bios=hd4,msdos1 --hint-efi=hd4,msdos1 --hint-baremetal=ahci4,msdos1  0c1a6d11-1ecd-4696-b90b-7764f36bc1ab
                else
                  search --no-floppy --fs-uuid --set=root 0c1a6d11-1ecd-4696-b90b-7764f36bc1ab
                fi
                echo    'Loading Linux linux ...'
                linux   /vmlinuz-linux root=/dev/mapper/archvg-root rw  loglevel=3 quiet cryptdevice=/dev/md127:cryptdev root=/dev/mapper/archvg-root libata.force=noncq
                echo    'Loading initial ramdisk ...'
                initrd  /initramfs-linux-fallback.img
        }
}

### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/20_linux_xen ###
### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/30_os-prober ###
### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  ${config_directory}/custom.cfg ]; then
  source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg;
fi
### END /etc/grub.d/41_custom ###

changes in /etc/mkinitcpio.conf

MODULES=(raid456 dm_crypt twofish_x86_64_3way dm_mod) # not sure if needed, gotta test it one by one
HOOKS=(base udev autodetect modconf block mdadm keyboard keymap consolefont encrypt lvm2 filesystems fsck)

I would still very much prefer to do it the systemd-way, can anyone see my mistake now?

schard

Forum Moderator

From: Hannover

Registered: 2016-05-06

Posts: 2,178

Website

Re: Timed out waiting - LVM on LUKS on mdadm with systemd and Grub

I'd suspect the autodetect hook being the culprit here.
You may want to try your old setup with the required hook keyboard moved before autodetect.

---

Inofficial first vice president of the Rust Evangelism Strike Force

stfischr

Member

Registered: 2020-04-29

Posts: 6

Re: Timed out waiting - LVM on LUKS on mdadm with systemd and Grub

Hi Everyone,

with the help of schrad and another user from the german forum i was able to get it running:
/etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="loglevel=3 quiet rd.luks.name=7fa28d88-133b-40f9-974c-5d0bec44c083=cryptdev rd.luks.options=discard root=/dev/archvg/root libata.force=noncq"

UUID= was wrong, been looking at the wrong example

HOOKS=(base systemd modconf block keyboard sd-vconsole autodetect sd-encrypt sd-lvm2 filesystems fsck shutdown)

and autodetect screws up the order, therefore it needs to be placed after the keyboard stuff like schrad suggested.

I wanted to edit the wiki but i can't log in, is there an extra registration needed? Maybe someone else would change the hooks-line to place autodetect after the keyboard stuff? https://wiki.archlinux.org/index.php/Dm … initcpio_2

Edit: and many thanks ofcourse

Last edited by stfischr (2020-11-16 17:50:42)

GaKu999

Member

From: US/Eastern

Registered: 2020-06-21

Posts: 696

Re: Timed out waiting - LVM on LUKS on mdadm with systemd and Grub

is there an extra registration needed?

Yes, you need to create an account of course.

Maybe someone else would change the hooks-line to place autodetect after the keyboard stuff?

The general case is to have autodetect in there, yet you have a hardware quirk that gives problems on that.
In that scenario, is up to you to trace the issue, and considering you already did, there's nothing else to do.
Thankfully you had the fallback initramfs.

...

On another note, I always generate a full initramfs (no autodetect) and only one, it's overall smaller than the typical 2, and allows to avoid hardware quirks that may arise.
(Also since one of my Archs is inside an USB key...you get the idea )

Last edited by GaKu999 (2020-11-16 22:56:49)

---

My repos • Some snippets

Heisenberg might have been here.