[partly SOLVED] problems booting encrypted raid (...add.with grub-gfx)

nexus7 · 2008-10-12 15:31:02

Hi folks!

Since I managed to setup an encrypted system properly I am now about to do the same with 2 identical fresh disks and simple raid1-mirroring (no lvm). This is the layout:

/dev/md0:   /dev/sda3, /dev/sdb3 for /
/dev/md1:   /dev/sda1, /dev/sdb1 for /boot
/dev/md2:   /dev/sda4, /dev/sdb4 for /home
/dev/md3:   /dev/sda2, /dev/sdb2 for swap

while md0, md2 und md3 are encrypted. These are the given wikis: raid mkinitcpio luks, installation media is the archlinux-2008.06-core-i686.

Setting the system up was no problem --- creating the md-arrays, then encryption, then installation. Anyhow, booting up is always ending like this:

...
:: Running Hook [raid].
md: linear personality registered for level -1
md: multipath personality registered for level -4
md: raid0 personality registered for level 0
md: raid1 personality registered for level 1
xor: automatically using best check checksumming function: pIII_sse
   pIII_sse : 2258.400 MB/sec
xor: using function: pIII_sse (2258.400 MB/sec)
async_tx: api initialized (async)
raid6: int32x1     442 MB/sec
raid6: int32x2     499 MB/sec
raid6: int32x4     678 MB/sec
raid6: int32x8     389 MB/sec
raid6: mmxx1      1503 MB/sec
raid6: mmxx2      1890 MB/sec
raid6: sse1x1      885 MB/sec
raid6: sse1x2     1567 MB/sec
raid6: sse2x1     1539 MB/sec
raid6: sse2x2     2017 MB/sec
raid6: using algorithm sse2x2 (2017 MB/s)
md: raid6 personality registered for level 6
md: raid5 personality registered for level 5
md: raid4 personality registered for level 4
md: raid10 personality registered for level 10
md: Will configure md1 (super-block) from /dev/sda1,/dev/sdb1, below.
md: Will configure md0 (super-block) from /dev/sda3,/dev/sdb3, below.
md: Loading md1: /dev/sda1
md: bind<sda1>
md: bind<sdb1>
md: raid1 personality registerered for level 1
raid1: raid set md1 active with 2 out of 2 mirrors
md: Loading md0: /dev/sda3
md: bind<sda3>
md: bind<sdb3>
raid1: raid set md0 active with 2 out of 2 mirrors
:: Running Hook [keymap]...done
:: Running Hook [encrypt]
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.13.0-ioctl (2007-10-18) initialised: dm-devel@redhat.com

A password is required to access the root volume:
Enter LUKS passphrase
device-mapper: table: 253:0: crypt: Error allocating crypto tfm
device-mapper: ioctl: error adding target to table
device-mapper: ioctl: device doesn't appear to be in the dev hash table.
Command failed: No key available with this passphrase.

Enter LUKS passphrase:

Meanwhile I succeeded in booting porperly with adding the specific filesystems into the MODULES=line in /etc/mkinitcpio.conf though that shouldn't be necessary because of the filesystems-hook. This was very fine, because I was beeing able to tune up my installation with X-stuff, DE, office-proggies and all.
Then I liked to have a more colorful boot-screen and installed grub-gfx; still booting normally, but no splashimage was showing up. Then I added "raid1" into the MODULES=-line, and from now on booting always hangs!

Nevertheless it's no problem getting into my system using the installation-cd and doing this:

modprobe raid1
mdadm -Ac partitions -m 0 /dev/md0
mdadm -Ac partitions -m 1 /dev/md1

modprobe dm-crypt
cryptsetup luksOpen /dev/md0 root

mount /dev/mapper/root /mnt
mount /dev/md1 /mnt/boot
mount -o bind /sys /mnt/sys
mount -o bind /dev /mnt/dev
mount -o bind /proc /mnt/proc

chroot /mnt

I've been trying alot with disabling hooks in grub, reinstalled the normal grub, reinstalled the kernel, building initrd's, changing mkinitcpio.conf-settings,and checking the output of # mdadm -D --scan >>/etc/mdadm.conf. Well, /etc/mdadm.conf is always the same (ARRAYS, UUIDs) except of additional metadata-stuff:

ARRAY /dev/md0 level=raid1 num-devices=2 [b]metadata=00.90[/b] UUID=1234abcd:...
ARRAY /dev/md1 level=raid1 num-devices=2 [b]metadata=00.90[/b] UUID=2a4b6e7f:...
...
...

which appeared again as ignored errors while biulding the initrds and booting (actually the manpage just says it's the defult version of mdadm). Again: mdadm -Q -D /dev/mdX and mdadm -E /dev/sdX don't show any complications, so I deleted this metadata. And of course belonging to how many arrays are actually set up, so many ARRAY-lines appear in mdadm.conf (with the same UUIDs than before, so I left all of the original working ARRAYs). Finally I even reinstalled the system two times.

Maybe something of interest: # grub-install doesn't work:

/dev/md1 does not have any corresponding BIOS drive.

, but installing grub within its shell into any of the harddrives' mbr does.

OK, here's some more data:

# /boot/grub/menu.lst:

# general configuration:
timeout   5
default   0
# splashimage /grub/splash.xpm.gz
color light-blue/black light-cyan/blue

# (0) Arch Linux
title  Arch Linux
root    (hd0,0)
kernel /vmlinuz26 root=/dev/md0 ro md=0,/dev/sda3,/dev/sdb3 md=1,/dev/sda1,/dev/sdb1 vga=775
initrd /kernel26.img

# /etc/mkinitcpio.conf
MODULES=""
# alternatively: MODULES="pata_acpi ata_generic scsi_mod ata_piix ext2 ext3 reiserfs" 
FILES="/etc/mdadm.conf"
HOOKS="base udev raid autodetect pata scsi usb usbinput keymap encrypt filesystems"
# root-array:
md=0,/dev/sda3,/dev/sdb3

# /etc/mdadm.conf
# The designation "partitions" will scan all partitions found in 
# /proc/partitions 
DEVICE partitions 
ARRAYs and UUIDs existing...
...

# /proc/partitons (for comparison from another ebcrypted system w/o raid):
major minor  #blocks  name 

   8     0  117220824 sda 
   8     1      96358 sda1 
   8     2     747022 sda2 
   8     3   39062047 sda3 
   8     4   77312812 sda4 
 254     0   39060539 dm-0 
 254     1   77311784 dm-1 
 254     2     747022 dm-2

-- the actual /proc/partitions contains as well all sd[a b]X and all md[0 1 2 3], but also loop[0 1] and only (1) dm-0 (because I actually mounted only /dev/mapper/root), I guess this is still OK.

I don't know what to do anymore Another combination of hooks or modules?...
So I hope somebody has got a proposal or the solution...

Last edited by nexus7 (2008-10-16 20:28:21)

nexus7 · 2008-10-13 19:08:31

I don't think the problem depends neither on grub or grub-gfx, therefor I changed the thread's topic.

Probably it is the order of the hooks and/or modules. I just have built another initrd with an additonal dm-mod and without filesystems like this:

# /etc/mkinitcpio.conf
MODULES="pata_acpi ata_generic scsi_mod ata_piix dm-mod ext2 ext3 reiserfs"
...
HOOKS="base udev raid autodetect pata scsi usb usbinput keymap encrypt"
...

Same output

And then:

# /etc/mkinitcpio.conf
MODULES="dm-crypt"
CRYPTO_MODULES="aes_i586 aes256_generic sha256_generic"
...
HOOKS="base udev raid autodetect pata scsi usb usbinput keymap filesystems"
...

-- kernel panic at boot

And then:

# /etc/mkinitcpio.conf
CRYPTO_MODULES="aes_i586 aes256_generic sha256_generic"
MODULES="pata_acpi ata_generic scsi_mod ata_piix dm-crypt dm-mod ext2 ext3 reiserfs"
...
HOOKS="base udev raid autodetect pata scsi usb usbinput keymap encrypt filesystems"
...

results into

...
:: Running Hook [encrypt]
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.13.0-ioctl (2007-10-18) initialised: dm-devel@redhat.com

A password is required to access the root volume:
Enter LUKS passphrase
device-mapper: table: 25[b]4[/b]:0: crypt: Error allocating crypto tfm
device-mapper: ioctl: error adding target to table
device-mapper: ioctl: device doesn't appear to be in the dev hash table.
Command failed: No key available with this passphrase.

Enter LUKS passphrase:

And then:

# /etc/mkinitcpio.conf
CRYPTO_MODULES="aes_i586 aes256_generic sha256_generic"
MODULES="raid1 raid456 pata_acpi ata_generic scsi_mod ata_piix dm-crypt dm-mod ext2 ext3 reiserfs"
...
HOOKS="base udev raid autodetect pata scsi usb usbinput keymap encrypt filesystems"
...

results again into

...
:: Running Hook [encrypt]
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.13.0-ioctl (2007-10-18) initialised: dm-devel@redhat.com

A password is required to access the root volume:
Enter LUKS passphrase
device-mapper: table: 253:0: crypt: Error allocating crypto tfm
device-mapper: ioctl: error adding target to table
device-mapper: ioctl: device doesn't appear to be in the dev hash table.
Command failed: No key available with this passphrase.

Enter LUKS passphrase:

wtf...?

Followed by thus using 1:

# /etc/mkinitcpio.conf
MODULES="pata_acpi ata_generic scsi_mod ata_piix dm-mod dm-crypt aes-i586 aes256-generic sha256-generic ext2 ext3 reiserfs"
...
HOOKS="base udev raid autodetect pata scsi usb usbinput keymap encrypt filesystems"
...

hey great!!! it WORKS

...but I guess my mkinitcpio.conf is somewhat overloaded~ Will it still work after putting it on diet?!? And what's about installing grub-gfx again...?
I think I'll have to go to sleep and to dream about it first~

Last edited by nexus7 (2008-10-13 20:41:27)

nexus7 · 2008-10-16 19:11:57

After fumbling around a lot with the configs and building another dozen of initrds with

mkinitcpio -g /boot/test.img

and booting them up I got a final solution!
The gist of the matter is including dm-mod in the MODULES-line of /etc/mkinitcpio.conf, though the basic system had been set up with modprobe dm-crypt for creating the encrypted partitions, and I don't use lvm as well.
Additionally, there must not be a line CRYPTO_MODULES="blah" at all, otherwise you'll end up within the passphrase-looping shown above!

Let's have a look how an ordinary boot up then looks like:

...
:: Running Hook [keymap]
...done.
:: Running Hook [encrypt]
A password is required to access the root volume.
Enter LUKS passphrase:

key slot 0 unlocked
:: Running Hook [filesystems]
...

(The keymap-hook is optional for people like me who use another than the US-keyboard; it must be before the encrypt hook.)
You may notice then as well the absence of

device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.13.0-ioctl (2007-10-18) initialised: dm-devel@redhat.com

while everything is running fine if you'd enter the correct password.

Finally, to speed up booting noticably and to suppress the harmless but annoying warning about missing padlock drivers at the same time, it is not necessary to load tons of modules explicitly (though it's not harmful) --- just two of them are sufficient! This is a brief listing of my actual /etc/mkinitcpio.conf's contents:

MODULES="dm-mod aes-generic"
BINARIES=""
FILES="/etc/mdadm.conf"
HOOKS="base udev raid autodetect pata scsi usb usbinput keymap encrypt filesystems"
md=0,/dev/sda3,/dev/sdb3

So, if you use aes-encryption it's enough to use either the aes-i586- or the aes-generic-module, even with a sha-enrcyption of swap. As said before, logging into your machine will succeed without that modules, but then that warnign appears. If you use other or additional methods then you'll probably also need to specify other drivers.

I don't know if it is necessary or even possible to shorten this config (it's your turn to find this out ), but with the right passphrase booting this raid is almost as fast as light!

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Hurray, the main part of my problem is now solved!
Another intention for this thread has been about how to get to work grub-gfx properly with this raid; it is booting smoothly, but no spalshimage is showing up~
But I'd like to have one --- noblesse oblige --- and Arch is noble!
...and I ilke eyecandy.

So here's the /boot/menu.lst:

# general configuration:
timeout   5
default   0
splashimage /grub/splash.xpm.gz
color light-blue/black light-cyan/blue

# (0) Arch Linux
title  Arch Linux
root    (hd0,0)
kernel /vmlinuz26 root=/dev/md0 ro md=0,/dev/sda3,/dev/sdb3 md=1,/dev/sda1,/dev/sdb1 vga=775
initrd /kernel26.img
...

The reason therefor is probalby this:

#  grub-install /dev/sda
/dev/md1 does not have any corresponding BIOS drive.

Grub could only be installed within its shell:

# grub

grub> root (hd0,0)
 Filesystem type is ext2fs, partition type 0xfd

grub> setup (hd0)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... yes
 Checking if "/grub/stage2" exists... yes
 Checking if "/grub/e2fs_stage1_5" exists... yes
 Running "embed /grub/e2fs_stage1_5 (hd0)"...  16 sectors are embedded.
succeeded
 Running "install /grub/stage1 (hd0) (hd0)1+16 p (hd0,0)/grub/stage2 /grub/menu.lst"... succeeded
Done.

grub>  quit

Any proposals?
Or should I give grub-gfxmenu a try...?

Cheers!

Last edited by nexus7 (2008-10-17 08:29:07)

Arch Linux

#1 2008-10-12 15:31:02

[partly SOLVED] problems booting encrypted raid (...add.with grub-gfx)

#2 2008-10-13 19:08:31

Re: [partly SOLVED] problems booting encrypted raid (...add.with grub-gfx)

#3 2008-10-16 19:11:57

Re: [partly SOLVED] problems booting encrypted raid (...add.with grub-gfx)

Board footer