You are not logged in.

#1 2011-09-14 16:53:06

KairiTech
Member
From: Toronto, Canada
Registered: 2011-06-04
Posts: 275

[Solved] Pure UEFI GRUB2 LVM2 on LUKS encrypted GPT 3xSSD RAID0

EDIT: Archboot 2011.11 „2k11-R7“ is out. No longer have to apply the efivars fix to the install script. It gets installed correctly now. Still get prompted to install GRUB2 twice though.

Okay! So let's get going...

First off, this is a complete rewrite or my original post about a pure UEFI install and boot.

I was initially able to install and boot using 'grub2-bios' mode but that wasn't good enough for me. After lots of research and online help I finally figured out how to install and boot using 'grub2-efi-x86_64'. You can read the posts following this one to get a glimpse into some of the banter that helped me figure all of this out.

There are several things that are key to a successful pure UEFI install of Arch Linux:

1. You'll need to boot the install media in UEFI mode. So what is that exactly? The current version of the Archboot ISO image (archlinux-2011.10-1-archboot.iso) fully supports UEFI but archlinux-2011.08.19 does not. Copy the Archboot iso image to a FAT formatted USB. (I seem to have more consistent success using 'unetbootin' rather than 'dd' though. On a side note: I find it a little odd that my ASUS UEFI motherboard will see any FAT formatted removable device as UEFI bootable (even if it contain no UEFI required file structure or files) but will not see any FAT formatted HDD or SSD the same way (More on this later.) The boot menu on my motherboard shows the USB twice in the list of available boot devices. Once as a "regular" USB and a second time as a "UEFI" USB. You'll need to figure out how to choose the UEFI option from your motherboard's boot menu.

2. The drive you intend to install to must use the 'GPT' partitioning scheme. Not MBR!

3. You will need a) FAT formatted partition with the "boot" flag set, b) a partition for 'grub2' (I use ext2 since it'd be static with little, if any, changes after the install) c) a partition for root (I use ext4) d) any other partitions that you usually use for your installs.

4. Obviously the boot partition will be mounted on /boot and the root partition on /. The FAT formatted partition MUST be mounted on /boot/uefi (make sure you mount /boot first). Another oddity I've discovered is that even though a size of 200MB is brandished about all over the internet as the minimum acceptable size for the FAT partition I have no problems using just 25MB. For the boot partition for 'grub2' I use 10MB.

5. efivars fix no longer needed with 2011.11 „2k11-R7“ There are a couple of bugs in the Archboot 2011.10 „2k11-R6“ script. You'll need to change

if [[ $(modprobe -q efivars) ]]; then

to

modprobe -q efivars

if [[ "$(lsmod | grep efivars)" ]]; then

and although Archboot recognizes that you booted in UEFI mode and asks you to confirm which version of 'grub2' you'd like to install and then installs it, it takes you to the bootloader selection a second time. I simple skipped the second request completely and it didn't create any issues. There is a patch available but I never could get it to work after I applied it.

6. Archboot will eventually create a file called 'grub.efi' needed for booting but my ASUS will have no part of it. The only way I can convince my motherboard to boot the install was to copy 'grub.efi' to 'shellx64.efi' and put 'shellx64.efi' onto the top level (not a subdirectory) of ANY FAT formatted partition on ANY device. Then when I boot my computer I have to go into the BIOS setup, choose Advanced Mode, select Exit, then choose "Launch EFI shell from filesystem device". I alluded to this previously but my ASUS motherboard does not see the drive I installed the bootloader to as "UEFI" bootable. I haven't had the courage to test what would happen if I had two or more distinct 'shellx64.efi' files on two or more separate FAT partitions. But I will eventually test it.

Now on to the specifics of my configuration of choice.

Be warned! It's complex. Still though, you might be able to cherry-pick some of what I've done for your own configuration. Also, rather than use Acrcboot to set everything up during the install I created a script so that I'd do it consistently every time and avoid silly typos that'd wreck the install. I'll explain and show below the relative portions of the script. The good thing is that since Archboot loads completely into memory I can save my script (and the Archboot setup script with the changes mentioned above) to the USB and once Archboot is up and running, exit the installer immediately, create a directory called '/mnt/USB' and mount the USB to it so I'll have access to both scripts. My script will be run before I run the modified Archboot setup script.

a) I want RAID. And RAID0 at that. Not so much interetsed in data security as I am in performance. Besides, I backup my backups. I have three SSDs and to make sure they all have the exact same partitioning scheme I define a function and call it for each SSD in turn.

set -o errexit							##### exit on any type of error so that the script does not run wild if I do crap
function partitionDRIVE {
   echo "|||||"
   parted /dev/sd$SDD mktable gpt				##### create an new empty GPT partition table
   parted /dev/sd$SDD mkpart vfat 1MB 12MB			##### create 1st partition as vfat for uefi
   parted /dev/sd$SDD name 1 uefi
   parted /dev/sd$SDD set 1 boot on				##### set the boot flag so it will be recognized as a UEFI SYSTEM PARTITION
   parted /dev/sd$SDD mkpart ext2 12MB 39MB			##### create 2nd partition for GRUB2 (no boot flag though)
   parted /dev/sd$SDD name 2 grub2
   parted /dev/sd$SDD set 2 raid off				##### shouldn't RAID this partition
   parted /dev/sd$SDD mkpart ext4 39MB 60GB			##### create partition for encrypted RAID0 array for root & data
   parted /dev/sd$SDD name 3 root
   parted /dev/sd$SDD set 3 raid on
}

SDD=a ; partitionDRIVE
SDD=b ; partitionDRIVE
SDD=c ; partitionDRIVE

This is what each SSD's partition scheme should look like:

Model: ATA KINGSTON SNV425S (scsi)
Disk /dev/sda: 64.0GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start   End     Size    File system  Name   Flags
 1      1049kB  11.5MB  10.5MB               uefi   boot
 2      11.5MB  38.8MB  27.3MB               grub2
 3      38.8MB  60.0GB  60.0GB               root   raid

I name the partition to be mounted on /boot 'grub2' because that's all it will contain.

b) I want system encryption. Not just file or directory. But I only want to enter one passphrase and not one for root and one for my data. So I use LVM2 to chop the RAID array into two volumes. One for root and the remainder for my data. And I want to enter the passphrase every time I boot so I do not create a 'keyfile'.

I don't use swap.
The LVM2 physical volume 'pv64SSDx3' will be on the RAID0 array.
The LVM2 volume group 'vg64SSDx3' will be in 'pv64SSDx3'.
The LVM2 logical volumes 'lvRoot' and 'lvMyStuff' will be in 'vg64SSDx3'.
Told you it was complex.

modprobe raid0 											##### load the RAID0 module just in case they aren't already loaded
mdadm --create /dev/md3 --level=0 --raid-devices=3 /dev/sda3 /dev/sdb3 /dev/sdc3		##### create the RAID device for root & data
modprobe dm_mod 										##### load the device mapper module
cryptsetup --cipher=aes-xts-plain --verify-passphrase --key-size=512 luksFormat /dev/md3	##### encrypt the device that will hold root & data
cryptsetup luksOpen /dev/md3 pv64SSDx3 								##### open the encrypted device (pv64SSDx3)
pvcreate /dev/mapper/pv64SSDx3 									##### create the physical volume
vgcreate vg64SSDx3 /dev/mapper/pv64SSDx3 							##### create the volume group
lvcreate --contiguous y --size 6G --name lvRoot vg64SSDx3 					##### create the logical volume for root
lvcreate --extents +100%FREE --name lvMyStuff vg64SSDx3 					##### create the logical volume for data (myStuff) with all the remaining space

After my script finishes I start the modified Archboot setup script. I set up the mountpoints like this:

/dev/sda2			/boot		formatted as ext2
/dev/sda1			/boot/uefi	formatted as FAT
/dev/mapper/vg64SSDx3-lvRoot	/		formatted as ext4
/dev/mapper/vg64SSDx3-lvMyStuff	/media/myStuff	formatted as ext4

Note that the first and second partitions of the second and third SSDs remain unused.

Proceed with the install as usual and when Archboot gets to the 'configure system' phase there are several key things that need to be changes or added. I store the changes in my script and pull them into the config files to avoid those nasty typos.

rc.conf
=======

change

USEDMRAID="no"

to

USEDMRAID="yes"

change

USELVM="no"

to

USELVM="yes"

mkinitcpio.conf
===============

Make sure that the modules listed below are in your "MODULES=" line. Your starting list may be different because of the motherboard you have. I tend to remove everything I absolutely don't need. The order is important so don't change it unless you know what you are doing.

MODULES="nouveau dm_mod dm_crypt aes_x86_64 raid0"

The "HOOKS=" line also need to be changed as follows. Again, your starting list may be different. I use comments in my script to remind me why I need those entries.

# base		Sets up all initial directories and installs base utilities and libraries. Always add this hook unless you know what you are doing. 
# udev		Adds udev to your image.
# autodetect	Shrinks your initramfs to a smaller size by autodetecting your needed modules. Be sure to verify included modules are correct and none are missing. This hook must be run before other subsystem hooks in order to take advantage of auto-detection. Any hooks placed before 'autodetect' will be installed in full. 
# sata		Adds serial ATA modules to the image. Use this if your root device is on a SATA disk. Also use the autodetect hook if you want to minimize your image size. 
# mdadm		This hook supersedes the above raid hook. It supports assembling the arrays from /etc/mdadm.conf, or autodetection during boot. 
# usbinput	Adds USB HID modules to the image. Use this if you have an USB keyboard and need it in early userspace (either for entering encryption passphrases or for failsafe mode). 
# keymap	Adds keymap and consolefonts from rc.conf. 
# encrypt	Adds the dm-crypt kernel module and the cryptsetup tool to the image. You need to have the cryptsetup package installed to use this. 
# lvm2		Adds the device mapper kernel module and the lvm tool to the image. You need to have the lvm2 package installed to use this. 
# filesystems	This includes necessary filesystem modules into your image. This hook is required unless you specify your filesystem modules in MODULES. 

HOOKS="base udev autodetect sata mdadm usbinput keymap encrypt lvm2 filesystems"

grub.cfg
========

You'll need to specify that root (and your data) is encrypted by adding 'cryptdevice=/dev/md3:vg64SSDx3' immediately before 'ro' on both kernel parameter lines as in the example below.

kernel /vmlinuz-linux root=/dev/mapper/vg64SSDx3-lvRoot cryptdevice=/dev/md3:vg64SSDx3 ro

At the end of it all, after I exit the installer, I copy 'grub.efi' to 'shellx64.efi' so I can use my ASUS M5A99X EVO UEFI BIOS to boot into pure UEFI. I just copy it onto the UEFI boot USB I mounted earlier with my script and the modifies archboot script. I figure if I ever needed to have another completely isolated pure UEFI install on another drive I'd use another USB flash drive.

cp /tmp/install/boot/efi/efi/grub/grub.efi /mnt/USB/shellx64.efi

Now I reboot and every time I go through BIOS setup, Advanced Mode, Exit, "Launch EFI shell from filesystem device" ... walla! I'm into pure UEFI Arch Linux. I'm pretty sure this is an ASUS-specific way to boot into pure UEFI. You'll need to figure out how to get your motherboard to boot from 'grub.efi' or whatever it needs to be called wherever it needs to be installed.

Thanks to everyone who helped me figure this out.

Now you try it and good luck!

One more thing. When I need Windows I run it in virtualbox so there's no dual-booting in any of this.

Last edited by KairiTech (2012-01-01 20:48:50)

Offline

#2 2011-09-14 17:03:15

skodabenz
Banned
From: Tamilnadu, India
Registered: 2010-04-11
Posts: 382

Re: [Solved] Pure UEFI GRUB2 LVM2 on LUKS encrypted GPT 3xSSD RAID0

@KairiTech:

1: No big deal. MBR to GPT conversion is smooth using gdisk. Better use cgdisk though.

2, 3, 4: No idea about RAID, LVM, LUKS. But grub2 1.99 does not support LUKS (upstream bzr repo does - use grub2-bios-bzr or grub2-efi-bzr from AUR). No idea what exactly is missing from 1.99 release that prevents grub2 from allowing LUKS though.

5, 6: For grub2 UEFI try Archboot iso since you need to boot thorough UEFI to setup the system in UEFI boot mode.


My new forum user/nick name is "the.ridikulus.rat" .

Offline

#3 2011-09-15 21:36:22

KairiTech
Member
From: Toronto, Canada
Registered: 2011-06-04
Posts: 275

Re: [Solved] Pure UEFI GRUB2 LVM2 on LUKS encrypted GPT 3xSSD RAID0

Well I got it all working with a couple of changes to my original script and partition scheme. I used Archboot 2011.08 „2k11-R5“ as my installer. I tried to do all of the partitioning, RAID, LUKS and LVM2 configuration with the installer but kept ending up with a "phanton" LVM2 physical volume that would fail to open after I entered the passphrase during boot but the boot would still succeed. I'm still very impressed with how well thought out the Archboot scripts are.

The basic premise remains the same though. I encrypted the RAID array and not the LVM2 volumes. That way I need only one passphrase for root and data since the LVM2 logical volumes are on the same RAID0 array.

Ever since the update to Linux 3 I could no longer get ARCH to install legacy GRUB or GRUB2 to all the drives by creating a RAID1 array for boot so I no longer create one and only install GRUB2 to the first drive.

GPT needs, at minimum, a 1MB partition with the bios_grub flag set to store what the MBR used to contain so I modified my partitioning scheme slightly.

See below for the new script I'm now using to set everything up before I run /arch/setup. I still have my BIOS based motherboard so I'm using grub2-bios. Will switch over to grub2-efi-x86_64 once I get the new UEFI motherboard.

# I use RAID0 - not so much interested in data storage security. I want performance and trust me. I learned my lesson about backups a long time ago. I backup my backups.

# I don't use SWAP - I have 8GB of RAM.

# As you can likely tell I have three 64GB SSDs.

# Root and my data are spread across all three disks. /boot is now on the first drive only.
#
#############################################
#
# partition all the drive like so...
#
# script to prepare hard disk for RAID system encryption with /arch/setup
#
# Partition Table: gpt
#
# Number  Start   End     Size    File system  Name  Flags
#  1      1049kB  2097kB  1049kB                     bios_grub
#  2      2097kB  75.5MB  73.4MB  ext2               raid
#  3      75.5MB  60.0GB  59.9GB                     raid

      parted /dev/sda p
      parted /dev/sda mktable gpt
      parted /dev/sda mkpart p ext2 1MB 2MB
      parted /dev/sda set 1 bios_grub on
      parted /dev/sda mkpart p ext2 2MB 75MB    
      parted /dev/sda set 2 raid off
      parted /dev/sda mkpart p ext4 75MB 60GB
      parted /dev/sda set 3 raid on

      parted /dev/sdb p
      parted /dev/sdb mktable gpt
      parted /dev/sdb mkpart p ext2 1MB 2MB
      parted /dev/sdb set 1 bios_grub on
      parted /dev/sdb mkpart p ext2 2MB 75MB
      parted /dev/sdb set 2 raid off
      parted /dev/sdb mkpart p ext4 75MB 60GB
      parted /dev/sdb set 3 raid on

      parted /dev/sdc p
      parted /dev/sdc mktable gpt
      parted /dev/sdc mkpart p ext2 1MB 2MB
      parted /dev/sdc set 1 bios_grub on
      parted /dev/sdc mkpart p ext2 2MB 75MB
      parted /dev/sdc set 2 raid off
      parted /dev/sdc mkpart p ext4 75MB 60GB
      parted /dev/sdc set 3 raid on
      
modprobe raid1
modprobe raid0 ##### load the RAID modules just in case they aren't already loaded

mdadm --create /dev/md3 --level=0 --raid-devices=3 /dev/sda3 /dev/sdb3 /dev/sdc3 ##### create the RAID device for root & data

modprobe dm_mod ##### load the device mapper module


cryptsetup --cipher=aes-xts-plain --verify-passphrase --key-size=512 luksFormat /dev/md3 ##### encrypt the device that will hold root & data

cryptsetup luksOpen /dev/md3 pv64SSDx3 ##### open the encrypted device (pv64SSDx3)

pvcreate /dev/mapper/pv64SSDx3 ##### create the physical volume

vgcreate vg64SSDx3 /dev/mapper/pv64SSDx3 ##### create the volume group

lvcreate --contiguous y --size 6G --name lvRoot vg64SSDx3 ##### create the logical volume for root

lvcreate --extents +100%FREE --name lvMyStuff vg64SSDx3 ##### create the logical volume for data (myStuff) with all the remaining space

vgchange -ay ##### activate volume group and logical volumes

#
# this is when you start /arch/setup
#
# choose the option to manually prepare hard drives using UUID
#
# /etc/fstab: static file system information should look like this...
#
# <file system>	<dir>	<type>	<options>	<dump>	<pass>
# tmpfs		/tmp	tmpfs	nodev,nosuid	0	0
# DEVICE DETAILS: /dev/mapper/vg64SSDx3-lvMyStuff UUID=2cbb85cd-5632-4a10-a809-0b6cb8d30950 LABEL=myStuff
# DEVICE DETAILS: /dev/mapper/vg64SSDx3-lvRoot    UUID=d4948528-a172-47e8-9ff3-1a3c66771e46 LABEL=root
# DEVICE DETAILS: /dev/sda2                       UUID=6bc71716-d3b2-4f7e-aa11-6ef0bc34b712 LABEL=boot

# UUID=2cbb85cd-5632-4a10-a809-0b6cb8d30950 /media/myStuff ext4 defaults 0 1
# UUID=6bc71716-d3b2-4f7e-aa11-6ef0bc34b712 /boot          ext2 defaults 0 1
# UUID=d4948528-a172-47e8-9ff3-1a3c66771e46 /              ext4 defaults 0 1

#############################################
#
# configure the system as indicated below (do not include the "###s")
#

### USEDMRAID="yes" ##### change "no" to "yes" in /etc/rc.conf
### USELVM="yes"
#
### MODULES="dm_mod dm_crypt aes_x86_64 raid1 raid0" ##### insert modules to be included in the boot image in/etc/mkinitcpio
#
# I have s USB keyboard so I need usbinput
# I prefer to type the passphrase every time so I need keymaps
# the order is important so don't change it unless you know what you are doing
#
# base		Sets up all initial directories and installs base utilities and libraries. Always add this hook unless you know what you are doing. 
# udev		Adds udev to your image.
# autodetect	Shrinks your initramfs to a smaller size by autodetecting your needed modules. Be sure to verify included modules are correct and none are missing. This hook must be run before other subsystem hooks in order to take advantage of auto-detection. Any hooks placed before 'autodetect' will be installed in full. 
# pata		Adds the new libata/PATA IDE modules to the image. Use this if your root device is on a IDE disk. Also use the autodetect hook if you want to minimize your image size 
# scsi		Adds SCSI modules to the image. Use this if your root device is on a SCSI disk. Also use the autodetect hook if you want to minimize your image size. 
# sata		Adds serial ATA modules to the image. Use this if your root device is on a SATA disk. Also use the autodetect hook if you want to minimize your image size. 
# mdadm		This hook supersedes the above raid hook. It supports assembling the arrays from /etc/mdadm.conf, or autodetection during boot. 
# usbinput	Adds USB HID modules to the image. Use this if you have an USB keyboard and need it in early userspace (either for entering encryption passphrases or for failsafe mode). 
# keymap	Adds keymap and consolefonts from rc.conf. 
# encrypt        Adds the dm-crypt kernel module and the cryptsetup tool to the image. You need to have the cryptsetup package installed to use this. 
# lvm2		Adds the device mapper kernel module and the lvm tool to the image. You need to have the lvm2 package installed to use this. 
# filesystems	This includes necessary filesystem modules into your image. This hook is required unless you specify your filesystem modules in MODULES. 
#
### HOOKS="base udev autodetect pata scsi sata mdadm usbinput keymap encrypt lvm2 filesystems" ##### insert the hooks for the boot image in/etc/mkinitcpio
#.

Last edited by KairiTech (2011-09-15 21:37:33)

Offline

#4 2011-10-05 00:49:26

KairiTech
Member
From: Toronto, Canada
Registered: 2011-06-04
Posts: 275

Re: [Solved] Pure UEFI GRUB2 LVM2 on LUKS encrypted GPT 3xSSD RAID0

See my comments here about my experience using the latest 2011.10-1, "2k11-R6" release booted from USB in UEFI mode.

Last edited by KairiTech (2011-11-15 16:28:15)

Offline

Board footer

Powered by FluxBB