[Solved] Installing and using GRUB on LVM

psycho · 2013-08-01 17:57:19

New installation, no more windows, using LVM.

pvdisplay :

  --- Physical volume ---
  PV Name               /dev/sda1
  VG Name               DesktopLVM
  PV Size               243.14 MiB / not usable 3.14 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              60
  Free PE               0
  Allocated PE          60
  PV UUID               9LXTZW-zZNI-dQ3W-RMTD-C3Fl-5blw-TbPwOo
   
  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               DesktopLVM
  PV Size               19.07 GiB / not usable 4.15 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              4882
  Free PE               0
  Allocated PE          4882
  PV UUID               bgu0dD-seVt-nz7i-3Vdi-PZ4U-MKKa-pk3i6V
   
  --- Physical volume ---
  PV Name               /dev/sda3
  VG Name               DesktopLVM
  PV Size               7.63 GiB / not usable 4.86 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              1952
  Free PE               0
  Allocated PE          1952
  PV UUID               lybKyF-aqIM-3tTA-FEH3-LRPn-cjI5-qqhG75
   
  --- Physical volume ---
  PV Name               /dev/sda4
  VG Name               DesktopLVM
  PV Size               438.82 GiB / not usable 3.84 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              112337
  Free PE               0
  Allocated PE          112337
  PV UUID               l1h3Hh-BkzT-Fj0q-9mDH-1IGx-DtI0-v8AGgS

lvdisplay :

  --- Logical volume ---
  LV Path                /dev/DesktopLVM/boot
  LV Name                boot
  VG Name                DesktopLVM
  LV UUID                Je7Lat-TOzj-vWei-5fLe-4cc8-ZAUF-J2IGAu
  LV Write Access        read/write
  LV Creation host, time archiso, 2013-08-01 16:54:03 +0000
  LV Status              available
  # open                 1
  LV Size                256.00 MiB
  Current LE             64
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:1
   
  --- Logical volume ---
  LV Path                /dev/DesktopLVM/root
  LV Name                root
  VG Name                DesktopLVM
  LV UUID                xRK2SN-4IG1-RjNV-RHAl-qeXL-R8F3-k4SDQt
  LV Write Access        read/write
  LV Creation host, time archiso, 2013-08-01 16:54:12 +0000
  LV Status              available
  # open                 1
  LV Size                20.00 GiB
  Current LE             5120
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:2
   
  --- Logical volume ---
  LV Path                /dev/DesktopLVM/var
  LV Name                var
  VG Name                DesktopLVM
  LV UUID                Ej5zmm-7Q2x-HubD-JCBp-rg3F-g1oo-4rGBEn
  LV Write Access        read/write
  LV Creation host, time archiso, 2013-08-01 16:54:21 +0000
  LV Status              available
  # open                 1
  LV Size                8.00 GiB
  Current LE             2048
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:3
   
  --- Logical volume ---
  LV Path                /dev/DesktopLVM/home
  LV Name                home
  VG Name                DesktopLVM
  LV UUID                jU5Fdz-Tgdl-wPeE-ePOB-NaZk-CsGN-Xc8rkn
  LV Write Access        read/write
  LV Creation host, time archiso, 2013-08-01 16:54:37 +0000
  LV Status              available
  # open                 1
  LV Size                437.50 GiB
  Current LE             111999
  Segments               4
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:4

The problem is when using grub-install :

sh-4.2# grub-install --recheck /dev/sda
WARNING: Failed to connect to lvmetad: No such file or directory. Falling back to internal scanning.
WARNING: Failed to connect to lvmetad: No such file or directory. Falling back to internal scanning.
WARNING: Failed to connect to lvmetad: No such file or directory. Falling back to internal scanning.
WARNING: Failed to connect to lvmetad: No such file or directory. Falling back to internal scanning.
WARNING: Failed to connect to lvmetad: No such file or directory. Falling back to internal scanning.
WARNING: Failed to connect to lvmetad: No such file or directory. Falling back to internal scanning.
WARNING: Failed to connect to lvmetad: No such file or directory. Falling back to internal scanning.
WARNING: Failed to connect to lvmetad: No such file or directory. Falling back to internal scanning.
WARNING: Failed to connect to lvmetad: No such file or directory. Falling back to internal scanning.
/usr/bin/grub-bios-setup: warning: your core.img is unusually large. It won't fit in the embedding area.
/usr/bin/grub-bios-setup: error: embedding is not possible, but this is required for RAID and LVM install.

cat /boot/grub/grub.cfg :

#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
insmod part_gpt
insmod part_msdos
if [ -s $prefix/grubenv ]; then
  load_env
fi
if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   set boot_once=true
else
   set default="0"
fi

if [ x"${feature_menuentry_id}" = xy ]; then
  menuentry_id_option="--id"
else
  menuentry_id_option=""
fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
  save_env saved_entry
  set prev_saved_entry=
  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
    save_env saved_entry
  fi
}

function load_video {
  if [ x$feature_all_video_module = xy ]; then
    insmod all_video
  else
    insmod efi_gop
    insmod efi_uga
    insmod ieee1275_fb
    insmod vbe
    insmod vga
    insmod video_bochs
    insmod video_cirrus
  fi
}

if [ x$feature_default_font_path = xy ] ; then
   font=unicode
else
insmod part_msdos
insmod part_msdos
insmod part_msdos
insmod part_msdos
insmod lvm
insmod ext2
set root='lvm/DesktopLVM-root'
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=root --hint='lvm/DesktopLVM-root'  b881e4e6-ad72-495c-8ae1-610ec15d27c8
else
  search --no-floppy --fs-uuid --set=root b881e4e6-ad72-495c-8ae1-610ec15d27c8
fi
    font="/usr/share/grub/unicode.pf2"
fi

if loadfont $font ; then
  set gfxmode=auto
  load_video
  insmod gfxterm
  set locale_dir=$prefix/locale
  set lang=en_US
  insmod gettext
fi
terminal_input console
terminal_output gfxterm
set timeout=5
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/10_linux ###
menuentry 'Arch Linux, with Linux core repo kernel' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-core repo kernel-true-b881e4e6-ad72-495c-8ae1-610ec15d27c8' {
	load_video
	set gfxpayload=keep
	insmod gzio
	insmod part_msdos
	insmod part_msdos
	insmod part_msdos
	insmod part_msdos
	insmod lvm
	insmod ext2
	set root='lvm/DesktopLVM-boot'
	if [ x$feature_platform_search_hint = xy ]; then
	  search --no-floppy --fs-uuid --set=root --hint='lvm/DesktopLVM-boot'  edf3011c-e581-448e-baad-d1e03d5fb8f6
	else
	  search --no-floppy --fs-uuid --set=root edf3011c-e581-448e-baad-d1e03d5fb8f6
	fi
	echo	'Loading Linux core repo kernel ...'
	linux	/vmlinuz-linux root=/dev/mapper/DesktopLVM-root ro  quiet
	echo	'Loading initial ramdisk ...'
	initrd	/initramfs-linux.img
}
menuentry 'Arch Linux, with Linux core repo kernel (Fallback initramfs)' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-core repo kernel-fallback-b881e4e6-ad72-495c-8ae1-610ec15d27c8' {
	load_video
	set gfxpayload=keep
	insmod gzio
	insmod part_msdos
	insmod part_msdos
	insmod part_msdos
	insmod part_msdos
	insmod lvm
	insmod ext2
	set root='lvm/DesktopLVM-boot'
	if [ x$feature_platform_search_hint = xy ]; then
	  search --no-floppy --fs-uuid --set=root --hint='lvm/DesktopLVM-boot'  edf3011c-e581-448e-baad-d1e03d5fb8f6
	else
	  search --no-floppy --fs-uuid --set=root edf3011c-e581-448e-baad-d1e03d5fb8f6
	fi
	echo	'Loading Linux core repo kernel ...'
	linux	/vmlinuz-linux root=/dev/mapper/DesktopLVM-root ro  quiet
	echo	'Loading initial ramdisk ...'
	initrd	/initramfs-linux-fallback.img
}

### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/20_linux_xen ###
### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/30_os-prober ###
### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  ${config_directory}/custom.cfg ]; then
  source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg;
fi
### END /etc/grub.d/41_custom ###

### BEGIN /etc/grub.d/60_memtest86+ ###
### END /etc/grub.d/60_memtest86+ ###

Partially related, fdisk -l :

Disk /dev/sda: 500.1 GB, 500107862016 bytes, 976773168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x1bbed4e9

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          63      498014      248976   8e  Linux LVM
/dev/sda2          498015    40499864    20000925   8e  Linux LVM
/dev/sda3        40499865    56500604     8000370   8e  Linux LVM
/dev/sda4        56500605   976773167   460136281+  8e  Linux LVM

Disk /dev/mapper/arch_root-image: 1494 MB, 1494220800 bytes, 2918400 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/DesktopLVM-boot: 268 MB, 268435456 bytes, 524288 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/DesktopLVM-root: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/DesktopLVM-var: 8589 MB, 8589934592 bytes, 16777216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/DesktopLVM-home: 469.8 GB, 469757853696 bytes, 917495808 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

How do i install grub now ?

L.E. : do not include the boot partition into LVM.

Last edited by psycho (2013-08-01 18:58:18)

WonderWoofy · 2013-08-01 18:40:31

Your first partition starts at block 63. Grub has gotten so bloated that it now writes to not only the MBR, but also to the space from the MBR (after sector 0) to the start of the first partition, which these days is sector 2048. This new start point is for alignment purposes.

You either need to recreate your first partition and let fdsik align it to sector 2048, or you need to create a grub bios boot partition (as though you were using GPT).

Honestly, although grub2 can handle this, I think putting /boot inside the LVM is a bad idea. It just makes you totally reliant on grub2 with no other bootloader choice. For example, if something in some poorly designed program decided to write some crap to the area between the MBR and the first partition (which applications have been known to do), and you had a USB flash drive with syslinux on it (which the Arch live media has), then you could not set it up to boot your disc.

I think you should use the pvremove command to take that first partition out temporarily, then create two new partitions. One for /boot and one to add back. But this is not going to be very straight forward since you already have four primary partitions. It might be better to convert your partitions to GPT (which is super easy using gdisk) because then you are limited to only 128 primary partitions instead of a measly four. If you do that though, then you will have to make three new partitions if you want to continue to use grub2. You will additionally need the grub bios partition mentioned above. The grub2 wiki page had more info on this. But then you should make the /boot partition and then the partition to add back.

At the very least, you are going to have to remove the first partition so that it can be resized, and then add it back. Hopefully, you have not already filled the LVM space, so that you can easily shift things around.

psycho · 2013-08-01 18:56:26

WonderWoofy wrote:

post that i read two times

Thank you very much for your response. After 28 hours of no sleep, kids yelling at cats in front of the flat, neighbours' dogs barking at ghosts and other things, LVM got me a terrible headache. I won't use LVM for this install, and i hope everything will be all right.

Again, thank you for your post.

WonderWoofy · 2013-08-01 19:13:38

Using LVM is fine, it is actually grub2 that IMO is the biggest issue here.

In fact, if it weren't for the LVM2 move to lvmetad bug that I experienced a few months ago, I would still be using it. Now I am using btrfs which offers all the awesomeness of LVM and more!

cfr · 2013-08-02 01:37:21

I'm curious why you created so many different partitions if you planned to use lvm. For example, I use lvm and have 3 partitions: 1 /boot, 1 EFI partition and 1 for LVM-on-LUKS. Then I use lvm to create logical volumes within the third partition. Doesn't it rather undermine the point of using lvm in the first place to put each logical volume on a different partition as you seem to have done? Or have I misunderstood?

WonderWoofy · 2013-08-02 02:46:35

On my server, I have each disk chunked up into four partitions. I have had issues with pvresize in the past. But I also created this when LVM was still fairly new to me in practice. So now that I look back at it, I probably could have figured out what the issue was and resized the PV to have shrunk it down. But this way, I can simply vgremove a quarter of the disk pretty easily. And the amount of lost storage space is pretty minimal in comparison to the overall available.

I'm not sure that this is why the OP did this, but that was my logic when I did something similar.

Arch Linux

#1 2013-08-01 17:57:19

[Solved] Installing and using GRUB on LVM

#2 2013-08-01 18:40:31

Re: [Solved] Installing and using GRUB on LVM

#3 2013-08-01 18:56:26

Re: [Solved] Installing and using GRUB on LVM

#4 2013-08-01 19:13:38

Re: [Solved] Installing and using GRUB on LVM

#5 2013-08-02 01:37:21

Re: [Solved] Installing and using GRUB on LVM

#6 2013-08-02 02:46:35

Re: [Solved] Installing and using GRUB on LVM

Board footer