[SOLVED] BTRFS Root on 2009 Macbook Pro hangs at scanning for btrfs

n8henrie · 2021-08-13 14:57:12

Hi all,

On my primary machine I've been very happy for a few years running Arch on BTRFS root, RAID1 across a few NVME drives, using Grub. I also have an old Macbook Pro that's been running Arch happily for a few years (2 actually). One of them has a mostly broken display and acts as a headless backup and Plex server; lately I've started running out of drive space frequently, so I decided to upgrade the old 360 GB spinning rust to a 500 GB SSD to liven things up and help with the space issues.

I've done 4 BTRFS root conversions now (one Arch, and Ubuntu / Void / Raspbian on a Pi) so I thought I was pretty comfortable with it, but I'm not having any luck this time.

I started by hooking up the new SSD, GPT format, adding a 550MB boot partition formatted as vfat / ef00, then my BTRFS root partition with the subvolume layout I like (@, @boot, @var, @home, @snapshots), then an 8 GB swap partition (8 GB ram, in case I want to support hibernation at some point).

This went fine.

I currently have the `@boot` subvol mounted to `/boot`, and mounted the EFI partition to `/boot/efi` (I have it mounted to `/efi` on my other machine, but the Macbook seems to need it on `/boot/efi` per reading lots of other threads)

Then I used rsync with the following flags to copy over the old hard drive's contents onto the new drive:

# rsync --archive --hard-links --acls --xattrs --sparse --partial --progress --verbose --info=progress2 -- /oldroot/ /newroot/

Then genfstab -U >> /mnt/btrfs-root/etc/fstab

Then I chroot into the new drive and

- edit / review fstab
- Add `BINARIES=("/usr/bin/btrfs")` and `HOOKS=(base keyboard btrfs udev autodetect modconf block filesystems fsck)` to `mkinitcpio.conf`

# pacman -Syu
# pacman -Syu linux linux-headers linux-lts linux-lts-headers intel-ucode os-prober efibootmgr btrfs-progs # to be safe and regenerate all the initramfs
...
# pacman -S grub
# pacman -S snapper grub-btrfs # a few tools I know I'll want
...
# mkinitcpio -P # once more just to be sure
==> Building image from preset: /etc/mkinitcpio.d/linux-lts.preset: 'default'
  -> -k /boot/vmlinuz-linux-lts -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-lts.img
==> Starting build: 5.10.56-1-lts
  -> Running build hook: [base]
  -> Running build hook: [keyboard]
==> WARNING: Possibly missing firmware for module: xhci_pci
  -> Running build hook: [btrfs]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating zstd-compressed initcpio image: /boot/initramfs-linux-lts.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux-lts.preset: 'fallback'
  -> -k /boot/vmlinuz-linux-lts -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-lts-fallback.img -S autodetect
==> Starting build: 5.10.56-1-lts
  -> Running build hook: [base]
  -> Running build hook: [keyboard]
==> WARNING: Possibly missing firmware for module: xhci_pci
  -> Running build hook: [btrfs]
  -> Running build hook: [udev]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: aic94xx
==> WARNING: Possibly missing firmware for module: wd719x
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating zstd-compressed initcpio image: /boot/initramfs-linux-lts-fallback.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
  -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
==> Starting build: 5.13.9-arch1-1
  -> Running build hook: [base]
  -> Running build hook: [keyboard]
==> WARNING: Possibly missing firmware for module: xhci_pci
  -> Running build hook: [btrfs]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating zstd-compressed initcpio image: /boot/initramfs-linux.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'fallback'
  -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-fallback.img -S autodetect
==> Starting build: 5.13.9-arch1-1
  -> Running build hook: [base]
  -> Running build hook: [keyboard]
==> WARNING: Possibly missing firmware for module: xhci_pci
  -> Running build hook: [btrfs]
  -> Running build hook: [udev]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
==> WARNING: Possibly missing firmware for module: aic94xx
==> WARNING: Possibly missing firmware for module: wd719x
  -> Running build hook: [filesystems]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating zstd-compressed initcpio image: /boot/initramfs-linux-fallback.img
==> Image generation successful
# grub-install --target=x86_64-efi --efi-directory=/boot/efi --removable --recheck
Installing for x86_64-efi platform.
Installation finished. No error reported.
# grub-mkconfig -o /boot/grub/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-linux-lts
Found initrd image: /boot/intel-ucode.img /boot/initramfs-linux-lts.img
Found fallback initrd image(s) in /boot:  intel-ucode.img initramfs-linux-lts-fallback.img
Found linux image: /boot/vmlinuz-linux
Found initrd image: /boot/intel-ucode.img /boot/initramfs-linux.img
Found fallback initrd image(s) in /boot:  intel-ucode.img initramfs-linux-fallback.img
Warning: os-prober will not be executed to detect other bootable partitions.
Systems on them will not be added to the GRUB boot configuration.
Check GRUB_DISABLE_OS_PROBER documentation entry.
Detecting snapshots ...
Info: Separate boot partition not detected 
Warning: snapper detected but config 'root' does not exist
No snapshots found.
If you think an error has occurred , please file a bug report at " https://github.com/Antynea/grub-btrfs "
Nothing to do. Abort.
done

Then reboot. Currently, with the above setup, I see the boot option list in the Apple Boot Manager. When I select it, it almost immediately hangs with minimal output; the last line is "scanning for btrfs filesystems" and never gets past that. As I noted, the display is pretty busted (for some reason the built-in Apple mini display port works great with a VGA adapter from the live image but doesn't show boot info when booting from hard disk -- I'm probably missing some kernel driver or something), but I don't see any other output beneath that.

For the first several attempts I would get a blank screen and the EFI partition wouldn't even show up as a boot option, which was when I did some reading and found that the Apple Boot Manager requires `/boot/efi/EFI/BOOT/BOOTX64.EFI `, and instead of just copying the file I found that adding the `--removable` flag to the `grub-install` command gets it to provide that file.

I assume that because it's hanging at `scanning for btrfs filesystems` that it's not finding my root for some reason; I've triple checked the UUIDs in my `/etc/fstab` and they are correct, and I can see that the correct subvol is listed in the `rootflags=subvol=` parts of `/boot/grub/grub.cfg`.

I can successfully "boot" with `systemd-nspawn --boot --directory=/mnt/btrfs-root/`; I see an error about `systemd-remount-fs.service`:

systemd-remount-fs[24]: mount: /: can't find UUID=c899ba80-df7c-4ea6-b29f-37deda5b1a1a

I'm not very familiar with systemd-nspawn, and it seems like this might be something about that environment, because `sudo fdisk -l` and `sudo blkid` are both empty (`sudo lsblk` still has output, though `sudo lsblk -o +UUID` doesn't show the UUIDs).

I'm sure there's lots of other output I could provide to be helpful; please let me know if posting my complete `mkinitcpio.conf`, `grub.cfg`, `/etc/default/grub`, etc. would be helpful, though I think I've listed any changes I've made.

One thing I'm wondering about is that `efibootmgr` seems to report a file that doesn't exist as the default for booting:

# efibootmgr -v
BootCurrent: 0000
BootOrder: 0000,0001
Boot0000* BOOT	HD(1,GPT,90242b16-0913-43aa-a730-16fa0dbfa55d,0x800,0x113000)/File(\EFI\BOOT\grubx64.efi)
Boot0001* arch	HD(1,GPT,90242b16-0913-43aa-a730-16fa0dbfa55d,0x800,0x113000)/File(\EFI\arch\grubx64.efi)
BootFFFF* 	PciRoot(0x0)/Pci(0x6,0x1)/USB(1,0)/HD(1,MBR,0xa71a00c,0x19b000,0x22000)/File(\EFI\BOOT\BOOTX64.efi)
# ls -l /boot/efi/EFI/BOOT/grubx64.efi
ls: cannot access '/boot/efi/EFI/BOOT/grubx64.efi': No such file or directory
# find /boot -iname '*.efi' -ls
        8    280 -rwxr-xr-x   1 root     root       286720 Aug 13 14:30 /boot/efi/EFI/BOOT/BOOTX64.EFI
     2860    280 -rw-r--r--   1 root     root       286720 Aug 13 14:30 /boot/grub/x86_64-efi/core.efi
     2861    280 -rw-r--r--   1 root     root       286720 Aug 13 14:30 /boot/grub/x86_64-efi/grub.efi

Also, both in and out of the chroot, `ls /sys/firmware/efi/efivars` shows plenty of output`, but I get this error:

# modprobe efivars
modprobe: FATAL: Module efivars not found in directory /lib/modules/5.12.15-arch1-1

Many thanks in advance for suggestions.

Last edited by n8henrie (2021-08-17 21:14:34)

n8henrie · 2021-08-13 21:22:51

I thought that perhaps I needed to move the `udev` hook before `btrfs` so it could look up the root by UUID, changing the active lines in my `mkinitcpio.conf` to be:

# sed -n '/^[^#]/p' /etc/mkinitcpio.conf
MODULES=()
BINARIES=(fsck fsck.btrfs btrfsck)
FILES=()
HOOKS=(base udev keyboard btrfs autodetect modconf block filesystems fsck)

Also did not work.

Then I suddenly realized that I have a second identical Macbook Pro, also running Arch, but with a working display -- so I took the drive back out and booted from it on the other laptop so I could better see the boot log.

It looks like I get a fairly normal GRUB prompt, and after it times out it goes to:

Starting version 249.3-1-arch
Scanning for Btrfs filesystems
_

where `_` is a blinking cursor. It then hangs there, indefinitely.

n8henrie · 2021-08-15 18:15:33

To try to see if it was a hardware issue with the Macbooks, I took the same hard drive and tried to boot from my existing Linux desktop, it hung at the same place.

I tried removing the `BTRFS` hook from `mkinitcpio.conf`, and then it just hangs at the line before: `Starting version 249.3-1-arch`

n8henrie · 2021-08-17 21:22:12

Brilliantly, I hadn't noticed the `quiet` in `GRUB_CMDLINE_LINUX` in the default `/etc/default/grub` (also could have just seen / edited it from the Grub menu).

Once I removed that, I could see the systemd output, which was eventually hanging on a service file that did not have a timeout set (was infinite).

Mounting the drive from another Arch device, then using `sudo systemd-nspawn --boot --directory=/mymountpoint`, I was able to `systemctl disable` that service file, which solved the issue. (I eventually just added `TimeoutStartSec` to that service file.)

In the end, I have a 2009 MBP happily booting from a zstd-compressed BTRFS root, including a compressed `@boot` subvolume, and separate `/efi`.

Arch Linux

#1 2021-08-13 14:57:12

[SOLVED] BTRFS Root on 2009 Macbook Pro hangs at scanning for btrfs

#2 2021-08-13 21:22:51

Re: [SOLVED] BTRFS Root on 2009 Macbook Pro hangs at scanning for btrfs

#3 2021-08-15 18:15:33

Re: [SOLVED] BTRFS Root on 2009 Macbook Pro hangs at scanning for btrfs

#4 2021-08-17 21:22:12

Re: [SOLVED] BTRFS Root on 2009 Macbook Pro hangs at scanning for btrfs

Board footer