Unable to boot secondary Arch partition

rhearmas · 2021-02-24 20:28:29

Hey, look, it's me again! Before I continue, please keep in mind that I will refer to my main Arch partition as P2, the secondary one as P3, and the boot partition as P1.

I've been trying to boot my secondary Arch partition for a while now, but unfortunately, I get this error every time I try to boot it:

[        0.444767] Initramfs unpacking failed: invalid magic at start of compressed archive
[        0.772683] List of all partitions:
[        0.772714] 103:00000 500107608 nvme0n1
[        0.772715]  (driver?)
[        0.772756]   103:00001 266240 nvme0n1p1 eeff6aa8-f7a7-4afc-8c53-6012d6cb3425
[        0.772757]
[        0.772810]   103:00002 432731136 nvme0n1p2 0693acf3-5b01-2145-a375-faf00674ce50
[        0.772810]
[        0.772863]   103:00003 67108864 nvme0n1p3 46Zaadoc-4c1b-4569-bfed-faa4a9eee2fa
[        0.772864]
[        0.772916] No filesystem could mount root, tried:
[        0.772916]
[        0.772956] Kernel panic not syncing: VFS: Unable to mount root fs on unknown-block(259,3)
[        0.773004] CPU: 5 PID: 1 Comm: swapper/0 Not tainted 5.4.85-gnu-1-Its #1
[        0.773042] Hardware name: ASUSTeK COMPUTER INC. VivoBook_ASUSLaptop X512FL_F512FL/X512FL, BIOS X512FL.303 08/26/2019
[        0.773099] Call Trace:
[        0.773140]  dump_stack+OX64/0x88
[        0.773164]  panic+0x10b/OxZe1
[        0.773188]  mount_block_root+0x2b8/Ox2dc
[        0.773216]  prepare_namespace+0x136/0x16c
[        0.773243]  ? rest_init+Oxa4/Oxa4
[        0.773266]  kernel_init+Oxa/0x10C
[        0.773290]  ret_from_fork+0x1f/0x40
[        0.773371] Kernel Offset: 0x21000000 from Oxffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[        0.773432] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(259,3) ]---

The strange thing about this is that P2 boots just fine. P3 is the one that causes this error to appear. I am using GRUB as my bootloader; my current configuration is here: http://ix.io/2QEL

Could I receive some assistance with resolving this issue? I have looked around on the forums here and nothing else posted seems to be working. I guess it's because it doesn't involve an issue similar to this one.

loqs · 2021-02-24 20:48:34

5.4.85-gnu-1-Its https://archlinux.org/news/moving-to-zs … kinitcpio/

bezdor · 2021-02-24 21:05:41

I think loqs is correct here. If you are not already, you should consider subscribing to the announcement mailing list

https://lists.archlinux.org/listinfo/arch-announce

This was announced some time back.

rhearmas · 2021-02-24 21:53:15

That makes sense. Since I can't boot P3, how will I be able to generate the mkinitcpio properly for that partition? Will I have to chroot into it and go from there?

EDIT: nevermind, got it working in the chroot environment.

Last edited by rhearmas (2021-02-24 21:57:59)

bezdor · 2021-02-24 21:57:04

rhearmas wrote:

That makes sense. Since I can't boot P3, how will I be able to generate the mkinitcpio properly for that partition? Will I have to chroot into it and go from there?

Yea, that seems like a good approach to me.

rhearmas · 2021-02-24 22:01:10

I used that command while in the chroot environment (mkinitcpio -P), and no errors were reported. After rebooting, I still seem to get that same error as mentioned in the OP.

loqs · 2021-02-24 22:06:32

rhearmas wrote:

I used that command while in the chroot environment (mkinitcpio -P), and no errors were reported.

What changes did you make to /etc/mkinitcpio.conf before you ran `mkinitcpio -P`?

rhearmas · 2021-02-24 22:07:56

loqs wrote:

What changes did you make to /etc/mkinitcpio.conf before you ran `mkinitcpio -P`?

I added COMPRESSION=zstd to the file. Every other compression method is commented out.

loqs · 2021-02-24 22:11:46

rhearmas wrote:

loqs wrote:
What changes did you make to /etc/mkinitcpio.conf before you ran `mkinitcpio -P`?
I added COMPRESSION=zstd to the file. Every other compression method is commented out.

As linux-lts moved to the 5.10 version, all official kernels of Arch Linux now support zstd compressed initramfs images, so mkinitcpio is switching to zstd compressed images by default with version 30, which is currently on [testing].
If, for any reason, you are using a kernel version prior to 5.9, make sure to change mkinitcpio.conf COMPRESSION to use one of the compressors supported, like gzip, otherwise you will not be able to boot images generated by mkinitcpio.

Emphasis added

rhearmas · 2021-02-24 23:30:44

loqs wrote:

As linux-lts moved to the 5.10 version, all official kernels of Arch Linux now support zstd compressed initramfs images, so mkinitcpio is switching to zstd compressed images by default with version 30, which is currently on [testing].
If, for any reason, you are using a kernel version prior to 5.9, make sure to change mkinitcpio.conf COMPRESSION to use one of the compressors supported, like gzip, otherwise you will not be able to boot images generated by mkinitcpio.

Oh... I thought the mentioning of zstd meant that it also directly supported that method. Whoops!

Anyways, after fixing that, I still get the error. This time, I had compression set to gzip. The error in OP is still present, even after running mkinitcpio -P via chroot.

loqs · 2021-02-24 23:35:23

From the live media make sure the system's root and boot partitions are mounted, chroot in and execute:

file /boot/*.img

The initrds should show "gzip compressed data"

rhearmas · 2021-02-24 23:37:30

I can confirm this. Both the main and fallback image do appear with that string.

loqs · 2021-02-24 23:41:23

Using the fallback initrd the result is the same? The issue has been present for 'a while', can you approximate it? Days, weeks, moths?

rhearmas · 2021-02-24 23:42:17

loqs wrote:

Using the fallback initrd the result is the same? The issue has been present for 'a while', can you approximate it? Days, weeks, moths?

- The fallback initrd had the same error.
- This has been going on for about 2 days.

loqs · 2021-02-24 23:45:55

From the chroot, mount each partition one a time, check if it has a boot directory, if it does does it contain initrds, if it does check with file what they are compressed with.
This is to check the initrd you updated is the one the boot loader is using.

rhearmas · 2021-02-24 23:52:50

P2 is normal, and so is P3. However, P1 only has linux and linux-fallback. I don’t seem to see linux-libre’s images in P1.

Quick update on this though: mounted P3 to /mnt and P1 to /mnt/boot via the live iso. Building the initrds from there still doesn’t seem to be effective, as unmounting and rebooting still gives me the same error.

Last edited by rhearmas (2021-02-24 23:58:45)

rhearmas · 2021-02-25 12:22:54

Bump. Still experiencing this issue.

seth · 2021-02-25 12:43:51

What do you mean by "P2 is normal"? Is there a /boot directory on "P2" and are there initramfs in that?

Boot the iso, don't chroot.
Run

mount P3 /mnt # substitute P3
ls -lR /mnt/boot
mount P1 /mnt/boot # substitute P1
ls -lR /mnt/boot # yes, again

Post the outputs.

Edit: also don't bump. Chances are that loqs is sleeping.
If the situation is pressing for you, try to use your brain harder.

Last edited by seth (2021-02-25 12:45:08)

rhearmas · 2021-02-25 13:00:10

seth wrote:

What do you mean by "P2 is normal"? Is there a /boot directory on "P2" and are there initramfs in that?
Boot the iso, don't chroot.
Run
mount P3 /mnt # substitute P3
ls -lR /mnt/boot
mount P1 /mnt/boot # substitute P1
ls -lR /mnt/boot # yes, again
Post the outputs.
Edit: also don't bump. Chances are that loqs is sleeping.
If the situation is pressing for you, try to use your brain harder.

My bad. P2 boots normally and doesn't have any issues.

The output of both ls commands is labeled below.
ls -lR /mnt/boot (while /dev/nvme0n1p3 (P3) is mounted): http://ix.io/2QOk
ls -lR /mnt/boot (while /dev/nvme0n1p1 (P1) is mounted): http://ix.io/2QOl

On the topic of the edit, sorry about the bump.

seth · 2021-02-25 13:26:11

P1

/mnt/boot:
total 149656
drwxr-xr-x 4 root root     4096 Feb 22 20:02 EFI
-rwxr-xr-x 1 root root  3342336 Dec 16 14:42 FSCK0000.REC
drwxr-xr-x 4 root root     4096 Dec 16 14:42 System Volume Information
-rwxr-xr-x 1 root root        0 Dec 16 14:42 config.txt
drwxr-xr-x 6 root root     4096 Feb 24 05:18 grub
-rwxr-xr-x 1 root root 11427813 Feb 24 23:27 initramfs-linux.img
-rwxr-xr-x 1 root root 35546412 Feb 24 23:28 initramfs-linux-fallback.img
-rwxr-xr-x 1 root root  6377268 Feb 24 23:54 initramfs-linux-libre-lts.img
-rwxr-xr-x 1 root root 24083065 Feb 24 23:54 initramfs-linux-libre-lts-fallback.img
-rwxr-xr-x 1 root root 11135741 Feb 24 23:27 initramfs-linux-lts.img
-rwxr-xr-x 1 root root 32548594 Feb 24 23:27 initramfs-linux-lts-fallback.img
-rwxr-xr-x 1 root root  3624960 Feb 17 08:15 intel-ucode.img
-rwxr-xr-x 1 root root  9405504 Feb 24 16:12 vmlinuz-linux
-rwxr-xr-x 1 root root  6408512 Feb 24 23:54 vmlinuz-linux-libre-lts
-rwxr-xr-x 1 root root  9319616 Feb 24 16:12 vmlinuz-linux-lts

P3

/mnt/boot:
total 42816
drwxr-xr-x 1 root root       18 Feb 25 12:48 EFI
-rwxr-xr-x 1 root root  3342336 Jan  1  1980 FSCK0000.REC
drwxr-xr-x 1 root root      110 Aug  3  2020 System Volume Information
-rwxr-xr-x 1 root root        0 Dec 25 21:22 config.txt
drwxr-xr-x 1 root root      512 Feb 18 05:53 efi
drwxr-xr-x 1 root root       16 Feb 24 04:45 grub
-rw-r--r-- 1 root root  6377268 Feb 24 23:53 initramfs-linux-libre-lts.img
-rw-r--r-- 1 root root 24083065 Feb 24 23:53 initramfs-linux-libre-lts-fallback.img
-rw-r--r-- 1 root root  3624960 Feb 17 08:15 intel-ucode.img
-rw-r--r-- 1 root root      344 Feb 18 20:17 refind_linux.conf
-rw-r--r-- 1 root root  6408512 Feb 24 04:45 vmlinuz-linux-libre-lts
…
/mnt/boot/efi:
total 145004
drwxr-xr-x 1 root root       18 Feb 25 12:48 EFI
-rwxr-xr-x 1 root root  3342336 Feb 18 05:53 FSCK0000.REC
drwxr-xr-x 1 root root      110 Feb 18 05:53 System Volume Information
-rwxr-xr-x 1 root root        0 Feb 18 05:53 config.txt
-rwxr-xr-x 1 root root 11144307 Feb 18 05:53 initramfs-linux.img
-rwxr-xr-x 1 root root 32531375 Feb 18 05:53 initramfs-linux-fallback.img
-rwxr-xr-x 1 root root  7814467 Feb 18 05:53 initramfs-linux-lts.img
-rwxr-xr-x 1 root root 29599586 Feb 18 05:53 initramfs-linux-lts-fallback.img
-rwxr-xr-x 1 root root 11289984 Feb 18 05:53 initramfs-linux-zen.img
-rwxr-xr-x 1 root root 33481901 Feb 18 05:53 initramfs-linux-zen-fallback.img
-rwxr-xr-x 1 root root        0 Feb 18 05:53 intel-ucode.img
-rwxr-xr-x 1 root root  9321824 Feb 18 05:53 vmlinuz-linux-lts
-rwxr-xr-x 1 root root  9942432 Feb 18 05:53 vmlinuz-linux-zen

The newest images are the linux-libre ones and they are on both partitions. You should look at all images (with "file") and the ones that are zstd compressed are the ones you're actually booting - regardless on which partition they are.

rhearmas · 2021-02-25 13:44:01

All of the images on all 3 partitions are gzip-compressed. Not a single one is compressed with zstd or other methods.

seth · 2021-02-25 14:39:23

Did you also check the older images in /mnt/boot/efi?
Please post an updated error message and also

lsblk -f

(to see whether there's maybe another pot. boot partition lingering around)

Last edited by seth (2021-02-25 14:39:34)

rhearmas · 2021-02-25 15:19:30

Yes, I checked /mnt/boot/efi, and every image there was also compressed with gzip.

Output of lsblk -f:

NAME        FSTYPE   FSVER LABEL UUID                                 FSAVAIL FSUSE% MOUNTPOINT
loop0       squashfs 4.0                                                    0   100% /var/lib/snapd/snap/snapd/9721
loop1       squashfs 4.0                                                    0   100% /var/lib/snapd/snap/core18/1885
loop2       squashfs 4.0                                                    0   100% /var/lib/snapd/snap/snapd/9607
loop3       squashfs 4.0                                                    0   100% /var/lib/snapd/snap/core18/1932
nvme0n1
├─nvme0n1p1 vfat     FAT32 BOOT  4281-C511                              95.9M    63% /boot
├─nvme0n1p2 ext4     1.0   ARCH  aeb4402d-be5f-4ace-82de-0ed06a191688  221.6G    40% /
└─nvme0n1p3 btrfs                57a43922-4ce3-490e-bd43-eba98dc6d4ae   40.4G    36% /mnt

There are no other drives that I use here. P1 is boot, P2 is the working Arch partition, and P3 is the unbootable one.

Last edited by rhearmas (2021-02-25 15:20:36)

seth · 2021-02-25 15:36:57

Your grub.cfg only has the parabola installation on 57a43922-4ce3-490e-bd43-eba98dc6d4ae so that would be what you need to fix, not arch?

a) do you try to boot parabola when the image decompression fails?
b) do you get a rescue shell?
c) In the assumed rescue shell, "zgrep -i decompress /proc/config.gz"

rhearmas · 2021-02-25 15:42:06

seth wrote:

Your grub.cfg only has the parabola installation on 57a43922-4ce3-490e-bd43-eba98dc6d4ae so that would be what you need to fix, not arch?
a) do you try to boot parabola when the image decompression fails?
b) do you get a rescue shell?
c) In the assumed rescue shell, "zgrep -i decompress /proc/config.gz"

Sorry about that, I didn't pay attention!

I select the boot option, and I get the kernel panic error in OP almost immediately after selecting it. Also, I do not get a rescue shell. Said error is all I get; I'm not given any opportunity to type any commands.

Arch Linux

#1 2021-02-24 20:28:29

Unable to boot secondary Arch partition

#2 2021-02-24 20:48:34

Re: Unable to boot secondary Arch partition

#3 2021-02-24 21:05:41

Re: Unable to boot secondary Arch partition

#4 2021-02-24 21:53:15

Re: Unable to boot secondary Arch partition

#5 2021-02-24 21:57:04

Re: Unable to boot secondary Arch partition

#6 2021-02-24 22:01:10

Re: Unable to boot secondary Arch partition

#7 2021-02-24 22:06:32

Re: Unable to boot secondary Arch partition

#8 2021-02-24 22:07:56

Re: Unable to boot secondary Arch partition

#9 2021-02-24 22:11:46

Re: Unable to boot secondary Arch partition

#10 2021-02-24 23:30:44

Re: Unable to boot secondary Arch partition

#11 2021-02-24 23:35:23

Re: Unable to boot secondary Arch partition

#12 2021-02-24 23:37:30

Re: Unable to boot secondary Arch partition

#13 2021-02-24 23:41:23

Re: Unable to boot secondary Arch partition

#14 2021-02-24 23:42:17

Re: Unable to boot secondary Arch partition

#15 2021-02-24 23:45:55

Re: Unable to boot secondary Arch partition

#16 2021-02-24 23:52:50

Re: Unable to boot secondary Arch partition

#17 2021-02-25 12:22:54

Re: Unable to boot secondary Arch partition

#18 2021-02-25 12:43:51

Re: Unable to boot secondary Arch partition

#19 2021-02-25 13:00:10

Re: Unable to boot secondary Arch partition

#20 2021-02-25 13:26:11

Re: Unable to boot secondary Arch partition

#21 2021-02-25 13:44:01

Re: Unable to boot secondary Arch partition

#22 2021-02-25 14:39:23

Re: Unable to boot secondary Arch partition

#23 2021-02-25 15:19:30

Re: Unable to boot secondary Arch partition

#24 2021-02-25 15:36:57

Re: Unable to boot secondary Arch partition

#25 2021-02-25 15:42:06

Re: Unable to boot secondary Arch partition

Board footer