Linux 6.7 kernel creating very large initramfs.

Scimmia · 2024-01-15 12:52:43

There are a couple of reports of the kms hook triggering this on the fallback because of nouveau. I just checked, I never switched to the kms hook when it was introduced, I just use the modules array for early kms, so that's why I'm not seeing it.

Trilby · 2024-01-15 14:28:10

Scimmia wrote:

Yes, this is an nvidia-only issue (even if, for some reason, you have nvidia installed on an Intel system).

Either this is wrong, or there is yet another issue. I have a pure intel system with no nvidia packages installed, but my fallback image is huge.

~~I also don't use a kms hook - I just include i915 in modules.~~ (edit: oops, I feel like an idiot - I was shelled into my other machine when I checked the mkinitcpio.conf. Somehow my local systems mkinitcpio.conf has been overwritten by the default. So I do have the kms hook for the moment).

Last edited by Trilby (2024-01-15 14:43:58)

Scimmia · 2024-01-15 14:30:36

You specifically said earlier that nvidia was being included in the fallback initramfs.

Trilby · 2024-01-15 14:45:23

Apparently I do have the kms hook, but no nvidia packages. All the nvidia content in the initramfs are provided by the linux or linux-firmware packages (which everyone would have installed).

Reconfiguring my mkinitcpio.conf to again exclude kms resulted in normal image sizes again. I'm not sure when / how my config got reset on this machine as streamlining that is one of the first things I do on every machine I own. Now the image listings are down from several thousand items to a couple hundred.

Last edited by Trilby (2024-01-15 14:53:19)

Scimmia · 2024-01-15 15:05:06

Ah, so it was nouveau, not nvidia, being included.

lenhuppe · 2024-01-15 16:59:53

Scimmia wrote:

Wait, so only AMD is immune? Strange that it wasn't caught in Testing.
Same statement, if you willing to test, use lsinitcpio

FYI: I have an AMD Ryzen and I was getting the "boot partition full" message" until I reinstalled with a 1GB esp partition instead of the 256MB I had been running on for some time.

Scimmia · 2024-01-15 17:02:16

Right, it'll hit the fallback if you're using the kms hook, no matter what system you're on.

raneon · 2024-01-15 17:52:00

I really wonder why this issue is not part of the "Latest News" section on archlinux.org... I must say this is annoying, I do not read the forum before updating my systems :-(

d.ALT · 2024-01-15 19:54:46

With no kms declared into HOOKS (Intel iGPU only):

     Total   Exclusive  Set shared  Filename
   7.03MiB       0.00B           -  /boot/intel-ucode.img
     0.00B       0.00B           -  /boot/refind_linux.conf.orig
     0.00B       0.00B           -  /boot/refind_linux.conf
  12.29MiB       0.00B           -  /boot/vmlinuz-linux-lts
  12.40MiB       0.00B           -  /boot/vmlinuz-linux
  13.33MiB       0.00B           -  /boot/vmlinuz-linux-zen
  14.51MiB    14.51MiB           -  /boot/initramfs-linux-lts.img
  41.95MiB    41.95MiB           -  /boot/initramfs-linux-lts-fallback.img
  15.23MiB    15.23MiB           -  /boot/initramfs-linux.img
  43.66MiB    43.66MiB           -  /boot/initramfs-linux-fallback.img
  15.46MiB    15.46MiB           -  /boot/initramfs-linux-zen.img
  44.84MiB    44.84MiB           -  /boot/initramfs-linux-zen-fallback.img
 220.70MiB   175.64MiB    45.05MiB  /boot

linux 6.7.arch3-1
linux-lts 6.6.11-2
linux-zen 6.7.zen3-1

ZeroLinux · 2024-01-15 21:13:19

Hello, please help me. I cannot update the system to the latest Linux 6.7 kernel.

$ lspci -k | grep -EA3 'VGA|3D|Display'
01:00.0 VGA compatible controller: NVIDIA Corporation GT218 [GeForce 210] (rev a2)
	Subsystem: Micro-Star International Co., Ltd. [MSI] N210 [Geforce 210] PCIe graphics adapter
	Kernel driver in use: nouveau
	Kernel modules: nouveau

in /etc/mkinitcpio.conf
MODULES="nouveau"
HOOKS="base udev autodetect modconf block filesystems keyboard fsck shutdown"

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda2        39G   31G  6.0G  84% /
/dev/sda1       289M   68M  206M  25% /boot
/dev/sda3       196G   89G   98G  48% /home

When I try to update

...
==> Creating zstd-compressed initcpio image: '/boot/initramfs-linux-fallback.img'
zstd: error 70 : Write error : cannot write block : No space left on device
bsdtar: Write error
bsdtar: Write error
==> ERROR: Image generation FAILED: 'sort reported an error'
error: command failed to execute correctly

I can connect to the computer only remotely. So I cannot resize /boot partition

nl6720 · 2024-01-15 21:19:35

mkinitcpio 37.2-1 should fix the issue for most people.

silvrax · 2024-01-15 22:06:54

I've upgraded again, using mkinitcpio 37.2-1, going from kernel 6.6.8 to 6.7.
Unfortunately, I still run out of space when generating the fallback image.
Some numbers.

6.6.8:
26 MiB initramfs-linux.img
78 MiB initramfs-linux-fallback.img

6.7 with previous mkinitcpio:
190 MiB initramfs-linux.img
243 MiB initramfs-linux-fallback.img

6.7 with mkinitcpio 37.2-1:
64 MiB initramfs-linux.img
118 MiB initramfs-linux-fallback.img

My /boot partition is 200 MiB and is separate from ESP. (That's the minimum documented in Partitioning wiki). Though df -h reports the file-system size (ext4) as 189MiB.
I do have kms hook enabled.

By my calculations I'm about 40MiB short for the fallback image.

cirrus9 · 2024-01-15 22:53:49

I upgraded to the new mkinitcpio 37.2-1, and all would be okay for me, even with KMS hook. However KMS hook still makes for larger initramfs images.

With KMS 144Mb for main, and 208Mb for fall back. Without KMS 101Mb for main, and 132Mb for fall back.

Since I don't need KMS for my Nvidia setup, I'm leaving KMS out.

The previous initramfs images (with the old mkinitcpio version) were much larger at 238Mb for main, and 270Mb for fallback.

These were all with 6.7 kernel.

Last edited by cirrus9 (2024-01-15 22:54:32)

xerxes_ · 2024-01-15 23:12:36

@silvrax: solutions are in post #15. You may disable fallback creation by editing mkinitcpio preset file, but you better have second, lts kernel (with also disabled fallback creation) and update them one a time, selectively.

Last edited by xerxes_ (2024-01-15 23:13:42)

glyn · 2024-01-16 11:28:55

Post #15 worked for me, thanks! I chose to edit `/etc/mkinitcpio.d/linux.preset` to change `fallback_options` from `"-S autodetect"` to `"-S autodetect -S kms"`.

I then regenerated the fallback image using `sudo mkinitcpio -p linux` and rebooted.

What should I do with fallback_options now? Leave it changed or change it back? I assume that it needs to stay changed until such time as there is a fix that reduces the size of the fallback image.

BTW here's a gentle write-up of the problem for anyone else who is new to a lot of this stuff.

Last edited by glyn (2024-01-16 12:51:09)

frostschutz · 2024-01-16 14:08:42

Make your boot partition larger, skimping on space here is just not worth the hassle in the long run. Kernel+initramfs sizes have been growing for years for various reasons. If it barely fits now, you'll run out of space at some point sooner or later.

If you never use the fallback image anyway, you can remove it altogether. Just have a usb rescue stick ready for emergencies. People tend forget about the fallback and never even try it and directly boot a live system to fix things. You need that in some situations anyway.

If your fallback works without KMS, that's fine too.

Technically some /boot space could be saved by not duplicating initramfs files at all. Grub+Kernel support concatenating initramfs, so the fallback image would only need files not already present in the regular initramfs anyway. So a fallback without KMS modules, could have KMS modules anyway by passing both regular and fallback image to the Grub initrd line. It's just difficult to implement, making a larger boot partition is simpler...

Last edited by frostschutz (2024-01-16 14:10:05)

Trilby · 2024-01-16 15:42:25

frostschutz wrote:

Make your boot partition larger... If it barely fits now, you'll run out of space at some point sooner or later.

Was this directed at a particular person's boot partition size, or meant as a more general statement? If the latter, I'm not sure you're grasping the scale of the problem many of us faced. This wasn't the old initramfs barely fitting, or the new one growing incrementally, but rather a 5-10 fold increase in size.

Having a boot partition for room for growth is wise, but not room for exponential cancerous growth.

archer666 · 2024-01-17 14:56:35

Was bitten by this too. zstd -> xz does basically nothing

I have already just the kernel+initramfs, no fallback, BECAUSE

/dev/nvme1n1p2 97280 60420 36860 63% /boot

/boot is not only just 100MB in size, but also sharing that 100MB with Windows.
I'm pretty positive Arch Linux docs DIDN'T mention 300MB minimum at the time
of the install (which was 2016 mind you "rolling release").

Only viable solution: cut kms from HOOKS.

And yeah, I do have

core/mkinitcpio 37.2-1 [installed]

still with KMS it burps out a 54MB initramfs. Without kms it's 7.5MB.

Scimmia · 2024-01-17 14:58:03

yes, the KMS hook drastically increases the initramfs size. That's normal and has little to do with this thread.

Last edited by Scimmia (2024-01-17 15:00:04)

PseudoSpock · 2024-03-06 19:29:58

This is a problem on ARM64, too. I'm running EndeavourOS under UTM (Mac port of QEMU) on an M1:

❯ neofetch --stdout
username@vm-m1 
----------------- 
OS: EndeavourOS Linux aarch64 
Host: QEMU Virtual Machine virt-7.2 
Kernel: 6.7.6-1-aarch64-ARCH 
Uptime: 10 mins 
Packages: 1397 (pacman) 
Shell: zsh 5.9 
Resolution: 3840x2097 
DE: Plasma 5.27.10 
WM: kwin 
WM Theme: ExposeAir 
Theme: [Plasma], Windows-7-2.1 [GTK2/3] 
Icons: Windows 7 Ultimate [Plasma], Windows 7 Ultimate [GTK2/3] 
Terminal: konsole 
Terminal Font: UbuntuMono Nerd Font 12 
CPU: (8) 
GPU: 00:02.0 Red Hat, Inc. Virtio 1.0 GPU 
Memory: 2943MiB / 46954MiB

Tried updating today, and package linux-aarch64-6.7.8-1-aarch64.pkg.tar.xz caused /boot to fill to 100% with oversized initramfs images.

/boot is only 200M, as this was installed via UTM's image gallery base Arch image, then using the EndeavourOS install script.

Under 6.7.6, I have initramfs size under 9 meg:

❯ du -s /boot/* | sort -n
4       /boot/startup.nsh
8368    /boot/initramfs-linux.img
14948   /boot/Image.gz
43300   /boot/Image
54312   /boot/dtbs

6.7.8 kernel update errored when /boot filled full, leaving me with a new image of over 80 meg.

Luckily I was able to revert... but had to do:

rm /boot/*initramfs*
cd /var/cache/pacman/pkg/
pacman -U file://linux-aarch64-6.7.6-1-aarch64.pkg.tar.xz # this fails on space due to fallback size, but gets the kernel itself installed...

mkdir /mnt/boot
cp -a /boot/. /mnt/boot/.

mount -o bind /mnt/boot /boot

pacman -U file://linux-aarch64-6.7.6-1-aarch64.pkg.tar.xz # succeeds, as this spot has plenty of space for initramfs and fallback...

umount /boot # undo bind mount

umount /boot # verify we aren't still bind mounted

mount /boot # proper location from /etc/fstab

rm /boot/*initramfs*

cp -a /mnt/boot/initramfs-linux.img /boot/.

Then test rebooted and whew, that worked. I went through all that just in case the exact location of the kernel on disk was somehow important to this bootloader (don't know), but usually initramfs files don't have such a restriction.

But yeah, I can't update my kernel until this is fixed.

Last edited by PseudoSpock (2024-03-06 19:32:16)

V1del · 2024-03-06 19:48:10

It is fixed, drop the kms hook from your mkinitcpio.conf and/or disable fallback image generation (which is pretty useless on a VM anyway, you're never going to need all that firmware)

PseudoSpock · 2024-03-06 19:56:54

@V1del: success via mkinitcpio removing kms.

❯ uname -r
6.7.8-1-aarch64-ARCH

torvic9 · 2024-03-06 20:04:17

I had a very similar problem but it was solved by running
'mkinitcpio -P'
once again after the update completed. The UKIs were restored as well.

Zibi1981 · 2024-03-06 21:21:16

frostschutz wrote:

Make your boot partition larger, skimping on space here is just not worth the hassle in the long run.

And what about people trying to run Arch on notebook with Windows preinstalled (dual-boot) and EFI partition already factory settled, without any chance of resizing it?

V1del · 2024-03-06 23:04:11

You can use your root or a different partition with a bootloader that can read linux filesystems, you're not forced to keep the initramfs or /boot for that matter on the ESP partition.

Arch Linux

#26 2024-01-15 12:52:43

Re: Linux 6.7 kernel creating very large initramfs.

#27 2024-01-15 14:28:10

Re: Linux 6.7 kernel creating very large initramfs.

#28 2024-01-15 14:30:36

Re: Linux 6.7 kernel creating very large initramfs.

#29 2024-01-15 14:45:23

Re: Linux 6.7 kernel creating very large initramfs.

#30 2024-01-15 15:05:06

Re: Linux 6.7 kernel creating very large initramfs.

#31 2024-01-15 16:59:53

Re: Linux 6.7 kernel creating very large initramfs.

#32 2024-01-15 17:02:16

Re: Linux 6.7 kernel creating very large initramfs.

#33 2024-01-15 17:52:00

Re: Linux 6.7 kernel creating very large initramfs.

#34 2024-01-15 19:54:46

Re: Linux 6.7 kernel creating very large initramfs.

#35 2024-01-15 21:13:19

Re: Linux 6.7 kernel creating very large initramfs.

#36 2024-01-15 21:19:35

Re: Linux 6.7 kernel creating very large initramfs.

#37 2024-01-15 22:06:54

Re: Linux 6.7 kernel creating very large initramfs.

#38 2024-01-15 22:53:49

Re: Linux 6.7 kernel creating very large initramfs.

#39 2024-01-15 23:12:36

Re: Linux 6.7 kernel creating very large initramfs.

#40 2024-01-16 11:28:55

Re: Linux 6.7 kernel creating very large initramfs.

#41 2024-01-16 14:08:42

Re: Linux 6.7 kernel creating very large initramfs.

#42 2024-01-16 15:42:25

Re: Linux 6.7 kernel creating very large initramfs.

#43 2024-01-17 14:56:35

Re: Linux 6.7 kernel creating very large initramfs.

#44 2024-01-17 14:58:03

Re: Linux 6.7 kernel creating very large initramfs.

#45 2024-03-06 19:29:58

Re: Linux 6.7 kernel creating very large initramfs.

#46 2024-03-06 19:48:10

Re: Linux 6.7 kernel creating very large initramfs.

#47 2024-03-06 19:56:54

Re: Linux 6.7 kernel creating very large initramfs.

#48 2024-03-06 20:04:17

Re: Linux 6.7 kernel creating very large initramfs.

#49 2024-03-06 21:21:16

Re: Linux 6.7 kernel creating very large initramfs.

#50 2024-03-06 23:04:11

Re: Linux 6.7 kernel creating very large initramfs.

Board footer