/boot is full (Solved)

Gabachin · 2023-05-18 05:07:32

I have a Dell XPS13 laptop, on which I did a manual vanilla Arch install with Hyprland. I installed systemd-boot as the bootloader, and gave the boot partition 512M of space. Sometime later, a pacman upgrade failed because /boot was full. I tried to resize with gparted, but got an error message. I then thought it would be easier to just uninstall system-boot and replace it with grub, but following the usual wiki procedure that I had used several times before with success, I still could not get the laptop to boot. I mounted the partitions correctly, updated my fstab file, chrooted as always, and installed grub successfully, that is, with no errors. But a reboot sent me straight to UEFI options and nothing else. I then reinstalled the system, this time using archinstall, and am now happily running my fresh install. But the installer also allotted 512M for /boot so I guess I am going to run into the same problem at some point, Why does this happen and what can I do about it?

❯ 
df
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        4.0M     0  4.0M   0% /dev
tmpfs           7.7G   45M  7.6G   1% /dev/shm
tmpfs           3.1G   15M  3.1G   1% /run
/dev/nvme0n1p2  468G  149G  296G  34% /
tmpfs           7.7G   19M  7.7G   1% /tmp
/dev/nvme0n1p1  510M  510M     0 100% /boot
tmpfs           1.6G   32K  1.6G   1% /run/user/1000


❯ la -la /boot
drwxr-xr-x    - root 31 Dec  1969 .
drwxr-xr-x    - root 17 May 22:44 ..
drwxr-xr-x    - root 17 May 05:15 EFI
drwxr-xr-x    - root 17 May 20:17 loader
.rwxr-xr-x 103M root 17 May 08:52 initramfs-custom.img
.rwxr-xr-x  67M root 17 May 08:57 initramfs-linux-fallback.img
.rwxr-xr-x 127M root 17 May 10:04 initramfs-linux-lts-fallback.img
.rwxr-xr-x 103M root 17 May 10:03 initramfs-linux-lts.img
.rwxr-xr-x 103M root 17 May 08:57 initramfs-linux.img
.rwxr-xr-x 7.3M root 16 May 13:11 intel-ucode.img
.rwxr-xr-x  13M root 17 May 05:13 vmlinuz-linux
.rwxr-xr-x  11M root 17 May 10:03 vmlinuz-linux-lts

A suggestion on this forum was to delete the fallback.img and then delete fallback in the PRESET line:

❯ cat /etc/mkinitcpio.d/linux.preset 
# mkinitcpio preset file for the 'linux' package

ALL_config="/etc/mkinitcpio.conf"
ALL_kver="/boot/vmlinuz-linux"
ALL_microcode=(/boot/*-ucode.img)

PRESETS=('default' 'fallback')

#default_config="/etc/mkinitcpio.conf"
default_image="/boot/initramfs-linux.img"
#default_uki="/efi/EFI/Linux/arch-linux.efi"
#default_options="--splash /usr/share/systemd/bootctl/splash-arch.bmp"

#fallback_config="/etc/mkinitcpio.conf"
fallback_image="/boot/initramfs-linux-fallback.img"
#fallback_uki="/efi/EFI/Linux/arch-linux-fallback.efi"
fallback_options="-S autodetect"

Is this a viable solution?

Last edited by Gabachin (2023-05-21 02:59:06)

seth · 2023-05-18 06:39:50

Why are your initramfs > 100MB??
Wha's the custom initramfs?

Gabachin · 2023-05-18 12:01:07

I installed the system using the archinstall shortcut, so these files were created by the script. Unfortunately, I did not have the sense to copy from cat the contents of /boot when I did the first install, which was manual, that is, done without using the installer. Can I just delete the custom.initramfs? It seems odd to me that the script did not create a partition large enough to avoid this problem in the first place.

Last edited by Gabachin (2023-05-18 12:02:58)

Trilby · 2023-05-18 12:44:17

Gabachin wrote:

I installed the system using the archinstall shortcut

The installer would not have created the custom entry. It also would not have created images anywhere near that big unless you did some customization (e.g., of mkinitcpio.conf)*. The extra "custom" entry is only a small part of the problem: all of your images are on the order of 7 times larger than they would be with default settings.

Gabachin wrote:

Unfortunately, I did not have the sense to copy from cat the contents of /boot when I did the first install, which was manual, that is, done without using the installer.

Can you restate that? Is there a typo there? I really can't parse that sentence.

Gabachin wrote:

Can I just delete the custom.initramfs?

If you are not using it, yes.

Gabacin wrote:

It seems odd to me that the script did not create a partition large enough to avoid this problem in the first place.

It created a partition large enough for typical usage, but you are clearly doing something quite atypical. You can certainly do so, but you can't expect automated tools to have default settings that cater to atypical usage.

*edit: or perhaps if you failed to make a recommended adjustment as noted below.

Last edited by Trilby (2023-05-18 12:55:26)

Scimmia · 2023-05-18 12:47:27

Using the kms hook with nvidia? That is default, although the wiki page tells you to remove it. It will make the initramfs' huge.

Gabachin · 2023-05-18 17:13:24

@Trilby: The first time I installed Arch, I did not use archinstall. The second time, I used the script.

When I did the manual install, the only directories in /boot were /EFI, which I did not create, and /loader, which I did: into which I placed

boot/loader/entries/arch.conf
title   Arch Linux
linux   /vmlinuz-linux
initrd  /intel-ucode.img
initrd  /initramfs-linux.img
llinux-lts vmlinuz-linux-lts
initrd  /initramfs-linux-lts.img
options root="LABEL=arch_os" rw

boot/loader/entries/arch-fallback.conf
title   Arch Linux (fallback initramfs)
linux   /vmlinuz-linux
initrd  /intel-ucode.img
initrd  /initramfs-linux-fallback.img
llinux-lts vmlinuz-linux-lts
initrd  /initramfs-linux-lts-fallback.img
options root="LABEL=arch_os" rw

I think I know what happened. I used a script ] to install hyprland. It installs and then later calls another script in my home directory and this script modified /boot. (At least, I think this is what happens.)

I really do not use the Nvidia GPU, so if I remove kernel mode setting from /etc/mkinitcpio.conf and just delete the files

.rwxr-xr-x 103M root 17 May 08:52 initramfs-custom.img
.rwxr-xr-x  67M root 17 May 08:57 initramfs-linux-fallback.img
.rwxr-xr-x 127M root 17 May 10:04 initramfs-linux-lts-fallback.img
.rwxr-xr-x 103M root 17 May 10:03 initramfs-linux-lts.img
.rwxr-xr-x 103M root 17 May 08:57 initramfs-linux.img
.rwxr-xr-x 7.3M root 16 May 13:11 intel-ucode.img
.rwxr-xr-x  13M root 17 May 05:13 vmlinuz-linux
.rwxr-xr-x  11M root 17 May 10:03 vmlinuz-linux-lts

,

would this be suffiicient?

I want to disable Nvidia but there is no option for this in BIOS and I have found the procedures outlined in the forum, up till now at least, too complicated to slog through (sigh.) The problem, of course, is that I mostly can only follow directions without really understanding what is going on underneath. Which is dangerous.

seth · 2023-05-18 18:28:31

I used a script ] to install hyprland

Somebody should be executed for that thing…
hyprland is in the repos.

Post your current /etc/mkinitcpio.conf

Gabachin · 2023-05-18 18:47:04

On my other laptop, I have hyprland-git with waybar. What I like most about it is not the bells and whistles, which are nice, but rather the fact that Wayland is fast and lowers my CPU temperature by 15C. I just thought Sol's rice would be cool to check out.

Here is my conf fle:

cat /etc/mkinitcpio.conf
MODULES=(nvidia nvidia_modeset nvidia_uvm nvidia_drm)
BINARIES=()
FILES=()
HOOKS=(base systemd autodetect keyboard sd-vconsole modconf block filesystems fsck)

Last edited by Gabachin (2023-05-18 18:49:55)

seth · 2023-05-18 18:54:57

This has nothing to do w/ hyprland, just that idiotic script you were running.

nvidia adds about 40MB, but I don't see how that would end you at 103 MB (also the initramfs-linux-fallback.img is "only" 67MB and supposed to be bigger that the non-fallback versions)

sudo -v
diff <(sudo lsinitcpio /boot/initramfs-linux-fallback.img | sort) <(sudo lsinitcpio /boot/initramfs-linux-lts-fallback.img | sort)

You can also extract the initramfs (lsinitcpio -x) and see what consumes the most size.

Gabachin · 2023-05-18 19:48:28

Running

 sudo -v

followed by

diff <(sudo lsinitcpio /boot/initramfs-linux-fallback.img | sort) <(sudo lsinitcpio /boot/initramfs-linux-lts-fallback.img | sort) | less

, I am getting a lot of output that I do not know how to interpret

12a13 
> etc/modprobe.d/supergfxd.conf
330a332
> usr/lib/modprobe.d/supergfxd-nvidiafb.conf
333,1074c335,1265
< usr/lib/modules/6.3.2-arch1-1/
< usr/lib/modules/6.3.2-arch1-1/kernel/
< usr/lib/modules/6.3.2-arch1-1/kernel/arch/
< usr/lib/modules/6.3.2-arch1-1/kernel/arch/x86/
< usr/lib/modules/6.3.2-arch1-1/kernel/arch/x86/crypto/
< usr/lib/modules/6.3.2-arch1-1/kernel/arch/x86/crypto/aesni-intel.ko
< usr/lib/modules/6.3.2-arch1-1/kernel/arch/x86/crypto/crc32c-intel.ko
< usr/lib/modules/6.3.2-arch1-1/kernel/arch/x86/crypto/crc32-pclmul.ko
< usr/lib/modules/6.3.2-arch1-1/kernel/arch/x86/crypto/sha256-ssse3.ko
< usr/lib/modules/6.3.2-arch1-1/kernel/arch/x86/crypto/sha512-ssse3.ko
< usr/lib/modules/6.3.2-arch1-1/kernel/crypto/
< usr/lib/modules/6.3.2-arch1-1/kernel/crypto/blake2b_generic.ko
< usr/lib/modules/6.3.2-arch1-1/kernel/crypto/ccm.ko
< usr/lib/modules/6.3.2-arch1-1/kernel/crypto/cmac.ko
< usr/lib/modules/6.3.2-arch1-1/kernel/crypto/crc32c_generic.ko
< usr/lib/modules/6.3.2-arch1-1/kernel/crypto/crc32_generic.ko
< usr/lib/modules/6.3.2-arch1-1/kernel/crypto/cryptd.ko
< usr/lib/modules/6.3.2-arch1-1/kernel/crypto/crypto_engine.ko
< usr/lib/modules/6.3.2-arch1-1/kernel/crypto/crypto_simd.ko

Is diff comparing the files of the two img files and listing them in descending order?

When I run

lsinitcpio -x

I get a lot of output that, again, I do not know how to read. for example:

❯lsinitcpio -x initramfs-linux.img
.......

usr/local/: Can't create 'usr/local'
usr/local/bin/: Can't create 'usr/local/bin'
usr/local/lib/: Can't create 'usr/local/lib'
usr/local/sbin/: Can't create 'usr/local/sbin'
usr/sbin: Can't create 'usr/sbin'
usr/share/: Can't create 'usr/share'
usr/share/kbd/: Can't create 'usr/share/kbd'
usr/share/kbd/keymaps/: Can't create 'usr/share/kbd/keymaps'
usr/share/kbd/keymaps/i386/: Can't create 'usr/share/kbd/keymaps/i386'
usr/share/kbd/keymaps/i386/include/: Can't create 'usr/share/kbd/keymaps/i386/include'
usr/share/kbd/keymaps/i386/include/compose.inc: Can't create 'usr/share/kbd/keymaps/i386/include/compose.inc'
usr/share/kbd/keymaps/i386/include/linux-keys-bare.inc: Can't create 'usr/share/kbd/keymaps/i386/include/linux-keys-bare.inc'
usr/share/kbd/keymaps/i386/include/linux-with-alt-and-altgr.inc: Can't create 'usr/share/kbd/keymaps/i386/include/linux-with-alt-and-altgr.inc'
usr/share/kbd/keymaps/i386/include/qwerty-layout.inc: Can't create 'usr/share/kbd/keymaps/i386/include/qwerty-layout.inc'
usr/share/kbd/keymaps/i386/qwerty/: Can't create 'usr/share/kbd/keymaps/i386/qwerty'
usr/share/kbd/keymaps/i386/qwerty/la-latin1.map: Can't create 'usr/share/kbd/keymaps/i386/qwerty/la-latin1.map'
var/: Can't create 'var'
var/run: Can't create 'var/run'
VERSION: Can't create 'VERSION'

I do not understand why the "extra files"

.rwxr-xr-x 103M root 17 May 08:52 initramfs-custom.img
.rwxr-xr-x  67M root 17 May 08:57 initramfs-linux-fallback.img
.rwxr-xr-x 127M root 17 May 10:04 initramfs-linux-lts-fallback.img
.rwxr-xr-x 103M root 17 May 10:03 initramfs-linux-lts.img
.rwxr-xr-x 103M root 17 May 08:57 initramfs-linux.img
.rwxr-xr-x 7.3M root 16 May 13:11 intel-ucode.img
.rwxr-xr-x  13M root 17 May 05:13 vmlinuz-linux
.rwxr-xr-x  11M root 17 May 10:03 vmlinuz-linux-lts

are there in the first place. When I install system-boot, I do it as outlined above and there are only the EFI and loader directories in boot.

What can I do to reduce the size of boot after all? Second stupid noob question: doesn't there exist a bootloader that can load the kernel from the root partition? Why is a separate partiton even necessary?

Last edited by Gabachin (2023-05-18 20:05:11)

seth · 2023-05-18 20:18:51

The only thing extra there is "initramfs-custom.img" and that's created by that nonsense script you ran.

usr/local/: Can't create 'usr/local'

Is because you're trying to run that in a path where you have no execution rights.

mkdir /tmp/initramfs
cd /tmp/initramfs
lsinitramfs -x /boot/initramfs-linux.img
du -hs *

What can I do to reduce the size of boot after all?

We're gonna figure why it's so blown up.

Why is a separate partiton even necessary?

Because it's not.

Gabachin · 2023-05-18 20:31:55

That makes sense. I thought there was an issue with permissions.

I am getting this when I run lsintramfs:

▒▓   /tmp/initramfs  
❯ lsinitramfs -x /boot/initramfs-linux.img
bash: lsinitramfs: command not found

seth · 2023-05-18 20:32:58

"lsinitcpio" … it's getting late

Gabachin · 2023-05-18 20:40:51

Ok, now it runs but gives no output.

▓   /tmp/initramfs  
❯ lsinitcpio -x /boot/initramfs-linux.img

░▒▓   /tmp/initramfs  
❯

seth · 2023-05-18 20:41:39

seth also wrote:

du -hs *

Gabachin · 2023-05-18 22:21:18

Here is the output:

❯ du -hs *
0	bin
4.0K	buildconfig
4.0K	config
0	dev
32K	etc
92K	init
16K	init_functions
0	lib
0	lib64
0	new_root
0	proc
0	run
0	sbin
0	sys
0	tmp
181M	usr
0	var
4.0K	VERSION

cd into usr:

░▒▓   /tmp/initramfs  
❯  cd usr/

░▒▓   /tmp/initramfs/usr  
❯ ls
drwxr-xr-x - paul 18 May 14:41 bin
drwxr-xr-x - paul 18 May 14:41 lib
lrwxrwxrwx 3 paul 18 May 14:41 lib64 -> lib
drwxr-xr-x - paul 18 May 14:38 local
lrwxrwxrwx 3 paul 18 May 14:41 sbin -> bin
drwxr-xr-x - paul 18 May 14:38 share

I find that

░▒▓   /tmp/initramfs/usr  
❯ du -hs *
2.7M	bin
269M	lib
0	lib64
0	local
0	sbin
24K	share

so lib seems to be the culprit. Repeating the procedure, find that /modules is 189M. Finally, find that in /modules:

91M	6.1.29-1-lts
98M	6.3.2-arch1-1

Last edited by Gabachin (2023-05-18 23:12:15)

seth · 2023-05-19 06:23:57

So you have the modules for both kernels in all initramfs…
Delete the initramfs-custom.img and run "sudo mkinitcpio -P" and check the new sizes.

Gabachin · 2023-05-19 13:13:08

But isn't this what happens when pacman updates systemd-boot? And I get the same problem, even after deleting the custom.img:

 zstd: error 70 : Write error : cannot write block : No space left on device

Can I just manually delete the modules that are duplicates? Or delete the fallbacks and run mkinitcpio again?

Last edited by Gabachin (2023-05-19 13:17:44)

seth · 2023-05-19 13:20:18

The idea was to empty some space and then hopefully rebuild less and smaller images.

You can also remove the custom image and then just "sudo mkinitcpio -p linux"

Also let's see

ls /etc/mkinitcpio.d
cat /etc/mkinitcpio.d/linux.preset

Edit

Can I just manually delete the modules that are duplicates? Or delete the fallbacks and run mkinitcpio again?

And what are those "duplicates"? mkinitcpio will rebuild teh fallbacks unless you change the preset - and that's ok.
What is not ok is most likely that "custom" image and most certainly not to have modules for all kernel versions in every single image.

Last edited by seth (2023-05-19 13:22:06)

Gabachin · 2023-05-19 13:29:02

Yes, I understand. OK, here is the output:

❯ ls /etc/mkinitcpio.d
.rw-r--r-- 584 root 17 May 10:03 linux-lts.preset
.rw-r--r-- 560 root 17 May 05:13 linux.preset

░▒▓   /boot  
❯ cat /etc/mkinitcpio.d/linux.preset
# mkinitcpio preset file for the 'linux' package

ALL_config="/etc/mkinitcpio.conf"
ALL_kver="/boot/vmlinuz-linux"
ALL_microcode=(/boot/*-ucode.img)

PRESETS=('default' 'fallback')

#default_config="/etc/mkinitcpio.conf"
default_image="/boot/initramfs-linux.img"
#default_uki="/efi/EFI/Linux/arch-linux.efi"
#default_options="--splash /usr/share/systemd/bootctl/splash-arch.bmp"

#fallback_config="/etc/mkinitcpio.conf"
fallback_image="/boot/initramfs-linux-fallback.img"
#fallback_uki="/efi/EFI/Linux/arch-linux-fallback.efi"
fallback_options="-S autodetect"

When I run sudo mkinitcpio -p linux, I get the same device is full error.

What if I delete fallbacks from the preset section, then delete them in /boot and run mkinitcpio again? Then, add fallbacks back and repeat-

Last edited by Gabachin (2023-05-19 13:31:08)

seth · 2023-05-19 13:41:01

Did you remove the initramfs-custom.img first?
You can delete every /boot/*.img and then run "mkinitcpio -P" (w/ or w/o the fallback preset in /etc/mkinitcpio.d/*.preset)
The interesting part is whether you again end up w/ modules for both kernels in all initramfs.

Gabachin · 2023-05-19 13:52:56

Yes I did delete the custom.img first. I then did rm /boot/*.img with sudo and ran mkinitcipio and got the same error.

> Creating zstd-compressed initcpio image: '/boot/initramfs-linux-fallback.img'
zstd: error 70 : Write error : cannot write block : No space left on device

My /boot directory is now

drwxr-xr-x    - root 17 May 05:15 EFI
drwxr-xr-x    - root 19 May 07:08 loader
.rwxr-xr-x  74M root 19 May 07:48 initramfs-linux-fallback.img
.rwxr-xr-x 103M root 19 May 07:04 initramfs-linux-fallback.img.tmp
.rwxr-xr-x 127M root 19 May 07:48 initramfs-linux-lts-fallback.img
.rwxr-xr-x 103M root 19 May 07:48 initramfs-linux-lts.img
.rwxr-xr-x 103M root 19 May 07:48 initramfs-linux.img
.rwxr-xr-x  13M root 17 May 05:13 vmlinuz-linux
.rwxr-xr-x  11M root 17 May 10:03 vmlinuz-linux-lts

░▒▓   /boot  
❯ du -hs *
200K	EFI
71M	initramfs-linux-fallback.img
99M	initramfs-linux-fallback.img.tmp
122M	initramfs-linux-lts-fallback.img
98M	initramfs-linux-lts.img
99M	initramfs-linux.img
36K	loader
12M	vmlinuz-linux
11M	vmlinuz-linux-lts

So I guess mkinitcipio is installing both kernels in all the intramfs.

Last edited by Gabachin (2023-05-19 13:54:41)

seth · 2023-05-19 13:56:05

initramfs-linux-fallback.img is probably only 71mb because this is where you're running out of space.

ls -l /lib/modules
rm /boot/*.img
mkinitcpio -v -p linux

Gabachin · 2023-05-19 14:10:44

OK; here is what I am getting:

❯ ls -l /lib/modules
drwxr-xr-x - root 17 May 10:03 6.1.29-1-lts
drwxr-xr-x - root 17 May 08:14 6.3.2-arch1-1

after rm /boot/*.img:

❯ ls
drwxr-xr-x    - root 17 May 05:15 EFI
drwxr-xr-x    - root 19 May 07:08 loader
.rwxr-xr-x 103M root 19 May 07:04 initramfs-linux-fallback.img.tmp
.rwxr-xr-x  13M root 17 May 05:13 vmlinuz-linux
.rwxr-xr-x  11M root 17 May 10:03 vmlinuz-linux-lts

 mkinitcpio -v -p linux
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
  -> -v -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img --microcode /boot/*-ucode.img
==> ERROR: Unable to write to '/boot/initramfs-linux.img'
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'fallback'
  -> -v -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-fallback.img -S autodetect --microcode /boot/*-ucode.img
==> ERROR: Unable to write to '/boot/initramfs-linux-fallback.img'

du -hs *
200K	EFI
99M	initramfs-linux-fallback.img.tmp
36K	loader
12M	vmlinuz-linux
11M	vmlinuz-linux-lts

Error writing the intramfs. Why is that?

Last edited by Gabachin (2023-05-19 14:11:46)

seth · 2023-05-19 14:23:36

Permission error? You didn't run that as root?

Arch Linux

#1 2023-05-18 05:07:32

/boot is full (Solved)

#2 2023-05-18 06:39:50

Re: /boot is full (Solved)

#3 2023-05-18 12:01:07

Re: /boot is full (Solved)

#4 2023-05-18 12:44:17

Re: /boot is full (Solved)

#5 2023-05-18 12:47:27

Re: /boot is full (Solved)

#6 2023-05-18 17:13:24

Re: /boot is full (Solved)

#7 2023-05-18 18:28:31

Re: /boot is full (Solved)

#8 2023-05-18 18:47:04

Re: /boot is full (Solved)

#9 2023-05-18 18:54:57

Re: /boot is full (Solved)

#10 2023-05-18 19:48:28

Re: /boot is full (Solved)

#11 2023-05-18 20:18:51

Re: /boot is full (Solved)

#12 2023-05-18 20:31:55

Re: /boot is full (Solved)

#13 2023-05-18 20:32:58

Re: /boot is full (Solved)

#14 2023-05-18 20:40:51

Re: /boot is full (Solved)

#15 2023-05-18 20:41:39

Re: /boot is full (Solved)

#16 2023-05-18 22:21:18

Re: /boot is full (Solved)

#17 2023-05-19 06:23:57

Re: /boot is full (Solved)

#18 2023-05-19 13:13:08

Re: /boot is full (Solved)

#19 2023-05-19 13:20:18

Re: /boot is full (Solved)

#20 2023-05-19 13:29:02

Re: /boot is full (Solved)

#21 2023-05-19 13:41:01

Re: /boot is full (Solved)

#22 2023-05-19 13:52:56

Re: /boot is full (Solved)

#23 2023-05-19 13:56:05

Re: /boot is full (Solved)

#24 2023-05-19 14:10:44

Re: /boot is full (Solved)

#25 2023-05-19 14:23:36

Re: /boot is full (Solved)

Board footer