Kernel and several other packages corrupted, system won't boot

cmecau · 2021-05-30 01:12:39

I've been having a recurring problem for a while that I've mostly ignored where when a large amount of disk activity is happening at once, the filesystem turns read only and I have to reboot. It had been quite a while since this last occurred, and I figured I could solve it later and just not do anything too disk intensive for a while. As of last night, it had been nearly a month since I last ran updates, so I thought I thought it was about time. Unfortunately, the problem with the filesystem occurred again in the middle of updates. I now cannot boot. When I try, I get a message saying that vmlinuz-linux is not found. I'll note that my system is on an ssd, and the main partition (both root and home) is encrypted with luks. Also important is that I use btrfs.

I've managed to chroot in to try and figure out how to fix this, and I have backed up my home directory to an external hard drive. Trying to complete the update results in triggering the filesystem to turn read only again. dmesg is giving me a bunch of btrfs warnings and errors. As an example:

[ 7619.547656] BTRFS warning (device dm-0): csum failed root 5 ino 23736501 off 3057704960 csum 0x8d68be8f expected csum 0xcdfee1c0 mirror 1
[ 7619.547676] BTRFS error (device dm-0): bdev /dev/mapper/cryptvolume errs: wr 0, rd 0, flush 0, corrupt 111, gen 0

It repeats like that with different numbers many times. I'd post the rest, but it's a lot and I don't have a good way to copy it.

Also, whenever I try to run pacman, I get two errors that seem troubling:

error: could not open file /var/lib/pacman/local/plasma-desktop-5.21.5-1/desc: No such file or directory
error: could not open file /var/lib/pacman/local/texlive-core-2021.58710-2/desc: No such file or directory

Trying to update or reinstall just linux results in a bunch of messages saying that files were empty and not checked.

Is this fixable at all, or do I need to wipe the drive and start over? If the former, how?

seth · 2021-05-30 08:10:52

It's fixable, https://wiki.archlinux.org/title/Pacman … l_database
But you need to figure why you keep getting FS errors - notably whether the disk is falling apart or whether this is maybe a luks or btrfs problem - there's little point in just fixing this over and over again and running into the same corrupt data situation every other day thereafter.

For SSDs the trimming strategy might be relevant, do you mount discard or noatime?
https://wiki.archlinux.org/title/Dm-cry … ives_(SSD)

If there's no indication in the logs of the underlying problem and you don't mind wiping the drive, I'd stress it w/ dd a bit (no FS) then create an unencrypted btrfs and bang that and then try an encrypted ext4 partition.
If nothing of this breaks under load, it may simply be the combination (though btrfs+luks is promised to be reliable since linux 3.2)

Arch Linux

#1 2021-05-30 01:12:39

Kernel and several other packages corrupted, system won't boot

#2 2021-05-30 08:10:52

Re: Kernel and several other packages corrupted, system won't boot

Board footer