[SOLVED] Unable to boot into Arch

ochinko · 2022-08-25 10:52:12

I have problems booting into Arch and every failed attempt at troubleshooting has made them worse.

Problem first started after I rebooted the laptop that I left running yay -Syu overnight and deleted some files from the EFI partition, to make space for a Pop! OS and Windows install to multi-boot to. Off memory, while loading to Arch, the laptop screen flashed two lines of

"failed to load kernel modules"

, and my startx command failed erroring about being unable to find my external monitor display.

I deleted and tried remaking the EFI partition after accidentally running rm -rf on /boot/efi/EFI/bootloaders/. I received a line of

error mkdir /var/lockdmraid

while running grub-mkconfig -o /boot/grub/grub.cfg, trying to reinstall Grub.

Since I am on BTRFS with Snapper's automatic timeline snapshots and snapshots after every package change, I rolled back to a previous snapshot by following the steps on the Arch Wiki Snapper page. I removed the subvolid mount flag on the @ sub volume, leaving it with a subvol flag.

Now when i try booting from Grub, I get

452 out of range 0x6e936e60

I still have a copy of the latest @ subvolume and the contents of EFI partition without the bootloaders folder, before I re-formatted it.

After mounting and chrooting on Arch install iso:

# lsblk -f

nvme0n1

├─nvme0n1p1
│                              510.7M     0% /boot/efi
├─nvme0n1p2
│                              334.8G    16% /.snapshots
│                                            /opt
│                                            /var
│                                            /home
│                                            /tmp
│                                            /
├─nvme0n1p3
│
├─nvme0n1p4
│
├─nvme0n1p5
│
├─nvme0n1p6
│
├─nvme0n1p7
│
├─nvme0n1p8
│
└─nvme0n1p9

# cat /etc/fstab
# Static information about the filesystems.
# See fstab(5) for details.

# <file system> <dir> <type> <options> <dump> <pass>
# /dev/nvme0n1p2
UUID=1c762d92-defc-45b9-ae02-87ca3527ddbc       /               btrfs           rw,noatime,compress=zstd:3,ssd,discard=async,space_cache=v2,commit=120,subvol=/@       0 0

# /dev/nvme0n1p1
UUID=D7CA-BBD6          /boot/efi       vfat           rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro    0 2

# /dev/nvme0n1p2
UUID=1c762d92-defc-45b9-ae02-87ca3527ddbc       /opt            btrfs           rw,noatime,compress=zstd:3,ssd,discard=async,space_cache=v2,commit=120,subvolid=259,subvol=/@opt        0 0

# /dev/nvme0n1p2
UUID=1c762d92-defc-45b9-ae02-87ca3527ddbc       /tmp            btrfs           rw,noatime,compress=zstd:3,ssd,discard=async,space_cache=v2,commit=120,subvolid=260,subvol=/@tmp        0 0

# /dev/nvme0n1p2
UUID=1c762d92-defc-45b9-ae02-87ca3527ddbc       /home           btrfs           rw,noatime,compress=zstd:3,ssd,discard=async,space_cache=v2,commit=120,subvolid=257,subvol=/@home       0 0

# /dev/nvme0n1p2
UUID=1c762d92-defc-45b9-ae02-87ca3527ddbc       /.snapshots     btrfs           rw,noatime,compress=zstd:3,ssd,discard=async,space_cache=v2,commit=120,subvolid=261,subvol=/@.snapshots 0 0

# /dev/nvme0n1p2
UUID=1c762d92-defc-45b9-ae02-87ca3527ddbc       /var            btrfs           rw,noatime,compress=zstd:3,ssd,discard=async,space_cache=v2,commit=120,subvolid=258,subvol=/@var        0 0
2

Last edited by ochinko (2022-09-22 12:57:29)

TKO__GLOBAL__ · 2022-08-26 21:29:45

I would uninstall then reinstall grub and run grub-install per the wiki and then grub-mkconfig (not clear if you did this). I would then regenerate your interamfs to be on the safe side with mkinitpcio -P.
If you continue to have trouble with /var/lock/dmraid, is there anything in this folder? In my system this is a symlink to /var/run/lock/dmraid. Maybe try moving both symlink and folder somewhere else as root and trying grub-mkconfig again.

ochinko · 2022-08-27 07:48:45

TKO__GLOBAL__ wrote:

I would uninstall then reinstall grub and run grub-install per the wiki and then grub-mkconfig (not clear if you did this). I would then regenerate your interamfs to be on the safe side with mkinitpcio -P. If you continue to have trouble with /var/lock/dmraid, is there anything in this folder? In my system this is a symlink to /var/run/lock/dmraid. Maybe try moving both symlink and folder somewhere else as root and trying grub-mkconfig again.

As I was typing up the grub error about my esp after following your steps, I realised my silly mistake of not updating the esp uuid to the new one.

Now when I boot into Arch, the screen flashes two lines of "[ERROR] Failed to start Load Kernel Modules." and I am unable to startx.

Xorg.0.log
[   736.361] 
X.Org X Server 1.21.1.4
X Protocol Version 11, Revision 0
[   736.365] Current Operating System: Linux Laptop 5.19.3-zen1-1-zen #1 ZEN SMP PREEMPT_DYNAMIC Sun, 21 Aug 2022 18:55:27 +0000 x86_64
[   736.365] Kernel command line: BOOT_IMAGE=/@/boot/vmlinuz-linux-zen root=UUID=1c762d92-defc-45b9-ae02-87ca3527ddbc rw rootflags=subvol=@ loglevel=3 quiet
[   736.368]  
[   736.369] Current version of pixman: 0.40.0
[   736.371] 	Before reporting problems, check http://wiki.x.org
	to make sure that you have the latest version.
[   736.371] Markers: (--) probed, (**) from config file, (==) default setting,
	(++) from command line, (!!) notice, (II) informational,
	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
[   736.376] (==) Log file: "/home/user/.local/share/xorg/Xorg.0.log", Time: Sat Aug 27 15:12:40 2022
[   736.378] (==) Using config file: "/etc/X11/xorg.conf"
[   736.380] (==) Using config directory: "/etc/X11/xorg.conf.d"
[   736.381] (==) Using system config directory "/usr/share/X11/xorg.conf.d"
[   736.382] (==) ServerLayout "layout"
[   736.382] (**) |-->Screen "nvidia" (0)
[   736.382] (**) |   |-->Monitor "<default monitor>"
[   736.382] (**) |   |-->Device "nvidia"
[   736.382] (**) |   |-->GPUDevice "intel"
[   736.382] (==) No monitor specified for screen "nvidia".
	Using a default monitor configuration.
[   736.382] (**) |-->Inactive Device "intel"
[   736.382] (==) Automatically adding devices
[   736.382] (==) Automatically enabling devices
[   736.382] (==) Automatically adding GPU devices
[   736.382] (==) Automatically binding GPU devices
[   736.382] (==) Max clients allowed: 256, resource mask: 0x1fffff
[   736.382] (WW) The directory "/usr/share/fonts/misc" does not exist.
[   736.382] 	Entry deleted from font path.
[   736.383] (WW) The directory "/usr/share/fonts/TTF" does not exist.
[   736.383] 	Entry deleted from font path.
[   736.383] (WW) The directory "/usr/share/fonts/OTF" does not exist.
[   736.383] 	Entry deleted from font path.
[   736.383] (WW) The directory "/usr/share/fonts/Type1" does not exist.
[   736.383] 	Entry deleted from font path.
[   736.384] (==) FontPath set to:
	/usr/share/fonts/100dpi,
	/usr/share/fonts/75dpi
[   736.384] (==) ModulePath set to "/usr/lib/xorg/modules"
[   736.384] (II) The server relies on udev to provide the list of input devices.
	If no devices become available, reconfigure udev or disable AutoAddDevices.
[   736.384] (II) Module ABI versions:
[   736.384] 	X.Org ANSI C Emulation: 0.4
[   736.384] 	X.Org Video Driver: 25.2
[   736.384] 	X.Org XInput driver : 24.4
[   736.384] 	X.Org Server Extension : 10.0
[   736.385] (++) using VT number 1

[   736.387] (II) systemd-logind: took control of session /org/freedesktop/login1/session/_31
[   736.388] (II) xfree86: Adding drm device (/dev/dri/card0)
[   736.388] (II) Platform probe for /sys/devices/pci0000:00/0000:00:02.0/drm/card0
[   736.389] (II) systemd-logind: got fd for /dev/dri/card0 226:0 fd 13 paused 0
[   736.393] (--) PCI:*(0@0:2:0) 8086:591b:103c:836b rev 4, Mem @ 0xb2000000/16777216, 0xc0000000/268435456, I/O @ 0x00005000/64, BIOS @ 0x????????/131072
[   736.393] (--) PCI: (1@0:0:0) 10de:1c8d:103c:836b rev 161, Mem @ 0xb3000000/16777216, 0xa0000000/268435456, 0xb0000000/33554432, I/O @ 0x00004000/128, BIOS @ 0x????????/524288
[   736.394] (WW) Open ACPI failed (/var/run/acpid.socket) (No such file or directory)
[   736.394] (II) LoadModule: "glx"
[   736.394] (II) Loading /usr/lib/xorg/modules/extensions/libglx.so
[   736.401] (II) Module glx: vendor="X.Org Foundation"
[   736.401] 	compiled for 1.21.1.4, module version = 1.0.0
[   736.401] 	ABI class: X.Org Server Extension, version 10.0
[   736.401] (II) LoadModule: "nvidia"
[   736.401] (II) Loading /usr/lib/xorg/modules/drivers/nvidia_drv.so
[   736.410] (II) Module nvidia: vendor="NVIDIA Corporation"
[   736.410] 	compiled for 1.6.99.901, module version = 1.0.0
[   736.410] 	Module class: X.Org Video Driver
[   736.410] (II) LoadModule: "modesetting"
[   736.410] (II) Loading /usr/lib/xorg/modules/drivers/modesetting_drv.so
[   736.414] (II) Module modesetting: vendor="X.Org Foundation"
[   736.414] 	compiled for 1.21.1.4, module version = 1.21.1
[   736.414] 	Module class: X.Org Video Driver
[   736.414] 	ABI class: X.Org Video Driver, version 25.2
[   736.414] (II) NVIDIA dlloader X Driver  515.65.01  Wed Jul 20 13:50:46 UTC 2022
[   736.414] (II) NVIDIA Unified Driver for all Supported NVIDIA GPUs
[   736.414] (II) modesetting: Driver for Modesetting Kernel Drivers: kms
[   736.414] (II) Loading sub module "fb"
[   736.414] (II) LoadModule: "fb"
[   736.414] (II) Module "fb" already built-in
[   736.414] (II) Loading sub module "wfb"
[   736.414] (II) LoadModule: "wfb"
[   736.414] (II) Loading /usr/lib/xorg/modules/libwfb.so
[   736.415] (II) Module wfb: vendor="X.Org Foundation"
[   736.415] 	compiled for 1.21.1.4, module version = 1.0.0
[   736.415] 	ABI class: X.Org ANSI C Emulation, version 0.4
[   736.415] (II) Loading sub module "ramdac"
[   736.415] (II) LoadModule: "ramdac"
[   736.415] (II) Module "ramdac" already built-in
[   737.035] (EE) NVIDIA: Failed to initialize the NVIDIA kernel module. Please see the
[   737.035] (EE) NVIDIA:     system's kernel log for additional error messages and
[   737.035] (EE) NVIDIA:     consult the NVIDIA README for details.
[   737.035] (II) modeset(G0): using drv /dev/dri/card0
[   737.658] (EE) NVIDIA: Failed to initialize the NVIDIA kernel module. Please see the
[   737.658] (EE) NVIDIA:     system's kernel log for additional error messages and
[   737.658] (EE) NVIDIA:     consult the NVIDIA README for details.
[   737.658] (WW) Falling back to old probe method for modesetting
[   737.658] (EE) No devices detected.
[   737.658] (EE) 
Fatal server error:
[   737.658] (EE) no screens found(EE) 
[   737.658] (EE) 
Please consult the The X.Org Foundation support 
	 at http://wiki.x.org
 for help. 
[   737.658] (EE) Please also check the log file at "/home/user/.local/share/xorg/Xorg.0.log" for additional information.
[   737.658] (EE) 
[   737.674] (EE) Server terminated with error (1). Closing log file.

There also wasn't dmraid in /var/lock when I checked.

TKO__GLOBAL__ · 2022-08-27 09:57:58

ochinko wrote:

As I was typing up the grub error about my esp after following your steps, I realised my silly mistake of not updating the esp uuid to the new one.

Good you could solve that.

ochinko wrote:

Now when I boot into Arch, the screen flashes two lines of "[ERROR] Failed to start Load Kernel Modules." and I am unable to startx.

Your log says right there that its the nvidia driver and there are additional logs to look at But reinstalling the driver would be my first step.

seth · 2022-08-27 12:35:08

Linux Laptop 5.19.3-zen1-1-zen

Especially the proper one, that would in this case be some -dkms version, eg. https://archlinux.org/packages/extra/x8 … idia-dkms/

Edit: dude.

deleted some files from the EFI partition
"failed to load kernel modules"
I deleted and tried remaking the EFI partition after accidentally running rm -rf on /boot/efi/EFI/bootloaders/
trying to reinstall Grub.
rolled back to a previous snapshot
removed the subvolid mount flag on the @ sub volume

tl;dr "fucked up my system", please post a complete system journal so we can see what's actually going on there.

Last edited by seth (2022-08-27 12:38:50)

ochinko · 2022-08-27 15:06:38

seth wrote:

tl;dr "fucked up my system", please post a complete system journal so we can see what's actually going on there.

How do I go about doing that? I read the Arch wiki on systemd/Journal but I’m slightly confused. How would I upload whats on journal file? Could I get some steps I could follow?

ochinko · 2022-08-27 15:13:46

seth wrote:

tl;dr "fucked up my system", please post a complete system journal so we can see what's actually going on there.

How do I go about doing that? I read the Arch wiki on systemd/Journal but I’m slightly confused. How would I upload whats on journal file? Could I get some steps I could follow?

seth wrote:

Linux Laptop 5.19.3-zen1-1-zen
Especially the proper one, that would in this case be some -dkms version, eg. https://archlinux.org/packages/extra/x8 … idia-dkms/

I am already on the latest nvidia-dkms version, 515.65.01-2

seth · 2022-08-27 15:15:04

There's no "journal file", you view the journal w/ journalctl, "sudo" unless your is some admin group (wiki) and "-b" for eg. the current boot.
If you cannot upload the journal from the booting system, you can access it form a live distro ("Specify a different journal to view") and to upload shell outputs and files from the commanline, see the 1st link below.

ochinko · 2022-08-27 16:24:04

Here it is. http://codepad.org/IX3INnvh

seth · 2022-08-27 18:49:57

That's only 738 lines and ends midway through initializing sda, you're either posting a partial output or the journal is truncated because you reboot w/ the power button.
In the latter case: don't. Try to reboot by frenetically pressing ctrl+alt+del and if that doesn't work either, https://wiki.archlinux.org/title/Keyboa … el_(SysRq)

ochinko · 2022-08-28 01:15:20

seth wrote:

That's only 738 lines and ends midway through initializing sda, you're either posting a partial output or the journal is truncated because you reboot w/ the power button.
In the latter case: don't. Try to reboot by frenetically pressing ctrl+alt+del and if that doesn't work either, https://wiki.archlinux.org/title/Keyboa … el_(SysRq)

Yes i reboot with the power button.
So what i have to do is

Boot up and startx
After startx error, reboot with ctrl+alt+del
Boot up and startx again
After error, sudo journalctl -b > journal.txt

seth · 2022-08-28 04:38:25

Boot up and startx
After error, sudo journalctl -b > journal.txt

will do.
Or if you cannot interact w/ the system after the error happens

Boot up and startx
After startx error, reboot with ctrl+alt+del
Boot up 
sudo journalctl -b -1 > journal.txt

ochinko · 2022-08-28 11:25:44

seth wrote:

Boot up and startx
After startx error, reboot with ctrl+alt+del
Boot up 
sudo journalctl -b -1 > journal.txt

Got it, thanks for clarifying. http://ix.io/48V1

I realised it was only 738 lines previously because of the pastebin server's limit.

seth · 2022-08-28 13:07:48

Aug 27 23:46:35 Laptop systemd-modules-load[278]: Failed to insert module 'nvidia_uvm': Invalid argument
                                                                                                        ===============
Aug 27 23:46:35 Laptop systemd[1]: systemd-modules-load.service: Main process exited, code=exited, status=1/FAILURE
Aug 27 23:46:35 Laptop systemd[1]: systemd-modules-load.service: Failed with result 'exit-code'.
Aug 27 23:46:35 Laptop systemd[1]: Failed to start Load Kernel Modules.

But the real problem is

Aug 27 23:46:35 Laptop kernel: failed to validate module [nvidia] BTF: -22

Which is down to a bug in the pahole package, make sure you've some 1.23 version and rebuild all dkms packages.

ochinko · 2022-08-28 23:18:02

seth wrote:

Which is down to a bug in the pahole package, make sure you've some 1.23 version and rebuild all dkms packages.

Rebuilding the dkms packages after checking my pahole is ver. 1.23 did the trick.
Thanks for your time and help seth and TKO__GLOBAL__.

seth · 2022-08-29 06:12:53

Great.

Please always remember to mark resolved threads by editing your initial posts subject - so others will know that there's no task left, but maybe a solution to find.
Thanks.

Arch Linux

#1 2022-08-25 10:52:12

[SOLVED] Unable to boot into Arch

#2 2022-08-26 21:29:45

Re: [SOLVED] Unable to boot into Arch

#3 2022-08-27 07:48:45

Re: [SOLVED] Unable to boot into Arch

#4 2022-08-27 09:57:58

Re: [SOLVED] Unable to boot into Arch

#5 2022-08-27 12:35:08

Re: [SOLVED] Unable to boot into Arch

#6 2022-08-27 15:06:38

Re: [SOLVED] Unable to boot into Arch

#7 2022-08-27 15:13:46

Re: [SOLVED] Unable to boot into Arch

#8 2022-08-27 15:15:04

Re: [SOLVED] Unable to boot into Arch

#9 2022-08-27 16:24:04

Re: [SOLVED] Unable to boot into Arch

#10 2022-08-27 18:49:57

Re: [SOLVED] Unable to boot into Arch

#11 2022-08-28 01:15:20

Re: [SOLVED] Unable to boot into Arch

#12 2022-08-28 04:38:25

Re: [SOLVED] Unable to boot into Arch

#13 2022-08-28 11:25:44

Re: [SOLVED] Unable to boot into Arch

#14 2022-08-28 13:07:48

Re: [SOLVED] Unable to boot into Arch

#15 2022-08-28 23:18:02

Re: [SOLVED] Unable to boot into Arch

#16 2022-08-29 06:12:53

Re: [SOLVED] Unable to boot into Arch

Board footer