You are not logged in.

Pages: 1

#1 2023-07-02 09:37:41

stefan
Member
Registered: 2013-03-22
Posts: 131

[solved] Upgrade might break X (linux-lts, nvidia-lts) and nftables

Update after solution was found: The upgrade was not executed correctly, because /boot was not mounted correctly, due to an earlier mistake that went unnoticed

Hello,

after running `pacman -Syu` today, I was unable to start X.

This might be related to NVIDIA drivers not working, although I do not see `nouveau` anywhere, and my setup was previously running.

In case you find the dates below inconsistent: I’ve repeated up- and downgrading the system (`pacman -Syu`, `pacman -U …`) to reproduce the issue and gather more logs.

The upgrade was:

[2023-07-02T08:54:27+0200] [PACMAN] Running 'pacman -Syu'
[2023-07-02T08:54:27+0200] [PACMAN] synchronizing package lists
[2023-07-02T08:54:29+0200] [PACMAN] starting full system upgrade
[2023-07-02T09:02:08+0200] [ALPM] running '60-mkinitcpio-remove.hook'...
[2023-07-02T09:02:08+0200] [ALPM] transaction started
[2023-07-02T09:02:08+0200] [ALPM] upgraded amd-ucode (20230625.ee91452d-2 -> 20230625.ee91452d-3)
[2023-07-02T09:02:08+0200] [ALPM] upgraded zlib (1:1.2.13-2 -> 1:1.2.13-3)
[2023-07-02T09:02:08+0200] [ALPM] upgraded kbd (2.6.0-1 -> 2.6.1-1)
[2023-07-02T09:02:08+0200] [ALPM] upgraded libp11-kit (0.24.1-1 -> 0.25.0-1)
[2023-07-02T09:02:09+0200] [ALPM] upgraded p11-kit (0.24.1-1 -> 0.25.0-1)
[2023-07-02T09:02:09+0200] [ALPM] upgraded ca-certificates-mozilla (3.90-1 -> 3.91-1)
[2023-07-02T09:02:09+0200] [ALPM] upgraded at-spi2-core (2.48.3-1 -> 2.48.3-2)
[2023-07-02T09:02:09+0200] [ALPM] upgraded libpaper (2.1.0-1 -> 2.1.1-1)
[2023-07-02T09:02:09+0200] [ALPM] upgraded libpng (1.6.40-1 -> 1.6.40-2)
[2023-07-02T09:02:09+0200] [ALPM] upgraded linux-firmware-whence (20230625.ee91452d-2 -> 20230625.ee91452d-3)
[2023-07-02T09:02:09+0200] [ALPM] upgraded linux-firmware (20230625.ee91452d-2 -> 20230625.ee91452d-3)
[2023-07-02T09:02:09+0200] [ALPM] upgraded linux-lts (6.1.36-1 -> 6.1.37-1)
[2023-07-02T09:02:10+0200] [ALPM] upgraded linux-lts-headers (6.1.36-1 -> 6.1.37-1)
[2023-07-02T09:02:10+0200] [ALPM] upgraded minizip (1:1.2.13-2 -> 1:1.2.13-3)
[2023-07-02T09:02:10+0200] [ALPM] upgraded nss (3.90-1 -> 3.91-1)
[2023-07-02T09:02:10+0200] [ALPM] upgraded nvidia-lts (1:535.54.03-4 -> 1:535.54.03-5)
[2023-07-02T09:02:10+0200] [ALPM] upgraded python-jaraco.functools (3.7.0-1 -> 3.8.0-1)
[2023-07-02T09:02:10+0200] [ALPM] upgraded taglib (1.13-1 -> 1.13.1-1)
[2023-07-02T09:02:10+0200] [ALPM] transaction completed
[2023-07-02T09:02:11+0200] [ALPM] running '30-systemd-update.hook'...
[2023-07-02T09:02:11+0200] [ALPM] running '40-update-ca-trust.hook'...
[2023-07-02T09:02:11+0200] [ALPM] running '60-depmod.hook'...
[2023-07-02T09:02:12+0200] [ALPM] running '90-mkinitcpio-install.hook'...
[2023-07-02T09:02:12+0200] [ALPM-SCRIPTLET] ==> Building image from preset: /etc/mkinitcpio.d/linux-lts.preset: 'default'
[2023-07-02T09:02:12+0200] [ALPM-SCRIPTLET] ==> Using default configuration file: '/etc/mkinitcpio.conf'
[2023-07-02T09:02:12+0200] [ALPM-SCRIPTLET]   -> -k /boot/vmlinuz-linux-lts -g /boot/initramfs-linux-lts.img --microcode /boot/amd-ucode.img
[2023-07-02T09:02:12+0200] [ALPM-SCRIPTLET] ==> Starting build: '6.1.37-1-lts'
[2023-07-02T09:02:12+0200] [ALPM-SCRIPTLET]   -> Running build hook: [base]
[2023-07-02T09:02:12+0200] [ALPM-SCRIPTLET]   -> Running build hook: [udev]
[2023-07-02T09:02:13+0200] [ALPM-SCRIPTLET]   -> Running build hook: [autodetect]
[2023-07-02T09:02:13+0200] [ALPM-SCRIPTLET]   -> Running build hook: [modconf]
[2023-07-02T09:02:13+0200] [ALPM-SCRIPTLET]   -> Running build hook: [kms]
[2023-07-02T09:02:14+0200] [ALPM-SCRIPTLET]   -> Running build hook: [keyboard]
[2023-07-02T09:02:14+0200] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: 'xhci_pci'
[2023-07-02T09:02:14+0200] [ALPM-SCRIPTLET]   -> Running build hook: [keymap]
[2023-07-02T09:02:14+0200] [ALPM-SCRIPTLET]   -> Running build hook: [consolefont]
[2023-07-02T09:02:14+0200] [ALPM-SCRIPTLET] ==> WARNING: consolefont: no font found in configuration
[2023-07-02T09:02:14+0200] [ALPM-SCRIPTLET]   -> Running build hook: [block]
[2023-07-02T09:02:14+0200] [ALPM-SCRIPTLET]   -> Running build hook: [encrypt]
[2023-07-02T09:02:14+0200] [ALPM-SCRIPTLET]   -> Running build hook: [filesystems]
[2023-07-02T09:02:14+0200] [ALPM-SCRIPTLET]   -> Running build hook: [fsck]
[2023-07-02T09:02:15+0200] [ALPM-SCRIPTLET] ==> Generating module dependencies
[2023-07-02T09:02:15+0200] [ALPM-SCRIPTLET] ==> Creating zstd-compressed initcpio image: '/boot/initramfs-linux-lts.img'
[2023-07-02T09:02:15+0200] [ALPM-SCRIPTLET] ==> Image generation successful
[2023-07-02T09:02:15+0200] [ALPM-SCRIPTLET] ==> Building image from preset: /etc/mkinitcpio.d/linux-lts.preset: 'fallback'
[2023-07-02T09:02:15+0200] [ALPM-SCRIPTLET] ==> Using default configuration file: '/etc/mkinitcpio.conf'
[2023-07-02T09:02:15+0200] [ALPM-SCRIPTLET]   -> -k /boot/vmlinuz-linux-lts -g /boot/initramfs-linux-lts-fallback.img -S autodetect --microcode /boot/amd-ucode.img
[2023-07-02T09:02:15+0200] [ALPM-SCRIPTLET] ==> Starting build: '6.1.37-1-lts'
[2023-07-02T09:02:15+0200] [ALPM-SCRIPTLET]   -> Running build hook: [base]
[2023-07-02T09:02:15+0200] [ALPM-SCRIPTLET]   -> Running build hook: [udev]
[2023-07-02T09:02:15+0200] [ALPM-SCRIPTLET]   -> Running build hook: [modconf]
[2023-07-02T09:02:15+0200] [ALPM-SCRIPTLET]   -> Running build hook: [kms]
[2023-07-02T09:02:16+0200] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: 'ast'
[2023-07-02T09:02:17+0200] [ALPM-SCRIPTLET]   -> Running build hook: [keyboard]
[2023-07-02T09:02:17+0200] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: 'xhci_pci'
[2023-07-02T09:02:18+0200] [ALPM-SCRIPTLET]   -> Running build hook: [keymap]
[2023-07-02T09:02:18+0200] [ALPM-SCRIPTLET]   -> Running build hook: [consolefont]
[2023-07-02T09:02:18+0200] [ALPM-SCRIPTLET] ==> WARNING: consolefont: no font found in configuration
[2023-07-02T09:02:18+0200] [ALPM-SCRIPTLET]   -> Running build hook: [block]
[2023-07-02T09:02:18+0200] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: 'aic94xx'
[2023-07-02T09:02:18+0200] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: 'bfa'
[2023-07-02T09:02:18+0200] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: 'qed'
[2023-07-02T09:02:18+0200] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: 'qla1280'
[2023-07-02T09:02:18+0200] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: 'qla2xxx'
[2023-07-02T09:02:18+0200] [ALPM-SCRIPTLET] ==> WARNING: Possibly missing firmware for module: 'wd719x'
[2023-07-02T09:02:19+0200] [ALPM-SCRIPTLET]   -> Running build hook: [encrypt]
[2023-07-02T09:02:19+0200] [ALPM-SCRIPTLET]   -> Running build hook: [filesystems]
[2023-07-02T09:02:19+0200] [ALPM-SCRIPTLET]   -> Running build hook: [fsck]
[2023-07-02T09:02:20+0200] [ALPM-SCRIPTLET] ==> Generating module dependencies
[2023-07-02T09:02:20+0200] [ALPM-SCRIPTLET] ==> Creating zstd-compressed initcpio image: '/boot/initramfs-linux-lts-fallback.img'
[2023-07-02T09:02:20+0200] [ALPM-SCRIPTLET] ==> Image generation successful

This looked pretty standard for me, nothing suspicious. But X then refused to start. It’s log said (I’m back to a working setup, so here’s the log from the failed start but I found an older log from a failed start):

# cat /var/log/Xorg.0.log
[    37.135] 
X.Org X Server 1.21.1.8
X Protocol Version 11, Revision 0
[    37.135] Current Operating System: Linux tauhou 6.1.36-1-lts #1 SMP PREEMPT_DYNAMIC Wed, 28 Jun 2023 19:23:07 +0000 x86_64
[    37.135] Kernel command line: root=/dev/mapper/7daafe2a-6355-4e37-a27e-88cabdbfc2b9 rw cryptdevice=/dev/disk/by-uuid/7daafe2a-6355-4e37-a27e-88cabdbfc2b9:7daafe2a-6355-4e37-a27e-88cabdbfc2b9 initrd=\amd-ucode.img initrd=\initramfs-linux-lts.img
[    37.135]  
[    37.135] Current version of pixman: 0.42.2
[    37.135]    Before reporting problems, check http://wiki.x.org
        to make sure that you have the latest version.
[    37.135] Markers: (--) probed, (**) from config file, (==) default setting,
        (++) from command line, (!!) notice, (II) informational,
        (WW) warning, (EE) error, (NI) not implemented, (??) unknown.
[    37.136] (==) Log file: "/var/log/Xorg.0.log", Time: Sun Jul  2 11:16:28 2023
[    37.136] (==) Using config file: "/etc/X11/xorg.conf"
[    37.136] (==) Using system config directory "/usr/share/X11/xorg.conf.d"
[    37.136] (==) ServerLayout "Layout0"
[    37.136] (**) |-->Screen "Screen0" (0)
[    37.136] (**) |   |-->Monitor "Monitor0"
[    37.136] (**) |   |-->Device "Device0"
[    37.136] (**) |-->Input Device "Keyboard0"
[    37.136] (**) |-->Input Device "Mouse0"
[    37.136] (==) Automatically adding devices
[    37.136] (==) Automatically enabling devices
[    37.136] (==) Automatically adding GPU devices
[    37.136] (==) Automatically binding GPU devices
[    37.136] (==) Max clients allowed: 256, resource mask: 0x1fffff
[    37.138] (WW) The directory "/usr/share/fonts/Type1" does not exist.
[    37.138]    Entry deleted from font path.
[    37.138] (WW) The directory "/usr/share/fonts/100dpi" does not exist.
[    37.138]    Entry deleted from font path.
[    37.138] (WW) The directory "/usr/share/fonts/75dpi" does not exist.
[    37.138]    Entry deleted from font path.
[    37.138] (==) FontPath set to:
        /usr/share/fonts/misc,
        /usr/share/fonts/TTF,
        /usr/share/fonts/OTF
[    37.138] (==) ModulePath set to "/usr/lib/xorg/modules"
[    37.138] (WW) Hotplugging is on, devices using drivers 'kbd', 'mouse' or 'vmmouse' will be disabled.
[    37.138] (WW) Disabling Keyboard0
[    37.138] (WW) Disabling Mouse0
[    37.138] (II) Module ABI versions:
[    37.138]    X.Org ANSI C Emulation: 0.4
[    37.138]    X.Org Video Driver: 25.2
[    37.138]    X.Org XInput driver : 24.4
[    37.138]    X.Org Server Extension : 10.0
[    37.138] (++) using VT number 1

[    37.139] (II) systemd-logind: took control of session /org/freedesktop/login1/session/_31
[    37.140] (--) PCI:*(6@0:0:0) 10de:1c03:1462:3283 rev 161, Mem @ 0xfb000000/16777216, 0x7fe0000000/268435456, 0x7ff0000000/33554432, I/O @ 0x0000e000/128, BIOS @ 0x????????/524288
[    37.140] (WW) Open ACPI failed (/var/run/acpid.socket) (No such file or directory)
[    37.140] (II) LoadModule: "glx"
[    37.141] (II) Loading /usr/lib/xorg/modules/extensions/libglx.so
[    37.147] (II) Module glx: vendor="X.Org Foundation"
[    37.147]    compiled for 1.21.1.8, module version = 1.0.0
[    37.147]    ABI class: X.Org Server Extension, version 10.0
[    37.147] (II) LoadModule: "nvidia"
[    37.147] (II) Loading /usr/lib/xorg/modules/drivers/nvidia_drv.so
[    37.164] (II) Module nvidia: vendor="NVIDIA Corporation"
[    37.164]    compiled for 1.6.99.901, module version = 1.0.0
[    37.164]    Module class: X.Org Video Driver
[    37.164] (II) NVIDIA dlloader X Driver  535.54.03  Tue Jun  6 22:23:02 UTC 2023
[    37.164] (II) NVIDIA Unified Driver for all Supported NVIDIA GPUs
[    37.164] (II) Loading sub module "fb"
[    37.164] (II) LoadModule: "fb"
[    37.164] (II) Module "fb" already built-in
[    37.164] (II) Loading sub module "wfb"
[    37.164] (II) LoadModule: "wfb"
[    37.164] (II) Loading /usr/lib/xorg/modules/libwfb.so
[    37.165] (II) Module wfb: vendor="X.Org Foundation"
[    37.165]    compiled for 1.21.1.8, module version = 1.0.0
[    37.165]    ABI class: X.Org ANSI C Emulation, version 0.4
[    37.166] (EE) NVIDIA: Failed to initialize the NVIDIA kernel module. Please see the
[    37.166] (EE) NVIDIA:     system's kernel log for additional error messages and
[    37.166] (EE) NVIDIA:     consult the NVIDIA README for details.
[    37.166] (EE) No devices detected.
[    37.166] (EE) 
Fatal server error:
[    37.166] (EE) no screens found(EE) 
[    37.166] (EE) 
Please consult the The X.Org Foundation support 
         at http://wiki.x.org
 for help. 
[    37.166] (EE) Please also check the log file at "/var/log/Xorg.0.log" for additional information.
[    37.166] (EE) 
[    37.168] (EE) Server terminated with error (1). Closing log file.

I have repeated `mkinitcpio -P` and rebooted, repeated installation of `nvidia-lts` (matching my `linux-lts`) and regenerated the initramfs and rebooted, but to no avail. It seemed impossible to load the nvidia driver.

The `nouveau` driver is not even installed. `lcpci` did not mention nouveau as it does in the other post.

# lspci -k | grep -A2 -E 'VGA|3D'
lspci: Unable to load libkmod resources: error -2
06:00.0 VGA compatible controller: NVIDIA Corporation GP106 [GeForce GTX 1060 6GB] (rev a1)
        Subsystem: Micro-Star International Co., Ltd. [MSI] GP106 [GeForce GTX 1060 6GB]
06:00.1 Audio device: NVIDIA Corporation GP106 High Definition Audio Controller (rev a1)

This might related: While X was not working, I’ve also had no internet access. The `nftables` service had failed:

# jc -u nftables.service | tail           # sorry, I have alias jc=journalctl
-- Boot c72c87c31120422d86e9965f9da9e67b --
Jul 02 10:28:24 tauhou systemd[1]: Starting Netfilter Tables...
Jul 02 10:28:24 tauhou nft[729]: mnl.c:61: Unable to initialize Netlink socket: Protocol not supported
Jul 02 10:28:24 tauhou systemd[1]: nftables.service: Main process exited, code=exited, status=3/NOTIMPLEMENTED
Jul 02 10:28:24 tauhou systemd[1]: nftables.service: Failed with result 'exit-code'.
Jul 02 10:28:24 tauhou systemd[1]: Failed to start Netfilter Tables.
-- Boot 4696b6a41cce469f98508a5c216c9c9d --
…

This is also working again after the downgrade.

I’ve upgraded again, followed by a downgrade as above, and both problems (X, nftables) showed up, and went away respectively. So this seems both to be related to the upgrade. I did not investigate the `nftables` issue further.

Anything I could do to improve the situation?

Thanks for helping!

Stefan

Last edited by stefan (2023-07-02 15:09:10)

Offline

#2 2023-07-02 12:44:01

Scimmia
Fellow
Registered: 2012-09-01
Posts: 13,727

Re: [solved] Upgrade might break X (linux-lts, nvidia-lts) and nftables

When things are broken, what does `uname -a` give?

The xorg log seems to indicate that you're still running the older kernel. Assuming that's after a reboot, it usually means that you didn't mount your /boot partition before upgrading, but could also mean that your bootloader is looking in the wrong place for the kernel.

Last edited by Scimmia (2023-07-02 12:45:48)

Offline

#3 2023-07-02 15:06:43

stefan
Member
Registered: 2013-03-22
Posts: 131

Re: [solved] Upgrade might break X (linux-lts, nvidia-lts) and nftables

Scimmia wrote:

it usually means that you didn't mount your /boot partition before upgrading

That’s it. I’ll mark the issue as solved.

I have indeed mis-configured `/etc/fstab` a couple of days ago, leading to `/boot` not being mounted anymore. It had no immediate adverse effects, so I did not notice, forgot about the editing, and I obviously did not make the connection today. Also, I vaguely remember that I've been bitten by a similar chain of events some time ago.

I’m really sorry for wasting your time.

I wonder how I could make this more obvious to me, because it will occur again, surely. The train of thought has to be: broken upgrade → is the new kernel actually running? → check for /boot being mounted.

Thanks for helping!
Stefan

Offline

Pages: 1

Board footer

Powered by FluxBB