You are not logged in.
- Topics: Active | Unanswered
#1 2024-03-02 22:19:59
- 0xba5ed
- Member
- Registered: 2024-03-02
- Posts: 4
[SOLVED] Another "Failed to mount /boot"/"Dependency check failed".
Hey guys, am feeling a bit silly asking for help with this (was hoping the act of writing out the request might trigger catching something obvious).
I have the issue of start-up falling back to emergency mode, after failing to mount swap, or boot partitions.
Despite seeing parity between uname -a, pacman -Q linux and file /boot/vm-linuz-linux, commands. I did attempt some reinstalls of the kernel, via an arch-chroot from a bootable usb, making sure everything was mounted.
Did check fstab for discrepancies. The root drive is found and mounted, and the two uuid's that are not, are at least accurately matching fstab and blkid queries.
Have posted command and file output below.
Some initial attempts to fix included reinstalling dbus-deamon, and going through udev configs to remove or repair invalid xml.
A quick flash of a bus fail error being the last thing I had seen during the reboot prior to problems, which was post an update where asked to make the switch and so had thought it dbus-broker related.
$ pacman -Q linux
linux 6.7.6.arch1-2$ uname -a
Linux mikasa 6.7.6-arch1-2 #1 SMP PREEMPT_DYNAMIC Thu, 29 Feb 2024 19:08:51 +0000 x86_64 GNU/Linux$ file /boot/vmlinuz-linux
/vmlinuz-linux: Linux kernel x86 boot executable bzImage, version 6.7.6-arch1-2 (linux@archlinux) #1 SMP PREEMPT_DYNAMIC Thu, 29 Feb 2024 19:08:51 +0000, RO-rootFS, swap_dev 0XC, Normal VGA- fstab
# <file system> <dir> <type> <options> <dump> <pass>
# /dev/nvme0n1p3
UUID=7a8fbeff-05fd-4a13-82b3-75599b814f71 / ext4 rw,relatime 0 1
# /dev/nvme0n1p1
UUID=EA10-F89D /boot vfat rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro 0 2
# /dev/nvme0n1p2
UUID=2247828e-47cd-4f9c-ad7b-7d821405b904 none swap defaults 0 0$ ls /dev/disk/by-uuid
2247828e-47cd-4f9c-ad7b-7d821405b904 7a8fbeff-05fd-4a13-82b3-75599b814f71 EA10-F89D$ blkid nvme0n1p1 &&...
/dev/nvme0n1p1: UUID="EA10-F89D" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="e4197941-1d76-a143-86b4-cac6cd2e9b12"
/dev/nvme0n1p2: UUID="2247828e-47cd-4f9c-ad7b-7d821405b904" TYPE="swap" PARTUUID="f50e9ebc-4b9e-6041-bd8d-32a83bcc5d04"
/dev/nvme0n1p3: UUID="7a8fbeff-05fd-4a13-82b3-75599b814f71" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="c1a2d1e8-57cc-a84d-8c75-ae40801ae7ac"$ lsblk -f
NAME FSTYPE FSVER LABEL UUID FSAVAIL FSUSE% MOUNTPOINTS
nvme0n1
├─nvme0n1p1 283.6M 44% /boot
├─nvme0n1p2 [SWAP]
└─nvme0n1p3 624.8G 26% /- mkinitpcio.conf (summarised)
MODULES=()
BINARIES=()
FILES=()
HOOKS=(base udev autodetect modconf block filesystems keyboard resume fsck)- grub (summarised)
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Arch"
GRUB_CMDLINE_LINUX_DEFAULT="loglevel=3 split_lock_detect=off"
GRUB_CMDLINE_LINUX="resume=UUID=2247828e-47cd-4f9c-ad7b-7d821405b904"
GRUB_PRELOAD_MODULES="part_gpt part_msdos"
GRUB_TIMEOUT_STYLE=menu
GRUB_TERMINAL_INPUT=console
GRUB_GFXMODE=auto
GRUB_GFXPAYLOAD_LINUX=keep
GRUB_DISABLE_RECOVERY=true
GRUB_COLOR_NORMAL="light-blue/black"
GRUB_COLOR_HIGHLIGHT="light-cyan/blue" - `split_lock_detect=off` was a recent addition, as were `pci=nomsi`, pci=noaer, `quiet`, recent removals for debugging.
For context, the laptop was a custom lenovo build from just after intel gen12 dropped, the hardware wasn't yet very well supported (lenovo stopped offering linux support on that model, switching to only win 10) so I had suppressed a lot of warnings and worked on issues as time allowed/where severe enough, or in most cases as patches and drivers were rolled out.
It's a cuda dev box, that has only ever had arch on it. So no consideration for windows or any other partition. Secure boot disabled. Kept minimal (no gnome or kde) but have faffed about with nvidia a bit and trying to wake from sleep.
- journalctl -b -7
- - full: https://0x0.st/HRF7.txt
- - slice:
Mar 01 02:18:33 mikasa systemd[1]: systemd-rfkill.service: Deactivated successfully.
Mar 01 02:19:56 mikasa systemd[1]: dev-disk-by\x2duuid-2247828e\x2d47cd\x2d4f9c\x2dad7b\x2d7d821405b904.device: Job dev-disk-by\x2duuid-2247828e\x2d47cd\x2d4f9c\x2dad7b\x2d7d821405b904.device/start timed out.
Mar 01 02:19:56 mikasa systemd[1]: Timed out waiting for device /dev/disk/by-uuid/2247828e-47cd-4f9c-ad7b-7d821405b904.
Mar 01 02:19:56 mikasa systemd[1]: Dependency failed for /dev/disk/by-uuid/2247828e-47cd-4f9c-ad7b-7d821405b904.
Mar 01 02:19:56 mikasa systemd[1]: Dependency failed for Swaps.
Mar 01 02:19:56 mikasa systemd[1]: swap.target: Job swap.target/start failed with result 'dependency'.
Mar 01 02:19:56 mikasa systemd[1]: dev-disk-by\x2duuid-2247828e\x2d47cd\x2d4f9c\x2dad7b\x2d7d821405b904.swap: Job dev-disk-by\x2duuid-2247828e\x2d47cd\x2d4f9c\x2dad7b\x2d7d821405b904.swap/start failed with result 'dependency'.
Mar 01 02:19:56 mikasa systemd[1]: dev-disk-by\x2duuid-2247828e\x2d47cd\x2d4f9c\x2dad7b\x2d7d821405b904.device: Job dev-disk-by\x2duuid-2247828e\x2d47cd\x2d4f9c\x2dad7b\x2d7d821405b904.device/start failed with result 'timeout'.
Mar 01 02:19:56 mikasa systemd[1]: dev-disk-by\x2duuid-EA10\x2dF89D.device: Job dev-disk-by\x2duuid-EA10\x2dF89D.device/start timed out.
Mar 01 02:19:56 mikasa systemd[1]: Timed out waiting for device /dev/disk/by-uuid/EA10-F89D.
Mar 01 02:19:56 mikasa systemd[1]: Dependency failed for /boot.
Mar 01 02:19:56 mikasa systemd[1]: Dependency failed for Local File Systems.
Mar 01 02:19:56 mikasa systemd[1]: local-fs.target: Job local-fs.target/start failed with result 'dependency'.
Mar 01 02:19:56 mikasa systemd[1]: local-fs.target: Triggering OnFailure= dependencies.
Mar 01 02:19:56 mikasa systemd[1]: boot.mount: Job boot.mount/start failed with result 'dependency'.
Mar 01 02:19:56 mikasa systemd[1]: Dependency failed for File System Check on /dev/disk/by-uuid/EA10-F89D.
Mar 01 02:19:56 mikasa systemd[1]: systemd-fsck@dev-disk-by\x2duuid-EA10\x2dF89D.service: Job systemd-fsck@dev-disk-by\x2duuid-EA10\x2dF89D.service/start failed with result 'dependency'.
Mar 01 02:19:56 mikasa systemd[1]: dev-disk-by\x2duuid-EA10\x2dF89D.device: Job dev-disk-by\x2duuid-EA10\x2dF89D.device/start failed with result 'timeout'.
Mar 01 02:19:56 mikasa systemd[1]: systemd-ask-password-console.path: Deactivated successfully.Searching the issue almost all results have a mismatch in kernel version where reinstalling the linux package from arch-chroot does the trick, in others a mismatch in UUID, or a custom hook.
Did look for some custom hooks, as thought I may have forgotten something from nvidia or a wake from sleep solution. (This is 1 of 3 arch boxes, and am losing track between them a little so clearly room to improve the system).
I couldn't see anything under /etc/initcpio/install/ or MODULES in conf. I have an install_notes.md where I try to keep track of any major reminders like that.
But maybe something, or somewhere, else I should be checking?
If solution is obvious don't hold back on letting me know, if need more info please ask.
Appreciate any time taken.
Last edited by 0xba5ed (2024-03-03 18:35:48)
Offline
#2 2024-03-02 23:57:24
- Scimmia
- Fellow
- Registered: 2012-09-01
- Posts: 11,592
Re: [SOLVED] Another "Failed to mount /boot"/"Dependency check failed".
You covered all of the common issues very well, great job researching that.
Is this something that happens on every boot, or only some of them? I notice the log posted was from a previous kernel pkgrel, not the one you gave in the previous commands, so I thought I'd ask. That ls /dev/disk/by-uuid was from a failed boot?
Offline
#3 2024-03-03 00:08:18
- 0xba5ed
- Member
- Registered: 2024-03-02
- Posts: 4
Re: [SOLVED] Another "Failed to mount /boot"/"Dependency check failed".
You covered all of the common issues very well, great job researching that.
Is this something that happens on every boot, or only some of them? I notice the log posted was from a previous kernel pkgrel, not the one you gave in the previous commands, so I thought I'd ask. That ls /dev/disk/by-uuid was from a failed boot?
Appreciate the reply and the kind words.
That is from the first boot it appeared. It has happened on every reboot after, was just unsure if worth spamming with the rest:
Including the last 3 for disclosure, happy to add any others.
edit:
Yes the ls /dev/disk/by-uuid was from a failed boot. After mounting and exiting the emergency shell.
Last edited by 0xba5ed (2024-03-03 00:11:11)
Offline
#4 2024-03-03 01:49:46
- Scimmia
- Fellow
- Registered: 2012-09-01
- Posts: 11,592
Re: [SOLVED] Another "Failed to mount /boot"/"Dependency check failed".
So the devices are available at that point; seems like some kind of delay, which is really strange as the drivers for the storage system are all loaded to access the root partition already. I'm not seeing anything suspicious in the log.
What kernel did you update from? Can you try the old one?
Offline
#5 2024-03-03 02:01:08
- 0xba5ed
- Member
- Registered: 2024-03-02
- Posts: 4
Re: [SOLVED] Another "Failed to mount /boot"/"Dependency check failed".
So the devices are available at that point; seems like some kind of delay, which is really strange as the drivers for the storage system are all loaded to access the root partition already. I'm not seeing anything suspicious in the log.
What kernel did you update from? Can you try the old one?
[2024-02-18T00:56:00+0000] [ALPM] upgraded linux (6.6.1.arch1-1 -> 6.7.5.arch1-1)
...
[2024-02-27T13:10:09+0000] [ALPM] upgraded linux (6.7.5.arch1-1 -> 6.7.6.arch1-1)*corrected and added additional context. Will have a look at reboots around those dates.
Ignoring an update that was pushed while attempting fixes.
[2024-03-01T06:16:23+0000] [ALPM] upgraded linux (6.7.6.arch1-1 -> 6.7.6.arch1-2)Will downgrade and report back, it's a solution I should have already tried so appreciate the patience in that regard.
Update
After referencing the boot times correctly. I can see the issue occurred while I had 6.7.5-arch1-1
- Last successful boot: Feb 26 16:17:22 (with 6.7.5-arch1-1)
Feb 26 16:17:23 mikasa systemd[1]: Mounting /boot...
...
Feb 26 16:17:23 mikasa systemd[1]: Mounted /boot.- Restart: Feb 27 12:11:10
- First failed boot: Feb 27 12:11:34 (with 6.7.5-arch1-1)
Feb 27 12:13:04 mikasa systemd[1]: Dependency failed for File System Check on /dev/disk/by-uuid/EA10-F89D.
Feb 27 12:13:04 mikasa systemd[1]: Dependency failed for /boot.---
Installed the lts, just as that's simple and I have some experience with it.
Journal from a failed boot with 6.6.19-1-lts:
- http://0x0.st/H7KM.txt
Is it worth specifically downgrading to 6.7.5, if I upgraded after the issue occurred?
As I don't have experience with dkms or that type of downgrade I think it will take a couple of hours to be confident in the execution/track the process.
As always appreciate the assistance.
Last edited by 0xba5ed (2024-03-03 04:23:04)
Offline
#6 2024-03-03 18:35:04
- 0xba5ed
- Member
- Registered: 2024-03-02
- Posts: 4
Re: [SOLVED] Another "Failed to mount /boot"/"Dependency check failed".
Marking solved
It seems like this was quite shallow. A bad udev rule, for usb added on the 27th.
Thought I had already deleted it while troubleshooting, moved the file out and rebooted straight to login. (6.7.7 and lts).
Just want to thank Scimmia, for being someone to bounce ideas off, and for being patient. Really appreciate the help.
Offline