You are not logged in.
Pages: 1
A warm hello to everyone on this great forum!
I've been following the forums as a ghost in order to solve my problems, but I cannot figure out a solution to this one:
I updated my kernel yesterday from version 4.13.3-1 to 4.13.4-1 and when I rebooted I got a 'Failed to start Load Kernel Modules'. I then followed the suggestion to re-update the linux package from a chroot.
On the next reboot, I got dropped into emergency shell with following messages
Warning: /lib/modules/4.13.4-1-ARCH/modules.devname not found - ignoring
starting version 234
ERROR: device '/dev/sda2' not found. Skipping fsck.
mount: /new_root: no filesystem type specified.
You are now bein dropped into an emergency shell.
sh: can't access tty; job control turned off
After all I tried afterwards, nothing changed in these messages.
First, I was looking at the directory that could not be found and found the missing file exactly there:
ls /lib/modules/4.13.4-1-ARCH/
modules.alias modules.dep modules.softdep
modules.alias.bin modules.dep.bin modules.symbols
modules.builtin.bin modules.devname modules.symbols.bin
Some people recommended me to check /etc/fstab, which looks okay to me.
cat /etc/fstab
# /dev/sda2 UUID=69d7787f-016b-4acf-92e2-79f423f89537
/dev/sda2 / ext4 rw,relatime,data=ordered0 1
/dev/sda5 /home ext4 rw,relatime,data=ordered0 2
For completeness, lsblk gives:
lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 298.1G 0 disk
|-sda1 8:1 0 12G 0 part [SWAP]
|-sda2 8:2 0 25G 0 part /
|-sda3 8:3 0 25G 0 part
|-sda4 8:4 0 1K 0 part
`-sda5 8:5 0 236.1G 0 part
sr0 11:0 1 1024M 0 rom
Trying to reinstall again gives me this:
sudo pacman -Syu linux
:: Synchronizing package databases...
core is up to date
extra is up to date
haskell-core is up to date
community is up to date
multilib is up to date
warning: linux-4.13.4-1 is up to date -- reinstalling
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...
Packages (1) linux-4.13.4-1
Total Installed Size: 92.71 MiB
Net Upgrade Size: 0.00 MiB
:: Proceed with installation? [Y/n] y
(1/1) checking keys in keyring [######################] 100%
(1/1) checking package integrity [######################] 100%
(1/1) loading package files [######################] 100%
(1/1) checking for file conflicts [######################] 100%
(1/1) checking available disk space [######################] 100%
:: Processing package changes...
(1/1) reinstalling linux [######################] 100%
>>> Updating module dependencies. Please wait ...
depmod: WARNING: could not open /lib/modules/4.13.4-1-ARCH/modules.order: No such file or directory
depmod: WARNING: could not open /lib/modules/4.13.4-1-ARCH/modules.builtin: No such file or directory
:: Running post-transaction hooks...
(1/2) Updating linux initcpios
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
-> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
==> Starting build: 4.13.4-1-ARCH
-> Running build hook: [base]
-> Running build hook: [udev]
-> Running build hook: [autodetect]
-> Running build hook: [modconf]
-> Running build hook: [block]
-> Running build hook: [filesystems]
-> Running build hook: [keyboard]
==> ERROR: module not found: `usbhid'
-> Running build hook: [fsck]
==> WARNING: No modules were added to the image. This is probably not what you want.
==> Creating gzip-compressed initcpio image: /boot/initramfs-linux.img
==> WARNING: errors were encountered during the build. The image may not be complete.
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'fallback'
-> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-fallback.img -S autodetect
==> Starting build: 4.13.4-1-ARCH
-> Running build hook: [base]
-> Running build hook: [udev]
-> Running build hook: [modconf]
-> Running build hook: [block]
-> Running build hook: [filesystems]
-> Running build hook: [keyboard]
==> ERROR: module not found: `usbhid'
-> Running build hook: [fsck]
==> WARNING: No modules were added to the image. This is probably not what you want.
==> Creating gzip-compressed initcpio image: /boot/initramfs-linux-fallback.img
==> WARNING: errors were encountered during the build. The image may not be complete.
error: command failed to execute correctly
(2/2) Arming ConditionNeedsUpdate...
Where the latter part looks much the same as running 'mkinitcpio -p linux' myself.
I also unsuccessfully tried to run depmod:
depmod -a
depmod: WARNING: could not open /lib/modules/4.4.0-83-generic/modules.order: No such file or directory
depmod: WARNING: could not open /lib/modules/4.4.0-83-generic/modules.builtin: No such file or directory
and indeed the indicated files are not present in this directory (I linked /lib/modules/4.4.0-83-generic to /lib/modules/4.13.4-1-ARCH).
Here are some of the posts I consulted (the ones I could still find).
https://www.soimort.org/notes/170407/
https://bbs.archlinux.de/viewtopic.php?id=26307
https://stackoverflow.com/questions/348 … a-modprobe
I read the wiki pages for the tools, but without any further insight. Most of the people in the forums having similar problems appeared to build their own kernels, which I didn't.
Thank you for reading this far. I would appreciate any help that leads me out of the valley of frustration. Please, don't hesitate to ask for further information and/or to refer to the guidelines in case I disregarded one.
Last edited by Dietaer (2017-10-11 08:41:57)
Offline
Broken kernel package from broken mirror?
This
depmod: WARNING: could not open /lib/modules/4.13.4-1-ARCH/modules.order: No such file or directory
depmod: WARNING: could not open /lib/modules/4.13.4-1-ARCH/modules.builtin: No such file or directory
should not happen and leads to
==> WARNING: No modules were added to the image. This is probably not what you want.
Check the cached package for the existence of the file, download one from the web interface for comparism and fix your mirrorlist in case.
Offline
Check the output of uname -a and of pacman -Q linux and ensure the version numbers match.
Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way
Offline
Thank you both, I'm delighted to see some replies so quickly!
@Seth
Check the cached package for the existence of the file, download one from the web interface for comparism and fix your mirrorlist in case.
I checked both the cached package and a package from the arch linux repository website and both times I could find the modules.order and modules.builtin files inside the directories I extracted the packages to.
find ./ -name modules.*
./usr/lib/modules/4.13.4-1-ARCH/modules.dep
./usr/lib/modules/4.13.4-1-ARCH/modules.symbols
./usr/lib/modules/4.13.4-1-ARCH/modules.order
./usr/lib/modules/4.13.4-1-ARCH/modules.devname
./usr/lib/modules/4.13.4-1-ARCH/modules.symbols.bin
./usr/lib/modules/4.13.4-1-ARCH/modules.dep.bin
./usr/lib/modules/4.13.4-1-ARCH/modules.builtin.bin
./usr/lib/modules/4.13.4-1-ARCH/modules.builtin
./usr/lib/modules/4.13.4-1-ARCH/modules.softdep
./usr/lib/modules/4.13.4-1-ARCH/modules.alias.bin
./usr/lib/modules/4.13.4-1-ARCH/modules.alias
Downloading the package from the arch repository from the web interface and installing it with
pacman -U linux-4.13.4-1-x86_64.pkg.tar.xz
which produced the same set of modules files in /lib/modules/<version num>-ARCH/ . Also, the same errors persist when calling pacman or mkinitcpio. I downloaded and installed a package from the arch linux archive (linux 4.9.9-2) in a similar fashion with similar results.
Following you second suggestion, I updated the pacman-mirrorlist package as suggested in the wiki with
pacman -Syu pacman-mirrorlist
but it was already up to date. I reinstalled the linux package again with the same outcome. Further, I changed the order of some of the mirrors in /etc/pacman.d/mirrorlist, repeated the procedure and got the same results. Could it be that I'm doing something dead wrong when installing packages?
@ewaller
I ran both commands
uname -a
Linux t420 4.4.0-83-generic #106-Ubuntu SMP Mon Jun 26 17:54:43 UTC 2017 x86_64 GNU/Linux
pacman -Q linux
linux 4.13.4-1
though uname only gives me info about the system I am running the chroot from (Ubuntu). The number given by pacman does match the number of the directory in /lib/modules/ (as shown in my original post). The version number 4.13.4-1 seems quite reasonable, since the current version of the linux project on GitHub is v4.14-rc4.
I was looking for another way to check the version of the installed kernel of the system I'm in with chroot, but I could not find any. Do you know any alternative?
Thanks for helping.
Offline
Assuming the system's file hierarchy is mounted under /mnt
$ file /mnt/boot/vmlinuz-linux
Edit:
Also the output of the following please
$ ls -la /mnt/usr/lib/modules
$ ls -la /mnt/usr/lib/modules/4.13.4-1-ARCH
Last edited by loqs (2017-10-10 19:48:34)
Offline
Some stupid NoExtract in /etc/pacman.conf ?
Offline
@loqs: Thanks a lot!
Running from outside the chroot you can see that I mounted the system on /mnt/arch
lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 298,1G 0 disk
├─sda1 8:1 0 12G 0 part [SWAP]
├─sda2 8:2 0 25G 0 part /mnt/arch
├─sda3 8:3 0 25G 0 part /
├─sda4 8:4 0 1K 0 part
└─sda5 8:5 0 236,1G 0 part /mnt/arch/mnt/home
sr0 11:0 1 1024M 0 rom
Inspecting /mnt/arch/boot/vmlinuz-linux
file /mnt/arch/boot/vmlinuz-linux
/mnt/arch/boot/vmlinuz-linux: Linux kernel x86 boot executable bzImage, version 4.13.4-1-ARCH (builduser@tobias) #1 SMP PREEMPT Thu Sep 28 08:3, RO-rootFS, swap_dev 0x5, Normal VGA
So, indeed, the version numbers match. So this does mean that the package that is supposed to be there really is there and I did not screw up the installation itself, right?
Edit
Also the output of the following please
This gives
ls -la /mnt/arch/usr/lib/modules
insgesamt 152
drwxr-xr-x 4 root root 4096 Okt 9 23:35 .
drwxr-xr-x 166 root root 139264 Okt 10 20:05 ..
drwxr-xr-x 4 root root 4096 Okt 10 21:14 4.13.4-1-ARCH
drwxr-xr-x 2 root root 4096 Okt 10 21:14 extramodules-4.13-ARCH
ls -la /mnt/arch/usr/lib/modules/4.13.4-1-ARCH/
insgesamt 4664
drwxr-xr-x 4 root root 4096 Okt 10 21:14 .
drwxr-xr-x 4 root root 4096 Okt 9 23:35 ..
drwxr-xr-x 22 root root 4096 Okt 10 21:14 build
lrwxrwxrwx 1 root root 25 Sep 28 08:41 extramodules -> ../extramodules-4.13-ARCH
drwxr-xr-x 13 root root 4096 Okt 9 23:35 kernel
-rw-r--r-- 1 root root 1108427 Sep 28 08:41 modules.alias
-rw-r--r-- 1 root root 1075390 Sep 28 08:41 modules.alias.bin
-rw-r--r-- 1 root root 3835 Sep 28 08:41 modules.builtin
-rw-r--r-- 1 root root 5583 Sep 28 08:41 modules.builtin.bin
-rw-r--r-- 1 root root 517363 Sep 28 08:41 modules.dep
-rw-r--r-- 1 root root 690812 Sep 28 08:41 modules.dep.bin
-rw-r--r-- 1 root root 411 Sep 28 08:41 modules.devname
-rw-r--r-- 1 root root 154822 Sep 28 08:41 modules.order
-rw-r--r-- 1 root root 577 Sep 28 08:41 modules.softdep
-rw-r--r-- 1 root root 527958 Sep 28 08:41 modules.symbols
-rw-r--r-- 1 root root 651696 Sep 28 08:41 modules.symbols.bin
So
the version numbers are the same
the files mkinitcpio complains about are present in the folder /mnt/arch/usr/lib/modules/4.13.4-1-ARCH/
However I am puzzled why there is a directory containing kernel modules in /usr/lib/modules as well as in /lib/modules. Even more, given that the modules they contain differ (the latter does not contain modules.order and modules.builtin). What are their respective reasons to exist?
Last edited by Dietaer (2017-10-10 20:11:42)
Offline
@Seth: Thank you!
Some stupid NoExtract in /etc/pacman.conf ?
cat /etc/pacman.conf | grep NoExtract
#NoExtract =
So it is commented out, correct?
In case the full pacman.conf gives some higher being an insight, here it is.
[root@t420 /]# cat /etc/pacman.conf
#
# /etc/pacman.conf
#
# See the pacman.conf(5) manpage for option and repository directives
#
# GENERAL OPTIONS
#
[options]
# The following paths are commented out with their default values listed.
# If you wish to use different paths, uncomment and update the paths.
#RootDir = /
#DBPath = /var/lib/pacman/
#CacheDir = /var/cache/pacman/pkg/
#LogFile = /var/log/pacman.log
#GPGDir = /etc/pacman.d/gnupg/
#HookDir = /etc/pacman.d/hooks/
HoldPkg = pacman glibc
#XferCommand = /usr/bin/curl -C - -f %u > %o
#XferCommand = /usr/bin/wget --passive-ftp -c -O %o %u
#CleanMethod = KeepInstalled
#UseDelta = 0.7
Architecture = auto
# Pacman won't upgrade packages listed in IgnorePkg and members of IgnoreGroup
#IgnorePkg =
#IgnoreGroup =
#NoUpgrade =
#NoExtract =
# Misc options
#UseSyslog
#Color
#TotalDownload
CheckSpace
#VerbosePkgLists
# By default, pacman accepts packages signed by keys that its local keyring
# trusts (see pacman-key and its man page), as well as unsigned packages.
SigLevel = Required DatabaseOptional
LocalFileSigLevel = Optional
#RemoteFileSigLevel = Required
# NOTE: You must run `pacman-key --init` before first using pacman; the local
# keyring can then be populated with the keys of all official Arch Linux
# packagers with `pacman-key --populate archlinux`.
#
# REPOSITORIES
# - can be defined here or included from another file
# - pacman will search repositories in the order defined here
# - local/custom mirrors can be added here or in separate files
# - repositories listed first will take precedence when packages
# have identical names, regardless of version number
# - URLs will have $repo replaced by the name of the current repo
# - URLs will have $arch replaced by the name of the architecture
#
# Repository entries are of the format:
# [repo-name]
# Server = ServerName
# Include = IncludePath
#
# The header [repo-name] is crucial - it must be present and
# uncommented to enable the repo.
#
# The testing repositories are disabled by default. To enable, uncomment the
# repo name header and Include lines. You can add preferred servers immediately
# after the header, and they will be used before the default mirrors.
#[testing]
#Include = /etc/pacman.d/mirrorlist
[core]
Include = /etc/pacman.d/mirrorlist
[extra]
Include = /etc/pacman.d/mirrorlist
#[community-testing]
#Include = /etc/pacman.d/mirrorlist
[haskell-core]
Server = http://xsounds.org/~haskell/core/$arch
[community]
Include = /etc/pacman.d/mirrorlist
# If you want to run 32 bit applications on your x86_64 system,
# enable the multilib repositories as required here.
#[multilib-testing]
#Include = /etc/pacman.d/mirrorlist
[multilib]
Include = /etc/pacman.d/mirrorlist
# An example of a custom package repository. See the pacman manpage for
# tips on creating your own repositories.
#[custom]
#SigLevel = Optional TrustAll
#Server = file:///home/custompkgs
Thank you all for helping.
Last edited by Dietaer (2017-10-10 20:01:54)
Offline
Either chroot in or from from the ubuntu environment with the system under /mnt
$ /mnt/usr/bin/pacman --root /mnt -Qo /lib/modules/4.13.4-1-ARCH/modules.order
$ /mnt/usr/bin/pacman --root /mnt -Qkk linux
Edit:
fixed order of arguments to pacman
Edit2:
not sure if the -Qo call should include the /mnt element of the path
Last edited by loqs (2017-10-10 20:09:00)
Offline
According to the previous posts, the file won't be even there.
In case, the major question is why it's not extracted (and yes, NoExtract is commented - would have been too easy, I guess)
pacman --root /mnt -S -v --debug linux 2>&1 | tee ~/verylonglogoflinuxinstallation.txt
Offline
Either chroot in or from from the ubuntu environment with the system under /mnt
$ /mnt/usr/bin/pacman --root /mnt -Qo /lib/modules/4.13.4-1-ARCH/modules.order $ /mnt/usr/bin/pacman --root /mnt -Qkk linux
From Ubuntu it does not appear to be possible to call pacman, because
sudo /mnt/arch/usr/bin/pacman --root /mnt/arch -Qo /lib/modules/4.13.4-1-ARCH/modules.order
/mnt/arch/usr/bin/pacman: error while loading shared libraries: libalpm.so.10: cannot open shared object file: No such file or directory
Hence I tried from within the chroot
/usr/bin/pacman -Qo /lib/modules/4.13.4-1-ARCH/modules.order
error: failed to read file '/lib/modules/4.13.4-1-ARCH/modules.order': No such file or directory
/usr/bin/pacman -Qkk linux
linux: 4810 total files, 0 altered files
The first command was expected to fail, since modules.order is missing from /lib/modules/4.13.4-1-ARCH/modules.order (as described above), right? So it makes sense that the second command reports no changed files.
Thanks for helping!
Offline
From within the chroot
ls -lad /lib
@seth /mnt/arch/usr/lib/modules/4.13.4-1-ARCH/ contains modules.order see post #7 update but /lib/modules/4.13.4-1-ARCH/modules.order is missing broken symlink?
Offline
Would be an explanation, but comment #1 also had
ls /lib/modules/4.13.4-1-ARCH/
modules.alias modules.dep modules.softdep
modules.alias.bin modules.dep.bin modules.symbols
modules.builtin.bin modules.devname modules.symbols.bin
Are they all re-generated in a post-hook?
Offline
A hook I do not believe so but possibly https://git.archlinux.org/svntogit/pack … ages/linux
Offline
According to the previous posts, the file won't be even there.
In case, the major question is why it's not extracted (and yes, NoExtract is commented - would have been too easy, I guess)pacman --root /mnt -S -v --debug linux 2>&1 | tee ~/verylonglogoflinuxinstallation.txt
Since pacman wouldn't run from Ubuntu, I ran the suggested command within the chroot. Hence I did not have to set --root, right? Or is there any reason to run pacman from outside of the chroot? So I executed this command instead:
pacman -S -v --debug linux 2>&1 | tee /mnt/home/martin/Schreibtisch/logLinuxInst.txt
The output (too large to post here) can be found at pastebin.ca.
Thank you, Seth!
Offline
From within the chroot
ls -lad /lib
@seth /mnt/arch/usr/lib/modules/4.13.4-1-ARCH/ contains modules.order see post #7 update but /lib/modules/4.13.4-1-ARCH/modules.order is missing broken symlink?
Which gives
ls -lad /lib
drwxr-xr-x 3 root root 4096 Oct 9 22:49 /lib
Thanks for going with me through this.
Offline
/lib being a directory rather than a symlink to /usr/lib is the cause of the issue. Please do not immediately delete the directory and make it a symlink without checking what is located under /lib and if it needs to be merged with /usr/lib.
Edit:
From within the chroot
pacman -Qkk filesystem
To check if any of the other symlinks are broken. Any idea how this might have happened?
Last edited by loqs (2017-10-10 20:54:35)
Offline
@loqs
Ok, because of eg. nvidia in the extramodules path (thus out of sync) calling depmod after the installation makes perfectly sense - just shipping the maps then does probably not ;-)
Offline
/lib being a directory rather than a symlink to /usr/lib is the cause of the issue. Please do not immediately delete the directory and make it a symlink without checking what is located under /lib and if it needs to be merged with /usr/lib.
I am confused as to what you mean by this. According to the linux specification about /usr/lib and /lib these two directories have separate functions. So as a consequence, I would assume that they are separate directories and not one being a link to the other. Am I missing something?
Edit:
From within the chrootpacman -Qkk filesystem
To check if any of the other symlinks are broken. Any idea how this might have happened?
Here is the execution of the command:
pacman -Qkk filesystem
warning: filesystem: /bin (File type mismatch)
warning: filesystem: /lib (File type mismatch)
backup file: filesystem: /etc/fstab (Modification time mismatch)
backup file: filesystem: /etc/fstab (Size mismatch)
backup file: filesystem: /etc/group (Modification time mismatch)
backup file: filesystem: /etc/group (Size mismatch)
backup file: filesystem: /etc/gshadow (Modification time mismatch)
backup file: filesystem: /etc/gshadow (Size mismatch)
backup file: filesystem: /etc/hosts (Modification time mismatch)
backup file: filesystem: /etc/hosts (Size mismatch)
backup file: filesystem: /etc/passwd (Modification time mismatch)
backup file: filesystem: /etc/passwd (Size mismatch)
backup file: filesystem: /etc/resolv.conf (Modification time mismatch)
backup file: filesystem: /etc/resolv.conf (Size mismatch)
backup file: filesystem: /etc/shadow (Modification time mismatch)
backup file: filesystem: /etc/shadow (Size mismatch)
backup file: filesystem: /etc/shells (Modification time mismatch)
backup file: filesystem: /etc/shells (Size mismatch)
filesystem: 105 total files, 2 altered files
For now, I am clueless why the links needed are broken, since I think - with relative but not absolute certainty - that I never modified the top-level of the filesystem. Are there any programs that could do so when being installed? I cannot quite see how this error could have been produced given that it occurred just after updating the linux package, but never before (for months).
Thank you, loqs.
Offline
The historic separation of /lib and /usr/lib was given up a couple of years ago.
/bin /sbin and /usr/bin should be the same as well.
If somehing had cloned /usr/lib into /lib (backup that followed symlinks?) this would have been no problem for the time being, but became one when you relied on new files installed into /usr/lib but expected in /lib by some tool.
Offline
The historic separation of /lib and /usr/lib was given up a couple of years ago.
/bin /sbin and /usr/bin should be the same as well.If somehing had cloned /usr/lib into /lib (backup that followed symlinks?) this would have been no problem for the time being, but became one when you relied on new files installed into /usr/lib but expected in /lib by some tool.
By saying many years ago, you allude to this announcement: https://www.archlinux.org/news/the-lib- … a-symlink/ ? I'm quite a newcomer to Arch Linux (using it for some 10 months now), so I find it curious that I could have separate /lib and /usr/lib directories, even years after that decision.
Anyway, it seems trying to understand why the state of my file system is as it is does not lead anywhere. Furthermore, this move appears to be common among many distros, e.g. Fedora.
So, I'm now trying to come up with a plan of how to proceed. I think, I have two options here:
Attempt to merge /lib into /usr/lib and similarly for /bin, where I should discard duplicates from /lib. Afterwards, I just create symbolic links /lib...
Backup all configurations and programs installed and reinstall Arch cleanly.
I'm tempted to just go with the latter, because it's less painful than trying to fix this broken system. Does this make sense? Is there an easier solution that I miss?
Thank you all for your patience and help.
Offline
You could try move lib to lib.bak bin to bin.bak then reinstall the filesystem package that should make the correct symlinks or fail if it can not then install the linux package and try booting.
If you can boot successfully you can start on the cleanup.
As to what caused would look at any scripts or make installs that you may have run as root.
Offline
You could try move lib to lib.bak bin to bin.bak then reinstall the filesystem package that should make the correct symlinks or fail if it can not then install the linux package and try booting.
If you can boot successfully you can start on the cleanup.
It worked and I'm back in my beloved Arch installation.
The cleanup is fairly limited, because /lib only contained directories with the faulty kernel modules and /bin only contained a bash executable.
I'll mark this thread as solved now.
Once more, I'd like to thank all of you for your support.
Enjoy Arch!
Offline
Pages: 1