You are not logged in.

#1 2020-12-19 15:02:25

B3l3tte
Member
Registered: 2016-10-26
Posts: 14

[SOLED] Early kernel panic, another "cannot load libcrypto.so" problem

Hello,

I have been struggling with an unbootable kernel since at least 3 weeks.

I have installed arch on a Dell Latitude 6530 since at least 3 or 4 years, and never had such a sticking problem, so I'm humbly asking for your help, since I don't have any more ideas to fix it.

A kernel panic happens just after GRUB loads linux and initramfs :

/init: error while loading shared libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory
Kernel panic - not syncing: Attempted to kill init! exitcode=0x00007f00
CPU: 0 PID: 1 Comm: init Not tainted 5.9.14-arch1-1 #1
Hardware name: Dell Inc. Latitude E6530/07Y85M, BIOS A22 11/30/2018
Call Trace:
 dump_stack+0x6b/0x83
 panic+0x112/0x2e8
 do_exit.cold+0x67/0xaa
 do_group_exit+0x33/0xa0
 __x64_sys_exit_group+0x14/0x20
 do_syscall_64+0x33/0x40
 entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x7ff042244361
[memdump]

Issue :
I cannot fix this libcrypto.so.1.1 issue. OpenSSL seems to be correctly installed. I don't have any issue using pacman in an arch-chroot. That is telling me that it maybe is a filesystem issue ?
What I tried so far :

  • I re-installed all packages with

    #pacman -Qqn | pacman -S -

    and

    #pacman -Qqm | yay -S -
  • I regenerated initramfs from live USB

  • I re-ran grub-install and grub-mkconfig

  • btrfs check and btrfs scrub do not report any error

  • Dell bios check util (check of memory, HDD, etc ...) do not report any error

Here are various data from my system :

  • UEFI mode, legacy BIOS management disabled.

  • Secure boot disabled

  • BTRFS without subvolumes setup (i didn't understand btrfs subvolumes at the time I installed the system)

  • No dual-boot

  • No disk encryption

System is up to date with

#yay -Syu

No error while updating. Some hooks fail at the end though when I updated from live key (like udev daemon reload but I think this is due to arch-chroot environment limitations).
Here is my last month pacman.log

Packages info :

#pacman -Qs grub
local/grub 2:2.04-8
#pacman -Qs linux
...
local/linux 5.9.14.arch1-1
local/linux-api-headers 5.8-1
local/linux-firmware 20201218.646f159-1
local/linux-headers 5.9.14.arch1-1
...
#pacman -Qs systemd
local/lib32-systemd 247.2-1
local/systemd 247.2-1
local/systemd-libs 247.2-1
local/systemd-numlockontty 0.1-12
local/systemd-sysvcompat 247.2-1
#pacman -Qs openssl
local/lib32-openssl 1:1.1.1.i-1
local/openssl 1.1.1.i-2
...

Disk partitioning : sda is my HDD, sdb is my arch live key

#lsblk -f
NAME        FSTYPE   FSVER LABEL       UUID                                 FSAVAIL FSUSE% MOUNTPOINT
loop0       squashfs 4.0                                                                   
sda                                                                                        
|-sda1      vfat     FAT16             7D00-EE5F                              32.9M    65% /boot
|-sda2      btrfs                      8ad9efa0-fe54-4f2b-afe2-4350d03f0ee0    293G    36% /
`-sda3      swap     1                 161416a5-b37f-4702-8fca-9eea04af5416                
sdb                                                                                        
|-sdb1      vfat     FAT32 ARCH_202012 C42C-4DB7                                           
`-sdb2      vfat     FAT32             3FC3-82BC                                 4G     0% /mnt
sr0                                                                                        
mmcblk0                                                                                    
`-mmcblk0p1 vfat     FAT32             28D1-73D9    

                                       

#fdisk -l
Disk /dev/sda: 465.76 GiB, 500107862016 bytes, 976773168 sectors
Disk model: WDC WD5000BPKT-7
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 4A3F293A-E4E4-426C-8A98-63EF246B3572

Device         Start       End   Sectors   Size Type
/dev/sda1       2048    197359    195312  95.4M EFI System
/dev/sda2     197360 968484863 968287504 461.7G Linux filesystem
/dev/sda3  968484864 976773119   8288256     4G Linux swap


Disk /dev/mmcblk0: 28.97 GiB, 31104958464 bytes, 60751872 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000

Device         Boot Start      End  Sectors Size Id Type
/dev/mmcblk0p1       8192 60751871 60743680  29G  c W95 FAT32 (LBA)


Disk /dev/sdb: 15.12 GiB, 16231956480 bytes, 31703040 sectors
Disk model: Flash Disk      
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: C995DDEE-EDC8-4110-A909-934C64E508A0

Device        Start      End  Sectors  Size Type
/dev/sdb1      2048 23314378 23312331 11.1G Microsoft basic data
/dev/sdb2  23314379 31702954  8388576    4G Microsoft basic data


Disk /dev/loop0: 559.54 MiB, 586719232 bytes, 1145936 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

                                       

#btrfs subvolume list /
ID 263 gen 283278 top level 5 path var/lib/portables
ID 264 gen 283278 top level 5 path var/lib/machines

#btrfs subvolume get-default /
ID 5 (FS_TREE)

                                       

#cat /etc/fstab
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
#
UUID=7D00-EE5F 								/boot	vfat 	defaults 0 2
UUID=8ad9efa0-fe54-4f2b-afe2-4350d03f0ee0 	/ 		btrfs 	defaults,noatime,compress=zstd,commit=120,subvolid=5 0 0
UUID=161416a5-b37f-4702-8fca-9eea04af5416 	none 	swap 	defaults 0 0

Full /boot/grub/grub.cfg
Current grub commandlines :

#cat /boot/grub/grub.cfg | grep linux
### BEGIN /etc/grub.d/10_linux ###
menuentry 'Arch Linux' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-8ad9efa0-fe54-4f2b-afe2-4350d03f0ee0' {
	echo	'Loading Linux linux ...'
	linux	/vmlinuz-linux root=UUID=8ad9efa0-fe54-4f2b-afe2-4350d03f0ee0 rw loglevel=3 resume=UUID=161416a5-b37f-4702-8fca-9eea04af5416 systemd.journald.forward_to_console=1 console=tty1 systemd.debug_shell
	initrd	/intel-ucode.img /initramfs-linux.img
submenu 'Advanced options for Arch Linux' $menuentry_id_option 'gnulinux-advanced-8ad9efa0-fe54-4f2b-afe2-4350d03f0ee0' {
	menuentry 'Arch Linux, with Linux linux' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-linux-advanced-8ad9efa0-fe54-4f2b-afe2-4350d03f0ee0' {
		echo	'Loading Linux linux ...'
		linux	/vmlinuz-linux root=UUID=8ad9efa0-fe54-4f2b-afe2-4350d03f0ee0 rw  loglevel=3 resume=UUID=161416a5-b37f-4702-8fca-9eea04af5416
		initrd	/intel-ucode.img /initramfs-linux.img
	menuentry 'Arch Linux, with Linux linux (fallback initramfs)' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-linux-fallback-8ad9efa0-fe54-4f2b-afe2-4350d03f0ee0' {
		echo	'Loading Linux linux ...'
		linux	/vmlinuz-linux root=UUID=8ad9efa0-fe54-4f2b-afe2-4350d03f0ee0 rw  loglevel=3 resume=UUID=161416a5-b37f-4702-8fca-9eea04af5416
		initrd	/initramfs-linux-fallback.img
	menuentry 'Arch Linux, with Linux linux (recovery mode)' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-linux-recovery-8ad9efa0-fe54-4f2b-afe2-4350d03f0ee0' {
		echo	'Loading Linux linux ...'
		linux	/vmlinuz-linux root=UUID=8ad9efa0-fe54-4f2b-afe2-4350d03f0ee0 rw single 
		initrd	/initramfs-linux-fallback.img
### END /etc/grub.d/10_linux ###
### BEGIN /etc/grub.d/20_linux_xen ###
### END /etc/grub.d/20_linux_xen ###

Initramfs config :

#cat /etc/mkinitcpio.conf
MODULES=()

BINARIES=()

FILES=()

HOOKS=(base systemd autodetect modconf block filesystems keyboard resume fsck)

OpenSSL files :

#ls -l /usr/lib/libcrypt*
-rwxr-xr-x 1 root root   43264 Oct 14 17:00 /usr/lib/libcrypt-2.32.so
lrwxrwxrwx 1 root root      17 Aug 27 09:25 /usr/lib/libcrypt.so -> libcrypt.so.2.0.0
lrwxrwxrwx 1 root root      16 Oct 14 17:00 /usr/lib/libcrypt.so.1 -> libcrypt-2.32.so
lrwxrwxrwx 1 root root      17 Aug 27 09:25 /usr/lib/libcrypt.so.2 -> libcrypt.so.2.0.0
-rwxr-xr-x 1 root root  186112 Aug 27 09:25 /usr/lib/libcrypt.so.2.0.0
lrwxrwxrwx 1 root root      16 Dec 12 06:34 /usr/lib/libcrypto.so -> libcrypto.so.1.1
-rwxr-xr-x 1 root root 2986744 Dec 12 06:34 /usr/lib/libcrypto.so.1.1
lrwxrwxrwx 1 root root      20 May 18  2020 /usr/lib/libcryptopp.so -> libcryptopp.so.8.2.0
lrwxrwxrwx 1 root root      20 May 18  2020 /usr/lib/libcryptopp.so.8 -> libcryptopp.so.8.2.0
-rwxr-xr-x 1 root root 5083048 May 18  2020 /usr/lib/libcryptopp.so.8.2.0
lrwxrwxrwx 1 root root      23 Sep  3 20:27 /usr/lib/libcryptsetup.so -> libcryptsetup.so.12.6.0
lrwxrwxrwx 1 root root      23 Sep  3 20:27 /usr/lib/libcryptsetup.so.12 -> libcryptsetup.so.12.6.0
-rwxr-xr-x 1 root root  482456 Sep  3 20:27 /usr/lib/libcryptsetup.so.12.6.0
lrwxrwxrwx 1 root root      19 Mar  8  2020 /usr/lib/libcryptui.so -> libcryptui.so.0.0.0
lrwxrwxrwx 1 root root      19 Mar  8  2020 /usr/lib/libcryptui.so.0 -> libcryptui.so.0.0.0
-rwxr-xr-x 1 root root   71368 Mar  8  2020 /usr/lib/libcryptui.so.0.0.0

#lddtree /usr/lib/libcrypto.so.1.1
/usr/lib/libcrypto.so.1.1 (interpreter => None)
    libdl.so.2 => /usr/lib/libdl.so.2
        ld-linux-x86-64.so.2 => /usr/lib/ld-linux-x86-64.so.2
    libpthread.so.0 => /usr/lib/libpthread.so.0
    libc.so.6 => /usr/lib/libc.so.6

Last edited by B3l3tte (2020-12-19 16:41:11)

Offline

#2 2020-12-19 15:12:14

Scimmia
Fellow
Registered: 2012-09-01
Posts: 11,539

Re: [SOLED] Early kernel panic, another "cannot load libcrypto.so" problem

Just to be sure, when you regenerated the initramfs from USB, you mounted both the root and boot partition, then arch-chrooted in, then regenerated it?

Post the entire grub config

Last edited by Scimmia (2020-12-19 15:13:20)

Offline

#3 2020-12-19 15:23:57

B3l3tte
Member
Registered: 2016-10-26
Posts: 14

Re: [SOLED] Early kernel panic, another "cannot load libcrypto.so" problem

Yes !
Here is my routine when booting the live key :

#loadkeys fr
#mount -o compress=zstd /dev/sda2 /mnt
#mount /dev/sda1 /mnt/boot
#arch-chroot /mnt

Full grub.cfg (edited OP as well)

Offline

#4 2020-12-19 15:54:57

seth
Member
Registered: 2012-09-03
Posts: 50,926

Re: [SOLED] Early kernel panic, another "cannot load libcrypto.so" problem

lsinitcpio /mnt/boot/initramfs-linux.img | grep crypto

When you run mkinitcpio from the arch-chroot, are there any error messages?

Online

#5 2020-12-19 16:23:48

B3l3tte
Member
Registered: 2016-10-26
Posts: 14

Re: [SOLED] Early kernel panic, another "cannot load libcrypto.so" problem

SOLVED, great thank you to you all smile

#lsinitcpio /boot/initramfs-linux.img | grep crypto
home/belette/.local/share/Steam/steamapps/common/HeroSiege/linux/libcrypto.so.1.1
usr/lib/libcrypt.so.2
usr/lib/libgcrypt.so.20

I actually had a copy of libcrypto inside a steam game folder for it to launch (see problems like this one).
Now I remember that I had a really hard time launching this game. The only thing that I tried that worked was to make a /etc/ld.so.conf.d/ file to add this copy to the ldconfig library list.
I removed the /etc/ld.so.conf.d/ file, reran ldconfig and regenerated initramfs.

Now :

#lsinitcpio /boot/initramfs-linux.img | grep crypto
usr/lib/libcrypto.so.1.1
usr/lib/libcrypt.so.2
usr/lib/libgcrypt.so.20

The system is booting correctly. Holly crap.

Offline

#6 2020-12-19 16:27:57

Scimmia
Fellow
Registered: 2012-09-01
Posts: 11,539

Re: [SOLED] Early kernel panic, another "cannot load libcrypto.so" problem

I've been around here a long time and this is a first for me. That one is...wow. Never would have guessed.

Last edited by Scimmia (2020-12-19 16:28:28)

Offline

Board footer

Powered by FluxBB