You are not logged in.

#1 2008-10-15 09:51:58

andrej.podzimek
Member
From: Zürich, Switzerland
Registered: 2005-04-10
Posts: 115

Kernel panic with vanilla kernel and mkinitcpio

I've been using a vanilla kernel together with mkinitcpio for ages. I have never encountered any severe problems like this one:

:: Loading Initramfs
/init: 36: replace: not found
/init: 36: replace: not found
export: 36: : bad variable name
Kernel panic - not syncing: Attempted to kill init!

This happened when I upgraded the kernel from 2.6.26.5 to 2.6.26.6. (Unbelievable...)
(BTW, the last two comments in http://bugs.archlinux.org/task/8832 seem to mention something similar to this problem.)

Tried to repeat the upgrade procedure, just to make sure I didn't forget anything important.
(Reiser4 patch, zcat /proc/config.gz ..., make oldconfig, make everything and istall modules.)
(That's what I do on three machines every week, so I don't think I made a serious mistake.)

Then I called

mkinitcpio -g /boot/myImage.img -c /path/to/mkinitcpio.conf -k `ls -l /usr/src/linux | cut -d'-' -f3,4`

std::answers {
        Yes, /usr/src/linux symlink is set correctly.
        Yes, I did run LILO afterwards and it completed successfully.
        Yes, the correct /boot partition was mounted.
        Yes, I checked twice that the program called „replace" was in the image.
)

There have been severe problems with klibc* packages recently. A piece of dependency hell... Now it seems that some klibc binaries are linked against different versions of the library. The output of prelink shows that clearly.

(...)
prelink: /usr/lib/klibc/bin/mkfifo: Using /lib/klibc-YKSdCBC5bWMDDPFRsTCT0bz4Wy4.so, not /lib/ld-linux.so.2 as dynamic linker
prelink: /usr/lib/klibc/bin/dd: Using /lib/klibc-YKSdCBC5bWMDDPFRsTCT0bz4Wy4.so, not /lib/ld-linux.so.2 as dynamic linker
prelink: /usr/lib/klibc/bin/ln: Using /lib/klibc-YKSdCBC5bWMDDPFRsTCT0bz4Wy4.so, not /lib/ld-linux.so.2 as dynamic linker
(...)
prelink: /usr/lib/klibc/bin/mdassemble: Using /lib/klibc-jfflyAahxqaliwAofrf_fdf5upI.so, not /lib/ld-linux.so.2 as dynamic linker
prelink: /usr/lib/klibc/bin/replace: Using /lib/klibc-jfflyAahxqaliwAofrf_fdf5upI.so, not /lib/ld-linux.so.2 as dynamic linker
prelink: /usr/lib/klibc/bin/parseblock: Using /lib/klibc-jfflyAahxqaliwAofrf_fdf5upI.so, not /lib/ld-linux.so.2 as dynamic linker
(...)

The (working) image for 2.6.26.5 contains /lib/klibc-tQg242GqA9bO1EKgzWDvsh1E7nc.so (Yeah, different from the two above!)
The (failing) image for 2.6.26.6 contains /lib/klibc-YKSdCBC5bWMDDPFRsTCT0bz4Wy4.so
There are quite many differences. Many files contained in one image are missing in the other one and vice versa.

This is the mkinitcpio.conf I use. FS and controller drivers are in the kernel. So are the LVM modules. (The lvm2 hook complains a bit, but that's no problem at all...) I only want the LVM volumes to be assembled. This is what always worked for me:

MODULES=""
BINARIES=""
FILES=""
HOOKS="base udev autodetect lvm2"

How could it happen that 2.6.26.5 works and 2.6.26.6 fails?
Is it possible that a dynamic linker failure (due to version mismatch) could result in the panic above, claiming that the binary is missing?
Now what should I do to make it work again?

Offline

#2 2008-10-15 21:53:07

andrej.podzimek
Member
From: Zürich, Switzerland
Registered: 2005-04-10
Posts: 115

Re: Kernel panic with vanilla kernel and mkinitcpio

Tried to recompile klibc (and edited PKGBUILD to set my kernel version string). This time I got a different kind of kernel panic, complainig about invalid format of the cpio archive. (Needless to say that the archive was created without error messages, it was readable by cpio and lilo had been run before reboot.)

Initcpio size should be ok. AFAIK, ramdisk size is set to 16 MB in my kernel config, whereas zcat /boot/myImage | wc -c says something like 1.7 MB. (Yes, that's pretty small (for my HOOKS line is pretty short) and it had always worked like that before.)

Well, this is a bad showstopper. I would like to update the kernel, since the chagelog from 2.6.26.5 to 2.6.26.6 is 81 kB long. But what shell I do to make it work?

Offline

#3 2008-10-16 07:24:14

brain0
Developer
From: Aachen - Germany
Registered: 2005-01-03
Posts: 1,382

Re: Kernel panic with vanilla kernel and mkinitcpio

andrej.podzimek wrote:

Tried to recompile klibc (and edited PKGBUILD to set my kernel version string). This time I got a different kind of kernel panic, complainig about invalid format of the cpio archive. (Needless to say that the archive was created without error messages, it was readable by cpio and lilo had been run before reboot.)

Do not try to recompile klibc, there is really much that can go wrong! To fix this error, just install the correct packages. Dependencies should ensure that all packages are correct, but apparently your klibc-extras package isn't. Did you use some evil -d tricks with pacman or just selectively installed some klibc packages instead of -Su'ing?

On x86_64:

[thomas@artin ~]$ pacman -Qi klibc|grep Provides
Provides       : klibc-088J8A5xVqZqloQrhjNKyTEVhY8
[09:21:42][thomas@artin ~]$ pacman -Qi $(pacman -Q | grep klibc | cut -d' ' -f1) | grep Depends
Depends On     : None
Depends On     : klibc  klibc-088J8A5xVqZqloQrhjNKyTEVhY8
Depends On     : klibc  klibc-088J8A5xVqZqloQrhjNKyTEVhY8  kbd
Depends On     : klibc  klibc-088J8A5xVqZqloQrhjNKyTEVhY8
Depends On     : coreutils  klibc  klibc-088J8A5xVqZqloQrhjNKyTEVhY8

On i686:

[thomas@artin ~]$ pacman -Qi klibc|grep Provides
Provides       : klibc-jfflyAahxqaliwAofrf_fdf5upI
[thomas@artin ~]$ pacman -Qi $(pacman -Q | grep klibc | cut -d' ' -f1) | grep Depends
Depends On     : None
Depends On     : klibc  klibc-jfflyAahxqaliwAofrf_fdf5upI
Depends On     : klibc  klibc-jfflyAahxqaliwAofrf_fdf5upI  kbd
Depends On     : klibc  klibc-jfflyAahxqaliwAofrf_fdf5upI
Depends On     : coreutils  klibc  klibc-jfflyAahxqaliwAofrf_fdf5upI

Initcpio size should be ok. AFAIK, ramdisk size is set to 16 MB in my kernel config, whereas zcat /boot/myImage | wc -c says something like 1.7 MB. (Yes, that's pretty small (for my HOOKS line is pretty short) and it had always worked like that before.)

This is no ramdisk, so ramdisk size doesn't matter. Our stock kernel doesn't even have built-in ramdisk support.

Offline

#4 2008-10-16 13:20:00

andrej.podzimek
Member
From: Zürich, Switzerland
Registered: 2005-04-10
Posts: 115

Re: Kernel panic with vanilla kernel and mkinitcpio

Reinstalling all the klibc* packages from the repository (and regenerating the image and running LILO) did not help. This time there were no complaints about missing files. It panicked due to the fact that the archive was corrupted and could not be read. At least that's what the last kernel messages were about. That's similar to what happens if you regenerate the image and forget to run LILO. But I did not forget to run it.

So I had to revert back to the previous kernel. I have always compiled nearly all the packages from ABS, including klibc, but have never seen such a problem before. Kernel updates were just a matter of rebooting the machine once...

Offline

Board footer

Powered by FluxBB