[SOLVED] Two kernels interfering

rcorre · 2017-02-01 02:41:31

I may have created a setup too complicated for my own good here...

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0 232.9G  0 disk 
├─sda1   8:1    0   512M  0 part /boot
├─sda2   8:2    0  31.5G  0 part /
├─sda3   8:3    0    88G  0 part /home
├─sda4   8:4    0     4G  0 part [SWAP]
├─sda5   8:5    0 108.5G  0 part 
└─sda6   8:6    0   450M  0 part 
sdb      8:16   0   477G  0 disk 
└─sdb1   8:17   0   477G  0 part

sda2-4 belong to my 'home' Archlinux.
sdb1 is an encrypted partition for my 'work' Archlinux.
sda1 is an EFI system partition, mounted as /boot for both installs.

This was working fine for a while, but after installing 4.9 on my work env, I can no longer boot into my home env:

Warning: /lib/modules/4.8.13-1-ARCH/modules.devname not found
...
ERROR: device '/dev/sda2 not found. Skipping fsck.
...

I booted back into my work install, rolled back the kernel to 4.8.13, and recompiled. I successfully rebooted in to my home environment only to find that now I could no longer boot into my work environment, because it could not load the encrypt hook.

Its worth noting that as I bounce between 4.8.13 and 4.9 each kernel recompile usually fails with something like:

/lib/modules/4.8.13-1-ARCH is not a valid kernel module directory

which I solve by symlinking /lib/modules/4.9-1-ARCH to /lib/modules/4.8.13-1-ARCH. I assume that's not a good idea, but its the only one I had.

In theory, my home kernel lives in /boot and my work kernel lives in /boot/here-img. In practice, I assume I screwed something up so they're interfering. Here's what my boot entries look like:

title 	Arch Linux
linux 	/vmlinuz-linux
initrd 	/intel-ucode.img
initrd 	/initramfs-linux.img
options root=/dev/sda2 rw

title 	HERE Arch Linux
linux 	/here-img/vmlinuz-linux
initrd 	/here-img/intel-ucode.img
initrd 	/here-img/initramfs-linux.img
options cryptdevice=/dev/sdb1:cryptroot root=/dev/mapper/cryptroot rw

All the paths look correct when building the kernel from either OS. Any idea what I might be doing wrong here?

Related: https://bbs.archlinux.org/viewtopic.php?id=220660

Last edited by rcorre (2017-02-05 20:20:29)

rcorre · 2017-02-02 14:26:58

I'm using gummiboot, if that matters. Right now I have to recompile the kernel every time I want to switch between my work and home install.

Trilby · 2017-02-02 15:17:22

Are you compiling your own kernel, or using the arch package? This all looks like you are using the kernel from the repos, but you twice refer to compiling and building the kernel. If you are building your own, are you using the ABS or traditional method?

If you are using repo packages, (how) are you getting updated kernels into /boot/here-img/ rather than /boot/ which is where pacman would place them?

I can't help but suspect you just manually moved or copied a kernel from /boot to /boot/here-img, but on the next update, pacman overwrote the kernel in /boot. If you want pacman to place the kernel in /boot/here-img you need to modify the package.

rcorre · 2017-02-02 16:40:54

Sorry for the confusion, I'm using the arch package in both cases.

I think the only thing I changed to handle the kernel locations was
/boot/loaders/entries. However, I thought it was placing the kernels in the
correct location because this is what I see when I install the kernel on my
work installation:

==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
  -> -k /boot/here-img/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/here-img/initramfs-linux.img
==> Starting build: 4.8.13-1-ARCH
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [encrypt]
  -> Running build hook: [lvm2]
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/here-img/initramfs-linux.img
==> Image generation successful

The access times confirm that only the images in /boot/here-img are getting
modified, the top-level images in /boot are untouched. However, in order to get the above to work I do need to symlink the
modules directory as I mentioned above, otherwise I see:

==> ERROR: '/lib/modules/4.8.13-1-ARCH' is not a valid kernel module directory

I just realized it says it is building 4.8.13, even though I'm installing the
package for 4.9.6. Why is it saying this, and why is it looking for
'/lib/modules/4.8.13-1-ARCH'?

Trilby · 2017-02-02 16:58:14

Is that mkinitcpio output from pacman, or did you run it manually? It if is from pacman (or likely in either case) you must have modified /etc/mkinitcpio/linux.preset to point to where the initramfs will be saved. But pacman will still place the new kernel (vmlinuz-linux) in /boot/.

As to symlinking module directories, there are so many problems with that that trying to figure out exactly why it leads to undersirable results seems a bit pointless. Just don't symlink the modules or module directories.

You need to go back to the beginning. Despite Scimmia suggesting a subdirectory of /boot/ as one of the viable options, simply creating the subdir is not sufficient. You will run into problems exactly like those described in this thread because pacman is putting the new kernel in /boot/. Of the two options Scimmia mentioned the first of using different kernel names is the only one that would occur to me and is far mode advisable in my opinion.

Actually, I suspect the "need" to symlink the module directory is due to the fact that your bootloader was pointing to a kernel in /boot/subdir/ but the new 4.9 kernel was installed in /boot/ so when you booted to the work install the older kernel was loaded from /boot/subdir but the new modules were used in building the initramfs.

You can confirm what the kernel versions are with `file`:

file /boot/vmlinuz-linux
file /boot/<subdir>/vmlinuz-linux

And `lsinitcpio` can be used on the initramfs images so see what kernel version modules are in it.

rcorre · 2017-02-02 17:08:21

That output is from pacman, but I believe I've tried it with mkinitcpio as well.
I'm sure symlinking is a bad idea, but it's the only way I can get things working for now.
mkdinitcpi/linux.preset points to /boot/here-img, I can't remember if I modified it or it was generated like that.

I can try using different kernel names instead of directories, but I don't really understand the difference. I don't think it is putting the new kernel in /boot, because I ran `ls -l` before and after `pacman`, and the access time of /boot/vmlinuz-linux is not changing while that of /boot/here-img/vmlinuz-linux is.

Trilby · 2017-02-02 17:14:09

If you are using the linux package from the repos then it is absolutely placing the kernel in /boot/. This is not a configurable option, that package contains the file /boot/vmlinuz-linux, not /boot/here-img/vmlinuz-linux. The initramfs is generated after the package installed and is placed according to your config file (which puts the img in your sub directory) - but the kernel itself ever only gets installed to /boot/. Period.

The output of `ls -l` is irrelevant. That tells you when the file was modified (e.g. when the kernel was built) not when it was installed. Use the `file` command as I suggested.

rcorre · 2017-02-02 18:05:32

Yup, you're right. Guess I need to start installing linux from the ABS now.

Is there any better way to manage two Arch installs on EFI? A way that doesn't involve installing the kernel manually from now on?
Everything I've read seems to suggest you don't want multiple EFI partitions, and you _do_ want your EFI partition mounted as /boot.
So it seems like I have to share the boot partition and the only way to change the kernel location is to modify the PKGBUILD.

I'm not complaining, just curious. I do appreciate the help, I think this was the answer I needed (trying it now).

Trilby · 2017-02-02 18:12:14

Do you need two different kernel versions? If it were me, I'd let one system manage the kernel, and just change the root partition in the boot loader. In your case, you'd likely want two mkinitcpio presets to create two separate initramfs images, one for the normal and one for the encrypted root. You could probably also have a single more general purpose initramfs that'd handle either setup - but two initramfs images with different names is trivial: just add a preset configuration.

Another simple alternative would be to use another ready-made kernel for one of the installs. Linux-lts is in the main repos, or you could use GraySky's linux-ck repo. As each of these has different name for the kernel, there'd be no issue having different kernels placed side-by-side in the same efi partition. There are also several other custom repos with prebuilt kernel variations.

rcorre · 2017-02-02 18:46:53

I was going based on this from my previous thread:

Scimmia wrote:

You can't have both of them using the same kernel image/initramfs

I don't think I actually need two different kernels though.
Good idea about using a different prebuilt kernel as well.

I'll report back with whatever ends up working. There seems to be pretty slim information out there regarding multiple linux installs on EFI.

Trilby · 2017-02-02 18:57:15

You can't have two independent installs each with pacman trying to manage the same kernel file. But you can use the same kernel with different root partitions quite easily. If it weren't for the encryption this would be absolutely trivial. The use of encryption on one but not the other just requires that you ensure *all* the needed modules for both root partitions are build in to the initramfs. This is easy to do, just you must configure the preset yourself.

rcorre · 2017-02-02 20:01:54

Actually, if I just let one system manage the kernel, how will /usr/lib/modules get updated in the other system? Does it matter? Is /usr/lib/modules only needed for generating the image?

Trilby · 2017-02-02 20:11:43

Perhaps someone else can comment on that. I've not done it myself, but allegedly, you should be able to boot into a root partition with no kernel installed within it. Perhaps this may only have limited functionality as modules could not be loaded after boot (so removable devices might not be functional).

positronik · 2017-02-02 21:20:47

I was wondering whether in this situation would be better to have another EFI partition in /dev/sdb and then select which archlinux install to boot via BIOS menu, or is there something wrong in this idea?

rcorre · 2017-02-03 14:31:01

I might have to try a second EFI partition. I'll have to figure out how to safely resize an encrypted partition though.

I tried installing an alternate kernel, but neither linux-lts nor linux-zen 'provide' linux, so it was breaking an nvidia dependency.
I ended up building a renamed kernel from ABS and added a 'provides' line, but after installing that I couldn't connect to the X server, even after rebuilding the nvidia module and rebooting:

[   178.978] 
X.Org X Server 1.19.1
Release Date: 2017-01-11
[   178.979] X Protocol Version 11, Revision 0
[   178.980] Build Operating System: Linux 4.8.14-1-ARCH x86_64 
[   178.980] Current Operating System: Linux heredesk 4.9.6-1-custom #1 SMP PREEMPT Thu Feb 2 18:03:19 EST 2017 x86_64
[   178.980] Kernel command line: initrd=\intel-ucode.img initrd=\initramfs-linux-custom.img cryptdevice=/dev/sdb1:cryptroot root=/dev/mapper/cryptroot rw
[   178.982] Build Date: 12 January 2017  07:11:40AM
[   178.982]  
[   178.983] Current version of pixman: 0.34.0
[   178.984] 	Before reporting problems, check http://wiki.x.org
	to make sure that you have the latest version.
[   178.984] Markers: (--) probed, (**) from config file, (==) default setting,
	(++) from command line, (!!) notice, (II) informational,
	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
[   178.986] (==) Log file: "/home/rodencor/.local/share/xorg/Xorg.0.log", Time: Fri Feb  3 09:15:29 2017
[   178.986] (==) Using system config directory "/usr/share/X11/xorg.conf.d"
[   178.986] (==) No Layout section.  Using the first Screen section.
[   178.986] (==) No screen section available. Using defaults.
[   178.986] (**) |-->Screen "Default Screen Section" (0)
[   178.986] (**) |   |-->Monitor "<default monitor>"
[   178.986] (==) No monitor specified for screen "Default Screen Section".
	Using a default monitor configuration.
[   178.986] (==) Automatically adding devices
[   178.986] (==) Automatically enabling devices
[   178.986] (==) Automatically adding GPU devices
[   178.986] (==) Max clients allowed: 256, resource mask: 0x1fffff
[   178.986] (WW) The directory "/usr/share/fonts/Type1/" does not exist.
[   178.986] 	Entry deleted from font path.
[   178.986] (WW) `fonts.dir' not found (or not valid) in "/usr/share/fonts/100dpi/".
[   178.986] 	Entry deleted from font path.
[   178.986] 	(Run 'mkfontdir' on "/usr/share/fonts/100dpi/").
[   178.986] (WW) `fonts.dir' not found (or not valid) in "/usr/share/fonts/75dpi/".
[   178.986] 	Entry deleted from font path.
[   178.986] 	(Run 'mkfontdir' on "/usr/share/fonts/75dpi/").
[   178.986] (==) FontPath set to:
	/usr/share/fonts/misc/,
	/usr/share/fonts/TTF/,
	/usr/share/fonts/OTF/
[   178.986] (==) ModulePath set to "/usr/lib/xorg/modules"
[   178.986] (II) The server relies on udev to provide the list of input devices.
	If no devices become available, reconfigure udev or disable AutoAddDevices.
[   178.986] (II) Loader magic: 0x821d60
[   178.986] (II) Module ABI versions:
[   178.986] 	X.Org ANSI C Emulation: 0.4
[   178.986] 	X.Org Video Driver: 23.0
[   178.986] 	X.Org XInput driver : 24.1
[   178.986] 	X.Org Server Extension : 10.0
[   178.987] (++) using VT number 1

[   178.987] (--) controlling tty is VT number 1, auto-enabling KeepTty
[   178.988] (II) systemd-logind: took control of session /org/freedesktop/login1/session/c1
[   178.988] (II) xfree86: Adding drm device (/dev/dri/card0)
[   178.989] (II) systemd-logind: got fd for /dev/dri/card0 226:0 fd 11 paused 0
[   178.989] (--) PCI: (0:0:2:0) 8086:0412:1458:d000 rev 6, Mem @ 0xf7400000/4194304, 0xd0000000/268435456, I/O @ 0x0000f000/64
[   178.989] (--) PCI:*(0:1:0:0) 10de:13c2:1043:8508 rev 161, Mem @ 0xf6000000/16777216, 0xe0000000/268435456, 0xf0000000/33554432, I/O @ 0x0000e000/128, BIOS @ 0x????????/131072
[   178.990] (WW) Open ACPI failed (/var/run/acpid.socket) (No such file or directory)
[   178.990] (II) LoadModule: "glx"
[   178.990] (II) Loading /usr/lib/xorg/modules/extensions/libglx.so
[   178.992] (II) Module glx: vendor="NVIDIA Corporation"
[   178.993] 	compiled for 4.0.2, module version = 1.0.0
[   178.993] 	Module class: X.Org Server Extension
[   178.993] (II) NVIDIA GLX Module  375.26  Thu Dec  8 17:59:51 PST 2016
[   178.993] (==) Matched intel as autoconfigured driver 0
[   178.993] (==) Matched nouveau as autoconfigured driver 1
[   178.993] (==) Matched nv as autoconfigured driver 2
[   178.993] (==) Matched modesetting as autoconfigured driver 3
[   178.993] (==) Matched fbdev as autoconfigured driver 4
[   178.993] (==) Matched vesa as autoconfigured driver 5
[   178.993] (==) Assigned the driver to the xf86ConfigLayout
[   178.993] (II) LoadModule: "intel"
[   178.993] (WW) Warning, couldn't open module intel
[   178.993] (II) UnloadModule: "intel"
[   178.993] (II) Unloading intel
[   178.993] (EE) Failed to load module "intel" (module does not exist, 0)
[   178.993] (II) LoadModule: "nouveau"
[   178.993] (WW) Warning, couldn't open module nouveau
[   178.993] (II) UnloadModule: "nouveau"
[   178.993] (II) Unloading nouveau
[   178.993] (EE) Failed to load module "nouveau" (module does not exist, 0)
[   178.993] (II) LoadModule: "nv"
[   178.993] (WW) Warning, couldn't open module nv
[   178.993] (II) UnloadModule: "nv"
[   178.993] (II) Unloading nv
[   178.993] (EE) Failed to load module "nv" (module does not exist, 0)
[   178.993] (II) LoadModule: "modesetting"
[   178.993] (II) Loading /usr/lib/xorg/modules/drivers/modesetting_drv.so
[   178.993] (II) Module modesetting: vendor="X.Org Foundation"
[   178.993] 	compiled for 1.19.1, module version = 1.19.1
[   178.993] 	Module class: X.Org Video Driver
[   178.993] 	ABI class: X.Org Video Driver, version 23.0
[   178.993] (II) LoadModule: "fbdev"
[   178.993] (WW) Warning, couldn't open module fbdev
[   178.993] (II) UnloadModule: "fbdev"
[   178.993] (II) Unloading fbdev
[   178.993] (EE) Failed to load module "fbdev" (module does not exist, 0)
[   178.993] (II) LoadModule: "vesa"
[   178.993] (WW) Warning, couldn't open module vesa
[   178.993] (II) UnloadModule: "vesa"
[   178.993] (II) Unloading vesa
[   178.993] (EE) Failed to load module "vesa" (module does not exist, 0)
[   178.993] (II) modesetting: Driver for Modesetting Kernel Drivers: kms
[   178.993] (II) modeset(G0): using drv /dev/dri/card0
[   178.993] (EE) No devices detected.
[   178.993] (EE) 
Fatal server error:
[   178.993] (EE) no screens found(EE) 
[   178.993] (EE) 
Please consult the The X.Org Foundation support 
	 at http://wiki.x.org
 for help. 
[   178.993] (EE) Please also check the log file at "/home/rodencor/.local/share/xorg/Xorg.0.log" for additional information.
[   178.993] (EE) 
[   178.996] (EE) Server terminated with error (1). Closing log file.

The 'Build Operating System' and 'Current operating System' don't match, and it fails to open a bunch of modules (I did update the entry in /boot/loader/entries to point to the alternate-named vmlinuz and initramfs).

Trilby · 2017-02-03 15:04:14

If you are going to use the lts kernel, you need the use the nvidia-lts package (also in the repos).

Similarly repo-ck has ck kernels and nvidia-ck packages.

rcorre · 2017-02-05 20:20:10

Using linux-lts on the second install is working so far. I just removed intel-ucode from the second install because I wasn't sure if that would interfere between the two.
Maybe I'll try dual efi partitions at some point, but this works for now. Thanks!

Slithery · 2017-02-05 20:47:43

You should reinstate microcode updates on your second install. They're only applied when you boot.

Trilby · 2017-02-05 20:49:08

Slithery, having the package installed on both systems would cause a file conflict. It only needs to be installed on one. It should be applied in both boot loader entries.

Arch Linux

#1 2017-02-01 02:41:31

[SOLVED] Two kernels interfering

#2 2017-02-02 14:26:58

Re: [SOLVED] Two kernels interfering

#3 2017-02-02 15:17:22

Re: [SOLVED] Two kernels interfering

#4 2017-02-02 16:40:54

Re: [SOLVED] Two kernels interfering

#5 2017-02-02 16:58:14

Re: [SOLVED] Two kernels interfering

#6 2017-02-02 17:08:21

Re: [SOLVED] Two kernels interfering

#7 2017-02-02 17:14:09

Re: [SOLVED] Two kernels interfering

#8 2017-02-02 18:05:32

Re: [SOLVED] Two kernels interfering

#9 2017-02-02 18:12:14

Re: [SOLVED] Two kernels interfering

#10 2017-02-02 18:46:53

Re: [SOLVED] Two kernels interfering

#11 2017-02-02 18:57:15

Re: [SOLVED] Two kernels interfering

#12 2017-02-02 20:01:54

Re: [SOLVED] Two kernels interfering

#13 2017-02-02 20:11:43

Re: [SOLVED] Two kernels interfering

#14 2017-02-02 21:20:47

Re: [SOLVED] Two kernels interfering

#15 2017-02-03 14:31:01

Re: [SOLVED] Two kernels interfering

#16 2017-02-03 15:04:14

Re: [SOLVED] Two kernels interfering

#17 2017-02-05 20:20:10

Re: [SOLVED] Two kernels interfering

#18 2017-02-05 20:47:43

Re: [SOLVED] Two kernels interfering

#19 2017-02-05 20:49:08

Re: [SOLVED] Two kernels interfering

Board footer