Can I compile a kernel in a VM?

bundito · 2019-02-23 19:38:52

Hi all. Relative newbie here. I've already got one custom kernel under my belt. I expected it, but it took hours to compile on my puny Core i5 machine.

In my basement is a Ubuntu machine that's not doing much but serving media. It's an AMD 12(?)-core machine with 16GB of RAM and a speedy SSD. I'm thinking it could come in useful for compiling kernels - but I'm pretty damned sure I can't compile an Arch Kernel natively on Ubuntu.

So I'm thinking about a VM. I might build one on the quick with Anarchy or Zen Installer.

Will I be able to compile a viable kernel this way? I figure I can let the box downstairs chew away while I keep using my laptop.

For the record, I'm trying various options trying to overcome a 'reboot hang' problem. Some have said it's an ACPI problem, others have said it's a crypto option in the kernel, others have said I need to sacrifice a rooster. This machine was built for Windows and has a BIOS with very limited options (no ACPI parameters, for example).

So, will this work? Or should I save my compiles for the laptop and kick them off and put on a movie instead?

Cheers.

-bundito

graysky · 2019-02-23 21:25:13

I have no experience with Anarchy or Zen, just VirtualBox... can you use lxc and compile in Arch on Ubuntu? Next question that comes to my mind is if you're just serving up files (NFS or Samba) why not do so from an Arch box? What does Ubuntu offer you that Arch does not?

Last edited by graysky (2019-02-23 21:26:04)

bundito · 2019-02-23 21:36:22

graysky wrote:

I have no experience with Anarchy or Zen, just VirtualBox... can you use lxc and compile in Arch on Ubuntu? Next question that comes to my mind is if you're just serving up files (NFS or Samba) why not do so from an Arch box? What does Ubuntu offer you that Arch does not?

The box is running Ubuntu just due to history. It's where I started and I've never rebuilt it with anything else. Simple laziness. I used to do all my Linux'ing from a MacBook using NoMachine to do it remotely. My basement (and my microwave cart full of equipment) isn't very well suited to extended periods of direct use.

Or maybe I'll try running a CK kernel... heh.

I'll keep plugging away.

Thanks.

Trilby · 2019-02-23 21:45:45

bundito wrote:

but I'm pretty damned sure I can't compile an Arch Kernel natively on Ubuntu.

Why? You should certainly be able to compile on Ubuntu just fine.

As for it taking that long, is this something you'd distribute, or is it just for your own use? If it's just for your own use, you should trim your config to build just what you use. Using a trimmed config - based largely on the result of `make localmodconfig` - I compiled a current kernel a couple weeks ago in about 30 minutes on a core2duo with 4GB of ram. Your 12 cores with 16GB of ram should beat the hell out of that.

graysky · 2019-02-23 21:45:47

...it's less work to put a basic Arch install (basic meaning you just need a file server which is trivial to setup) on a new machine than it is to even begin researching VMs

progandy · 2019-02-23 21:50:25

There is an archlinux docker container called archlinux/base available, maybe test compiling it with that.

Last edited by progandy (2019-02-23 21:51:19)

bundito · 2019-02-23 21:56:12

Maybe I should try recompiling and use `localmodconfig` to pare things down. It's just for me, trying to resolve this damned hang-on-reboot problem.

I'll ask elsewhere about that problem.

Thanks, all. I'll keep you posted.

-bundito

progandy · 2019-02-23 22:04:33

If you use localmodconfig for more than debugging, then take care to include modules for usb drives and such. They might not be loaded if you haven't plugged one in since the last reboot. That is probably not an issue while you are debugging, but it might annoying during regular use.

In the future use modprobed-db to collect the loaded modules during normal usage, then you are always ready to compile with localmodconfig.

Last edited by progandy (2019-02-23 22:07:37)

Lone_Wolf · 2019-02-24 11:03:28

You could also try DistCC .
The arch box would be master while the ubuntu box would be one of the slaves.

eschwartz · 2019-02-24 18:58:08

https://wiki.archlinux.org/index.php/In … ting_Linux

You can install Arch to a small partition and then use systemd-nspawn to enter the container, which is a lot less fragile than virtual machines. Systemd is the default on Ubuntu since 16.04 if I remember correctly. Alternatively, keep using chroot as suggested in the wiki guide.

Once inside of a minimal Arch container, you can simply run makepkg using the kernel PKGBUILD you are trying to build. You can make use of a manually copied kernel .config from your first localmodconfig exercise on your primary (Arch) computer.

...

In the future, I'd like it to be possible to natively build Arch containers on Debian/Ubuntu. This is already possible on Gentoo and Fedora which provide the pacman command... Jelle is trying to get pacman packaged for Debian too: https://github.com/jelly/pacman-deb

bundito · 2019-02-25 20:25:03

progandy wrote:

In the future use modprobed-db to collect the loaded modules during normal usage, then you are always ready to compile with localmodconfig.

I don't want to get too far off topic, but can you point me to some directions on how to use modprobed-db? I got it to collect some of my running modules, but I don't understand how to use the "recall" option - or some other option - into a config file for compiling a kernel. I'm compiling the traditional way, by the way.

Thanks!

Trilby · 2019-02-25 20:26:33

Read the link. You don't "recall into a config", you just use the recall command to load all the modules, then `make localmodconfig` will detect them.

bundito · 2019-02-25 20:29:08

Sorry for the newbie question. I'll RTFM more carefully. I was trying this at the end of a long day yesterday.

Apologies.

Head_on_a_Stick · 2019-02-25 21:10:01

eschwartz wrote:

use systemd-nspawn to enter the container

Note that the `systemd-nspawn` command is supplied separately by the systemd-container package in Ubuntu & Debian.

Debian also has `arch-chroot` in buster now:

https://packages.debian.org/buster/arch-install-scripts

The package has no dependencies so it should be possible to install the .deb in an Ubuntu system.

eschwartz wrote:

I'd like it to be possible to natively build Arch containers on Debian/Ubuntu

I just unpack one of the Arch bootstrap images when I want an Arch container in my Debian system.

eschwartz · 2019-02-25 21:26:48

Head_on_a_Stick wrote:

eschwartz wrote:
use systemd-nspawn to enter the container
Note that the `systemd-nspawn` command is supplied separately by the systemd-container package in Ubuntu & Debian.
Debian also has `arch-chroot` in buster now:
https://packages.debian.org/buster/arch-install-scripts
The package has no dependencies so it should be possible to install the .deb in an Ubuntu system.

It does not contain pacstrap, and the "arch-chroot" script should be renamed to debian-chroot as it contains assorted patches geared to the highly specific needs of Debian.

Trilby · 2019-02-26 01:13:00

So this whole thread seems to be about how to get arch vms or containers running on ubuntu all to satisfy the assumption that an arch kernel cannot be built in ubuntu ... but that initial assumption is false.

Is there any point in using a container/vm/etc? If all you need is to compile a kernel, you just need the kernel sources, any arch-specific patches, an appropriate kernel config, and a compiler. Nothing about that process requires an arch system containerized or otherwise.

eschwartz · 2019-02-26 01:35:25

Trilby wrote:

Is there any point in using a container/vm/etc? If all you need is to compile a kernel, you just need the kernel sources, any arch-specific patches, an appropriate kernel config, and a compiler. Nothing about that process requires an arch system containerized or otherwise.

tar'ing it up as a pacman-compatible package rather than cp'ing a tree full of kernel image + module.ko files.

Trilby · 2019-02-26 01:40:06

Ah, good point. But if `make localmodconfig` is to be used anyways, that could be replaced with `make localyesconfig` to generate the .config in arch, then use that to compile in Ubuntu. Then there's only one file to cp over.

Head_on_a_Stick · 2019-02-26 19:40:27

eschwartz wrote:

Head_on_a_Stick wrote:
Debian also has `arch-chroot` in buster now:
https://packages.debian.org/buster/arch-install-scripts
The package has no dependencies so it should be possible to install the .deb in an Ubuntu system.
It does not contain pacstrap

Yes but a bootstrap image is perfectly suited for use as a container, instructions here for the benefit of the OP:

https://wiki.alpinelinux.org/wiki/Runni … Arch_Linux

^ That includes the commands needed to un-comment a mirrror and comment out the CheckSpace option in pacman.conf, which seem to be missing from the ArchWIki page that you linked earlier, pacman will not work otherwise.

eschwartz wrote:

the "arch-chroot" script should be renamed to debian-chroot as it contains assorted patches geared to the highly specific needs of Debian

https://www.debian.org/Bugs/Reporting

eschwartz · 2019-02-26 19:59:00

Head_on_a_Stick wrote:

Yes but a bootstrap image is perfectly suited for use as a container, instructions here for the benefit of the OP:
https://wiki.alpinelinux.org/wiki/Runni … Arch_Linux
^ That includes the commands needed to un-comment a mirrror and comment out the CheckSpace option in pacman.conf, which seem to be missing from the ArchWIki page that you linked earlier, pacman will not work otherwise.

Our wiki mentions the need to uncomment a mirror, but does not mention anything about CheckSpace, which I believe is because for the target environment -- a partitioned install -- it is not needed. Feel free to update it with a suitable explanation...

eschwartz wrote:
the "arch-chroot" script should be renamed to debian-chroot as it contains assorted patches geared to the highly specific needs of Debian
https://www.debian.org/Bugs/Reporting

Meh. Why do I care if *their* fork has terrible advertising for *their* purposes?

Head_on_a_Stick · 2019-02-26 20:02:14

eschwartz wrote:

Our wiki mentions the need to uncomment a mirror

Oh yes, so it does, my apologies.

eschwartz wrote:

but does not mention anything about CheckSpace, which I believe is because for the target environment -- a partitioned install -- it is not needed. Feel free to update it with a suitable explanation

No, you're right, it is not needed for the purposes of that page (but it is needed to set up a container).

eschwartz wrote:

Why do I care if *their* fork has terrible advertising for *their* purposes?

halasz · 2019-02-26 20:45:55

bundito wrote:

Will I be able to compile a viable kernel this way?

Sure you will be able to. You can build them anywhere. I build my kernels on a Hyper-V VM. They finish eventually although it takes more than 7 hours. I do find the makepkg -s will fail if you don't give it enough RAM.

[adam@Transmission trunk]$ makepkg -s
==> Making package: linux-custom 4.20.12.arch1-1 (Tue 26 Feb 2019 05:35:35 PM CET)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
  -> Cloning archlinux-linux git repo...
Cloning into bare repository '/mnt/sdb1/build/linux/trunk/archlinux-linux'...
remote: Enumerating objects: 375239, done.
remote: Counting objects: 100% (375239/375239), done.
remote: Compressing objects: 100% (86844/86844), done.
fatal: Out of memory, calloc failed
fatal: index-pack failed
==> ERROR: Failure while downloading archlinux-linux git repo
    Aborting...

I'm sure 512MB is insufficient and 4GB is excessive. Somewhere in-between is the cutoff...

Arch Linux

#1 2019-02-23 19:38:52

Can I compile a kernel in a VM?

#2 2019-02-23 21:25:13

Re: Can I compile a kernel in a VM?

#3 2019-02-23 21:36:22

Re: Can I compile a kernel in a VM?

#4 2019-02-23 21:45:45

Re: Can I compile a kernel in a VM?

#5 2019-02-23 21:45:47

Re: Can I compile a kernel in a VM?

#6 2019-02-23 21:50:25

Re: Can I compile a kernel in a VM?

#7 2019-02-23 21:56:12

Re: Can I compile a kernel in a VM?

#8 2019-02-23 22:04:33

Re: Can I compile a kernel in a VM?

#9 2019-02-24 11:03:28

Re: Can I compile a kernel in a VM?

#10 2019-02-24 18:58:08

Re: Can I compile a kernel in a VM?

#11 2019-02-25 20:25:03

Re: Can I compile a kernel in a VM?

#12 2019-02-25 20:26:33

Re: Can I compile a kernel in a VM?

#13 2019-02-25 20:29:08

Re: Can I compile a kernel in a VM?

#14 2019-02-25 21:10:01

Re: Can I compile a kernel in a VM?

#15 2019-02-25 21:26:48

Re: Can I compile a kernel in a VM?

#16 2019-02-26 01:13:00

Re: Can I compile a kernel in a VM?

#17 2019-02-26 01:35:25

Re: Can I compile a kernel in a VM?

#18 2019-02-26 01:40:06

Re: Can I compile a kernel in a VM?

#19 2019-02-26 19:40:27

Re: Can I compile a kernel in a VM?

#20 2019-02-26 19:59:00

Re: Can I compile a kernel in a VM?

#21 2019-02-26 20:02:14

Re: Can I compile a kernel in a VM?

#22 2019-02-26 20:45:55

Re: Can I compile a kernel in a VM?

Board footer