Clarification on how mkinitcpio.conf works.

B-Con · 2008-05-30 10:01:44

I've been looking at mkinitcpio, udev, etc, more in-depth for the first time as of late. I've read the pertinent wiki articles but I have some questions I'd like to know the answers to before I gallivant about pretending to understand how these things work...

1) Just what, exactly, *is* a hook for initramfs, as specified in mkinitcpio.conf?
2) What's the difference between modules that get loaded by mkinitcpio.conf and those that get loaded by rc.conf? They both end up in userland, correct?
2a) If putting modules in mkinitcpio.conf makes them somehow more accessible / faster to load for interaction with the kernel, what's the advantage to leaving modules you know you'll always need in the rc.conf modules list? Is it just to avoid the necessity of rebuilding the initramfs?
3) (Following from #1...) If udev is not included as a hook in mkinitcpio.conf, will udev do anything if MOD_AUTOLOAD is set to "yes"?
3a) What does udev even do as a hook? Given a the default empty modules list in mkinitcpio.conf, I assume udev is responsible for all modules outside of the MODULES array loaded at that stage?
4) I hear of people say things like "compiling blah [where "blah" is usually a module(?) of some sort] into the kernel" -- this is NOT the same thing as including it as a module in the initramfs, right?

Just trying to get everything straight. Answers to any of those questions would help.

foxbunny · 2008-05-30 11:42:57

2) What's the difference between modules that get loaded by mkinitcpio.conf and those that get loaded by rc.conf? They both end up in userland, correct?

AFAIK (and I haven't done my homework...yet), those loaded by mkinitcpio.conf are necessary for the init kernel (the one that boots the "real" kernel), and the real kernel (the one that runs your actual system) will use modules in rc.conf. I'm not really sure myself, but I think it works something like this:

PWR--> init kernel (loads basic drivers necessary for real kernel to boot) --> real kernel (loads everything *you* need).

Um, okay, forget the above. It's BS. Here's the real stuff:

http://www.ibm.com/developerworks/linux … linuxboot/

During the boot of the kernel, the initial-RAM disk (initrd) that was loaded into memory by the stage 2 boot loader is copied into RAM and mounted. This initrd serves as a temporary root file system in RAM and allows the kernel to fully boot without having to mount any physical disks. Since the necessary modules needed to interface with peripherals can be part of the initrd, the kernel can be very small, but still support a large number of possible hardware configurations. After the kernel is booted, the root file system is pivoted (via pivot_root) where the initrd root file system is unmounted and the real root file system is mounted.

The initrd is initcpio in Arch (I think).

Here's a more complete overview of the boot process. I read it (I swear!) but I forgot what it said.

http://tldp.org/HOWTO/From-PowerUp-To-B … HOWTO.html

Last edited by foxbunny (2008-05-30 11:51:01)

bender02 · 2008-05-30 14:25:44

To clarify a bit: there's not "init kernel" and "real kernel". The kernel that gets loaded from grub *is* the kernel that runs the system. Initramfs (called initcpio in arch; or also initrd (which I think is the older version of initramfs)) *just* makes it possible to load some extra modules before the root filesystem gets mounted. This is necessary on kernels (like arch stock) which have most of the drivers compiled as modules - you obviously need to have modules for HD and filesystem loaded before you can mount the root filesystem.

So initcpio contains a bunch of modules, a script that loads them, and of course some binaries necessary for module loading.

B-Con wrote:

1) Just what, exactly, *is* a hook for initramfs, as specified in mkinitcpio.conf?

Hook is just a little script that gets called during the creation process of initcpio (by mkinitcpio). Its role is to pick the modules to be included in the initcpio.

B-Con wrote:

2) What's the difference between modules that get loaded by mkinitcpio.conf and those that get loaded by rc.conf? They both end up in userland, correct?
2a) If putting modules in mkinitcpio.conf makes them somehow more accessible / faster to load for interaction with the kernel, what's the advantage to leaving modules you know you'll always need in the rc.conf modules list? Is it just to avoid the necessity of rebuilding the initramfs?

There's no difference. As explained above, you just need to load some modules before you can mount the root filesystem. If you don't have to, there's no point in having initcpio.

B-Con wrote:

3) (Following from #1...) If udev is not included as a hook in mkinitcpio.conf, will udev do anything if MOD_AUTOLOAD is set to "yes"?
3a) What does udev even do as a hook? Given a the default empty modules list in mkinitcpio.conf, I assume udev is responsible for all modules outside of the MODULES array loaded at that stage?

There are two instances of udev run during the boot process, and they are (almost) completely unrelated. The purpose of running udev in initramfs is that it creates the device nodes (like /dev/sda1). This is necessary if you want to mount that drive of course
[EDIT: yea, so udev hook does not include any modules into initcpio; instead it includes the udev binary]

The purpose of udev which is run afterwards, from the usual rc.sysinit script, is to load modules for hardware you've got on your machine. This is affected by MODULES, MOD_AUTOLOAD and all that in rc.conf.

B-Con wrote:

4) I hear of people say things like "compiling blah [where "blah" is usually a module(?) of some sort] into the kernel" -- this is NOT the same thing as including it as a module in the initramfs, right?

No, it's almost the opposite The thing is: after the boot, you want to end up with a running kernel with all the drivers included. If you compile the drivers in, they're there since the moment grub loads your kernel, and no extra steps are necessary. If they're not compiled in, ie they're compiled as modules, they need to be loaded into memory during the boot process. Most of them are loaded via udev during the init process; but some (HD, filesystem) need to be loaded before you can even mount anything - and that's where you need initcpio.

Last edited by bender02 (2008-05-30 14:27:43)

peets · 2008-05-30 22:28:15

nice explanation bender02! These things were always mysteries to me.

B-Con · 2008-05-31 05:58:24

Very nice explanation, bender. Thanks for the links, foxbunny. :-)

So, if I wanted to figure out what all modules are necessary for initcpio, what would I do? As I understand it, the modules array will list modules to be explicitly loaded, the hooks array will call scripts to determine further modules are necessary (for each category), and a udev hook will cause udev to execute at load time to find any other potentially necessary miscellaneous, but unspecified thus far, modules.

Last edited by B-Con (2008-05-31 07:36:38)

JeremyTheWicked · 2008-05-31 07:33:37

Thank you all for informative posts! I've been following this thread b/c I didn't understand it, too.

bender02 · 2008-05-31 15:45:02

B-Con: For further investigation, there are couple of things you can do:

Disclaimer: suggestions below are for fun, you don't actually gain anything (definitely not a significant speedup).

- look at the documentation for kernel source to figure out which module does what, and understand which are necessary.

- 'lsmod' tells you which modules are loaded at the moment - so take that list as your starting point and understand which module does what

- look at what's actually inside initcpio, that is, unpack it somewhere, for instance like this: [as root: mkdir /tmp/initrd ; cd /tmp/initrd ; zcat /boot/kernel26.img | cpio -iv --no-absolute-filenames ] (kernel26.img is just gzip/cpio compressed directory). Investigate. By the way, the main script that's called during the boot is called 'init'. Modules are (as usual) in /tmp/initrd/lib/modules/2.6.25-ARCH/*. So this gives you a basic list of modules that are used now. Start with it, understand which are for what purpose.

- once you think you know which ones, you can remove all the HOOKS (except udev) and put the modules into MODULES (in mkinitcpio.conf). Regenerate kernel26.img. [Be sure to have a backup of a working one that you can boot with.] Try to reboot.

- by the way, if you edit things in /tmp/initrd manually, and would like to assemble that into an initramfs image back again, use for instance this: [as root: cd /tmp/initrd ; find . | cpio -o -H newc > /tmp/initrd.temp ; gzip -c /tmp/initrd.temp > /boot/kernel26.img ] Change filenames as needed

EDIT: the --no-absolute-filenames parameter to cpio is *crucial*, otherwise you end up with destroyed system (cpio would decompress the initcpio into /, overwriting important files).
EDIT2: Disclaimer added.

Last edited by bender02 (2008-05-31 15:49:44)

B-Con · 2008-05-31 21:18:32

Thanks for all the help, I'll have to sit and pick things apart.

awagner · 2008-06-01 21:13:06

Sorry, there's one thing I don't get (but thanks for the great info first of all!!):

bender02 wrote:

...
There's no difference. As explained above, you just need to load some modules before you can mount the root filesystem. If you don't have to, there's no point in having initcpio.
...
There are two instances of udev run during the boot process, and they are (almost) completely unrelated. The purpose of running udev in initramfs is that it creates the device nodes (like /dev/sda1). This is necessary if you want to mount that drive of course
[EDIT: yea, so udev hook does not include any modules into initcpio; instead it includes the udev binary]

So, supposed I have all boot-related drivers already compiled into the kernel. Then I don't need to load any modules. Do I still need an initcpio -- for udev creating my device nodes? (Other ways of creating these?)

Thanks,
Andreas

klixon · 2008-06-01 21:29:51

If you have compiled everything needed for boot into the kernel, you don't need an initramfs. The kernel creates the needed devices to get to init and udev (which with an initramfs would be the second run) does the rest
After i baked my own kernel, with nothing compiled as modules, it also shaved a tasty 4.5 secs off my udev processing time, but ymmv there. This is an ooold box

awagner · 2008-06-02 07:04:57

klixon wrote:

If you have compiled everything needed for boot into the kernel, you don't need an initramfs. The kernel creates the needed devices to get to init and udev (which with an initramfs would be the second run) does the rest.

Well, it doesn't here. I get an error about not being able to access root partition together with strangely named suggestions for which partition to use there. I didn't take notes - it seems I'll have to go through the kernel config again.

Thanks anyway,
Andreas

PS. Just to make sure: having support for initrd/initramfs in the kernel doesn't hurt, even if one has none of those, right? (And to take it into account, I'd just leave the initrd command in grub's menu.lst away - or is there something more/else I should do?)

schivmeister · 2008-06-02 07:44:08

It's recommended to NOT go with static things (eg. device nodes for disks) these days. We're in the modern age, dynamic is where we should be. Without init image, udev, autoloading does not occur. This is important, especially for laptops and systems where there's potential for regular hardware changes (inserting a USB drive is a hardware change; add-on). With static devs, it's quite troublesome to create nodes manually. Without udev, it's quite troublesome to search for the relevant module every time a new device needs to come in.

klixon · 2008-06-02 07:54:03

awagner wrote:

Well, it doesn't here. I get an error about not being able to access root partition together with strangely named suggestions for which partition to use there. I didn't take notes - it seems I'll have to go through the kernel config again.
Thanks anyway,
Andreas

PS. Just to make sure: having support for initrd/initramfs in the kernel doesn't hurt, even if one has none of those, right? (And to take it into account, I'd just leave the initrd command in grub's menu.lst away - or is there something more/else I should do?)

It does not hurt to leave the option enabled in the kernel config, but you should remove the initrd-line from you grub/menu.lst from the boot-option with your home-brewed kernel.
You are sure you compiled the drivers for your ata-controler and your root filesystem?
Did you go the sata- or ide-route for your controler and did you specify /dev/hdxx or /dev/sdxx accordingly?
If you went sata, did you also enable scsi/harddisk and scsi/cdrom support?

schivmeister wrote:

It's recommended to NOT go with static things (eg. device nodes for disks) these days. We're in the modern age, dynamic is where we should be. Without init image, udev, autoloading does not occur. This is important, especially for laptops and systems where there's potential for regular hardware changes (inserting a USB drive is a hardware change; add-on). With static devs, it's quite troublesome to create nodes manually. Without udev, it's quite troublesome to search for the relevant module every time a new device needs to come in.

You are quite right. Therefore I only mentioned to compile the modules statically into the kernel. This way, udev from an initramfs is not necessary. The udev that runs from init still runs and creates all other necessary device-nodes.

Last edited by klixon (2008-06-02 07:55:28)

awagner · 2008-06-02 13:53:09

klixon wrote:

You are sure you compiled the drivers for your ata-controler and your root filesystem?
Did you go the sata- or ide-route for your controler and did you specify /dev/hdxx or /dev/sdxx accordingly?
If you went sata, did you also enable scsi/harddisk and scsi/cdrom support?

right on the spot. I had been SOO sure that I've compiled them in, but lsmod and /proc/config.gz tell me otherwise: pata_via and pata_acpi loaded as modules (when booting the same kernel with initrd). I've reconfigured the kernel and now it works.

However, it dawned on me that I would not only be missing v86d (at least early in the boot process; that would not be so tough), but also the resume hooks. Now that's something I can't afford. Thanks a lot for the help tho, it really has cleared up a lot of things for me.

CU,
Andreas

klixon · 2008-06-02 18:06:55

You should still be able to have those. Just don't include the pata/sata, filesystem and udev hooks in mkinitcpio.conf and build an initramfs. Load it with your kernel and things /should/ be dandy (i have no experience with this though, so ymmv)

Arch Linux

#1 2008-05-30 10:01:44

Clarification on how mkinitcpio.conf works.

#2 2008-05-30 11:42:57

Re: Clarification on how mkinitcpio.conf works.

#3 2008-05-30 14:25:44

Re: Clarification on how mkinitcpio.conf works.

#4 2008-05-30 22:28:15

Re: Clarification on how mkinitcpio.conf works.

#5 2008-05-31 05:58:24

Re: Clarification on how mkinitcpio.conf works.

#6 2008-05-31 07:33:37

Re: Clarification on how mkinitcpio.conf works.

#7 2008-05-31 15:45:02

Re: Clarification on how mkinitcpio.conf works.

#8 2008-05-31 21:18:32

Re: Clarification on how mkinitcpio.conf works.

#9 2008-06-01 21:13:06

Re: Clarification on how mkinitcpio.conf works.

#10 2008-06-01 21:29:51

Re: Clarification on how mkinitcpio.conf works.

#11 2008-06-02 07:04:57

Re: Clarification on how mkinitcpio.conf works.

#12 2008-06-02 07:44:08

Re: Clarification on how mkinitcpio.conf works.

#13 2008-06-02 07:54:03

Re: Clarification on how mkinitcpio.conf works.

#14 2008-06-02 13:53:09

Re: Clarification on how mkinitcpio.conf works.

#15 2008-06-02 18:06:55

Re: Clarification on how mkinitcpio.conf works.

Board footer