AIF and kickstart (automated installation over network)?

V01D · 2010-05-04 18:05:38

Hi,
I'm currently wondering if AIF is capable of letting me create a simple kickstart (a-la RedHat, or even Ubuntu now) installation system. What I mean by this is something like:
- set a master server (which has a tftp server providing a bootable image and kernel, and a mirror of arch repositories)
- boot any other node, connected to the same network as master, over the network (ie: PXE boot)
- this node starts the image it retrieves from master and automatically configures network (detect required kernel modules and then use dhcp to configure itself)
- the node now retrieves a configuration file which tells the installation system how to partition, which packages to install, etc (this config file should be retrieved through HTTP in the master, so this configuration file can be dynamic based on which node is requesting it)
- the node installs as per instructed in configuration file and then reboots
- since the node has "boot disk first and then network" it will boot straight into arch and it will be completely functional

In the past I created my own set of scripts and a custom installation image but it turned out to be a real pain to chase Arch development. Now AIF seems like the correct tool, but I don't know if it allows a REALLY unattended installation like the one described here, or even if its planned to do so.

I ask this because I'm currently using my custom automated installation system on a HPC cluster here at work, and the time to upgrade to a newer version is nearing. I was considering moving to Ubuntu which seems to provide some kickstart support, but creating packages and configuring the system is a real pain. So it would be nice to know if this can be achieved with Arch.

Thanks,
Matt

Dieter@be · 2010-05-05 17:46:50

all of this is currently possible (with testing images from http://build.archlinux.org/isos/), except:

1) automatically starting the automatic installation after booting (I guess what's needed for this is puting the right command in /etc/rc.local of the environment it will load)
2) making sure it will boot from disk instead of pxe at reboot (assuming the bios is configured to boot from pxe first, otherwise this is obviously not a problem as the boot procedure will happen normally)

V01D · 2010-05-05 18:06:21

Dieter@be wrote:

all of this is currently possible (with testing images from http://build.archlinux.org/isos/), except:
1) automatically starting the automatic installation after booting (I guess what's needed for this is puting the right command in /etc/rc.local of the environment it will load)
2) making sure it will boot from disk instead of pxe at reboot (assuming the bios is configured to boot from pxe first, otherwise this is obviously not a problem as the boot procedure will happen normally)

Thank you for responding.
When you say "possible" you mean it is already implemented? I couldn't figure that out from the docs in the wiki. Or is it that this functionality is not yet documented?
I mean, autodetection, retrieval of the config file over HTTP, etc. Is all that implemented? That would be awesome.

2) is not a problem since the order in BIOS is "disk, pxe", so when the disk is fresh it boots from pxe and installs, then it reboots and boots into the freshly installed Arch.
For (1), I nice feature would be a boot parameter like "autostart=1", which is trivial to implement.

Thanks,
Matt

Dieter@be · 2010-05-05 19:29:48

V01D wrote:

Dieter@be wrote:
all of this is currently possible (with testing images from http://build.archlinux.org/isos/), except:
1) automatically starting the automatic installation after booting (I guess what's needed for this is puting the right command in /etc/rc.local of the environment it will load)
2) making sure it will boot from disk instead of pxe at reboot (assuming the bios is configured to boot from pxe first, otherwise this is obviously not a problem as the boot procedure will happen normally)
Thank you for responding.
When you say "possible" you mean it is already implemented? I couldn't figure that out from the docs in the wiki. Or is it that this functionality is not yet documented?
I mean, autodetection, retrieval of the config file over HTTP, etc. Is all that implemented? That would be awesome.

yes implemented, just not available in an official release yet. which is also why the documention is hidden now:
http://projects.archlinux.org/aif.git/t … n_guide_en
not much info there though. this is a handy script: http://wiki.archlinux.org/index.php/Arc … pxe-server

how the actual installation goes is entirely up to you. it's pretty trivial to autodetect your mac address/ip/hardware/.., fetch configuration over http etc.

V01D wrote:

2) is not a problem since the order in BIOS is "disk, pxe", so when the disk is fresh it boots from pxe and installs, then it reboots and boots into the freshly installed Arch.

aha. good. problem solved

V01D wrote:

For (1), I nice feature would be a boot parameter like "autostart=1", which is trivial to implement.

okay, patch welcome

V01D · 2010-05-05 20:49:39

Well, everything looks very promising. I hope that I can use this new installation framework in the near future, when the time to upgrade the cluster comes.

What I've already implemented (and would need some reworking to adapt it to this framework) is a simple system which would run on the master node and serves the config files. This is basically a Ruby-based CGI script, which reads XML-based configuration files. I'll put it on GitHub when things settle with this new AIF.

Matt

V01D · 2010-05-06 18:48:33

I was thinking, since I should be doing some changes to the installer scripts (like, for instance, automatically starting aif, and even adding the code to retrieve configuration from HTTP), is there an easy way to recreate the installation image with these modifications? How is that supposed to be done?

rransom · 2010-05-06 19:55:06

V01D wrote:

I was thinking, since I should be doing some changes to the installer scripts (like, for instance, automatically starting aif, and even adding the code to retrieve configuration from HTTP), is there an easy way to recreate the installation image with these modifications? How is that supposed to be done?

Images are built with archiso-git (in AUR). You may also need to modify and recompile packages (such as the package containing init scripts).

V01D · 2010-05-06 22:10:01

Ok, no problem. As long as there's a tool to do so, no problem. I'll stay alert for the release of this new aif version.

V01D · 2010-05-08 23:12:57

BTW, dieter, I'm thinking on eventually forking your repo at GitHub mainly to start adding the missing parts for a fully automated installation (like configuring network automatically and optionally auto-starting aif with a certain procedure when a specific boot parameter is used).
Are there any plans already to complete automated installtion (I cant figure out from the code how much is left to reach that... I mean, 0 user intervention) in some way? Maybe I can see what the plans are and start from there instead of deciding myself on how these steps should be implemented.

Dieter@be · 2010-05-09 08:21:55

V01D wrote:

BTW, dieter, I'm thinking on eventually forking your repo at GitHub mainly to start adding the missing parts for a fully automated installation (like configuring network automatically and optionally auto-starting aif with a certain procedure when a specific boot parameter is used).
Are there any plans already to complete automated installtion (I cant figure out from the code how much is left to reach that... I mean, 0 user intervention) in some way? Maybe I can see what the plans are and start from there instead of deciding myself on how these steps should be implemented.

well, just look at the example config files for the automatic procedure. they work fine if launched with aif -p automatic -c $path_to_config
so you just need to write a new automatic procedure which can fetch the config over http. maybe make it do a http get request to $url?mac=$yourmac&cpu=yourcpu&<moreparams>
once booted, the live system will keep the ip it received from dhcp during boot so that's not really a problem.
i'm not sure yet what's the best way to set $url, maybe also through a boot argument received from the pxe server?
and you also need to make it automatically start aif -p <your-procedure-here> on boot, through the boot option mechanism. i think that's it.

Last edited by Dieter@be (2010-05-09 08:37:58)

V01D · 2010-05-09 14:57:01

Oh, ok. I thought that all the code that actually configured your system (like editing rc.conf and all config files wasn't implemented yet for autoconfiguration). I was expecting there would be configuration variables which I could set and the scripts would take care of modifying the files. Is that how it works or the idea is for me to edit those files on my own?

An regarding the IP, I'm not so sure. PXE gives the computer IP just so it can retrieve the necessary images, but once the kernel is booted the network interface is unconfigured. Obviously It's just a matter of running DHCP again.

Dieter@be · 2010-05-09 15:15:43

V01D wrote:

Oh, ok. I thought that all the code that actually configured your system (like editing rc.conf and all config files wasn't implemented yet for autoconfiguration). I was expecting there would be configuration variables which I could set and the scripts would take care of modifying the files. Is that how it works or the idea is for me to edit those files on my own?

that's up to you.
there is already some code which you can reuse - most of it is just an easy sed expression anyway - but nothing is stopping you from pulling a complete rc.conf over http/svn/git/.. and putting that on the target system.

V01D wrote:

An regarding the IP, I'm not so sure. PXE gives the computer IP just so it can retrieve the necessary images, but once the kernel is booted the network interface is unconfigured. Obviously It's just a matter of running DHCP again.

pxe does not give ip's. dhcp is used, pxe boots the system. i know that through ipappend (isolinux option) the ip kernel option is added, so at the least the early userspace is aware of the ip (and sets it, so that it can load the nbd volume over network to boot), it could be that the network initscript then resets it again, but i'm not sure.

V01D · 2010-05-09 21:00:41

Dieter@be wrote:

that's up to you.
there is already some code which you can reuse - most of it is just an easy sed expression anyway - but nothing is stopping you from pulling a complete rc.conf over http/svn/git/.. and putting that on the target system.

Ah, ok. That's what I was already doing with my ad-hoc installation system. The problem is that it became tedious to maintain each configuration file up-to-date and consistent with what the package used. What I liked about kickstart is that all that configuration file handling was already taken care of and there was a simple .cfg file which the user provided to make the whole installation.

Anyhow, I'll see what I can do. Thank you for the information

Dieter@be wrote:

pxe does not give ip's. dhcp is used, pxe boots the system. i know that through ipappend (isolinux option) the ip kernel option is added, so at the least the early userspace is aware of the ip (and sets it, so that it can load the nbd volume over network to boot), it could be that the network initscript then resets it again, but i'm not sure.

Ok, I didn't know about ipappend. Good to know that. Makes things simpler.

Arch Linux

#1 2010-05-04 18:05:38

AIF and kickstart (automated installation over network)?

#2 2010-05-05 17:46:50

Re: AIF and kickstart (automated installation over network)?

#3 2010-05-05 18:06:21

Re: AIF and kickstart (automated installation over network)?

#4 2010-05-05 19:29:48

Re: AIF and kickstart (automated installation over network)?

#5 2010-05-05 20:49:39

Re: AIF and kickstart (automated installation over network)?

#6 2010-05-06 18:48:33

Re: AIF and kickstart (automated installation over network)?

#7 2010-05-06 19:55:06

Re: AIF and kickstart (automated installation over network)?

#8 2010-05-06 22:10:01

Re: AIF and kickstart (automated installation over network)?

#9 2010-05-08 23:12:57

Re: AIF and kickstart (automated installation over network)?

#10 2010-05-09 08:21:55

Re: AIF and kickstart (automated installation over network)?

#11 2010-05-09 14:57:01

Re: AIF and kickstart (automated installation over network)?

#12 2010-05-09 15:15:43

Re: AIF and kickstart (automated installation over network)?

#13 2010-05-09 21:00:41

Re: AIF and kickstart (automated installation over network)?

Board footer