You are not logged in.

#1 2013-11-15 17:07:39

pluc
Member
Registered: 2013-11-08
Posts: 9

Automatic deployment for end user box

Hello,

I'm not a system admin, but I must automatically install a set of computer that will be used by end users. I currently Arch/Debian/Ubuntu on different machines for development. I'm interested in installing Arch for production, but I would like to have some feedbacks on automatic install scripts. I like arch for it's recent and minimal package set, there will be no automatic updates so stability should not be an issue once I set up a working environment.

What I need is to install and configure a new PC without any interaction : partitioning, base system, post-installation scripts (user management, custom libraries compilation, etc). Booting from an USB key is acceptable, though I guess it's probably better using PXE.

For debian I know preseeding [0] and FAI [1]. For Arch I found AIF [2] which is marked as deprecated and replaced by install scripts [3].

Do you guys have experience to share, and interesting links ?

[0] https://wiki.debian.org/DebianInstaller/Preseed
[1] http://fai-project.org/
[2] https://wiki.archlinux.org/index.php/AI … ation_File
[3] https://projects.archlinux.org/arch-ins … ripts.git/

Edit : I thought I would complete a bit.

It's sill under heavy development, but for the moment I'll get machines shipped with Windows8 installed, probably on a GTP partitionned disk.
As a first step I'm gonna have to boot from an iso image on a DVD or usb stick. I need to perform the following steps :

  • Partition the disk(s)

  • Install the base system

  • Install a set of generic packages, including a desktop and graphic drivers

  • Setup everything to boot (fstab, bootload, etc)

  • Setup users, additional mounts, permissions, network, etc

  • Compile additional custom packages

When the system reboot, it has to be ready to use. The best would be a single application boot instead of a desktop environment, but this is another problem.

All this should be done without internet, I don't want to get packages from arch mirrors using internet, but directly inside the ISO or from a local server. I can't have different kernels, library versions, ...
For the base system and the generic packages I guess I can setup a local mirror ? And is it possible to ship everything within the ISO for the first tries ?

Last edited by pluc (2013-11-15 17:32:05)

Offline

#2 2013-11-15 17:50:16

drcouzelis
Member
From: Connecticut, USA
Registered: 2009-11-09
Posts: 4,092
Website

Re: Automatic deployment for end user box

pluc wrote:

I like arch for it's recent and minimal package set, there will be no automatic updates so stability should not be an issue once I set up a working environment.

Is this the only reason you chose Arch Linux for this project?

All this should be done without internet, I don't want to get packages from arch mirrors using internet, but directly inside the ISO or from a local server. I can't have different kernels, library versions, ...

Everything you described sounds like what Debian stable or CentOS was designed for. Just put the disc in the computer, install, and the computer is ready to use. Each computer will have the same software and each will be supported for years.

If a user wants to install a new application (for example, Chromium web browser) six months after you setup Arch Linux, will you use the Internet to install it? You can't (safely) install current software in Arch Linux without doing a complete update, which now means one computer has different libraries and software on it than the other Arch Linux computers...

Offline

#3 2013-11-15 19:14:19

ANOKNUSA
Member
Registered: 2010-10-22
Posts: 2,141

Re: Automatic deployment for end user box

pluc wrote:

I'm not a system admin, but I must automatically install a set of computer that will be used by end users.

This sentence is self-contradictory.  You're the one choosing and overseeing the installation of the systems; whether think so or not, your are voluntarily making yourself responsible for them, regardless of which distribution you choose to use. Who do you think will be called upon if and when something goes wrong?  Would it be fair for you to install and set up everything, then expect someone else to take up the responsibility and retroactively learn what you did up to that point?

Anyway, what you're looking for is a static release that can be automatically installed, without a connection to the internet, requiring no user intervention.  Arch is none of those things. You can, of course, install Arch on one system, create a user account, then use Archiso to create an installation image to deploy to other machines, assuming each machine is 100% identical.  But then why not use something like Debian? You can download .iso images containing their entire Stable repository branch, burn them to multiple DVDs, and have the entire distribution on disc until the next update.

Offline

#4 2013-11-15 20:05:35

pluc
Member
Registered: 2013-11-08
Posts: 9

Re: Automatic deployment for end user box

drcouzelis wrote:

Is this the only reason you chose Arch Linux for this project?

Everything you described sounds like what Debian stable or CentOS was designed for. Just put the disc in the computer, install, and the computer is ready to use. Each computer will have the same software and each will be supported for years.

If a user wants to install a new application (for example, Chromium web browser) six months after you setup Arch Linux, will you use the Internet to install it? You can't (safely) install current software in Arch Linux without doing a complete update, which now means one computer has different libraries and software on it than the other Arch Linux computers...

Well I guess I have not been very clear. About the choice :

  • It's not fixed, I can still use Debian (or Debian-based), I hope you will help me decide

  • About other distros (Red Hat derived, Suse, etc) : I never used them (exept CentOS a while ago), so I'd prefer not to. I used Debian for years, and I'm kinda new to Arch (but love it yikes)

  • I didn't installed Debian for a while, but as far as I remember, every iso install is full of packages I don't use or don't want, Ubuntu being the worst

  • Recent kernel and packages is important, because I need some recent versions of libs, and nvidia drivers. This would require additional work if packaged versions are too old.

  • I kinda like the simplicity of makepkg, I felt I could use it

  • Good wiki

As for your second remark, this is clear in my post. I want an automatic installation.

For your third remark, the PC will be like a "black box". Users will not have any permission to install stuff, or upgrade anything. They will just use one application that we develop. Think of it as an ATM machine.
If I wan't to upgrade some packages to a new version of my software, well I have to find a way to make it part of an automatic update I guess, and yes through internet, but again not a public mirror, from my own server with a predefined package set.

Hope this is more clear smile

ANOKNUSA wrote:
pluc wrote:

I'm not a system admin, but I must automatically install a set of computer that will be used by end users.

This sentence is self-contradictory.  You're the one choosing and overseeing the installation of the systems; whether think so or not, your are voluntarily making yourself responsible for them, regardless of which distribution you choose to use. Who do you think will be called upon if and when something goes wrong?  Would it be fair for you to install and set up everything, then expect someone else to take up the responsibility and retroactively learn what you did up to that point?

Anyway, what you're looking for is a static release that can be automatically installed, without a connection to the internet, requiring no user intervention.  Arch is none of those things. You can, of course, install Arch on one system, create a user account, then use Archiso to create an installation image to deploy to other machines, assuming each machine is 100% identical.  But then why not use something like Debian? You can download .iso images containing their entire Stable repository branch, burn them to multiple DVDs, and have the entire distribution on disc until the next update.

Good point, I'm not a system admin and I have to do it (for the close future at least). This is part of working in a very small start up smile It's not definitive, but I need to do it for a beta test phase, so it must not be perfect, but fullfill my needs.

I'll dig into Arch iso for the first step. All machines should be identical for the moment, except the disks UUID of course.

Now this is not the hard part I think. I also need to add some shared libraries compiled from source. I guess I can put everything within the iso. Now I'm afraid the full system is bigger than a DVD. Is setting up a server with a fixed package repository including custom packages a good answer to this ?

For debian stable, as I answerd to drcouzelis : First stable is too old, I think even gcc is too old for my needs. And second, I still need to custom the installation to make it fully automatic and include my softwares.

Anyway thanks smile

Offline

#5 2013-11-15 20:29:17

teateawhy
Member
From: GER
Registered: 2012-03-05
Posts: 1,138
Website

Re: Automatic deployment for end user box

Sorry for the shameless promotion, but look at my signature.

Offline

#6 2013-11-16 11:27:28

jrussell
Member
From: Cape Town, South Africa
Registered: 2012-08-16
Posts: 510

Re: Automatic deployment for end user box

Have a look at Debian wheezy (stable) and its backports, Im running wheezy with the 3.10 backported kernel, no problems.


bitcoin: 1G62YGRFkMDwhGr5T5YGovfsxLx44eZo7U

Offline

#7 2013-11-16 15:29:18

nomorewindows
Member
Registered: 2010-04-03
Posts: 3,362

Re: Automatic deployment for end user box

If you have a PXE installation, and you didn't want to worry about permanent installation on said boxes, you'd only have to worry about the PXE installation.  It would be easy to update from the Arch box it would be stored on.  You wouldn't even have any evidence of touching what was already on the disk.

Last edited by nomorewindows (2013-11-16 15:48:01)


I may have to CONSOLE you about your usage of ridiculously easy graphical interfaces...
Look ma, no mouse.

Offline

#8 2013-11-18 15:50:31

pluc
Member
Registered: 2013-11-08
Posts: 9

Re: Automatic deployment for end user box

@teateawhy I'll have a look, seems I could incorporate it in the iso the make things easier

@jrussell I could use wheezy, jessie or even sid, but the main interest here is the automatic installation

@nomorewindows I don't quite understand what you're saying (I'm not english native). I talked about PXE for the initial installation, to avoid making big iso files and boot from USB/DVD. Or at least a simple pacman mirror list to download intermediate packages.

Anyway I'm playing a bit with archiso, seems quite modular from what I've seen smile

Offline

#9 2013-11-18 17:09:21

ANOKNUSA
Member
Registered: 2010-10-22
Posts: 2,141

Re: Automatic deployment for end user box

pluc wrote:

...This is part of working in a very small start up...

If your cohorts have any money at all going into this, or your professional reputation is staked on it in any way, don't use Arch.  I might not say that if you were an experienced, professional administrator, but you said you aren't.  Arch is designed with individual users who take responsibility for their individual systems in mind; building an organization full of fallible (possibly ignorant?) people on top of it isn't a good idea.

Offline

#10 2013-11-18 19:57:03

pluc
Member
Registered: 2013-11-08
Posts: 9

Re: Automatic deployment for end user box

ANOKNUSA wrote:
pluc wrote:

...This is part of working in a very small start up...

If your cohorts have any money at all going into this, or your professional reputation is staked on it in any way, don't use Arch.  I might not say that if you were an experienced, professional administrator, but you said you aren't.  Arch is designed with individual users who take responsibility for their individual systems in mind; building an organization full of fallible (possibly ignorant?) people on top of it isn't a good idea.

Hiring a sys admin is possible, just not at the moment. Money comes and goes, sometimes you have to pass an alpha test phase to proves the concept is valuable so you confort your investors.
So the thing is, I have a pre-release of our soft, for which I know all the needed packages along with their versions. I need to install it on ~12 machines that will go to some testers.

At this point I just need to automate the install process and if possible all the post install steps : additional packages compilations, user managment, etc etc. With a pre-built iso this usually means :

For debian :

  • Select country/language / click Next

  • Select keymap / click Next

  • Configure partitions withing gparted / click Next

  • Enter a hostname / click Next

  • Enter a new user / click Next

  • Enter the password / click Next

  • ...

  • ...

  • reboot

  • login

  • apt-get install 800mb worth of packages

  • ...

  • Compile my stuff

  • ...

For Arch:

Now obviously this is a tedious task to repeat, all I wan't is to automate this.

Concerning your statement on Arch, and keeping in mind the testers will be logged on an account without permissions whatsoever, and no automatic updates, what will be the difference with another distrib ? Except maybe the lack of security updates provided for Debian for example, if enabled ?
edit: Maybe can you elaborate on what makes Arch bad for this task, taking this in consideration, and the installed system would be made of fully a working environment, see also: https://wiki.archlinux.org/index.php/Fu … with_rsync

Anyway I still take propositions for existing solutions. I don't refuse any distribution, as long as it proposes what I described, and a) if it's a "stable-like" distrib the packages are not too old (current debian should be enought for most of the common packages), b) if its a "rolling-like" distrib, the package versions can be fixed smile

Last edited by pluc (2013-11-18 20:11:40)

Offline

#11 2013-11-18 20:07:11

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 30,424
Website

Re: Automatic deployment for end user box

You don't need to do any point and clicking with debian (or any other distro, for that matter). You use a tool like Puppet to automate the builds and ongoing maintenance: there are probably dozens of recipes on the webs that you can tweak if you need hardware specific customizations...

e.g.: http://www.50ply.com/blog/2012/07/16/au … nd-puppet/


Arch + dwm   •   Mercurial repos  •   Surfraw

Registered Linux User #482438

Offline

#12 2013-11-18 20:07:15

drcouzelis
Member
From: Connecticut, USA
Registered: 2009-11-09
Posts: 4,092
Website

Re: Automatic deployment for end user box

pluc wrote:

all I wan't is to automate this.

The only professional experience I have with this is with Red Hat Enterprise Linux. For your information, it's called "Kickstart". A kickstart script is a text document that can be created by hand or using the simple GUI. It will automate the entire installation process, including anything you want to setup post-installation.

Offline

#13 2013-11-18 20:16:27

nomorewindows
Member
Registered: 2010-04-03
Posts: 3,362

Re: Automatic deployment for end user box

pluc wrote:

@teateawhy I'll have a look, seems I could incorporate it in the iso the make things easier

@jrussell I could use wheezy, jessie or even sid, but the main interest here is the automatic installation

@nomorewindows I don't quite understand what you're saying (I'm not english native). I talked about PXE for the initial installation, to avoid making big iso files and boot from USB/DVD. Or at least a simple pacman mirror list to download intermediate packages.

Anyway I'm playing a bit with archiso, seems quite modular from what I've seen smile

PXE not only can install clients from the internet directly, but you can also have a permanent PXE installation on your main station which can install clients from there in lieu of making ISOs, but can also run on a daily basis from one PXE installation being the same on all installations.  It can PXE boot several machines up and running, no extra installation needed.  But then, one application on one machine can be accessed such as through SSH.


I may have to CONSOLE you about your usage of ridiculously easy graphical interfaces...
Look ma, no mouse.

Offline

#14 2013-11-24 11:28:48

pluc
Member
Registered: 2013-11-08
Posts: 9

Re: Automatic deployment for end user box

jasonwryan wrote:

You don't need to do any point and clicking with debian (or any other distro, for that matter). You use a tool like Puppet to automate the builds and ongoing maintenance: there are probably dozens of recipes on the webs that you can tweak if you need hardware specific customizations...

e.g.: http://www.50ply.com/blog/2012/07/16/au … nd-puppet/

drcouzelis wrote:
pluc wrote:

all I wan't is to automate this.

The only professional experience I have with this is with Red Hat Enterprise Linux. For your information, it's called "Kickstart". A kickstart script is a text document that can be created by hand or using the simple GUI. It will automate the entire installation process, including anything you want to setup post-installation.

Thanks, great links. That was the next step after pre-seeding I guess. Puppet looks really nice, a bit too much for the short term, but definitly something to look into for production.

I've been playing with archiso, and I have a few questions :

  • What are the limitations when I'm in the first chroot (following  the wiki installation, after the pacstrap and the arch-chroot) ? What can I install or not ?

  • With the link of jason, I guess I can use systemd to launch a script after a first reboot, to finish up installation of what I could not do in the chroot, right ?

  • And finally, I'm not sure how to tell arch iso to start a script directly ?

I'm considering using teateawhy script, just without internet connection. The approch I used is to copy it directly in the arch iso step, then call it on boot..

Last edited by pluc (2013-11-24 13:15:10)

Offline

Board footer

Powered by FluxBB