figtree - an experimental AIF module

altercation · 2011-09-02 06:32:18

figtree is still in testing, this is an announcement for feedback. Clone/fork and test if interested. README on github has usage details.

https://github.com/altercation/figtree

figtree is an AIF module I've been working on with some useful (?) features:

Can be run from a remote path. Boot from a standard Arch Linux installation image and remotely access your custom installation profile with no customization of the ISO.
Remote source for both the procedure and profiles can be a version control system. Currently only github has been tested, but figtree has been designed to support git (github/other), mercurial (bitbucket/other), svn (google-code/other), and wget as a fallback option.
Intelligent sourcing of files (if you've already specified a remote path for the procedure, you can use a relative path for the profile). Procedures and profiles can be local, remote, both, and from different remote locations.
Supports automatic AUR package installation (see issues)
Loads both remote and local profiles.
Profiles can be standard AIF profiles or "figtree profiles" which support more complex system configuration at install time.
Profiles can link to other profiles, creating a config tree of profiles.
Profiles can use custom commands to add to:
- add to the install package list
- customize system variables in arbitrary config files
- add overlay files
- add kernel parameters
- add custom system commands to be run at the end of the install

An example:

From a clean Arch Linux ISO boot:

# aif -p partial-configure-network
# aif -p http://github.com/altercation/figtree/raw/master/procedures/automatic \
      -c profiles/generic

The above should actually work, but it's just the same as the generic AIF example, you can see other profile examples include profiles which link other profiles in the github /profiles directory.

More command line examples are in the github readme here https://github.com/altercation/figtree ).

Profiles, Scope, Linking

One of the key motivations behind figtree was to enable the "stacking" of configuration profiles. It is common to have different hardware but similar desktop environments, application sets and user configurations. Thus one might have a configuration tree like this:

    Specific-System
      |
      +--Systems/Manufacturer/Model
      |
      +--Environments/Type/Variant
      |
      +--Applicatons/Category/Set
      |
      +--Users/You

A desktop and laptop might thus look like this:

    profiles/es/es-laptop               profiles/es/es-desktop
      |                                   |
      +--systems/lenovo/x220              +--systems/generic/pc
      |                                   |
      +--environments/xmonad/es           +--environments/xmonad/es
      |                                   |
      +--applicatons/cli/basics           +--applications/cli/basics
      +--applicatons/cli/media            +--applications/cli/media
      |                                   +--applicatoins/gui/video
      |                                   |
      +--users/es                         +--users/es

Status:

Alpha.

Installs from github or local (other dvcs not yet tested)
AUR works-ish. Seeing I/O problems. Why? makepkg paths? Who knows. I have to figure that out.
automatic and partial-update-overlay procedure work but partial-conform-system is in progress

Last edited by altercation (2011-09-02 07:19:26)

Dieter@be · 2011-09-02 14:37:03

On my Arch systems, I like to try out new software, and often change which sofware I use (window manager, mail client, browsers, ..)
I also used to create aif profiles to make my setup exactly how I wanted it, but what's the point if you're changing all the time?
The way I see it, there should be a way to allow me to manually change the state of my machine (add/remove packages, change configs, home directory layout), and then to transfer that system state into a high level aif profile (update package list, update configs in version control, etc)

altercation · 2011-09-02 16:31:26

(Dieter, I'm going to post some of the issues we were discussing on mail here, apologies for duplication but we can pick up on this public channel)

For anyone that can shed light on the AUR related error I'm seeing, it can currently be reproduced from a clean Arch ISO boot using:

# aif -p partial-configure-network
# aif -p http://github.com/altercation/figtree/r … /automatic -c profiles/aur-test

That profile installs just base group and weechat-git (IRC client) from AUR. Many AUR installs work fine but weechat-git is a bigger build. The error I'm seeing is hundreds of:

Buffer I/O error

sprinkled with an occasional

EXT4-fs This should not happen!!

This occurs in the middle of makepkg cloning the git repo, so I'm guessing it has to do with where makepkg is cloning to (a cache dir running out of space). I am reading up more on makepkg but welcome feedback on this error.

What's confusing about this is that I always assumed that makepkg is just cloning to the src directory, but maybe it's using some intermediary location?

Last edited by altercation (2011-09-02 16:38:53)

altercation · 2011-09-02 16:37:30

Dieter@be wrote:

On my Arch systems, I like to try out new software, and often change which sofware I use (window manager, mail client, browsers, ..)
I also used to create aif profiles to make my setup exactly how I wanted it, but what's the point if you're changing all the time?
The way I see it, there should be a way to allow me to manually change the state of my machine (add/remove packages, change configs, home directory layout), and then to transfer that system state into a high level aif profile (update package list, update configs in version control, etc)

I do the same. The partial-update-overlay (and other partials which I haven't completed) are designed to allow the user to quickly snapshot their current system state.

There is a stub partial-package-diff, the purpose of which is to list out the diff between what a figtree config-tree set would install and what is currently installed (diff both ways, what's missing and what's extra).

By adding in a scan of the /etc/rc.conf state and comparing to the values in the profile, pretty much the entire system state can be captured.

I focused on support github specifically for the same reason, so that the system state can be version controlled.

It's possible that the whole state is just one profile, of course. But I find that when I set up my arch systems, about 90% of the initial setup is really just minor (though sometimes tedious) hardware tweaks. That's one of the reasons I wanted to break out the configs into a tree, so that the hardware specific configuration can be kept somewhat separate if desired (again, it can all be bundled up into one profile as well).

Dieter@be · 2011-09-02 16:51:14

altercation wrote:

(...)
Buffer I/O error
sprinkled with an occasional
EXT4-fs This should not happen!!
(...)

this sounds very much like an Archiso topic. You could ask about this on the releng mailing list.

Dieter@be · 2011-09-02 17:28:21

altercation wrote:

I do the same. The partial-update-overlay (and other partials which I haven't completed) are designed to allow the user to quickly snapshot their current system state.
There is a stub partial-package-diff, the purpose of which is to list out the diff between what a figtree config-tree set would install and what is currently installed (diff both ways, what's missing and what's extra).
By adding in a scan of the /etc/rc.conf state and comparing to the values in the profile, pretty much the entire system state can be captured.
I focused on support github specifically for the same reason, so that the system state can be version controlled.
It's possible that the whole state is just one profile, of course. But I find that when I set up my arch systems, about 90% of the initial setup is really just minor (though sometimes tedious) hardware tweaks. That's one of the reasons I wanted to break out the configs into a tree, so that the hardware specific configuration can be kept somewhat separate if desired (again, it can all be bundled up into one profile as well).

interesting, I never had the courage to build a full "system state back to aif profile" mechanism

altercation · 2011-09-02 18:40:32

Dieter@be wrote:

interesting, I never had the courage to build a full "system state back to aif profile" mechanism

it's funny, since when I first saw AIF's architecture that's the first thing I thought of. It's really nicely designed.

My only concern in this venture is that I don't stray from the simplicity of Arch, and it's possible to get down in the weeds too much and I don't want to add complexity. I think i've walked that line pretty well so far. Again, AIF's modularity has been a real boon in that area.

Arch Linux

#1 2011-09-02 06:32:18