semi-automatic arch installation and configuration

leonixyz · 2020-12-23 15:15:38

I already asked this on #archlinux@freenode, and got a pretty interesting answer, but I'd like to collect more feedback here.

I want to have a written definition of my system on some repository and be able to install arch semi-automatically, having my desired packages installed and configured and end up with a system ready to use. My goal is to find the easiest way to be able to:

manually download the latest installation iso
boot it
start a custom automatic process

This custom process should allow me to do either of:

reinstall arch on my laptop
install an equivalent system on a VM, or any another machine having a reasonably similar hardware

My idea was to write a simple bash script to perform the basic steps of an installation (partitioning, pacstrap base base-devel, grub-install, etc), then manually boot the live environment and execute the script.

Someone on freenode suggested a brilliant way about how to proceed afterwards, similar to what has been done here https://github.com/Earnestly/pkgbuilds/ … g/PKGBUILD:

By writing a custom pkgbuild file, listing all my desired packages as dependencies (e.g. firefox, gimp, libreoffice, etc), then supply a set of custom config files to overwrite system defaults, i should be able to maintain in a single repository the exact definition of my system. This pkgbuild file can then be built and installed on new systems, or changed and rebuilt/reinstalled on my existing system.

Do you see any problem/drawbacks in what explained above?
Do you know better alternatives to achieve my goal?

Bare in mind that I don't need something hyper-complex and flexible to use on 10M cloud VMs, I just want a way to deal with my own personal system.

Thanks in advance

Last edited by leonixyz (2020-12-23 15:20:21)

Slithery · 2020-12-23 16:12:37

For 1 and 2 there's no need to manually download the latest iso, just use the netboot image instead. These can be booted either from a USB or directly over your network if you have PXE set up.

leonixyz · 2020-12-23 17:34:07

Slithery wrote:

For 1 and 2 there's no need to manually download the latest iso, just use the netboot image instead. These can be booted either from a USB or directly over your network if you have PXE set up.

Thank you. Yes, any way of booting is fine for me. The thing i'm unsure about is the part 3

zpg443 · 2020-12-23 22:54:30

For part 3, why not just use rsync and restore?

Trilby · 2020-12-23 23:00:26

The metapackage with all the packages you want as dependencies is a bad idea. Your pacman database will be pretty screwy then as all the packages that should be installed explicitly will be installed as dependencies. If you ever chose to remove one, you'll need to remove the meta package which will result in all the other packages being orphaned.

Just follow the guidance in the wiki and generate a list of explicitly installed packages (`pacman -Qeq > pkg.list`) then on the new system install everything on that list (pacman -S - < pkg.list) and there is no need to screw over your local database in the process. And frankly this is easier than listing them all out in a depends array in a PKGBUILD anyways.

You can use the package for your configs - but then that's only for config files outside your home directory: given that most of these are owned by other packages (but see the comment below to address some of these), the utility of that meta-package approach ~~starts to fall apart~~ diminishes. You could just keep a dot-files repo somewhere (e.g., github).

EDIT: while the config.d approach can help for several configs, others do not have that option, so while saying the config-package approach "falls apart" may be overstatement, it's still only useful for a subset (those with config.d) of a subset (those not under your home directory) of config files. Given that you'll therefore need other supplemental approaches anyways, it seems more efficient to me to just use the other approach for everything in the first place.

Last edited by Trilby (2020-12-24 12:54:15)

Alad · 2020-12-24 12:06:59

One thing you can do to add more config files to the package is to use files in .d directories - which are usually not part of other packages. E.g. /etc/pacman.d, /etc/sudoers.d/, etc.

leonixyz · 2020-12-24 13:30:38

First, thank you all for answering.

Trilby wrote:

The metapackage with all the packages you want as dependencies is a bad idea. Your pacman database will be pretty screwy then as all the packages that should be installed explicitly will be installed as dependencies. If you ever chose to remove one, you'll need to remove the meta package which will result in all the other packages being orphaned.

I see what you're saying about the screwed pacman db, but I'm still not 100% convinced.

Say I choose to remove Chromium: First I remove it from the depends, then I bump up the pkgver. At that point I simply rebuild and reinstall the package on top of my system. It will be recognized by pacman as an upgrade, and as a result Chromium (and all its dependencies not used by other packages) will be uninstalled automatically.

Say, instead, I want to add Inkscape: I do the same thing (bump pkgver and add a new item to the depends array and rebuild). Rebuilding and reinstalling will take care of the rest.

Is the above true? Maybe i'm missing some pacman internals, but if not so, I find quite useful to have `pacman -Qeq` showing my metapackage + all other packages that I installed "manually", which are those that I need to later integrate in my PKGBUILD. To me, this comes especially handy when dealing with packages that need system-wide config, like daemons.

For example, say for some obscure reason I need samba (smbd specifically). I need to configure it system-wide. First I install it manually, then I tweak the config to my like. Finally (when I have time and after testing that the config is stable/suitable to my needs) I integrate the new dependency and its config in my metapackage, and rebuild it. From that point on, I have a "reproducible build" of my system.

Trilby wrote:

You could just keep a dot-files repo somewhere (e.g., github).

yes, that is something I'd need anyway

Trilby wrote:

EDIT: while the config.d approach can help for several configs, others do not have that option, so while saying the config-package approach "falls apart" may be overstatement, it's still only useful for a subset (those with config.d) of a subset (those not under your home directory) of config files. Given that you'll therefore need other supplemental approaches anyways, it seems more efficient to me to just use the other approach for everything in the first place.

This part is not clear to me: can't I replace a system default config with my own one, using the pkgbuild approach?
Say I state openssh as one of my dependencies. In my package() function I replace /etc/ssh/sshd_config. Isn't that possible?

Scimmia · 2020-12-24 13:32:23

leonixyz wrote:

Say I choose to remove Chromium: First I remove it from the depends, then I bump up the pkgver. At that point I simply rebuild and reinstall the package on top of my system. It will be recognized by pacman as an upgrade, and as a result Chromium (and all its dependencies not used by other packages) will be uninstalled automatically.

No it won't. Try it.

leonixyz · 2020-12-24 13:32:30

Alad wrote:

One thing you can do to add more config files to the package is to use files in .d directories - which are usually not part of other packages. E.g. /etc/pacman.d, /etc/sudoers.d/, etc.

Yes, thank you Alad, I already started moving my config to config.d wherever possible. This is extremely useful.

leonixyz · 2020-12-24 13:33:46

Scimmia wrote:

No it won't. Try it.

uuuh, alright... thanks

I knew I was missing some big point.

Thanks

Trilby · 2020-12-24 13:42:48

Yes, I didn't say it was impossible. But instead of the single step:

pacman -Rsn chromium

You have to do many steps some interactive:

cd /path/to/PKGBUILD
# yup, you'll have to keep that PKGBUILD around somewhere
vim PKGBUILD
# move to the line with chromium, delete that line, save and exit ...
makepkg -si
pacman -Rsn chromium

Even if that last `pacman -Rsn chromium` command were not needed, the second code block would still be a lot more work than the first.

I didn't intend to for my point to be "100% convicing" that the PKGBUILD approach is a bad idea as I've yet to see any reason why it's a good idea. It adds work, adds complexity, adds potential problems, and it really does nothing for you that couldn't be done much simpler without these issues.

I don't want to be too harsh about Mr E's PKGBUILD approach - it was a creative and very interesting idea. But for your use case, it just doesn't make any sense to me.

Consider these alternatives:
1) A repo with all your config files and a simple text file listing explicit packages
2) A repo with a subset of a subset of config files and a PKGBUILD including a list of what should be explicitly installed packages (but will not end up being so) that in turn needs to be built and installed

What could be the advantage of the second one? The only possible advantage would be that it could be somewhat automated by using makepkg. But then you realize it's really only a very hackish approach to making makpkg run some shell script commands burried in a package function and constrained by all the limitations of a package. If only there were another way to automate shell script commands ... oh yeah, there is:

3) A repo with all your config files, a simple text file listing explicit packages, and a regular shell script that calls pacman to install the desired packages (appropriately as-explicit by default) and also moves all the config files into place (yes, all of them, not just a subset of a subset of them like the PKGBUILD would).

You could even keep all the config files in a filesystem tree within the repo (under a directory "configroot" in the example below) that would make the script just a couple of lines: 1 line for the pacman command, and one line to move all other content to the root of the running filesystem:

pacman -S - < explicit.pkgs
tar -cf - -C configroot . | tar -xpf - -C /

Last edited by Trilby (2020-12-24 13:58:11)

leonixyz · 2020-12-24 14:02:14

Trilby wrote:

...

Thanks man, excellent answer.

leonixyz · 2020-12-24 18:43:34

Trilby wrote:

pacman -S - < explicit.pkgs

This command executes perfectly fine after installing the base system and rebooting, but if I try to execute it into the arch-chroot, it fails after a certain amount of packages with extremely weird error messages:

# arch-chroot /mnt pacman -S - < packages.list
...
GPGME error: System error w/o errno
GPGME error: System error w/o errno
GPGME error: System error w/o errno
GPGME error: System error w/o errno
GPGME error: System error w/o errno
GPGME error: System error w/o errno
GPGME error: System error w/o errno
GPGME error: System error w/o errno
GPGME error: System error w/o errno
GPGME error: System error w/o errno
GPGME error: System error w/o errno
...

if I split my list in two parts, it works

# wc -l packages.list 
161 packages.list
# tail -n 81 packages.list > packages2.list
# head -n 80 packages.list > packages1.list 
# arch-chroot /mnt pacman -S - < packages1.list
...
# arch-chroot /mnt pacman -S - < packages2.list
...

Last edited by leonixyz (2020-12-24 20:55:03)

GSMiller · 2020-12-25 22:33:22

leonixyz in the above there is not enough context to answer: all output, list of packages might be required.

Are you trying to create an Arch installer? Trilby did not say it clearly, but I think he means it is not a good idea.

Arch Linux

#1 2020-12-23 15:15:38

semi-automatic arch installation and configuration

#2 2020-12-23 16:12:37

Re: semi-automatic arch installation and configuration

#3 2020-12-23 17:34:07

Re: semi-automatic arch installation and configuration

#4 2020-12-23 22:54:30

Re: semi-automatic arch installation and configuration

#5 2020-12-23 23:00:26

Re: semi-automatic arch installation and configuration

#6 2020-12-24 12:06:59

Re: semi-automatic arch installation and configuration

#7 2020-12-24 13:30:38

Re: semi-automatic arch installation and configuration

#8 2020-12-24 13:32:23

Re: semi-automatic arch installation and configuration

#9 2020-12-24 13:32:30

Re: semi-automatic arch installation and configuration

#10 2020-12-24 13:33:46

Re: semi-automatic arch installation and configuration

#11 2020-12-24 13:42:48

Re: semi-automatic arch installation and configuration

#12 2020-12-24 14:02:14

Re: semi-automatic arch installation and configuration

#13 2020-12-24 18:43:34

Re: semi-automatic arch installation and configuration

#14 2020-12-25 22:33:22

Re: semi-automatic arch installation and configuration

Board footer