ABS Pacman wrapper?

steeps · 2022-02-07 15:38:15

I feel I broadly understand makepkg/ABS/AUR, and how they all fit into the ecosystem. I'm using an AUR helper (yay) to manage my AUR packages, which is working fine, but to build packages from the official repo and install them, I haven't discovered a better workflow than asp -> makepkg -> pacman.

Is there anyone aware of a tool that automates this process? My best guess at the moment would be to somehow jury-rig an AUR helper to use the official repos rather than the AUR, but this sounds dubious at best, and I don't know if it'd work. I feel it would be fairly straightforward to script something up (my next step), but of course it'd take some effort to make it robust, add dependency management, updates, etc.. I ask because I recently gave FreeBSD a go, and discovered the synth tool which I was pretty impressed with --- I feel something which provides similar functionality with Arch would be a "nice-to-have". Having googled around, there seem to have existed tools in the past, but none in the present (for example, bauerbill --- whilst the last update on the AUR wasn't all that long ago, the upstream URL seems to be dead).

Not to worry if no-one is aware of anything, just thought I'd enquire.

Trilby · 2022-02-07 15:43:19

What would you want this tool to do? A vast majority of aur helpers features wouldn't apply to the ABS as they are already covered by pacman. Searching, dependency handling, etc. What's left?

If you want to (re)build/install a repo package, it's just two commands: asp and makepkg. If you wanted you could make your own wrapper script for this that would be all of two or three lines.

steeps · 2022-02-07 16:02:23

So what I had in mind was FreeBSD-style "rebuilding everything". If I take a package which has dependencies, and use makepkg, then either it'll complain it's missing dependencies, or with --syncdeps, it'll download the binaries with pacman. It'd be good if I had a tool which could recursively build and install dependencies, rather than stopping building one-level-deep. It'd also then be great to have the option to remove the build dependencies post build. Does that make sense?

Last edited by steeps (2022-02-07 16:12:12)

Trilby · 2022-02-07 16:36:36

steeps wrote:

It'd also then be great to have the option to remove the build dependencies post build.

There is. Please take some time to read the man / wiki page for makepkg. There are flags for removing build dependencies and for installing built packages without a separate call to pacman.

steeps wrote:

It'd be good if I had a tool which could recursively build and install dependencies, rather than stopping building one-level-deep.

Ok, that makes sense. Kinda'. If your goal is to build most/all packages from source, then why are you using arch linux? A source based distro would be a better fit for you (e.g., gentoo or one of it's derivatives).

What would be the point of automating such source-builds? If you aren't modifying the source of the PKGBUILD, you'd get basically the same thing that's already available in the repo. The only thing I see that you could benefit from would be the use of CFLAGS from makepkg.conf. But the benefits of this tend to be trivial for the average package. Perhaps rebuilding the occasional specific package just to get your preferred build flags would serve a purpose - but then you'd not care about rebuilding the dependencies, just the target package that needs to be optimized.

Last edited by Trilby (2022-02-07 16:43:46)

steeps · 2022-02-07 16:57:57

Trilby wrote:

steeps wrote:
It'd also then be great to have the option to remove the build dependencies post build.
There is. Please take some time to read the man / wiki page for makepkg. There are flags for removing build dependencies and for installing built packages without a separate call to pacman.

Ah, missed that, sorry about that. Thanks for pointing it out.

Trilby wrote:

The only thing I see that you could benefit from would be the use of CFLAGS from makepkg.conf.

Yeah, I agree. Still, I feel it'd be a neat thing to be able to do if you wanted to do that --- by the popularity of Gentoo, FreeBSD, etc., it's evident that enough people enjoy compiling all/most of their own packages from source.

Trilby · 2022-02-07 17:55:14

I don't know of a tool that would meet your exact goals, but I suspect it should be pretty easy to make your own script for it. Certainly there's a little more to it than the two-to-three-liner I originally proposed, though.

What makes this a good bit easier than an AUR helper with those features is that so long as you are rebuilding packages that you already have installed, build-order and dependency management are non-issues. (Build order in particular is the most non-trivial part of an aur helper for these goals).

So the tasks are "simply" to have asp download all the relevant package sources, then loop through directories building and installing each target. These could be kept as separate steps so you'd "asp export <whatever>" anytime you wanted to add a package to the build queue, the run another command for your builder script which might be as simple as the following:

#!/bin/sh

aspdir=/path/to/asp-downloaded-sources

for dir in ${aspdir}/*; do
   cd dir
   makepkg
done

pacman -U ${aspdir}/*/*.pkg.tar.zst

Here I opted to not use the -i flag for makepkg so that all packages could be installed in a single transaction at the end. The only reason for this is to minimize interactivity requiring only one prompt for a password. This could be an issue as there may be makedepends that are not yet installed, so perhaps the following just before the loop above would do:

for dir in ${aspdir}/*; do
   cd dir
   makepkg --printsrcinfo | sed -n 's/\s*makedepends = //p'
done | pacman -S --asdeps -

Then remove them again after the build with `pacman -Qdtq | pacman -Rsn -`.

Of course this doesn't add every dependency to the build queue (aka the asp directory). But another little script could handle that pretty well.

As I tinker with this, I realize that using asp may actually be a bit of a hinderance, as asp isn't really equipped to update the build directories (which would be needed to get updates but avoid unnecessary rebuilds). So a better approach would be a direct git / snv clone for each package. That way you can run a "pull" in each directory and attempt to run makepkg - if there is nothing to do, makepkg will report the package as up to date and exit.

Anyhow, hopefully some of this brainstorming could be useful. This could be an interesting little script to put together.

Last edited by Trilby (2022-02-07 17:55:33)

steeps · 2022-02-07 18:32:58

That's really helpful --- cheers for discussing Trilby. Will have a play over the next week and will report back if I come up with anything robust.

Alad · 2022-02-07 20:35:18

Build order in particular is the most non-trivial part of an aur helper for these goals

Well, since tsort is a thing, it's not really. But since most AUR helpers don't work with local repositories, pacman -U does make things more complex.

Having googled around, there seem to have existed tools in the past, but none in the present (for example, bauerbill --- whilst the last update on the AUR wasn't all that long ago, the upstream URL seems to be dead).

At the risk of touting my own horn, with aurutils you can treat a set of git repositories like AUR packages. The AUR-specific parts are decoupled, and you can combine tsort with the SRCINFO tools to retrieve the dependency order, and build the products to a local repository. You can also review official changes in PKGBUILDs interactively before merging them with your own PKGBUILDs. It supports chroots as well, e.g. to avoid hidden dependencies. I guess aurutils is the closest thing to synth there is for Arch-based systems.

See https://github.com/AladW/aurutils/issues/860 for some more discussion.

It does depend to what extent you want to build packages locally. It isn't always sufficient to only go by metadata in the PKGBUILD, and you need more complex tooling in that case. See for example https://lists.archlinux.org/pipermail/a … 30674.html

Last edited by Alad (2022-02-07 20:42:40)

Arch Linux

#1 2022-02-07 15:38:15

ABS Pacman wrapper?

#2 2022-02-07 15:43:19

Re: ABS Pacman wrapper?

#3 2022-02-07 16:02:23

Re: ABS Pacman wrapper?

#4 2022-02-07 16:36:36

Re: ABS Pacman wrapper?

#5 2022-02-07 16:57:57

Re: ABS Pacman wrapper?

#6 2022-02-07 17:55:14

Re: ABS Pacman wrapper?

#7 2022-02-07 18:32:58

Re: ABS Pacman wrapper?

#8 2022-02-07 20:35:18

Re: ABS Pacman wrapper?

Board footer