You are not logged in.

#1 2016-03-27 03:14:27

Alad
Wiki Admin/IRC Op/TU
From: Bagelstan
Registered: 2014-05-04
Posts: 1,874
Website

aurutils - helper tools for the aur

Why another AUR helper

With the AUR4 update quartering available AUR helpers, I found none which fitted the following criteria:

- Be modular, instead of monolithic
- Handle most user PKGBUILDs reliably
- Use a local repository

As well as few more uncommon features, such as topological sorting, PCRE searches, and devtools (systemd-nspawn) support.

In 2016, there's a lot of tools to simplify managing packages in Arch, so "starting late" has its benefits. Taking a modular approach also allows uses outside the AUR, such as building package chains from github.

Why a local repository

Some benefits of using a repository, rather than foreign packages:

- It handles the issue of installing split packages trivially
- pacman --files support
- All packages work with --asdeps
- Multiple repos for different tasks, e.g. version control packages
- Allows to easily distribute packages across machines
- AUR packages can be updated without having them installed
- You get to learn the methods of Arch Linux package maintainers

How to use it

Two packages are available on the AUR: aurutils and aurutils-git. I've kept the PKGBUILD simple, so you may not need all dependencies (for example, aurqueue only relies on coreutils and awk). See aurutils(7) for example uses, and details on setting up a local repository.

Caveats

The documentation needs improvements, but I've tried to keep the code clear and accessible. See the issue tracker for progress:

https://github.com/AladW/aurutils/issues

Cheers smile

Last edited by Alad (2016-05-18 14:12:25)


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby
Honest Alad's Package Emporium—Now with added bugs! (Grand reopening: December 1st 2018)

Offline

#2 2016-04-09 13:32:51

Alad
Wiki Admin/IRC Op/TU
From: Bagelstan
Registered: 2014-05-04
Posts: 1,874
Website

Re: aurutils - helper tools for the aur

aurutils is at 0.5.1 now. Some highlights since the initial release:

  • git diffs are now displayed together with build files

  • Update local repo packages in one step with aursync -u

  • Validation of tsort input via datamash

  • pkgbase, long format and raw support for aursearch

  • Ability to ignore packages with aursync -i

  • New script: aurstranger (update wrapper for other helpers)

  • Improved error reporting and general reliability

Last edited by Alad (2016-04-09 13:40:46)


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby
Honest Alad's Package Emporium—Now with added bugs! (Grand reopening: December 1st 2018)

Offline

#3 2016-04-10 20:29:54

Spyhawk
Member
Registered: 2006-07-07
Posts: 485

Re: aurutils - helper tools for the aur

Still no love for aurutils out there? Well, let me be the first to add a comment in this thread then.

Aurutils has been handily crafted after careful analysis of existing helpers and thorough consideration of what a "helper done right" should be. Helpers come and go, aurutils is still quite young but I'm pretty sure it will be around for a very long time. Definitely worth a look.

Offline

#4 2016-04-14 11:46:47

Alad
Wiki Admin/IRC Op/TU
From: Bagelstan
Registered: 2014-05-04
Posts: 1,874
Website

Re: aurutils - helper tools for the aur

Thanks spyhawk. Listening to common complaints on IRC also helped much. tongue

I wanted to try asciinema, so I've made two screencasts previewing the 0.7.0 release:

https://ptpb.pw/t/t58p
https://ptpb.pw/t/jBt1


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby
Honest Alad's Package Emporium—Now with added bugs! (Grand reopening: December 1st 2018)

Offline

#5 2016-04-14 13:55:04

runical
Member
From: The Netherlands
Registered: 2012-03-03
Posts: 896

Re: aurutils - helper tools for the aur

I finally got around to testing this and it works quite well (I already had a local repo set up, but this works better than my solution). Only the documentation seems to be lacking. Any way I could help with that?

Offline

#6 2016-04-14 14:10:13

Alad
Wiki Admin/IRC Op/TU
From: Bagelstan
Registered: 2014-05-04
Posts: 1,874
Website

Re: aurutils - helper tools for the aur

Yeah, there's an open issue for that:

https://github.com/AladW/aurutils/issues/76

Would be great to get some help on this (feel free to use whatever format you prefer, I can typeset it to groff).


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby
Honest Alad's Package Emporium—Now with added bugs! (Grand reopening: December 1st 2018)

Offline

#7 2016-04-14 14:17:04

runical
Member
From: The Netherlands
Registered: 2012-03-03
Posts: 896

Re: aurutils - helper tools for the aur

I'll see what I can do later today then. Seems pandoc also handles groff, so that should be alright then smile

Offline

#8 2016-04-14 15:17:24

Earnestly
Member
Registered: 2011-08-18
Posts: 805

Re: aurutils - helper tools for the aur

Try to avoid converting a format to groff with tools like pandoc, it will make a mess of it.  Just use whatever format you like and it'll be converted into a style defined by man-pages(7), no problem.

Offline

#9 2016-04-14 16:01:45

runical
Member
From: The Netherlands
Registered: 2012-03-03
Posts: 896

Re: aurutils - helper tools for the aur

Noted. I'll write things in markdown then and look into man-pages then. Thanks for the warning.

Offline

#10 2016-04-19 13:44:05

Alad
Wiki Admin/IRC Op/TU
From: Bagelstan
Registered: 2014-05-04
Posts: 1,874
Website

Re: aurutils - helper tools for the aur

0.7.0 was released:

https://github.com/AladW/aurutils/issue … s%3Aclosed

* man pages for most programs (thanks to runical)
* improved error reporting and structure (thanks to Earnestly)
* repofind -c removes local repo packages from /var/cache/pacman (built packages are kept in PKGDEST)
* repofind -u warns on local repo packages which are not in the AUR
* repoman respects packages in IgnorePkg and IgnoreGroup
* aurchain warns on dependencies that are both missing in AUR and the official repositories
* aurbuild passes all options after -- to either makepkg or makechrootpkg


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby
Honest Alad's Package Emporium—Now with added bugs! (Grand reopening: December 1st 2018)

Offline

#11 2016-05-11 19:44:09

Alad
Wiki Admin/IRC Op/TU
From: Bagelstan
Registered: 2014-05-04
Posts: 1,874
Website

Re: aurutils - helper tools for the aur

0.9.0 was released. This is a pre-release and currently requires aria2-git (1.23 should be released May 15th). Testing/bug reports welcome.

https://github.com/AladW/aurutils/releases/tag/0.9.0

Note: aursync now assumes the package pool is in /var/cache/pacman/<reponame>/. If you use it, see the new man page aurutils(7) for information on how to migrate.

Last edited by Alad (2016-05-11 19:47:11)


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby
Honest Alad's Package Emporium—Now with added bugs! (Grand reopening: December 1st 2018)

Offline

#12 2016-05-16 20:21:00

runical
Member
From: The Netherlands
Registered: 2012-03-03
Posts: 896

Re: aurutils - helper tools for the aur

Hey Alad, I appreciate the work you are putting in, but is there a solid reason why you would need to use two directories for one repository? This makes making a repository available across multiple computers a less straight cut use case (especially for me, as I want to sync the repo to several computers).

Last edited by runical (2016-05-16 20:21:18)

Offline

#13 2016-05-16 21:06:30

Alad
Wiki Admin/IRC Op/TU
From: Bagelstan
Registered: 2014-05-04
Posts: 1,874
Website

Re: aurutils - helper tools for the aur

repose creates symbolic links in the root of the repository to all packages, or reflinks if you prefer. Are the packages not accessible this way? As to the reasoning, see:

https://github.com/AladW/aurutils/releases/tag/0.9.0
https://github.com/AladW/aurutils/issues/129

It solved the issue of mismatches between pool and cache when rebuilding (same pkgver)  in an elegant way, and you get some added benefits like pacman -Sc removing old versions, or packages not being stored twice twice (e.g. for packages like Darkmod at 2.6 GB this would be especially noticeable).

That said, this is only "enforced" in aursync, and disabling it means commenting a single line, replacing with pool=$root :

https://github.com/AladW/aurutils/blob/ … rsync#L190

However as I removed the code which deleted cached packages while building (makepkg installs needed dependencies during the process), you'd have to increment pkgrel with setconf when rebuilding. See:

https://github.com/AladW/aurutils/issues/125

If you have a better suggestion to the above problematic, I'm happy to hear it as well.

Last edited by Alad (2016-05-16 21:11:58)


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby
Honest Alad's Package Emporium—Now with added bugs! (Grand reopening: December 1st 2018)

Offline

#14 2016-05-17 02:54:25

Alad
Wiki Admin/IRC Op/TU
From: Bagelstan
Registered: 2014-05-04
Posts: 1,874
Website

Re: aurutils - helper tools for the aur

Actually, it might be possible to have both the root and pool in /var/cache/pacman/foo. If this actually works, please open a github issue.

Last edited by Alad (2016-05-17 02:55:14)


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby
Honest Alad's Package Emporium—Now with added bugs! (Grand reopening: December 1st 2018)

Offline

#15 2016-05-17 09:46:12

runical
Member
From: The Netherlands
Registered: 2012-03-03
Posts: 896

Re: aurutils - helper tools for the aur

I'll take a look tonight. It just struck me as strange to separate the pool from the database. Thanks for the quick overview!

Offline

#16 2016-05-17 11:23:18

Alad
Wiki Admin/IRC Op/TU
From: Bagelstan
Registered: 2014-05-04
Posts: 1,874
Website

Re: aurutils - helper tools for the aur

It works for me:

https://github.com/AladW/aurutils/commi … e85ad85a9c

So when you test, please do so with the latest git checkout. smile


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby
Honest Alad's Package Emporium—Now with added bugs! (Grand reopening: December 1st 2018)

Offline

#17 2016-05-18 11:23:37

Alad
Wiki Admin/IRC Op/TU
From: Bagelstan
Registered: 2014-05-04
Posts: 1,874
Website

Re: aurutils - helper tools for the aur

I've went ahead and released 1.0.0, which also includes GPG signing.

As above, aursync now assumes Server matches CacheDir; when updating from 0.8.x, just uncomment CacheDir in /etc/pacman.conf, and add a second entry CacheDir = /path/to/your/repo.

CleanMethod = KeepCurrent is also a good idea to easily remove old versions from the pool with pacman -Sc.

Last edited by Alad (2016-05-18 11:23:46)


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby
Honest Alad's Package Emporium—Now with added bugs! (Grand reopening: December 1st 2018)

Offline

#18 2016-05-18 15:46:00

runical
Member
From: The Netherlands
Registered: 2012-03-03
Posts: 896

Re: aurutils - helper tools for the aur

I didn't get around to testing yesterday, but the new version works great! I have added the repositories as CacheDirs and everything works, including the syncing. Thanks for the work you are putting in!

Offline

#19 2016-05-19 19:46:38

beelzebub
Member
Registered: 2009-07-04
Posts: 4

Re: aurutils - helper tools for the aur

installed it and used it today...really nice work

Offline

#20 2016-05-28 05:19:58

Alad
Wiki Admin/IRC Op/TU
From: Bagelstan
Registered: 2014-05-04
Posts: 1,874
Website

Re: aurutils - helper tools for the aur

Thanks, glad I didn't break it. wink

1.0.1 was released.

https://github.com/AladW/aurutils/releases/tag/1.0.1

Note that with systemd 230 release, you now need to add a machine id to an existing nspawn container:

sudo systemd-machine-id-setup --root=/var/lib/aurbuild/root

Last edited by Alad (2016-05-28 05:32:03)


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby
Honest Alad's Package Emporium—Now with added bugs! (Grand reopening: December 1st 2018)

Offline

#21 2016-07-28 15:02:00

Alad
Wiki Admin/IRC Op/TU
From: Bagelstan
Registered: 2014-05-04
Posts: 1,874
Website

Re: aurutils - helper tools for the aur

1.2.3 now supports i686 containers on x86_64 (or generally, any supported architecture by setarch) with aurbuild -m <arch>. To avoid creating a new container for the current architecture, move the files:

# mkdir /var/lib/aurbuild/$(uname -m)
# mv /var/lib/aurbuild/* /var/lib/aurbuild/$(uname -m)

Last edited by Alad (2016-07-28 15:28:02)


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby
Honest Alad's Package Emporium—Now with added bugs! (Grand reopening: December 1st 2018)

Offline

#22 2016-08-27 21:04:04

PUE
Member
From: Earth/Europe/Austria
Registered: 2009-05-09
Posts: 51

Re: aurutils - helper tools for the aur

I tried to use aurutils, but without success so far. Can you please write up a 'getting started' guide?
I followed the man page and ended up with the following error when I tried to use aursync -c (after installing the required 'devtools'):

ERROR: '/var/lib/aurbuild/x86_64/root' does not appear to be an Arch chroot.

I tried to specify CHROOT (according to the devtools wiki page), but as far as I could tell, it had no effect.

I tried without '-c' but aborted when aursync showed me the dependencies. On each subsequent run, aursync claimed that the package was already up-to-date. The only solution I could come up with was to delete the cache directory and create it again. Back to square one.

I was hoping for a better cower. smile

Offline

#23 2016-08-27 21:59:46

Alad
Wiki Admin/IRC Op/TU
From: Bagelstan
Registered: 2014-05-04
Posts: 1,874
Website

Re: aurutils - helper tools for the aur

Well, man 7 aurutils is supposed to be the getting started guide...

Regarding the error, it may happen when the chroot wasn't set up properly (e.g., the process was interrupted). Note that aurbuild - which does the builds for aursync - ignores the CHROOT variable.

Please remove the /var/lib/aurbuild/ directory, and post the output of the following:

pacconf
bash -x aursync <insert aur packages you want to build>
bash -x aursync -c <insert aur packages you want to build>

Last edited by Alad (2016-08-27 22:00:39)


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby
Honest Alad's Package Emporium—Now with added bugs! (Grand reopening: December 1st 2018)

Offline

#24 2016-08-28 14:07:42

PUE
Member
From: Earth/Europe/Austria
Registered: 2009-05-09
Posts: 51

Re: aurutils - helper tools for the aur

Thanks Alad, simply removing /var/lib/aurbuild/ did the trick.
I did follow 'man 7 aurutils' but it still seems a bit overwhelming for a simple user.
How the wrong setup occured I do not know, it is a fresh Arch install and I only follwed instructions.
The improper package that showed up after I aborted the build also worries me a bit and smells like a bug. After all, I aborted before installing dependencies, yet there was a package in the cache.

Offline

#25 2016-08-30 02:25:32

Alad
Wiki Admin/IRC Op/TU
From: Bagelstan
Registered: 2014-05-04
Posts: 1,874
Website

Re: aurutils - helper tools for the aur

I did follow 'man 7 aurutils' but it still seems a bit overwhelming for a simple user.

The page uses terminology similar to repose, so it might help to read man repose first. Help:Reading might also help, as I follow a style similar to the wiki's. If you have particular suggestions on any parts that are unclear, feel free to share them.

The improper package that showed up after I aborted the build also worries me a bit and smells like a bug. After all, I aborted before installing dependencies, yet there was a package in the cache.

Well it's hard to tell what it is if you don't give me some logs, like the ones mentioned above. wink At this time, I don't even know which AUR package we're talking about.


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby
Honest Alad's Package Emporium—Now with added bugs! (Grand reopening: December 1st 2018)

Offline

Board footer

Powered by FluxBB