aurutils - helper tools for the aur

vap0rtranz · 2020-01-06 01:12:16

Are you using a custom XferCommand in your pacman.conf? curl and the internal downloader both support file:/// urls, but e.g. wget does not.

Ah, yes I'm using wget. My ISP is a crappy DSL with high TTLs and dropped packets, so I switched to wget after having issues syncing with pacman.

Reverting back to pacman's default XferCommand fixed this. Ty!

hakunamatata · 2020-01-15 08:41:56

I am using aurutils since quite some time now. I have setup two local repositories, one for vcs packages and a "normal" one. Recently, I have noticed that when I run the sync command using the --rebuild flag it tries to install spotify and note (in both repositories):

$ aur sync -d aur -cu --rebuild *

==> Using [aur] repository
  -> note: (none) -> 1.3.22-1
  -> spotify: (none) -> 1:1.1.10.546-2
==> Retrieving package files
From https://aur.archlinux.org/spotify
 = [up to date]      master     -> origin/master
From https://aur.archlinux.org/note
 = [up to date]      master     -> origin/master
Press any key to continue or ctrl+d to abort

$ aur sync -d aur-vcs -cu --rebuild *

==> Using [aur-vcs] repository
  -> note: (none) -> 1.3.22-1
  -> spotify: (none) -> 1:1.1.10.546-2
==> Retrieving package files
From https://aur.archlinux.org/spotify
 = [up to date]      master     -> origin/master
From https://aur.archlinux.org/note
 = [up to date]      master     -> origin/master
Press any key to continue or ctrl+d to abort

I have had spotify installed at some point. But since then I have removed it from the system. Unfortunatelly, I don't remember how exactly I installed it at the time - it might be possible that I have used yay to install it but I don't remember.

If I list all the packages in the repositories, neither spotify nor note does appear:

$ aur repo -d aur --list
$ aur repo -d aur-vcs --list

And if I run pacman to list foreign packages, nothing is listed:

$ pacman -Qm

Any ideas, what might be the problem and how it can be resolved?

Last edited by hakunamatata (2020-01-15 10:11:40)

Alad · 2020-01-15 15:09:23

Is spotify still in aur.db or aur-vcs.db? That's where aur-sync -u looks for updates.

Also, the * expands to whatever files are in the directory you run aur-sync from (usual shell behavior).

hakunamatata · 2020-01-15 19:13:52

Alad wrote:

Is spotify still in aur.db or aur-vcs.db? That's where aur-sync -u looks for updates.

Could not find in these files.

Alad wrote:

Also, the * expands to whatever files are in the directory you run aur-sync from (usual shell behavior).

You're right. In fact I still had a file called 'spotify' in my home dir, where I ran the commands. Instead when I explicitly list the packages it works as expected, thank you very much.

Angel_LB · 2020-01-20 16:00:33

Hello,

I have a problem, and I am not sure where I fumbled! No I don' think arch or aurutils are broken, I'm sure I did something silly..!

Every time I sync a package with "aur sync", I keep getting this error. :
==> ERROR: '/net/Vault/custompkgs/custom.db' does not have a valid database archive extension.

I rebuild my db multiple time, even changed path, looking online for hints on how to make a local repo and I can't figure out what I keep doing wrong.

 ~> aur repo
/net/Vault/custompkgs/custom.db

Should this command output the .db or .db.tar string ? Both files exists, btw, along with .files and .files.tar

I've been adding packages manually for months using repo-add after each build, successfully, but I'd like to fix this configuration so I don't have to, like everyone else.

Thanks!

eschwartz · 2020-01-20 16:02:52

repo-add requires you to operate on custom.db.tar, aurutils simply uses repo-add.

Angel_LB · 2020-01-20 20:45:19

From what I understand, 'aur sync' uses 'repo-add' in the end, but is not passing the correct argument, hence the error message. I am trying to figure out why.

When I use 'aur repo', as posted earlier, I get a '..custom.db' output and I was wondering if it was right or if I should be expecting a '..custom.db.tar' instead, which is the full db archive extension I have.
I figured aur sync must be using its aur repo sibling for such information.

I am trying to narrow down where my problem is.
I can't figure out if 'aur repo' is supposed to output a db or an archive db filename, which would point me somewhere in my research.

And thanks for the comment .

Alad · 2020-01-20 22:53:01

aur-build resolves the symbolic link provided by aur-repo.

It was introduced here https://github.com/AladW/aurutils/commi … 76efeb8823 which is questionable, feel free to send a PR to revert the change.

Last edited by Alad (2020-01-20 22:53:16)

Angel_LB · 2020-02-02 18:08:39

Hello Alad, and thanks for the reply.

Well, I don't know if this is good or not (sourced by aur-repo), but I would like to conform with what ever is expected. Redoing my local repo is not an issue -- I'd like to know what I should do!

aur-repo does output a ".db" filename here, how can I make it output the full database archive filename ".db.tar" so aur-build is satisfied?

Thanks!

Alad · 2020-02-04 10:29:00

readlink -e -- "$(aur repo)"

Last edited by Alad (2020-02-04 10:29:27)

f33dm3bits · 2020-05-05 17:16:14

Hello Alad,

I came across your tool for managing aur packages by using a local repo. I setup my custom repo and added the packages according to the section "Pacman Configuration" in man aur. Under the section optional I added the CachDir and CleanMethod options:

[aur]
SigLevel = Optional TrustAll
Server = file:///home/$USER/Downloads/aurpkgs
CacheDir = /var/cache/pacman/aurpkgs
CleanMethod = KeepCurrent

When I run a pacman -S packagename. I get the following warnings:

warning: config file /etc/pacman.conf, line 104: directive 'CacheDir' in section 'aur' not recognized.
warning: config file /etc/pacman.conf, line 105: directive 'CleanMethod' in section 'aur' not recognized.

When I check the man for pacman.conf the options do exist and I find this at the top of the man page:
Note: Each directive must be in CamelCase. If the case isn’t respected, the directive won’t be recognized. For example. noupgrade or NOUPGRADE will not work.

I literately copy/pasted the options so my casing for those options are correct. Am I missing something or not understanding something correctly?

Although my CacheDir option isn't recognized by pacman, the built packages seem to have been added to the location I created
my custom repo: /home/$USER/Downloads/aurpkgs. I used aur sync pkgname to sync my packages to my custom repo.

eschwartz · 2020-05-05 18:30:05

The problem seems self-evident, you're trying to list global options in a local scope and they aren't recognized at local scope.

f33dm3bits · 2020-05-05 18:34:44

I seemed to have read passed an import part in the man aur:

[options]
         CacheDir = /var/cache/pacman/pkg
         CacheDir = /var/cache/pacman/custom
         CleanMethod = KeepCurrent

I missed the [options] section. I changed it to:

[aur]
SigLevel = Optional TrustAll
Server = file:///home/$USER/Downloads/aurpkgs
[options]
CacheDir = /var/cache/pacman/aurpkgs
CleanMethod = KeepCurrent

That seems to have fixed it. So I guess that was a PEBKAC of mine. Did that address the global local scope problem?

eschwartz · 2020-05-05 18:37:04

Yes, that's precisely what I meant.

[options] is for global options, [reponame] scopes a setting to a specific repo.

Last edited by eschwartz (2020-05-05 18:37:56)

f33dm3bits · 2020-05-05 18:59:49

I guess it would be better then to also keep the default pacman CacheDir defined?
So like this?

[options]
CacheDir = /var/cache/pacman/pkg
CacheDir = /var/cache/pacman/aurpkgs

*edit*

Seems I should be fine if I use either one since the man page of pacman.conf says they are done in the order they are listed.
So may as well use the default path only then.

Learned something new about pacman.conf in the process of trying to figure something out for aurutils

Last edited by f33dm3bits (2020-05-05 19:18:19)

luscinius · 2020-05-21 15:29:59

Hi

I have started using aurutils recently. I really like the idea, and I am gradually rebuilding the packages I installed with yay. Need help with the following:

1. Some packages fail to build in a clean chroot. For examples, environment-modules-tcl needs procps-ng (ps binary) to build. In such cases should I submit a comment on the AUR page for the package specifying that? Am I right that the clean chroot is created using makrchroot command from devtools package (what set of packages is installed?), and all packages are expected to build using this environment?

2. Is there a way to use the chroot in a location other than /var/lib/aurbuild/x86_64/ by default (not passing the -D option). I know that it is possible to create an alias/bash function/wrapper script. The question is whether there is some config option to alter the default location.

Thanks for any help.

Trilby · 2020-05-21 15:41:36

luscinius wrote:

1. Some packages fail to build in a clean chroot.

Those are packaging bugs, report them to the AUR maintainers.

luscinius wrote:

For examples, environment-modules-tcl needs procps-ng (ps binary) to build. In such cases should I submit a comment on the AUR page for the package specifying that?

Assuming you mean env-modules-tcl, there are several other problems with that PKGBUILD. Yes, report them to the maintainer. Though it seems the dependency issue was reported to them a few months ago and they've not yet fixed it.

Alad · 2020-05-22 09:24:50

2. Is there a way to use the chroot in a location other than /var/lib/aurbuild/x86_64/ by default (not passing the -D option). I know that it is possible to create an alias/bash function/wrapper script. The question is whether there is some config option to alter the default location.

No. The design allows you to add configuration systems of arbitrary complexity by yourself. For example, you could override `/usr/bin/aur` in `/usr/local/bin/aur` and make it read a file like `aurutils.ini`:

[build]
chroot
noconfirm
directory=/var/lib/your/face/x9000

[sync]
ignore=mongodb

/usr/local/bin/aur

# /usr/bin/aur until line 28

opts=()
case $1 in
  # bundle sync and build options, fall through to specific sync options
  build|sync) while IFS='=' read -r key val; do if [[ $val ]]; then opts+=("$key" "$val"); else opts+=("$key"); fi; done < <(pacini --section=build aurutils.ini) ;;&
  sync) while IFS='=' read -r key val; do if [[ $val ]]; then opts+=("$key" "$val"); else opts+=("$key"); fi; done < <(pacini --section=sync aurutils.ini) ;;
  pkglist) # similar for other aur commands. care should be taken with tools used by other commands, such as aur-fetch and aur-depends
esac

if type -P "aur-$1" >/dev/null; then
    exec "aur-$1" "${opts[@]}" "${@:2}"
else
    printf >&2 '%s: %q is not an aur command\n' "$argv0" "$1"
    exit 1
fi

Since I don't use such a configuration system it doesn't make sense for me to add it by default.

That said, there's environment variables for source files and local repositories, i.e. AURDEST, AUR_DBROOT and AUR_REPO. I guess aur-chroot could support a similar variable for the chroot directory.

edit: toyed some more with the idea, see https://gist.github.com/AladW/83ab3f1e5 … 447363f86a

Last edited by Alad (2020-05-22 16:15:21)

MountainX · 2020-09-21 00:52:00

Alad wrote:

edit: toyed some more with the idea, see https://gist.github.com/AladW/83ab3f1e5 … 447363f86a

Thank you! That's very helpful. I have a few questions.

in my config file, should I prefix the options with dashes? For example:

[sync]
--ignore=brother-ql1060n

Or like this without the dashes?

[sync]
ignore=brother-ql1060n

Can the options in the config file be combined with command line options? For example, with the above in a config file, can I run:

aur sync -K ~/.config/aurutils/aurutils.conf -u

I noticed in your example config system wrapper that you define a default config file location as so:

XDG_CONFIG_HOME=${XDG_CONFIG_HOME:-$HOME/.config}
config_file=$XDG_CONFIG_HOME/aurutils/aurutils.conf

However, that default value doesn't seem to be used, because the case statement only does read_options() if the -K option and a file name is given on the command line, right?

I made an attempt to fix this here: https://gist.github.com/MountainX/8efe6 … 50c583d738

This version does read the default config file and it reads and sets the keys and values. However, it doesn't seem to be working completely correctly and I haven't figured out why. (I'm still learning aurutils.)

MountainX · 2020-09-21 00:57:08

see edits below. I have a working wrapper with one caveat.

I am getting unexpected results from the --ignore option with aur sync. I need to ignore the package brother-ql1060n (for reasons I explain in the comments on that package).

The working version is brother-ql1060n-1.0.5r0-1, while the current version brother-ql1060n-1.0.6r0-1 is broken.

However, when I run this command:

aur sync --ignore = brother-ql1060n -u

The result is:

1. Adding package 'brother-ql1060n-1.0.6r0-1-x86_64.pkg.tar.zst'
2. Removing existing entry 'brother-ql1060n-1.0.5r0-1'

That's exactly what I do not wish to have happen. Have I misunderstood this option? Or am I using it incorrectly?

Here is the entire output:

$ aur sync --ignore = brother-ql1060n -u
==> Using [myaur] repository
warning: ignoring = package
  -> brother-ql1060n: 1.0.5r0-1 -> 1.0.6r0-1
==> Retrieving package files
From https://aur.archlinux.org/brother-ql1060n
 = [up to date]      master     -> origin/master
Running makepkg --clean --syncdeps
==> Making package: brother-ql1060n 1.0.6r0-1 (Sun 20 Sep 2020 08:11:24 PM EDT)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
  -> Found ql1060nlpr-1.0.6-0.i386.rpm
  -> Found ql1060ncupswrapper-1.0.6-0.i386rpm.tar.gz
  -> Found LICENSE
==> Validating source files with sha256sums...
    ql1060nlpr-1.0.6-0.i386.rpm ... Passed
    ql1060ncupswrapper-1.0.6-0.i386rpm.tar.gz ... Passed
    LICENSE ... Passed
==> Extracting sources...
  -> Extracting ql1060nlpr-1.0.6-0.i386.rpm with bsdtar
  -> Extracting ql1060ncupswrapper-1.0.6-0.i386rpm.tar.gz with bsdtar
==> Starting prepare()...
==> Entering fakeroot environment...
==> Starting package()...
==> Tidying install...
  -> Removing libtool files...
  -> Purging unwanted files...
  -> Removing static library files...
  -> Stripping unneeded symbols from binaries and libraries...
  -> Compressing man and info pages...
==> Checking for packaging issues...
==> Creating package "brother-ql1060n"...
  -> Generating .PKGINFO file...
  -> Generating .BUILDINFO file...
  -> Adding install file...
  -> Generating .MTREE file...
  -> Compressing package...
==> Leaving fakeroot environment.
==> Finished making: brother-ql1060n 1.0.6r0-1 (Sun 20 Sep 2020 08:11:26 PM EDT)
==> Cleaning up...
==> Extracting myaur.db.tar.zst to a temporary location...
==> Extracting myaur.files.tar.zst to a temporary location...
==> Adding package 'brother-ql1060n-1.0.6r0-1-x86_64.pkg.tar.zst'
  -> Computing checksums...
  -> Removing existing entry 'brother-ql1060n-1.0.5r0-1'...
  -> Creating 'desc' db entry...
  -> Creating 'files' db entry...
==> Creating updated database file '/home/usercommon/cloud/HollyOaks/repos/myaur/x86_64/myaur.db.tar.zst'
:: Synchronizing package databases...
 myaur                                                                                                                       23.3 KiB  0.00   B/s 00:00 [#############################################################################################] 100%
:: Synchronizing package databases...
 myaur                                                                                                                        5.6 KiB  1868 KiB/s 00:00 [#############################################################################################] 100%
:: Starting full system upgrade...
 there is nothing to do

EDIT 1: I think the issue may be the spaces here:

--ignore = brother-ql1060n

I assumed spaces were OK because the wrapper in your earlier comment includes spaces around the options it adds. Maybe that's a bug?

EDIT 2: This version of the wrapper is working with a caveat:
https://gist.github.com/MountainX/8efe6 … 50c583d738
It works if given a new name like /usr/local/bin/aur2. However, it give this error if named /usr/local/bin/aur.

parse error: Invalid numeric literal at line 1, column 7
Cannot write to ‘-’ (Broken pipe).

I handled trimming of spaces for key and value in the wrapper. And the config file contents do need the dashes like this:

[sync]
--ignore=brother-ql1060n

With those fixes, I get the desired results:

$ aur2 sync -u
config file is: ~/.config/aurutils/aurutils.conf
adding option --ignore=brother-ql1060n
==> Using [myaur] repository
warning: ignoring brother-ql1060n package
  -> brother-ql1060n: 1.0.5r0-1 -> 1.0.6r0-1
    there is nothing to do

The remaining issue is how to name it as suggested in https://bbs.archlinux.org/viewtopic.php … 9#p1906209

Last edited by MountainX (2020-09-21 01:52:04)

HeptaSean · 2020-10-09 13:33:36

aur sync --chroot --upgrades

always prints out

warning: couldn't find or create package cache, using /tmp/ instead

during its run and it, in fact, seems to download all packages anew (at least after a reboot).

Is there some way to get it to use the “normal” package cache of pacman?

Stefan Husmann · 2020-10-09 14:22:49

Read the manpage about CacheDir.

HeptaSean · 2020-10-09 14:32:23

Stefan Husmann wrote:

Read the manpage about CacheDir.

That's not really it.

Of course, I have

[options]
CacheDir = /var/cache/pacman/pkg
CacheDir = /var/cache/pacman/aur
CleanMethod = KeepCurrent

there, This works well to put the compiled packages into /var/cache/pacman/aur and so, but it is not honoured by the pacman inside the chroot(s) during building the packages.

Alad · 2020-10-10 17:16:46

Try with 3.0.0rc and put your container pacman.conf to /usr/share/devtools/pacman-aur.conf.

HeptaSean · 2020-10-11 00:41:05

Alad wrote:

Try with 3.0.0rc and put your container pacman.conf to /usr/share/devtools/pacman-aur.conf.

Thank you very much! This works.

Seems a little strange having to create a file in /usr/share, though. After all, the FHS is pretty clear:

/usr is shareable, read-only data. That means that /usr should be shareable between various FHS-compliant hosts and must not be written to. Any information that is host-specific or varies with time is stored elsewhere.

There is some processing happening, which is described by man aur-chroot:

       -C FILE, --pacman-conf=FILE
              The  pacman.conf(5)  file  used  inside  the  container.  Defaults  to
              /usr/share/devtools/pacman-extra.conf.

              This  file  is processed with pacman-conf to bind-mount listed file://
              repositories.

This behaves odd if there are multiple CacheDir lines in the file (even ones commented out). The file in the root chroot then contains all collected cache paths in every one of these CacheDir lines and the file in the build chroot then seems to do another round of collection and concatenates these identical paths.

Moreover, since the processing adds all file:// repositories, I had to remove it with respect to my main /etc/pacman.conf file to prevent the path from being doubled.

All in all, it would seem easier to me to just not process the conf file at all and leave it to the users to create a sensible one (preferably in /etc, not in /usr).

Arch Linux

#376 2020-01-06 01:12:16

Re: aurutils - helper tools for the aur

#377 2020-01-15 08:41:56

Re: aurutils - helper tools for the aur

#378 2020-01-15 15:09:23

Re: aurutils - helper tools for the aur

#379 2020-01-15 19:13:52

Re: aurutils - helper tools for the aur

#380 2020-01-20 16:00:33

Re: aurutils - helper tools for the aur

#381 2020-01-20 16:02:52

Re: aurutils - helper tools for the aur

#382 2020-01-20 20:45:19

Re: aurutils - helper tools for the aur

#383 2020-01-20 22:53:01

Re: aurutils - helper tools for the aur

#384 2020-02-02 18:08:39

Re: aurutils - helper tools for the aur

#385 2020-02-04 10:29:00

Re: aurutils - helper tools for the aur

#386 2020-05-05 17:16:14

Re: aurutils - helper tools for the aur

#387 2020-05-05 18:30:05

Re: aurutils - helper tools for the aur

#388 2020-05-05 18:34:44

Re: aurutils - helper tools for the aur

#389 2020-05-05 18:37:04

Re: aurutils - helper tools for the aur

#390 2020-05-05 18:59:49

Re: aurutils - helper tools for the aur

#391 2020-05-21 15:29:59

Re: aurutils - helper tools for the aur

#392 2020-05-21 15:41:36

Re: aurutils - helper tools for the aur

#393 2020-05-22 09:24:50

Re: aurutils - helper tools for the aur

#394 2020-09-21 00:52:00

Re: aurutils - helper tools for the aur

#395 2020-09-21 00:57:08

Re: aurutils - helper tools for the aur

#396 2020-10-09 13:33:36

Re: aurutils - helper tools for the aur

#397 2020-10-09 14:22:49

Re: aurutils - helper tools for the aur

#398 2020-10-09 14:32:23

Re: aurutils - helper tools for the aur

#399 2020-10-10 17:16:46

Re: aurutils - helper tools for the aur

#400 2020-10-11 00:41:05

Re: aurutils - helper tools for the aur

Board footer