aurutils - helper tools for the aur

chicha · 2018-01-11 17:22:36

Ichimonji10 wrote:

aursync does recursive AUR dependency handling. To build $pkg and all of its AUR dependencies, the following should suffice:
aursync "$pkg"

It is not working with gdm-plymouth I get the error :

aursync -c gdm-plymouth
.....
.....
error: target not found: plymouth
==> ERROR: 'pacman' failed to install missing dependencies.
...

I suspect this is due to the nature of gdm-plymouth package (multiple package with dependencies on one of its package).
Can you reproduce it on your side ?

Last edited by chicha (2018-01-11 17:22:50)

Ichimonji10 · 2018-01-11 20:26:18

Oddly, pacsearch can't find plymouth.

$ pacsearch plymouth
$

EDIT:

chicha wrote:

Can you reproduce it on your side?

Yes, I can. Of interest, aursync plymouth && aursync gdm-plymouth works. As an immediate hack to get gdm-plymouth installed, you can try that. I'm not sure what effect the -c flag will have. As in, will the two builds use separate containers? I need to learn more about systemd-nspawn.

Perhaps plymouth has some broken metadata in its package.

Last edited by Ichimonji10 (2018-01-12 15:04:03)

Alad · 2018-01-12 20:43:13

Seems to work for other people in #aurutils-complaints #archlinux. What version are you using? Please don't truncate logs.

Ichimonji10 · 2018-01-12 20:54:24

aurutils version 1.5.3-5.

$ aursync gdm-plymouth
  -> Using [aur] repository
==> Resolving dependencies...
  -> gdm-plymouth 0 -> 3.26.2.1-2
  -> libgdm-plymouth 0 -> 3.26.2.1-2
  -> plymouth 0 -> 0.9.3-3
==> Retrieving build files...
From https://aur.archlinux.org/gdm-plymouth
 = [up to date]      master     -> origin/master
From https://aur.archlinux.org/plymouth
 = [up to date]      master     -> origin/master
==> Making package: gdm-plymouth 3.26.2.1-2 (Fri Jan 12 15:49:41 EST 2018)
==> Checking runtime dependencies...
==> Installing missing dependencies...
error: target not found: plymouth
==> ERROR: 'pacman' failed to install missing dependencies.

And for what it's worth:

$ pacman -Qs plymouth
$ paclist aur
hivex 1.3.14-3
libguestfs 1.36.11-1
mnemosyne 2.6-2
ms-sys 1:2.4.1-2
perl-sys-virt 3.0.0-3
systemd-boot-pacman-hook 2-1
vim-badwolf-git 1:14fd1e1-1
vim-hemisu-git v3.4.4.g37ea6aa-1
vim-mark 2.8.5-1

chicha?

Last edited by Ichimonji10 (2018-01-12 20:54:43)

chicha · 2018-01-13 09:21:49

$ paclist aur
aurutils 1.5.3-5

$ pacman -Q repose devtools expac
repose 7.1-2
devtools 20171108-1
expac 8-1

$ aursync -c gdm-plymouth
  -> Using [aur] repository
==> Resolving dependencies...
  -> gdm-plymouth 0 -> 3.26.2.1-2
  -> libgdm-plymouth 0 -> 3.26.2.1-2
  -> plymouth 0 -> 0.9.3-3
==> Retrieving build files...
Depuis https://aur.archlinux.org/gdm-plymouth
 = [à jour]          master     -> origin/master
Depuis https://aur.archlinux.org/plymouth
 = [à jour]          master     -> origin/master
:: Synchronizing package databases...
 core                                                       126.8 KiB  1321K/s 00:00 [################################################] 100%
 extra                                                     1626.9 KiB  2.29M/s 00:01 [################################################] 100%
 community                                                    4.0 MiB  2.09M/s 00:02 [################################################] 100%
 aur                                                          4.0 KiB  0.00B/s 00:00 [################################################] 100%
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...

Packages (3) glib2-2.54.3-1  libsystemd-236.0-3  systemd-236.0-3

Total Download Size:    7.05 MiB
Total Installed Size:  33.87 MiB
Net Upgrade Size:       0.22 MiB

:: Proceed with installation? [Y/n] 
:: Retrieving packages...
(3/3) checking keys in keyring                                                       [################################################] 100%
(3/3) checking package integrity                                                     [################################################] 100%
(3/3) loading package files                                                          [################################################] 100%
(3/3) checking for file conflicts                                                    [################################################] 100%
:: Processing package changes...
(1/3) upgrading glib2                                                                [################################################] 100%
(2/3) upgrading libsystemd                                                           [################################################] 100%
(3/3) upgrading systemd                                                              [################################################] 100%
:: Running post-transaction hooks...
(1/4) Updating udev hardware database...
(2/4) Updating system user accounts...
(3/4) Creating temporary files...
(4/4) Arming ConditionNeedsUpdate...
==> Synchronizing chroot copy [/var/lib/aurbuild/x86_64/root] -> [chicha]...done
:: Synchronizing package databases...
 core                                                       126.8 KiB  1349K/s 00:00 [################################################] 100%
 extra                                                     1626.9 KiB  2.11M/s 00:01 [################################################] 100%
 community                                                    4.0 MiB  2.11M/s 00:02 [################################################] 100%
 aur is up to date
:: Starting full system upgrade...
 there is nothing to do
==> Making package: gdm-plymouth 3.26.2.1-2 (Sat Jan 13 10:19:56 CET 2018)
==> Retrieving sources...
  -> Found gdm-3.26.2.1.tar.xz
  -> Found 0002-Xsession-Don-t-start-ssh-agent-by-default.patch
  -> Found gdm.sysusers
==> Validating source files with sha256sums...
    gdm-3.26.2.1.tar.xz ... Passed
    0002-Xsession-Don-t-start-ssh-agent-by-default.patch ... Passed
    gdm.sysusers ... Passed
==> Making package: gdm-plymouth 3.26.2.1-2 (Sat Jan 13 10:19:57 CET 2018)
==> Checking runtime dependencies...
==> Installing missing dependencies...
error: target not found: plymouth
==> ERROR: 'pacman' failed to install missing dependencies.
==> ERROR: Build failed, check /var/lib/aurbuild/x86_64/chicha/build

Alad · 2018-01-14 13:44:19

aursync checks what packages to build by going over the repo db directly:

https://github.com/AladW/aurutils/blob/ … rcheck#L11
https://github.com/AladW/aurutils/blob/ … rsync#L189

So since you used --chroot, it's possible the changes to the repository weren't propagated to pacman (and thus to makepkg -s). That should be fixed by running "pacsync <yourrepohere>" first.

That's the best guess I have with the little information to go from here. Uninstall repose, rebuild the db, and try the git version if pacsync didn't help.

Last edited by Alad (2018-01-14 13:58:06)

progandy · 2018-01-14 16:34:46

The problem with gdm-plymouth is the package order after filtering for pkgbase. libgdm-plymouth is a split package from gdm_plymouth, but it doesn't depend on plymouth and is listed before plymouth. The split package is now inserted in its place and the second occurence of gdm_plymouth is removed.

This is with a fresh install of aursync 1.5.3-2, empty local repository, nothing plymouth related installed at all.

$ bash -c "set -x; source /bin/aursync" aursync gdm-plymouth
...
$ cd /tmp/aursync.3kaFTqQV
$ cat queue_0
check
dconf
glib2
gobject-introspection
intltool
systemd
yelp-tools
docbook-xsl
libdrm
pango
gnome-session
gnome-shell
libgdm-plymouth
plymouth
upower
xorg-server
xorg-server-xwayland
xorg-xhost
xorg-xrdb
gdm-plymouth

$ cat queue_1 
gdm-plymouth
plymouth

Edit: I assume that an additional call to aurqueue here might fix the problem.

Last edited by progandy (2018-01-14 16:44:17)

Alad · 2018-01-14 18:30:05

Can you check if it works with the git version? If so, it would be great if you could git bisect commits for aurchain and aursync. If the issue persists in git, please file it on github and I'll see what I can do.

Last edited by Alad (2018-01-14 18:35:15)

progandy · 2018-01-14 19:12:26

Alad wrote:

Can you check if it works with the git version? If so, it would be great if you could git bisect commits for aurchain and aursync. If the issue persists in git, please file it on github and I'll see what I can do.

It seems to be working since commit 91188d22a8b2c7cabd6f8fb49409250c6eab3b66 (aursync: use aurchain -t)

I'll have to file another bug report, though. Since ad85184d52814dba08251439e80b441ee31cea88 I always get "There is nothing to do" after resolving dependencies.

Alad · 2018-01-14 21:04:22

Of course both are monolithic beasts of commits... Thanks for investigating!

@chicha: Please use the -git version for now. It's probably less effort to just release 1.6.0 than to try and fix 1.5.3.

chicha · 2018-01-15 07:47:07

@Alad, @progandy : thank you for your investigation and your time. On my side I solved the issue by installing plymouth first flagging it "asdeps". I will switch to the git version until the 1.6.0 is out.

Lucki · 2018-01-17 00:49:52

Hi Alad,

is there a reason you're checking aur dependencies only against the given repository? Line 231: aurcheck -d "$repo" -r "$root" -c
I've followed the recommendation in your setup guide and have two repositories for aur and aur-vcs.
If there's a dependent non-vcs package which is already built in the aur repository aursync want to build it again into the aur-vcs repository. In the end aur dependencies would possibly be duplicated in both repositories. Using double build time and disk space.

I would expect that packages are checked against both repositories and aursync don't build them again if there's already the required package in a different than the given repository.
This would also add to the option --no-ver. When no version check is made the given package and all dependencies will be rebuild and added to the given repository instead of being rebuild and added to the repository where the last version was. (I haven't tested this)

Is this expected behavior?

For testing I've used aurutils-git:

lucki@Archlinux ~ % aursync -cs --repo aur-vcs osu-lazer-git    
  -> Using [aur-vcs] repository
==> Resolving dependencies...
  -> mono-stable N/A -> 5.4.1.6-1
  -> nuget4 N/A -> 4.4.1-0
==> Retrieving build files...

lucki@Archlinux ~ % pacman -Ss mono-stable
aur/mono-stable 5.4.1.6-1
    Stable version of free .NET implementation.

lucki@Archlinux ~ % pacman -Ss nuget4
aur/nuget4 4.4.1-0
    Package manager for .NET.

lucki@Archlinux ~ % aursync -cs --repo aur osu-lazer-git
  -> Using [aur] repository
==> Resolving dependencies...
  -> osu-lazer-git N/A -> 2018.113.0_9_g25d444554-1
==> Retrieving build files...

Alad · 2018-01-17 18:56:12

Some related issues:

https://github.com/AladW/aurutils/issues/105
https://github.com/AladW/aurutils/issues/178

In particular, local repos may need to be self-contained, such as when sharing them with others. You can use e.g. --ignore to achieve the behavior you describe.

johnramsden · 2018-01-25 03:44:54

Is there a way to build packages in tmpfs, while still using chroot? If you set certain flags in /etc/makepkg.conf, will they be respected? Meaning can I set BUILDDIR=/tmp/makepkg, and my packages will be built there? Or will I have to do something to mount the tmpfs directory in the chroot?

My gut tells me I can't just set it in makepkg.conf - if I can't what would you recommend to build packages in memory?

eschwartz · 2018-01-25 03:54:11

aurutils uses makechrootpkg for its chroot builds, and makepkg.conf is only ever loaded by makepkg *inside* the chroot.

But, makechrootpkg has the -r parameter for specifying the directory that the chroot is in. And, aurutils hardcodes /var/lib/aurbuild/$(uname -m) as that directory.

You can, however, mount that directory as a tmpfs. See for example /usr/lib/systemd/system/tmp.mount which automates this for /tmp
For example, https://pkgbuild.com (the Arch Linux build server) does this for /var/lib/archbuild

johnramsden · 2018-01-25 06:29:47

@Eschwartz thanks that's helpful. I guess I'll just mount a tmpfs directory.

johnramsden · 2018-01-25 06:44:22

Actually, would it be possible to mount a directory into the chroot directory? So that only the single directory where things are getting compiled in is running in tmpfs? I'm also wondering how this works with the temp option.

The problem I have is I manually need to install some packages into the chroot that don't get installed automatically, so I can't leave the entire chroot in volatile storage.

Alad · 2018-01-25 09:17:57

Eschwartz wrote:

And, aurutils hardcodes /var/lib/aurbuild/$(uname -m) as that directory.

I'm not hardcoding anything, it's just a default. You can specify the chroot directory from aurbuild, aurbuild_chroot, and aursync.

johnramsden wrote:

Actually, would it be possible to mount a directory into the chroot directory? So that only the single directory where things are getting compiled in is running in tmpfs? I'm also wondering how this works with the temp option.

Not sure. You could somehow try that the root container is stored on disk and the rsync'ed clones go to tmpfs.

Last edited by Alad (2018-01-25 09:18:20)

johnramsden · 2018-01-25 17:55:39

@Alad Cool! Thanks.

eschwartz · 2018-01-25 23:26:39

Alad wrote:

Eschwartz wrote:
And, aurutils hardcodes /var/lib/aurbuild/$(uname -m) as that directory.
I'm not hardcoding anything, it's just a default. You can specify the chroot directory from aurbuild, aurbuild_chroot, and aursync.
johnramsden wrote:
Actually, would it be possible to mount a directory into the chroot directory? So that only the single directory where things are getting compiled in is running in tmpfs? I'm also wondering how this works with the temp option.
Not sure. You could somehow try that the root container is stored on disk and the rsync'ed clones go to tmpfs.

...

I am not sure how I missed that. I've gotten too used to reading the code instead of the manpage, I think, and then I get error-prone.

Cody Learner · 2018-02-26 09:32:48

Nice job Alad + team!

I finally got around to setting up a pacman repo called aur for aurutils. (pacman and AUR in the same sentence just "sounds" wrong!) I have a few hours invested at this point and doing some testing. Very impressed with the results so far.

Only tested the basics, aursync to install monero-wallet-qt and played around manually with aurqueue. Tried setting a custom AURDEST in my .bashrc, but it didn't set on my last test install... Getting late and I'm tired.

My go to test for scripting a dependency details funct effort was monero-wallet-qt. It seemed to have something strange with dependencies going on. When I did a test install the first time, (not using aurutils) I'll just say it at it turned into somewhat of a pita to install.

Using ~~aurqueue~~ aurchain -a to see what it found for deps came up with 3 or 4 extras.... or so I thought and that's in addition to 18 being listed. Seems aurqueue was smart enough to list the dep's for the other package that was going to need installed!

Then aursync monero-wallet-qt did everything in one step, except nudge me to fill in my password prompt on the last package to install. It was going so well and takes a long time to build, I got distracted. Pretty sure the password prompt timed out? The package was built and ready for install so I did another aursync monero-wallet-qt. Got the following message. I installed it with pacman all is good. My own fault and need to spend more time with aurutils....

$ aursync monero-wallet-qt
  -> Using [aur] repository
==> Resolving dependencies...
    there is nothing to do

Did I miss an uninstall command? Something that would remove the git related and whatever other stuff along with an auto pacman -Rns package?

Last edited by Cody Learner (2018-02-27 02:54:19)

Alad · 2018-02-26 10:57:19

When the sudo prompt timed out probably the step that was missing was to propogate the database changes to pacman. You can repeat that step with "pacsync aur". I can expand on sudo timeouts later

Did I miss an uninstall command? Something that would remove the git related and whatever other stuff along with an auto pacman -Rns package?

Not sure what you mean?

Cody Learner · 2018-02-26 17:16:00

Thanks Alad,

I ran:

$ sudo pacsync aur
aur is up to date

I may have ran some pacman commands after the sudo timeout last night and before pacsync this morning. Could the pacman -U for the install have possibly updated it?

Did some reading up on sudo timeout... The default 5 min seems reasonable. I guess one could use -T to set it if actually needed.

quote sudo man:

If authentication is required, sudo will exit if the user's password is not entered within a configurable time limit. This limit is policy-specific; the default password prompt timeout for the sudoers security policy is 5 minutes.
-T timeout, --command-timeout=timeout

On the uninstall question, should have been "remove package". I meant to say I didn't see a remove package command in aurutils yet. One could use pacman to remove the package, then manually deal with any leftovers as preferred. I was just curious if something in aurutils automated the process as described.

On second thought, that would be going way beyond pacmans normal remove package process. Are package created dir's/files under users home ever cleaned up by package removal?

Edit: more questions

I thought I had a problem with pacman databases as pacman -Qqm didn't show packages installed by aursync. But this is by design with the additional user created pacman repo? So if this is correct, aursift will come in handy. My little "package" script uses pacman -Qqm and -Qqn to check for install, etc... Doesn't give correct results now. Probably make use of aursift in it now.

$ pacman -Qqn | aursift 2>/dev/null
cower
libmonero-wallet
monero-wallet-qt

Last edited by Cody Learner (2018-02-26 21:54:08)

Alad · 2018-02-27 08:15:24

I may have ran some pacman commands after the sudo timeout last night and before pacsync this morning. Could the pacman -U for the install have possibly updated it?

You can check if the package is in your repo with pacman -Si, e.g. pacman -Si monero-wallet-qt. If it's not there and pacsync reports "up to date" I assume the build process was aborted before repo-add was run. You can add it again with:

repo-add /path/to/aur.db /path/to/package.tar.xz

Replacing the "/path/to" with the path to your repo.

I guess one could use -T to set it if actually needed.

As I read it, -T just exits the command if the given timeout is reached, compare timeout(1).

On the uninstall question, should have been "remove package". I meant to say I didn't see a remove package command in aurutils yet. One could use pacman to remove the package, then manually deal with any leftovers as preferred. I was just curious if something in aurutils automated the process as described.

There's some reports on the aurutils github concerning this but there's no real need to add this to aurutils. I use pacman -Sc with KeepInstalled (deletes all repo packages that aren't installed, another benefit of using a CacheDir) and remove stale entries with repoctl. If you use repose, you can just rerun it as described in aurutils(7).

https://github.com/cassava/repoctl

Are package created dir's/files under users home ever cleaned up by package removal?

No, not really.

I thought I had a problem with pacman databases as pacman -Qqm didn't show packages installed by aursync. But this is by design with the additional user created pacman repo? So if this is correct, aursift will come in handy. My little "package" script uses pacman -Qqm and -Qqn to check for install, etc... Doesn't give correct results now. Probably make use of aursift in it now.

You can use aursift but it's simpler to list the repository (as seen by pacman) directly.

pacman -Sl aur

effeeffe · 2018-02-28 10:12:19

Hi, I have a problem with updating the spotify package (though it worked in the past). From today I get the following message:

$ aursync -u
  -> Using [aur] repository
==> Resolving dependencies...
  -> spotify 1.0.70.399-1 -> 1.0.72.117-1
==> Retrieving build files...
From https://aur.archlinux.org/spotify
 = [up to date]      master     -> origin/master
fatal: couldn't read spotify/.git/packed-refs: Not a directory

Other packages update fine. I don't really use any "advanced" feature of aurutils, and IIRC I followed the instructions after installing aurutils.
Did I mess up? Is it a problem with aurutils or with the package?

Arch Linux

#126 2018-01-11 17:22:36

Re: aurutils - helper tools for the aur

#127 2018-01-11 20:26:18

Re: aurutils - helper tools for the aur

#128 2018-01-12 20:43:13

Re: aurutils - helper tools for the aur

#129 2018-01-12 20:54:24

Re: aurutils - helper tools for the aur

#130 2018-01-13 09:21:49

Re: aurutils - helper tools for the aur

#131 2018-01-14 13:44:19

Re: aurutils - helper tools for the aur

#132 2018-01-14 16:34:46

Re: aurutils - helper tools for the aur

#133 2018-01-14 18:30:05

Re: aurutils - helper tools for the aur

#134 2018-01-14 19:12:26

Re: aurutils - helper tools for the aur

#135 2018-01-14 21:04:22

Re: aurutils - helper tools for the aur

#136 2018-01-15 07:47:07

Re: aurutils - helper tools for the aur

#137 2018-01-17 00:49:52

Re: aurutils - helper tools for the aur

#138 2018-01-17 18:56:12

Re: aurutils - helper tools for the aur

#139 2018-01-25 03:44:54

Re: aurutils - helper tools for the aur

#140 2018-01-25 03:54:11

Re: aurutils - helper tools for the aur

#141 2018-01-25 06:29:47

Re: aurutils - helper tools for the aur

#142 2018-01-25 06:44:22

Re: aurutils - helper tools for the aur

#143 2018-01-25 09:17:57

Re: aurutils - helper tools for the aur

#144 2018-01-25 17:55:39

Re: aurutils - helper tools for the aur

#145 2018-01-25 23:26:39

Re: aurutils - helper tools for the aur

#146 2018-02-26 09:32:48

Re: aurutils - helper tools for the aur

#147 2018-02-26 10:57:19

Re: aurutils - helper tools for the aur

#148 2018-02-26 17:16:00

Re: aurutils - helper tools for the aur

#149 2018-02-27 08:15:24

Re: aurutils - helper tools for the aur

#150 2018-02-28 10:12:19

Re: aurutils - helper tools for the aur

Board footer