You are not logged in.

#101 2017-12-30 00:35:21

johnramsden
Member
Registered: 2017-08-07
Posts: 31

Re: aurutils - helper tools for the aur

I'm really impressed with this tool and how it can build packages in containers. I like the idea of keeping build dependencies separate from the final packages. Thanks for your work on this great utility Alad.

I'm wondering what the recommended way would be to migrate all packages that one might have had installed from the AUR to the local repository. Is there a recommended way to do this besides getting a list of all installed AUR packages and rebuilding, and reinstalling them?

Offline

#102 2017-12-31 02:01:31

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: aurutils - helper tools for the aur

If your previous AUR packages were saved via makepkg.conf(5) PKGDEST, or alternatively via some AUR helper that offers a separate option for saving the built package files, then all you need to do is move the the saved packages to the custom repo directory configured via aurutils, and `repo-add` the old files.

aurutils will then check the new custom repo entries for updates, just like any other AUR package in its repo.

If you allowed the older packages to be purged, for example building and storing them in /tmp and allowing them to be deleted at the next reboot, then you will have to rebuild them with aurutils in order to add them to your custom repo.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#103 2017-12-31 02:16:32

johnramsden
Member
Registered: 2017-08-07
Posts: 31

Re: aurutils - helper tools for the aur

Okay thanks, I'll just get a list and rebuild them. I was using pacaur and purging the older packages.

Offline

#104 2018-01-01 19:51:56

Ichimonji10
Member
Registered: 2010-11-07
Posts: 32
Website

Re: aurutils - helper tools for the aur

This is a nice set of tools, Alad. Thanks for making aurutils.

Several folks (e.g. in this thread and on aurutils #247) have expressed a desire for more or different "getting started" documentation. I understand that aurutils(7) is supposed to fulfill that role, and in the end, it worked for me. But I think I understand where other folks are coming from. I think people are wanting a story, e.g. "aurutils can populate a local repository with packages from the AUR, which pacman can then install. Here's how that might be done..."

Just something to chew on.

I whipped up an Ansible role that packages and installs software from the AUR, using aurutils. Y'all might find it of interest. I'm interested in hearing what I've done wrong.

Offline

#105 2018-01-01 23:21:12

bulletmark
Member
From: Brisbane, Australia
Registered: 2013-10-22
Posts: 649

Re: aurutils - helper tools for the aur

Ichimonji10 wrote:

"aurutils can populate a local repository with packages from the AUR, which pacman can then install."

Oh, so that's what aurutils does!?

Offline

#106 2018-01-01 23:33:53

Ichimonji10
Member
Registered: 2010-11-07
Posts: 32
Website

Re: aurutils - helper tools for the aur

It's one possible use. They could also be used to populate a remote repository. Or they could be used piecemeal by another application. Or something else.

EDIT: But yes, that's how I use it.

Last edited by Ichimonji10 (2018-01-02 01:32:16)

Offline

#107 2018-01-05 04:51:13

bgdawes
Member
Registered: 2017-02-01
Posts: 10

Re: aurutils - helper tools for the aur

I realize this is a dumb question, before I ask it, but can anyone let me know if I'm missing any steps to completely remove a package? I took the following steps below:

# pacman -Rsn foo
$ repo-remove /var/cache/pacman/custom/custom.db.tar foo
# pacman -Syu

Everything looks great after these steps, package is uninstalled and removed from my custom repository. However, I still see remnants of the package in:

/var/lib/aurbuild/x86_64/username/build
/var/lib/aurbuild/x86_64/username/build/usr/share
/var/lib/aurbuild/x86_64/username/build/var/lib/pacman/local
~/.cache/aursync/foo

Any obvious suggestions about what I'm doing wrong?


"The problem with the world is that the intelligent people are full of doubts, while the stupid ones are full of confidence."
- Charles Bukowski

Offline

#108 2018-01-05 06:23:27

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: aurutils - helper tools for the aur

~/.cache contains the cached PKGBUILD files, in case you decide to redownload and rebuild the package. /var/lib/aurbuild is the transient build directory used for chroot builds; as soon as you build another package with aurutils, it will be purged and used to create the other package -- at which point the reference will go away.
If you want, you can delete the package from ~/.cache/aursync.

tl;dr You didn't miss anything, nor did you do anything wrong.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#109 2018-01-05 13:57:58

phw
Member
Registered: 2013-05-27
Posts: 318

Re: aurutils - helper tools for the aur

Just wanted to say thanks for these tools. I had previously been using pacaur and manual git clone / makepkg for some packages, and when I first heard about aurutils maintaining a local repository felt a bit overkill. But after having a fresh new install on a new laptop I decided to got with aurutils, and I must say I love it. Due to the setup required and multiple separate tools it is a bit more work to get into it then other aur helpers or even basic manual building, but having the separate tools is quite flexible.

And a custom repository is awesome. I frequently switch between packages I test, and having e.g. both the stable and -git package in the repository makes it super easy to switch between them (as opposed to looking for the package file on my disk and / or rebuilding the package).

Offline

#110 2018-01-05 20:03:39

bgdawes
Member
Registered: 2017-02-01
Posts: 10

Re: aurutils - helper tools for the aur

Thanks for the response Eschwartz! I also really appreciate the background for context as well. I'm still relatively new to the linux world and this is very helpful!


"The problem with the world is that the intelligent people are full of doubts, while the stupid ones are full of confidence."
- Charles Bukowski

Offline

#111 2018-01-07 18:13:16

ewaller
Administrator
From: Pasadena, CA
Registered: 2009-07-13
Posts: 19,739

Re: aurutils - helper tools for the aur

Alad, I am loving these tools.  Thank you.

One question.  I note that when I sync things that are from source management systems (git, svn, hg) they sync (I beleive) to the HEAD of the source tree, not the version advertised by the AUR tools.  I am on board with this, but what is the behavior of aursync/aurcheck when the HEAD moves, but the AUR is not updated?  Do things rebuild in this case, or need this be performed manually?

ewaller@turing ~ 1050 %aurcheck -d my_foreign -a | grep \<\-
                                                   
avrdude-svn 20171129.1399-1 <- 20160328.1388-1
chessx-svn 1.4.8.2420-1 <- 1.3.4.2124-1
lemonbar-xft-git 268.043ad47-1 <- 245.38f69d8-1
libfprint-git 1:0.7.0.r7.gd35da0c-1 <- 1:0.7.0.r4.g69de32f-1
ltspice 4.23k-1 <- 4.23h-1
pyside2-common-git 2.0.0.r5383.2f4bfa56-1 <- 2.0.0.r5325.ad14f649-1
python-pyside2-git 2.0.0.r5383.2f4bfa56-1 <- 2.0.0.r5325.ad14f649-1
python-shiboken2-git 2.0.0.r5383.2f4bfa56-1 <- 2.0.0.r5325.ad14f649-1
python2-pyside2-git 2.0.0.r5383.2f4bfa56-1 <- 2.0.0.r5325.ad14f649-1
python2-shiboken2-git 2.0.0.r5383.2f4bfa56-1 <- 2.0.0.r5325.ad14f649-1
shiboken2-git 2.0.0.r5383.2f4bfa56-1 <- 2.0.0.r5325.ad14f649-1
urxvt-resize-font-git 14.97c91ec-1 <- 10.2bbde29-1
ewaller@turing ~ 1051 %

Having recently changed over, I just don't have enough data points to see a pattern based on my observations.

Last edited by ewaller (2018-01-07 18:16:01)


Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way

Offline

#112 2018-01-07 19:41:29

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: aurutils - helper tools for the aur

aurutils behaves no different from pacaur or yaourt in this respect, in that aursync -u will only detect available updates if the AUR is updated.

Both pacaur and yaourt have --devel flags which essentially say, "try updating the package anyway if it matches some arbitrary heuristic which is probably a VCS package".
aurutils does not (yet) implement special handling of VCS packages. Instead, you can explicitly list them using `aursync --no-ver "${pkglist[@]}"`.

Also see https://github.com/AladW/aurutils/issues/227 and https://github.com/AladW/aurutils/issues/65


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#113 2018-01-07 20:33:55

ewaller
Administrator
From: Pasadena, CA
Registered: 2009-07-13
Posts: 19,739

Re: aurutils - helper tools for the aur

Thanks Eschwartz,  Makes sense.
... And congrats on becoming a TU.


Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way

Offline

#114 2018-01-09 15:38:21

Lucki
Member
Registered: 2014-03-06
Posts: 10

Re: aurutils - helper tools for the aur

Hi Alad,
I'm trying out these tools and everything worked out so far. But there's something I can't figure out which is related to issue 242 when building in a container an unedited makepkg.conf is used:
The issue is about aursync and you've implemented -M in aurbuild. How can I set the makepkg.conf that aursync should use or forward to aurbuild? I think I need this to set an additional -T0 switch for faster xz compression and to set my GPG-Key while using aursync -cs. (With the stable package it was possible to use GPGKEY=XXXXX aursync -cs, that's not possible with the git version anymore.)

Anyway, thanks for these interesting tools!

Offline

#115 2018-01-09 21:29:15

carlitos
Member
Registered: 2016-01-20
Posts: 16

Re: aurutils - helper tools for the aur

Hi guys,

I've just recently moved from pacaur and now am getting to know these excellent tools Alad made. One question, if a package was updated with aursync -u in my custom repo, why am I still seeing the old pkg? I saw output like removing existing entry (in .db I suppose ) run paccache -rvvvk1 but it is still there. repo-add -R did not help either ( file cannot be found - so it's not in repo db, right? )
Can I simply delete the old pkg at this point?

Thanks


“We are what we repeatedly do. Excellence is not an act, but a habit.”

Offline

#116 2018-01-09 21:57:35

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: aurutils - helper tools for the aur

Yes, the old package files can be removed. repo-add does not remove them by default, and neither does aurutils (what would you do if you needed to downgrade for some reason?)


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#117 2018-01-09 22:31:50

carlitos
Member
Registered: 2016-01-20
Posts: 16

Re: aurutils - helper tools for the aur

Thanks for the clarification.
I thought tools like paccache ( and pkgcacheclean ) could clean up my custom repo as well as my regular pacman repo. [ I always do paccache -rvvvk1 ( and use downgrade/r if needed or snapper snapshots ) for regular pkgs ] I keep latest 3 of each my custom package in the cloud -to save space and practice cli rclone.

Thanks

Last edited by carlitos (2018-01-09 22:32:46)


“We are what we repeatedly do. Excellence is not an act, but a habit.”

Offline

#118 2018-01-09 23:09:40

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: aurutils - helper tools for the aur

paccache only cleans directories that are configured as a pacman.conf CacheDir. Which I do for my custom repo, but if you only use it as Server = file:/// then pacman will "download" the package extremely fast and make a copy in /var/cache/pacman/pkg/ and paccache will only see the extra copy.

Having it clean out your custom repo is not one of its design goals, just a happy coincidence. wink

...

The other option is to use repo-add -R at the time it initially adds the new version. Which would require modifying aurutils itself, as that is not currently exposed via an option.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#119 2018-01-10 16:12:11

Alad
Wiki Admin/IRC Op
From: Bagelstan
Registered: 2014-05-04
Posts: 2,407
Website

Re: aurutils - helper tools for the aur

SolarAquarion wrote:

I'm trying to build habs and i'm having issues with aurqueue, specifically wanting to build the PKGBUILDs in the pkgbuild directories that are created

Circular dependency. Fix your pkgbuilds.

Eschwartz wrote:

If you allowed the older packages to be purged, for example building and storing them in /tmp and allowing them to be deleted at the next reboot, then you will have to rebuild them with aurutils in order to add them to your custom repo.

Should not be required unless the installed packages are damaged, see paccheck + bacman.

Eschwartz wrote:

The other option is to use repo-add -R at the time it initially adds the new version. Which would require modifying aurutils itself, as that is not currently exposed via an option.

The amount of options to aursync/aurbuild is already daunting...

Eschwartz wrote:

aurutils does not (yet) implement special handling of VCS packages. Instead, you can explicitly list them using `aursync --no-ver "${pkglist[@]}"`.

Also see https://github.com/AladW/aurutils/issues/227 and https://github.com/AladW/aurutils/issues/65

I'd rather see https://bugs.archlinux.org/task/56602 fixed than implement hacks client-side.

Lucki wrote:

How can I set the makepkg.conf that aursync should use or forward to aurbuild? I think I need this to set an additional -T0 switch for faster xz compression and to set my GPG-Key while using aursync -cs.

https://github.com/AladW/aurutils/commi … 4de482e129

Lucki wrote:

(With the stable package it was possible to use GPGKEY=XXXXX aursync -cs, that's not possible with the git version anymore.)

Not sure why it would no longer be possible, can you provide some detail?

https://github.com/AladW/aurutils/blob/ … build#L109


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby

Offline

#120 2018-01-10 19:27:01

Lucki
Member
Registered: 2014-03-06
Posts: 10

Re: aurutils - helper tools for the aur

Hi Alad,
thanks for your answer and fast implementation!

I've done some further testing and found an error at your linked line 109. There's

gpg --listkeys

which should actually be

gpg --list-keys

. That's why GPGKEY=XXXXX aursync -cs don't work anymore.

However, if I'm using aursync -cs -M /etc/makepkg.conf where makepkg.conf has the same key set inside as GPGKEY="XXXXX" (and correctly used by makepkg --sign) the wrong key gets selected. Maybe I'm wrong here but shouldn't the same key be used here as with the other command above?
So for your troubleshooting:
1)-4) all ok here, I'm using aurutils-git to test smile
5) Maybe? Do I have to do special configuration inside the container?

Logs (--list-keys already fixed):

Last edited by Lucki (2018-01-10 20:03:18)

Offline

#121 2018-01-10 23:50:12

Alad
Wiki Admin/IRC Op
From: Bagelstan
Registered: 2014-05-04
Posts: 2,407
Website

Re: aurutils - helper tools for the aur

Lucki wrote:

I've done some further testing and found an error at your linked line 109.

Wow, not sure how that one slipped through the cracks... Good catch!

https://github.com/AladW/aurutils/commi … b00ac2391d

Lucki wrote:

However, if I'm using aursync -cs -M /etc/makepkg.conf where makepkg.conf has the same key set inside as GPGKEY="XXXXX" (and correctly used by makepkg --sign) the wrong key gets selected. Maybe I'm wrong here but shouldn't the same key be used here as with the other command above?

You're not wrong, there's however 3 methods available for signing:

1. makepkg --sign
2. manual signing
3. repo-add --sign

Method 2 is used because of FS #49946 which is fixed in pacman 5.1. Method 3 does not respect configuration in makepkg.conf.

I guess you could parse the given makepkg.conf for GPGKEY (using pacini), but then you'd have to consider even more cases: makepkg.conf given through -M or any of the standard locations used by makepkg. It may be better to add something to the man page instead.

Tracked in: https://github.com/AladW/aurutils/issues/272

Last edited by Alad (2018-01-10 23:58:11)


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby

Offline

#122 2018-01-11 01:00:42

Lucki
Member
Registered: 2014-03-06
Posts: 10

Re: aurutils - helper tools for the aur

So, if I understand that right:
You're using method 2 at the moment and want to switch to method 1 when pacman 5.1 with the fix ships out.
So this will be some time on hold. Got it!

Thanks for the explanation smile

Edit: Seems like I missed that method 3 is also used a few lines later. So setting the key as enviroment variable will also be needed with pacman 5.1 and the change to makepkg --sign…
If makepkg tells us which key is used to sign it would also be possible to catch the output and give that key with --key <key> to repo-add - instead of manually scraping the config file.

Last edited by Lucki (2018-01-11 01:28:18)

Offline

#123 2018-01-11 02:55:06

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: aurutils - helper tools for the aur

Alad wrote:
Eschwartz wrote:

If you allowed the older packages to be purged, for example building and storing them in /tmp and allowing them to be deleted at the next reboot, then you will have to rebuild them with aurutils in order to add them to your custom repo.

Should not be required unless the installed packages are damaged, see paccheck + bacman.

That will be difficult once pacman 5.1 removes it, since it has not been ported to pacman-contrib.

Alad wrote:
Eschwartz wrote:

The other option is to use repo-add -R at the time it initially adds the new version. Which would require modifying aurutils itself, as that is not currently exposed via an option.

The amount of options to aursync/aurbuild is already daunting...

I make no judgment calls on whether it *should*, merely on what the concept would entail. smile

Alad wrote:
Eschwartz wrote:

aurutils does not (yet) implement special handling of VCS packages. Instead, you can explicitly list them using `aursync --no-ver "${pkglist[@]}"`.

Also see https://github.com/AladW/aurutils/issues/227 and https://github.com/AladW/aurutils/issues/65

I'd rather see https://bugs.archlinux.org/task/56602 fixed than implement hacks client-side.

If only I really understood php well enough to implement this (or most anything else, really)...


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#124 2018-01-11 10:14:07

chicha
Member
From: France
Registered: 2007-04-20
Posts: 271

Re: aurutils - helper tools for the aur

Hello,

Do you plan to add recursive aur dependency handling within aursync ?
Else is

aurchain foobar | while read -r pkg _; do aursync -c $pkg; done

the recommended way to build an aur package and all its related aur dependencies, please ?

Offline

#125 2018-01-11 16:28:31

Ichimonji10
Member
Registered: 2010-11-07
Posts: 32
Website

Re: aurutils - helper tools for the aur

aursync does recursive AUR dependency handling. To build $pkg and all of its AUR dependencies, the following should suffice:

aursync "$pkg"

Offline

Board footer

Powered by FluxBB