Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

karol · 2010-12-22 09:17:23

Xyne wrote:

The other discrepancies are indeed due to the way it searches the AUR. The AUR does not understand regular expressions so searching for e.g. "-mt$" will not return anything because no package contains "$" in its name. You have to pass the results through grep to find the ones that end in "-mt".

I see you fixed it now for non-AUR searches and I'm happy with that :-)

Xyne wrote:

I may rework the way it searches though now that I'm thinking about it. In the meantime, please update to the latest version of perl-xyne-* from my site and let me know if it works more as expected.

Updated, works great. Thanks again, Xyne.
I like the separation of results from repos from those from AUR.

[karol@black ~]$ bauerbill -Ssq --aur -- -mt
haskell-mtl
ypbind-mt
courier-mta

chicken-random-mtzig
dbus-core-mt
eclipse-mtj
...

I may be wrong but I haven't seen it before - good job.

Xyne wrote:

Also note that you do not need to use "sudo" for searches.

For some reason I put 'sudo' in front of every 'bauerbill / pacman -S[x]'. I'm dumb.

pogeymanz · 2010-12-22 18:14:17

Xyne wrote:

Army wrote:
I just wanted to give clyde and bauerbill a try, because aurbuild doesn't do the job good enough for me anymore.
So I activated bauerbill's AUR support and chose a user for the build process in the config file. Then I ran a bauerbill -Syu. First the repos were synced, everything fine so far, but as soon as bauerbill starts looking for updates in the AUR, the RAM usage grows and grows, more than 300mb!! Why is that? Did I do something wrong? If not, how can I help to solve this?
That's a problem with Perl and threading. Each AUR query is run in a parallel thread, but each thread duplicates all variables in memory. I have a plan to deal with that, but it will be a while before I can implement it.

Just wondering if this plan is solidifying at all?

It's just that updating my system slows me down to a chug with bauerbill and I'm far too hyperactive to let bauerbill do its thing without me trying to do 10 others. So this is a big reason for me to stay with yaourt for updates even though it's horrible if I have a bunch of AUR packages to update.

Xyne · 2010-12-22 18:29:01

pogeymanz wrote:

Xyne wrote:
That's a problem with Perl and threading. Each AUR query is run in a parallel thread, but each thread duplicates all variables in memory. I have a plan to deal with that, but it will be a while before I can implement it.

Just wondering if this plan is solidifying at all?
It's just that updating my system slows me down to a chug with bauerbill and I'm far too hyperactive to let bauerbill do its thing without me trying to do 10 others. So this is a big reason for me to stay with yaourt for updates even though it's horrible if I have a bunch of AUR packages to update.

Have you tried using the "--threads" option, e.g. "bauerbill --threads 1 ..."? Start at 1 and work up from there to find a balance between speed and memory usage. I added that as an intermediate solution.

As for the plan, it is solidifying, but very slowly. It feels as though I'm climbing several mountains but only have time to take a few steps every week, and sometimes not even that.

ngoonee · 2010-12-23 10:22:44

bauerbill -S dwarffortress
--> resolving lib32-gtk2... installed
--> resolving lib32-mesa... installed
--> resolving lib32-sdl_image... found in [multilib]
--> resolving lib32-libsndfile... installed
--> resolving lib32-openal... installed
--> resolving lib32-libxdamage... installed
--> resolving lib32-ncurses... installed
--> resolving lib32-sdl_ttf... found in [multilib]
--> resolving lib32-sdl... found in [multilib]
--> resolving lib32-libpng... installed
--> resolving lib32-libjpeg... installed
--> resolving lib32-libtiff... installed
--> resolving lib32-zlib... installed
--> resolving sdl_image... installed
--> resolving lib32-libxext... installed
--> resolving lib32-libxrender... installed
--> resolving lib32-libx11... installed
--> resolving sdl... installed (provided by sdl-pulse)
--> resolving lib32-freetype2... installed
--> resolving sdl_ttf... installed

Happens on -Syu etc. as well, it resolves dependencies and doesn't actually install anything. Fully updated system from [testing] and [xyne-any]. sudo pacman -S ** works.

donbex · 2010-12-23 14:49:39

First of all, I would like to thank Xyne for his work.

I've been using bauerbill for a while and I'm really happy with it.
Today, though, I am experiencing an odd bug. When trying to install an aur package it correctly resolves the dependecies, but it doesn't switch to makepkg. E.g.:

$sudo bauerbill -S --aur canto

WARNING: canto does not match any package or group in sync db
--> Searching for AUR packages...
--> Loading AUR taurball data...
WARNING: about to source /tmp/bauerbill/build/aur/canto/PKGBUILD
view [v]  backup [b]  edit [e]  continue [ok]  skip [s]  abort [x]  v
view [v]  backup [b]  edit [e]  continue [ok]  skip [s]  abort [x]  ok
--> resolving ncurses... installed
--> resolving python2... installed
--> resolving python2-chardet... installed
canto

Manually calling makepkg works as expected.

I really don't have a clue about solving this issue...

splippity · 2010-12-23 16:16:29

ngoonee wrote:

bauerbill -S dwarffortress
--> resolving lib32-gtk2... installed
--> resolving lib32-mesa... installed
--> resolving lib32-sdl_image... found in [multilib]
--> resolving lib32-libsndfile... installed
--> resolving lib32-openal... installed
--> resolving lib32-libxdamage... installed
--> resolving lib32-ncurses... installed
--> resolving lib32-sdl_ttf... found in [multilib]
--> resolving lib32-sdl... found in [multilib]
--> resolving lib32-libpng... installed
--> resolving lib32-libjpeg... installed
--> resolving lib32-libtiff... installed
--> resolving lib32-zlib... installed
--> resolving sdl_image... installed
--> resolving lib32-libxext... installed
--> resolving lib32-libxrender... installed
--> resolving lib32-libx11... installed
--> resolving sdl... installed (provided by sdl-pulse)
--> resolving lib32-freetype2... installed
--> resolving sdl_ttf... installed

Happens on -Syu etc. as well, it resolves dependencies and doesn't actually install anything. Fully updated system from [testing] and [xyne-any]. sudo pacman -S ** works.

Im having the same problem.

Xyne · 2010-12-23 19:10:59

It should be fixed now (perl-xyne-arch>=2010.12.23.1).
Sorry about that. It was due to leftover debugging code.

donbex · 2010-12-24 14:24:03

Xyne wrote:

It should be fixed now (perl-xyne-arch>=2010.12.23.1).
Sorry about that. It was due to leftover debugging code.

Thank you, it is. ^^

osmano807 · 2010-12-24 20:27:21

Is there anything I can do to reduce memory consumption? Enough to destabilize the system when beginning an update...

pogeymanz · 2010-12-24 20:39:07

osmano807 wrote:

Is there anything I can do to reduce memory consumption? Enough to destabilize the system when beginning an update...

Use the --threads option to limit the threads to 1 or 2. I've been running with --threads 2 and it's much nicer.

osmano807 · 2010-12-24 22:35:24

pogeymanz wrote:

Use the --threads option to limit the threads to 1 or 2. I've been running with --threads 2 and it's much nicer.

It worked, thanks:)

Last edited by osmano807 (2010-12-24 22:35:53)

karol · 2011-01-04 22:57:29

I'm trying to get http://aur.archlinux.org/packages/xnvie … p/PKGBUILD working on my i686: makepkg has no problems with it, bauerbill has.
I have 'CARCH="i686"'in my /etc/makepkg.conf and I get:

WARNING: about to source /tmp/bauerbill/build/aur/xnviewmp/PKGBUILD
view [v] backup [b] edit [e] continue [ok] skip [s] abort [x] ok
--> resolving lib32-glib2... unresolvable
--> resolving lib32-libpng... unresolvable
--> resolving lib32-expat... unresolvable
--> resolving bash... installed
--> resolving libarchive>=2.7.1... installed
--> resolving libfetch>=2.25... installed
--> resolving pacman-mirrorlist... installed
WARNING: unresolved dependencies: xnviewmp -> lib32-expat (not found)
:: Do you want to ignore the above packages for this operation? [Y/n]

What am I doing wrong? Is the PKGBUILD malformed?

skunktrader · 2011-01-04 23:02:04

multilib ?

karol · 2011-01-04 23:05:46

skunktrader wrote:

multilib ?

Huh?
I don't have multilib enabled - I'm on i686 so the lib32- packages are not needed.

makepkg installs the package OK, but bauerbill says it needs a bunch of lib32- packages.

ngoonee · 2011-01-04 23:47:17

karol wrote:

I'm trying to get http://aur.archlinux.org/packages/xnvie … p/PKGBUILD working on my i686: makepkg has no problems with it, bauerbill has.
I have 'CARCH="i686"'in my /etc/makepkg.conf and I get:
WARNING: about to source /tmp/bauerbill/build/aur/xnviewmp/PKGBUILD
view [v] backup [b] edit [e] continue [ok] skip [s] abort [x] ok
--> resolving lib32-glib2... unresolvable
--> resolving lib32-libpng... unresolvable
--> resolving lib32-expat... unresolvable
--> resolving bash... installed
--> resolving libarchive>=2.7.1... installed
--> resolving libfetch>=2.25... installed
--> resolving pacman-mirrorlist... installed
WARNING: unresolved dependencies: xnviewmp -> lib32-expat (not found)
:: Do you want to ignore the above packages for this operation? [Y/n]
What am I doing wrong? Is the PKGBUILD malformed?

Has to do with bauerbill not being able to parse those sections which basically go

if x86_64 then;

and so on. For example with wine-pulse, everytime there's an update I must manually edit the PKGBUILD to remove those if lines and the i686 lines.

Xyne knows about this, and says he will not change it because one of the principles is that bauerbill should not execute the PKGBUILD (or something like that), which would be necessary for all these hacky ways of changing depends and makedepends.

Of course, that would mean that bauerbill couldn't handle the [multilib] PKGBUILDs =p

karol · 2011-01-04 23:52:14

Ah, OK. Thanks for the explanation, ngoonee. As it's trivial to edit the PKGBUILD it's not a big problem.

Xyne · 2011-01-05 12:32:11

ngoonee wrote:

For example with wine-pulse, everytime there's an update I must manually edit the PKGBUILD to remove those if lines and the i686 lines.

Why don't you use the AutoPatch option for that? You should only have to edit it once, unless the maintainer is overhauling the PKGBUILD with each update.

ngoonee wrote:

Xyne knows about this, and says he will not change it because one of the principles is that bauerbill should not execute the PKGBUILD (or something like that), which would be necessary for all these hacky ways of changing depends and makedepends.

That's not exactly the reason. It's true that bauerbill does not actually execute any bash code itself. I wrote a simplified parser that handles the most common stuff but writing a full bash parser that can handle all of those "hacks" is not easy.

If there were no split packages then it would be possible to just source PKGBUILDs to extract the values, but split packages bury variables inside of functions (the different build and package functions for each split package). Sourcing the PKGBUILD would not interpolate those variables because the code in the functions is not run. The only way to get those values is to run the code in the functions, but that would start building or packaging the package, which we obviously don't want to do at that point.

I am working on a solution and have been for a while, but I haven't had any real free time in months and what I have in mind requires mountains of time and work. You'll understand why when I finally manage to release it, but I have no idea when that will be. This will be fixed eventually though, and in the meantime the AutoPatch option should hide the problem for all but the first build.

Last edited by Xyne (2011-01-05 12:33:42)

ngoonee · 2011-01-05 16:48:28

Ah, you know, I just never thought of using autopatch for that . It'll fit nicely I'm sure (will check the next time wine-pulse upgrades). Thanks Xyne.

pogeymanz · 2011-01-07 00:28:39

Is there a good way to make bauerbill operate on AUR only? As in

bauerbill -Su

would only upgrade AUR packages, and

bauerbill -Ss

would only search AUR packages.

I love the way bauerbill operates, but I'm trying to be a bit more modular in my administration, so I'm preferring to use pacman + AUR helper.

For now, I just point bauerbill to an empty file for its pacman.conf, but it complains about not having any repos configured. Not a big deal really, just a little unclean looking.

Keep up the excellent work.

Xyne · 2011-01-07 16:57:34

No, there is no way to disable pacman in bauerbill, but it only downloads the packages, All installation is done directly with pacman, so I really can't see any reason to disable the repos, plus it needs them to manage dependency resolution when building packages.

ngoonee · 2011-01-07 23:03:50

--> Searching for upgradable AUR packages...
Use of uninitialized value $json in substitution (s///) at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 284.
Use of uninitialized value $json in substr at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 421.
Use of uninitialized value in numeric lt (<) at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 291.
Use of uninitialized value $json in substr at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 439.
Use of uninitialized value $json in substr at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 443.
Use of uninitialized value $json in substr at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 447.
Use of uninitialized value $json in substitution (s///) at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 284.
Use of uninitialized value $json in substr at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 421.
Use of uninitialized value in numeric lt (<) at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 291.
Use of uninitialized value $json in substr at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 439.
Use of uninitialized value $json in substr at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 443.
Use of uninitialized value $json in substr at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 447.
Use of uninitialized value $json in substitution (s///) at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 284.
Use of uninitialized value $json in substr at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 421.
Use of uninitialized value in numeric lt (<) at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 291.
Use of uninitialized value $json in substr at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 439.
Use of uninitialized value $json in substr at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 443.
Use of uninitialized value $json in substr at /usr/share/perl5/vendor_perl/Xyne/Common/JSON.pm line 447.

Fully updated from [xyne-any] - this started happening after latest update (to 2011.01.07.1-1).

ngoonee@ngoonee-laptop ~ % pacman -Qi perl-xyne-arch
Name           : perl-xyne-arch
Version        : 2011.01.07.1-1
URL            : http://xyne.archlinux.ca/projects/perl-xyne-arch
Licenses       : GPL
Groups         : None
Provides       : None
Depends On     : perl  perl-libwww  perl-xyne-common>=1.0.3
Optional Deps  : sudo
                 perl-crypt-ssleay: for reflector support
                 python: for rankmirrors support
Required By    : bauerbill  powerpill  reflector
Conflicts With : None
Replaces       : None
Installed Size : 408.00 K
Packager       : Xyne
Architecture   : any
Build Date     : Fri 07 Jan 2011 08:08:08 AM MYT
Install Date   : Fri 07 Jan 2011 10:14:17 AM MYT
Install Reason : Installed as a dependency for another package
Install Script : No
Description    : Xyne's Arch-specific Perl modules.

ngoonee@ngoonee-laptop ~ % pacman -Qi perl-xyne-common
Name           : perl-xyne-common
Version        : 2011.01.07.1-1
URL            : http://xyne.archlinux.ca/projects/perl-xyne-common
Licenses       : GPL
Groups         : None
Provides       : None
Depends On     : perl
Optional Deps  : None
Required By    : bauerbill  perl-xyne-arch  powerpill
Conflicts With : None
Replaces       : None
Installed Size : 128.00 K
Packager       : Xyne
Architecture   : any
Build Date     : Fri 07 Jan 2011 08:08:10 AM MYT
Install Date   : Fri 07 Jan 2011 10:14:16 AM MYT
Install Reason : Installed as a dependency for another package
Install Script : No
Description    : Xyne's common Perl modules.

ngoonee@ngoonee-laptop ~ % cat /var/log/pacman.log | grep 2011.01.07.1-1
[2011-01-07 10:14] upgraded perl-xyne-common (2010.12.21.1-3 -> 2011.01.07.1-1)
[2011-01-07 10:14] upgraded perl-xyne-arch (2010.12.26.1-1 -> 2011.01.07.1-1)

Xyne · 2011-01-08 19:55:12

I can't reproduce that error here but I've added a check that should catch it. Btw, the error was generated by a module in perl-xyne-common, not perl-xyne-arch. The last update included a rewrite of the JSON parser and pointed Reflector at the official mirrorlist API on archlinux.org.

ngoonee · 2011-01-11 06:12:43

Ah sorry forgot to report back. Yep that works (at least the error doesn't show up anymore).

flan_suse · 2011-01-20 10:56:18

Not sure if this has been covered, but is it normal behaviour for bauerbill to invoke sudo when removing a package, but not when installing a package?

If I do the following, I am prompted by sudo for the root password:

bauerbill -Rs packagename

But if I do this, I am told I have no write permissions (so I need to preceed the command with sudo, manually):

bauerbill -S packagename

* drops privileges to build when run as root, checks for sudo to install packages when run as non-root

This seems to only work when removing packages, not when installing them.

ngoonee · 2011-01-20 13:13:38

It works here for installing AND removing.

Arch Linux

#926 2010-12-22 09:17:23

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#927 2010-12-22 18:14:17

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#928 2010-12-22 18:29:01

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#929 2010-12-23 10:22:44

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#930 2010-12-23 14:49:39

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#931 2010-12-23 16:16:29

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#932 2010-12-23 19:10:59

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#933 2010-12-24 14:24:03

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#934 2010-12-24 20:27:21

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#935 2010-12-24 20:39:07

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#936 2010-12-24 22:35:24

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#937 2011-01-04 22:57:29

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#938 2011-01-04 23:02:04

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#939 2011-01-04 23:05:46

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#940 2011-01-04 23:47:17

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#941 2011-01-04 23:52:14

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#942 2011-01-05 12:32:11

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#943 2011-01-05 16:48:28

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#944 2011-01-07 00:28:39

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#945 2011-01-07 16:57:34

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#946 2011-01-07 23:03:50

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#947 2011-01-08 19:55:12

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#948 2011-01-11 06:12:43

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#949 2011-01-20 10:56:18

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#950 2011-01-20 13:13:38

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

Board footer