You are not logged in.

#1 2015-12-04 18:19:30

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

Project page: http://xyne.archlinux.ca/projects/bauerbill/
AUR page: https://aur.archlinux.org/packages/bauerbill/

Bauerbill is a versatile ABS and AUR helper. It is an extension of Powerpill that can build packages from AUR and ABS with full dependency resolution. It provides trust management (e.g. trust some users for all packages, trust others for certain packages, trust others for particular releases that you have verified) and provides extensive support for user hooks to apply patches and affect other changes while building packages. It correctly handles all metadata (e.g. split packages, architecture-specific metadata): see the table.

Being an extension of Powerpill, it also speeds up downloads of databases and packages.

Original Post
It's been a while since I've posted anything new but 'tis this season! I've finally managed to rewrite a (imo) worthy successor to Bauerbill. big_smile

For those of you who've never heard of the original Bauerbill*, it was an extension of Powerpill that could build packages from ABS and the AUR (nearly) transparently. It was in the same class of AUR helpers as yaourt. This iteration is slightly different. It generates build scripts that can be configured with custom commands and hooks per package.


It's fresh off the keyboard so expect some bugs. Please report what you find (or just give feedback in general). Check the project page for information about what it can do and how you can configure it. Things will likely change a bit in the coming weeks as I play around with it and get feedback. The main work is done but there is a lot of polishing left.


* Rhymes with Powerpill, means Bill the Builder... the name is clunky but it amuses me, and you can alias it to "bb" wink

Last edited by Xyne (2016-04-05 16:17:15)


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#2 2015-12-08 10:06:22

SanskritFritz
Member
From: Budapest, Hungary
Registered: 2009-01-08
Posts: 1,923
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

WOW. Welcome back, Bauerbill.
Thanks Xyne.


zʇıɹɟʇıɹʞsuɐs AUR || Cycling in Budapest with a helmet camera || Revised log levels proposal: "FYI" "WTF" and "OMG" (John Barnette)

Offline

#3 2015-12-14 22:24:49

SanskritFritz
Member
From: Budapest, Hungary
Registered: 2009-01-08
Posts: 1,923
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

How do I set the working directory? I would like to reuse my existing one.


zʇıɹɟʇıɹʞsuɐs AUR || Cycling in Budapest with a helmet camera || Revised log levels proposal: "FYI" "WTF" and "OMG" (John Barnette)

Offline

#4 2015-12-14 22:25:33

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

I have added some usage examples to the webpage here.

I have also added a "--build-vcs" option to rebuild VCS packages. The patterns for VCS packages can be configured in bauerbill.json. You can easily add a custom package to the array to force it to always be rebuilt with that option.

SanskritFritz wrote:

How do I set the working directory? I would like to reuse my existing one.

What's the structure of your working directory? If it just contains the package directories then you should be able to run the download and build scripts in it. You would also use the --build-dir option in that case with the wrapper script.

If you want custom SRCDEST and PKGDEST then you can create a custom makepkg configuration file and add it to the makepkg commands (either the default or a per-package command) in bauerbill.json.

I will probably add some more configuration options for per-package build locations. I need to think about how to do it.


Feel free to make suggestions. Same for anything else in the package, e.g. the bb-wrapper script. I may create a second one without "sudo", or make it optional in the script via a command line argument.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#5 2015-12-15 12:17:05

SanskritFritz
Member
From: Budapest, Hungary
Registered: 2009-01-08
Posts: 1,923
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

--build-dir was my solution, thanks, Xyne.


zʇıɹɟʇıɹʞsuɐs AUR || Cycling in Budapest with a helmet camera || Revised log levels proposal: "FYI" "WTF" and "OMG" (John Barnette)

Offline

#6 2015-12-15 21:07:03

SanskritFritz
Member
From: Budapest, Hungary
Registered: 2009-01-08
Posts: 1,923
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

I get an error when running this:

~> bauerbill -Su --aur-only

WARNING: multiinfo query (kde-wallpapers): no results
...
WARNING: Installed orphans: kde-wallpapers ...
Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.5/site-packages/Bauerbill.py", line 1362, in <module>
    run_main()
  File "/usr/lib/python3.5/site-packages/Bauerbill.py", line 1346, in run_main
    return main(args)
  File "/usr/lib/python3.5/site-packages/Bauerbill.py", line 1324, in main
    scripts['download'] = download_script(bb_config, pargs, build_pkgs)
  File "/usr/lib/python3.5/site-packages/Bauerbill.py", line 836, in download_script
    'info' : sh_info_comments(pkgs),
  File "/usr/lib/python3.5/site-packages/Bauerbill.py", line 645, in sh_info_comments
    maintainers=' '.join(pkg.maintainers()),
TypeError: sequence item 0: expected str instance, NoneType found

zʇıɹɟʇıɹʞsuɐs AUR || Cycling in Budapest with a helmet camera || Revised log levels proposal: "FYI" "WTF" and "OMG" (John Barnette)

Offline

#7 2015-12-15 21:52:08

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

fixed (I think)


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#8 2015-12-15 23:32:13

SanskritFritz
Member
From: Budapest, Hungary
Registered: 2009-01-08
Posts: 1,923
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

Xyne wrote:

fixed (I think)

Confirmed, thanks!


zʇıɹɟʇıɹʞsuɐs AUR || Cycling in Budapest with a helmet camera || Revised log levels proposal: "FYI" "WTF" and "OMG" (John Barnette)

Offline

#9 2015-12-16 10:01:17

SanskritFritz
Member
From: Budapest, Hungary
Registered: 2009-01-08
Posts: 1,923
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

There is a false warning, when checking for installed non existing packages.
Example:

WARNING: multiinfo query (pzl_games): no results

This is probably because pzl_games is a base package, but it does exist, but of course it cannot be installed directly, for it provides plz_* packages (https://aur.archlinux.org/pkgbase/pzl_games/).


zʇıɹɟʇıɹʞsuɐs AUR || Cycling in Budapest with a helmet camera || Revised log levels proposal: "FYI" "WTF" and "OMG" (John Barnette)

Offline

#10 2015-12-16 13:40:46

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

That's not a false warning. When running a system upgrade, the query arguments are taken directly from the names of installed foreign packages. If you have a package named "pzl_games" installed then it will search for it on the AUR if it can't be found in the repos. Even if it coincides with an existing package base, there is no package with that name.

If there is no way to upgrade the package via pacman or bauerbill then you could just add it to the list of ignored packages to avoid the warning (assuming that you want to keep it installed).


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#11 2015-12-16 14:55:47

SanskritFritz
Member
From: Budapest, Hungary
Registered: 2009-01-08
Posts: 1,923
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

Oh, you're right, sorry. I checked it again, and it was indeed installed. I accidentally installed it long time ago according to pacman.log. Sorry for the noise.


zʇıɹɟʇıɹʞsuɐs AUR || Cycling in Budapest with a helmet camera || Revised log levels proposal: "FYI" "WTF" and "OMG" (John Barnette)

Offline

#12 2015-12-18 02:57:33

rlevArch
Member
Registered: 2011-12-04
Posts: 35

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

Hi Xyne,

Just getting back into using my arch system after many years. I'm having trouble with bauerbill, getting the following error:

~  bauerbill -Syu
ERROR: LockError(/var/lib/pacman/db.lck): failed to lock database [[Errno 13] Permission denied: '/var/lib/pacman/db.lck'] 

db.lck does not exist in that directory, and pacman is not running elsewhere. If I run pacman -Syu or use pacaur -Syu, everything updates correctly.

_________
Edit: nvm, looks like using bb-wrapper instead of bauerbill alone does the trick.

Last edited by rlevArch (2015-12-18 03:00:39)

Offline

#13 2015-12-18 20:39:42

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

Hi rlevArch,

The error says that you do not have permission to create the file, not that it exists. "bauerbill -Syu" requires root privileges, just like "pacman -Syu'.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#14 2016-01-12 23:34:30

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

Made some changes:

  • Renamed option --build-targets to --build.

  • Added option --nobqd for generating build scripts without pre-emptive dependency installation. The scripts can now be created without ever needing root.

  • Added possibility to trust maintainer-pkgbase combinations with bb-query_trust.

  • Added colorized output to bb-query_trust (screenshot below).

bb-query_trust-screenshot.png


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#15 2016-01-14 04:58:39

SanskritFritz
Member
From: Budapest, Hungary
Registered: 2009-01-08
Posts: 1,923
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

Handling of orphaned packages is a bit problematic. When I run ./download.sh I get this:

INFO: searching AUR
INFO: found gtk-theme-orion at https://aur4.archlinux.org/cgit/aur.git/snapshot/gtk-theme-orion.tar.gz
INFO: found lib32-libpng12 at https://aur4.archlinux.org/cgit/aur.git/snapshot/lib32-libpng12.tar.gz
INFO: found libpng12 at https://aur4.archlinux.org/cgit/aur.git/snapshot/libpng12.tar.gz
usage: bb-query_trust [-h] [-c <path>] [-l <path>]
                      <package base> <modification time> <maintainer>
                      [<maintainer> ...]
bb-query_trust: error: the following arguments are required: <maintainer>

It would be nice to know which one is orphan and honestly I don't know what bauerbill should do. What should happen with orphan packages?


Also, when I press ctrl-c at the prompt, I'd expect bauerbill to exit, instead it continues:

Untrusted combination:
  Package Base:         lib32-libpng12
  Modification Time:    1451369309 [2015-12-29 06:08:29 GMT]
  Maintainers:
    arthurzam

You may inspect the files at the following location before proceeding:
  /home/frank/build/lib32-libpng12

Options:
  P: trust this combination once and proceed
  M: add maintainer(s) to list of trusted users
  C: add combination to list of trusted combinations

  Press any other key if you do not trust this combination.

[PMC] ^CUntrusted combination:
  Package Base:         libpng12
  Modification Time:    1451369269 [2015-12-29 06:07:49 GMT]
  Maintainers:
    arthurzam

You may inspect the files at the following location before proceeding:
  /home/frank/build/libpng12

Options:
  P: trust this combination once and proceed
  M: add maintainer(s) to list of trusted users
  C: add combination to list of trusted combinations

  Press any other key if you do not trust this combination.

zʇıɹɟʇıɹʞsuɐs AUR || Cycling in Budapest with a helmet camera || Revised log levels proposal: "FYI" "WTF" and "OMG" (John Barnette)

Offline

#16 2016-01-14 07:54:57

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

Orphans and ctrl+c should now be handled correctly. Thanks for reporting.

edit
Added support for "-Qu --aur" and new option `--bb-quiet` to suppress AUR query warnings.

Last edited by Xyne (2016-01-14 10:26:35)


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#17 2016-01-23 20:12:20

tedbell
Member
Registered: 2012-08-04
Posts: 167

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

When updatign aur packages I get this error:

Running makepkg as root is not allowed as it can cause permanent,
catastrophic damage to your system.

How can I update AUR packages with bauerbill as user?

Offline

#18 2016-01-24 18:52:37

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

How are you running bauerbill? It doesn't run the generated AUR scripts automatically so you must be trying to run them directly as root.

Please see the usage examples on the project page. Basically there are two main ways to do it. Either run bauerbill as root then run the generate build scripts as a normal user, or just use the bb-wrapper script, e.g.

bb-wrapper -Syu --aur

My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#19 2016-01-25 00:52:55

tedbell
Member
Registered: 2012-08-04
Posts: 167

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

Xyne wrote:

How are you running bauerbill? It doesn't run the generated AUR scripts automatically so you must be trying to run them directly as root.

Please see the usage examples on the project page. Basically there are two main ways to do it. Either run bauerbill as root then run the generate build scripts as a normal user, or just use the bb-wrapper script, e.g.

bb-wrapper -Syu --aur

I was running it as

sudo bb-wrapper -Syu --aur

thanks for the suggestions

Offline

#20 2016-01-25 21:30:37

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

Since Friday's version bb-wrapper should drop privileges when necessary when run as root.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#21 2016-02-13 17:12:43

Alad
Wiki Admin/IRC Op
From: Bagelstan
Registered: 2014-05-04
Posts: 2,407
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

Hi Xyne,

When retrieving the AUR deps for plasma-desktop-git with bauerbill --aur --nobqd plasma-desktop-git -S, it seems to miss two makedepends:

extra-cmake-modules-git
kdoctools-git

Depends list from download.sh: https://ptpb.pw/Ei96
Depends list from tsort: https://ptpb.pw/j7PW
download.sh: https://ptpb.pw/XV7l/sh

Last edited by Alad (2016-02-13 17:14:54)


Mods are just community members who have the occasionally necessary option to move threads around and edit posts. -- Trilby

Offline

#22 2016-02-14 06:23:11

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

It should be fixed now. It will also generate the clean-up script now for nobqd to remove leftover deps and makedeps.
edit: Both bauerbill and pm2ml need to be updated.

Last edited by Xyne (2016-02-14 06:28:22)


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#23 2016-02-14 11:38:38

Spyhawk
Member
Registered: 2006-07-07
Posts: 485

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

I know you have some concern about putting your work on github or similar social forges for some reason, but I really feel the biggest drawback of your projects is that they're developed being closed doors. Sometimes, I wish I could have a look at what the last commit is and what it brings, how you fixed a certain issue without having to diff the provided tarballs, or just have a look at your release map and planned features, or other similar stuff as done by developers and enthusiasts in the 2010'. This is specially true for project that could be built-on like your python3-aur package.

This said, thank you for all your work. I'm a very happy user of aurtomatic among other tools, and I'm also quite impressed by bauerbill too.

Edit: Also, I just found out that bauerbill has some difficulties installing split packages. The "python-novaclient" PKGBUILD provides both python-novaclient and python2-novaclient. I tried installing python-novaclient with bb-wrapper -S --aur --nobqd python-novaclient, but get many errors along the way:

=> Removing installed dependencies...
checking dependencies...
error: failed to prepare transaction (could not satisfy dependencies)
:: python-oslo-i18n: removing python-pbr breaks dependency 'python-pbr>=1.6'
:: python2-oslo-i18n: removing python2-pbr breaks dependency 'python2-pbr>=1.6'
:: python2-oslo-i18n: removing python2-six breaks dependency 'python2-six>=1.9.0'
==> WARNING: Failed to remove installed dependencies.

then:

warning: cannot resolve "python2-debtcollector>=0.3.0", a dependency of "python2-oslo-utils"
:: The following package cannot be upgraded due to unresolvable dependencies:
      python2-oslo-utils

:: Do you want to skip the above package for this upgrade? [y/N] y

and later on:

==> Installing python-novaclient package group with pacman -U...
loading packages...
resolving dependencies...
warning: cannot resolve "python-oslo-serialization>=1.10.0", a dependency of "python-novaclient"
warning: cannot resolve "python-oslo-utils>=3.2.0", a dependency of "python-novaclient"
warning: cannot resolve "python2-oslo-serialization>=1.10.0", a dependency of "python2-novaclient"
warning: cannot resolve "python2-oslo-utils>=3.2.0", a dependency of "python2-novaclient"
:: The following packages cannot be upgraded due to unresolvable dependencies:
      python-novaclient  python2-novaclient

:: Do you want to skip the above packages for this upgrade? [y/N] y
looking for conflicting packages...
 there is nothing to do
~/build
error: target not found: python-oslo-serialization
error: target not found: python-oslo-utils

On a side note, it seems bauerbill is currently unable to install only one of the provided split packages (either python-novaclient or python2-novaclient) and wants to install both of them.

Last edited by Spyhawk (2016-02-14 15:30:53)

Offline

#24 2016-02-14 19:13:33

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

The warnings are due to makepkg failing to distinguish between deps and makedeps and therefore trying to remove them all after the package has been installed. It correctly detects the deps and leaves them installed but issues the warning nevertheless. You can ignore the warnings or file a bug report on the pacman bug tracker.

The issue with split packages is that makepkg no longer recognized the "--pkg" option to select which split package to install. I'm looking in to it and will file a bug report if it really has been removed.

edit
Bug report: https://bugs.archlinux.org/task/48183

I will post an update if I find a reliable work-around.

Last edited by Xyne (2016-02-14 19:21:59)


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#25 2016-02-14 20:03:17

Spyhawk
Member
Registered: 2006-07-07
Posts: 485

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

Xyne wrote:

The issue with split packages is that makepkg no longer recognized the "--pkg" option to select which split package to install. I'm looking in to it and will file a bug report if it really has been removed.

edit
Bug report: https://bugs.archlinux.org/task/48183

I will post an update if I find a reliable work-around.

Yes, this has been known for a few months now. See pacman-dev and following messages for the explanation.
The workaround is to simply use "pacman -U" to install the selected subpackage instead of using "makepkg -i --pkg subpackage".

Offline

Board footer

Powered by FluxBB