Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

GSF1200S · 2017-10-27 05:10:58

Not sure what happened here. Apparmor and Google Chrome needed to be upgraded but Bauerbill failed with:

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 1151, in <module>
    sys.exit(run_main())
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 1132, in run_main
    return main(args)
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 1117, in main
    bb.generate_build_scripts(build_pkgs, build_deps)
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 646, in generate_build_scripts
    scripts['download'] = self.download_script(build_pkgs)
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 731, in download_script
    self.config, self.powerpill.pargs['pacman_config'], 'predownload', pbs[pkgbase]
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 302, in sh_hooks_block
    for pkg in iterate_hook_targets(pbs, as_dict=as_dict):
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 285, in iterate_hook_targets
    for pkg in sorted(pbs):
TypeError: '<' not supported between instances of 'AurBuildablePkg' and 'AurBuildablePkg'
bash: /tmp/bbscripts/download.sh: No such file or directory
sed: can't read build.sh: No such file or directory
bash: /tmp/bbscripts/build.sh: No such file or directory

Further is there anyway to install an AUR group, or all the packages of a split package? If I use bauerbill to install "apparmor" it will install just this package despite building the rest that I need built as well. I just used pacman -U manually so I wouldnt have to use bauerbill a bunch of times for all the splits.

Xyne · 2017-11-01 19:07:45

The comparison error should be fixed with today's update.

Pacman inteprets all arguments as package names so Bauerbill does the same. If you want to install all packages from the apparmor package base, there are two workarounds:

Explicitly list the packages when you first install it (upgrades will autodetect previously installed packages).
Use makepkg instead of makepkgx in bauerbill.json. You can set it as a default command to always install all packages in a package base, or add a custom command for apparmor and any other package for which you want to install the full base.

Example of a default command:

  "makepkg commands" : {
    "build" : {
      "common arguments" : [],
      "default" : ["makepkg", "-irs"]
    }
  }

Example of a custom command for apparmor:

  "makepkg commands" : {
    "build" : {
      "custom" : [
        ["AUR/apparmor", ["makepkg", "-irs"]]
      ]
  }

GSF1200S · 2017-11-03 01:51:21

Excellent- that custom command is just what I needed. Thanks for the fix and the info

Spyhawk · 2017-12-06 16:43:26

FYI, this makepkg patch by eschwartz should alleviate the need to readd `--pkg` back into makepkg, and it might provide a good alternative to patching makepkg. It's not in master yet but shouldn't take long to be merged.

SolarAquarion · 2017-12-19 15:23:36

Xyne, do you know if it's possible for you to add "package completion" of aur packages and pacman packages into bauerbill

Last edited by SolarAquarion (2017-12-19 15:23:55)

eschwartz · 2017-12-19 16:29:56

It surely is, but most AUR helpers don't try to offer completion for AUR packages as that would involve constantly redownloading a huge file containing the list of package names, and package name completion for AUR packages is kind of pointless.

Package name completion for repository packages should be as simple as hooking into pacman's own completion functions.

SolarAquarion · 2017-12-25 21:16:01

It's more in terms of not wanting to complete "files" after i do something like bauerbill -S ....

eschwartz · 2017-12-25 22:06:27

Hmm, well that should be excluded anyway regardless of whether package names are completed.

effeeffe · 2018-01-08 09:49:12

Hi Xyne, it looks like there is a newline missing between the downloading process of different packages:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0==> Making package: whatsapp-desktop 0.4.2-2 (lun  8 gen 2018, 10.32.38, CET)

nplatis · 2018-01-14 08:12:44

Hello,
I started using bauerbill today, and it seems to fail to download .sig files for the repos (indeed, the .sig files do not seem to be where requested, for example http://ftp.otenet.gr/linux/archlinux/mu … lib.db.sig

Anything I am doing wrong?

dapolinario · 2018-01-14 15:20:54

nplatis wrote:

Hello,
I started using bauerbill today, and it seems to fail to download .sig files for the repos (indeed, the .sig files do not seem to be where requested, for example http://ftp.otenet.gr/linux/archlinux/mu … lib.db.sig
Anything I am doing wrong?

Add this line in your pacman.conf to each repository (core, extra, community):

SigLevel = PackageRequired

eschwartz · 2018-01-14 21:12:34

dapolinario, that is already the default unless you've done something ill-advised like editing the default pacman.conf without having any idea what it all means.

So I desperately help that that advice wasn't actually necessary.

nplatis · 2018-01-14 23:16:11

Eschwartz wrote:

dapolinario, that is already the default unless you've done something ill-advised like editing the default pacman.conf without having any idea what it all means.
So I desperately help that that advice wasn't actually necessary.

In fact this is not the default.
pacman.conf contains the instruction

SigLevel = Required DatabaseOptional

and does not contain any relevant instruction for the repos.

My pacman.conf is not touched in this respect.
And notice that I am not able to download .sig files for the repos, not the packages.

dapolinario · 2018-01-15 00:33:27

Default pacman.conf -> pacman.conf

eschwartz · 2018-01-15 00:48:54

I am aware of the default quite well. The difference between "PackageRequired" and "Required DatabaseOptional" is that if the database .sig is available but invalid, this will be considered a fatal error. So the basic behavior defaults to much the same thing.

Making this change would help you, if the repository is lying about having a signature file, and giving you an error page that is not properly marked with an HTTP 404 error status.
Or, if Bauerbill was so broken that it tried to download .sig files and failed. Which would be weird, as Bauerbill would have to be completely broken on a conceptual level and this breakage would be the default behavior for every single Arch user, without exception. The official repos have *never* had .sig files available, and the default pacman.conf has traditionally attempted to find .sig files and forgive their total 404 absence, and anything that failed to reproduce that behavior could only do so if it (i.e. Bauerbill) was only ever used on systems and by users that had a different pacman.conf

I'd think even if Xyne didn't notice that, at least one user somewhere would.

But if you're saying that that is so nevertheless, okay...

Xyne · 2018-01-15 01:25:27

Eschwartz wrote:

Or, if Bauerbill was so broken that it tried to download .sig files and failed. Which would be weird, as Bauerbill would have to be completely broken on a conceptual level and this breakage would be the default behavior for every single Arch user, without exception.

Bauerbill and Powerpill are conceptually broken on a fundamental level then. The downloading is handled by Aria2 via an intermediate metalink. If the metalink includes signature files that don't exist then Aria2c will exit with an error after the other downloads complete. I am not aware of an option to configure conditional downloads that ignore 404 errors for the signature files. It could check the exit code and verify that only optional files are missing before proceeding but a simpler solution is to just set the correct SigLevel setting for the repo. A repo that randomly offers signatures would be broken with regard to security (an attacker could simply inject an unsigned repo, and the downloader would have to deal with removing old signature files before each database refresh to prevent erroneous mismatches). Regardless of Bauerbill/Powerpill's behavior, PackageRequired is the correct SigLevel setting for the official repos and using it avoids the issue of dealing with optional downloads.

[core]
SigLevel = PackageRequired
Include = /etc/pacman.d/mirrorlist

"completely broken on a conceptual level"

eschwartz · 2018-01-15 01:32:29

So, do you automatically attempt to download the .sig files for people who use the default pacman.conf, and then fail with unrealistic error messages?

Your "simpler solution" is what I call "laziness", or "not being a proper wrapper for pacman". I'd call that pretty broken for something that intends to be a download wrapper around pacman.

If you dislike the idea of pacman providing the ability to mark signatures as optional, I encourage you to file a bugreport or raise a discussion on pacman-dev to tell everyone about how "A repo that randomly offers signatures would be broken with regard to security", rather than being unfaithful to pacman's spec. Maybe then you could fix it for everyone.

...

In the meantime, perhaps powerpill should, you know, document the need for non-default pacman.conf
I don't see it anywhere on the project page or the manpage.

Last edited by eschwartz (2018-01-15 01:50:21)

eschwartz · 2018-01-15 01:57:15

Perhaps aria2 could use an --on-download-error event hook, but it looks like it just reports the first of the failing files. So using the RPC interface to extract more details would probably be the most flexible way of getting proper statuses. Failing signature files could then be cross-referenced with the repositories whose resolved SigLevel include DatabaseOptional. Similar logic can probably be used when wrapping `pacman -U https://...`

This could also allow removing old signature files for failed downloads that correspond to successful database downloads.

Last edited by eschwartz (2018-01-15 01:59:22)

Xyne · 2018-01-21 03:58:48

This will indeed be easier with the RPC interface. It's already integrated in the Pacboy backend along with several other improvements, but there is still a lot to do and I just don't have the time at the moment.

I will add a note to the Powerpill project page about Pacman SigLevels. It is already mentioned on the wiki with a link to a forum post from 2013. Despite your condescending attitude, I agree with you on code correctness and I will reconsider my options given the current implementation. Nevertheless, if this was really a usability issue then it would have come up more than 2 or 3 times over the past 5 years.

A word of advice for dealing with people in general, especially given your new role as a TU: be nice and assume good intentions until proven otherwise. Disparaging remarks, in particular those laden with smug hyperbole, rarely elicit a positive and constructive response, which is what you should aim for.

And please use the edit button to append information when no replies have been posted.

jorges · 2018-03-12 15:29:46

Hi, recently switched to bauerbill, so I'm still finding my way in. This morning I needed to install some packages, but bauerbill failed to download the packages, due to the proxy server that's in place at work. Is there any way of making bauerbill aware of the proxy server?

eschwartz · 2018-03-12 16:06:03

What is wrong with the standard https://wiki.archlinux.org/index.php/Proxy_settings ?

jorges · 2018-03-12 17:43:39

Eschwartz wrote:

What is wrong with the standard https://wiki.archlinux.org/index.php/Proxy_settings ?

Thanks!. I did follow that guide some time ago to set things up, but didn't need the part related to sudo. Maybe that's the culprit here. I do have gnome propperly set for gui apps, and in terminals I do have the appropriate env variables defined, but bauerbill seemed to ignored them. I'll explore that route and report back.

jorges · 2018-03-15 12:53:15

jorges wrote:

Eschwartz wrote:
What is wrong with the standard https://wiki.archlinux.org/index.php/Proxy_settings ?
Thanks!. I did follow that guide some time ago to set things up, but didn't need the part related to sudo. Maybe that's the culprit here. I do have gnome propperly set for gui apps, and in terminals I do have the appropriate env variables defined, but bauerbill seemed to ignored them. I'll explore that route and report back.

Just wanted to confirm that the info pointed by Eschwartz in the wiki solved the issue I had. Specifically, adding "Defaults env_keep += "http_proxy https_proxy ftp_proxy"" to the sudo configuration file did the trick. Thanks again!

GSF1200S · 2018-04-17 04:16:28

Bauerbill has been awesome, though for some reason the last couple times Apparmor has needed an update Bauerbill has failed. If I invoke with

bb-wrapper -S --aur apparmor

it works fine. Figured it was a one-off issue the last few times (no problems upgrading anything else) so decided not to bother you with it. Today Apparmor was to be upgraded again, and Bauerbill bailed out with the following:

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 1151, in <module>
    sys.exit(run_main())
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 1132, in run_main
    return main(args)
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 1117, in main
    bb.generate_build_scripts(build_pkgs, build_deps)
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 646, in generate_build_scripts
    scripts['download'] = self.download_script(build_pkgs)
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 731, in download_script
    self.config, self.powerpill.pargs['pacman_config'], 'predownload', pbs[pkgbase]
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 302, in sh_hooks_block
    for pkg in iterate_hook_targets(pbs, as_dict=as_dict):
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 285, in iterate_hook_targets
    for pkg in sorted(pbs):
  File "/usr/lib/python3.6/site-packages/XCPF/ArchPkg.py", line 296, in __lt__
    name_cmp = cmp(self.qualified_pkgname(), other.qualified_pkgname())
NameError: name 'cmp' is not defined

Not sure if this is due to my launching bauerbill from an update script, whether its due to a faulty config on my part, or whether its a bug. As before, it worked fine when I used -S --aur apparmor instead of -Su --aur.

Its really no big deal to me- only one package does this and it doesn't take much time to fix. I'm mainly posting in case it reveals a problem you want to fix.

Again, thanks for the AUR helper- I've used it for awhile now and think its great!

Xyne · 2018-04-17 04:44:15

A new version of python3-xcpf that resolves the issue should be up in a few minutes. Thanks for reporting it!

Arch Linux

#201 2017-10-27 05:10:58

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#202 2017-11-01 19:07:45

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#203 2017-11-03 01:51:21

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#204 2017-12-06 16:43:26

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#205 2017-12-19 15:23:36

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#206 2017-12-19 16:29:56

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#207 2017-12-25 21:16:01

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#208 2017-12-25 22:06:27

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#209 2018-01-08 09:49:12

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#210 2018-01-14 08:12:44

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#211 2018-01-14 15:20:54

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#212 2018-01-14 21:12:34

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#213 2018-01-14 23:16:11

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#214 2018-01-15 00:33:27

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#215 2018-01-15 00:48:54

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#216 2018-01-15 01:25:27

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#217 2018-01-15 01:32:29

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#218 2018-01-15 01:57:15

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#219 2018-01-21 03:58:48

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#220 2018-03-12 15:29:46

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#221 2018-03-12 16:06:03

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#222 2018-03-12 17:43:39

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#223 2018-03-15 12:53:15

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#224 2018-04-17 04:16:28

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#225 2018-04-17 04:44:15

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

Board footer