Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

runical · 2017-01-07 12:24:15

That is unfortunate. I do not use the software myself, so I'm not sure I'll be of much help further, but did you try resetting the config/databases etc?

Good luck with your problem!

OJaksch · 2017-01-07 12:34:19

runical wrote:

did you try resetting the config/databases etc?

Yes, I resetted the configs to default and to the same as the other arch boxes, but same result.
According to the database I couldn't find any; bauerbill says

sqlite3.OperationalError: attempt to write a readonly database

but what database and where is it?

runical · 2017-01-07 12:54:53

Maybe in ~/.cache? (As I said, I don't use the software)

Your best bet might be to wait for Xyne to wake up. He's in North America IIRC.

OJaksch · 2017-01-07 13:09:08

runical wrote:

Maybe in ~/.cache?

Great, I love you, pal! There was a folder ~/.cache/AUR in users home, but owned by root. Must be by mistake in the past.
Deleted, bauerbill is working.

Again, thanks a lot for pointing me to the right direction!

Xyne · 2017-01-07 13:58:40

If you are wrapping it in yaourt, just use powerpill instead. Yaourt only invokes the internal pacman wrapper with pacman arguments (afaik), so the only difference from pacman will be the parallel downloads. Powerpill is simpler than bauerbill and necessarily installed as it is a dependency of bauerbill.

If the permission error occurs again, let me know. It may be caused by buggy permission management in another app that uses the same database (e.g. aurquery).

OJaksch · 2017-01-07 16:18:44

Xyne wrote:

If you are wrapping it in yaourt, just use powerpill instead. Yaourt only invokes the internal pacman wrapper with pacman arguments (afaik), so the only difference from pacman will be the parallel downloads. Powerpill is simpler than bauerbill and necessarily installed as it is a dependency of bauerbill.
If the permission error occurs again, let me know. It may be caused by buggy permission management in another app that uses the same database (e.g. aurquery).

Thanks for your suggestion.
I don't believe that there is any buggy permission management (left); I'm using bauerbill/powerpill at dozens of arch installations w/o any issues - except that one I mentioned earlier. And I'm sure this was my mistake.

carbolymer · 2017-01-16 22:48:57

I've got an issue with the update of aur packages using bauerbill, after

bauerbill -Syu --aur

I am getting following error:

aria2 will resume download if the transfer is restarted.
If there are any errors, then see the log file. See '-l' option in help/man page for details.
[2017-01-16 23:39:32] ERROR: PowerpillError: aria2c exited with 19

The full log: http://pastebin.com/u3u6QVas

Also, what can I do about these errors during signatures downloading? I've already set sig levels:

SigLevel    = Required DatabaseNever
LocalFileSigLevel = Optional
# RemoteFileSigLevel = Required

(modyfing RemoteFileSigLevel does not change anything).

Last edited by carbolymer (2017-01-16 22:49:18)

Xyne · 2017-01-16 23:46:53

Use SigLevel = PackageRequired for the official repos, e.g

[core]
SigLevel = PackageRequired
Include = /etc/pacman.d/mirrorlist

Handling of optional signals will be improved in the next major update.

timofonic · 2017-01-18 16:23:44

I'm using your repos to install bauerbill.

When I run bauerbill, I have the following error:

$ bauerbill 
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 38, in <module>
    import AUR.PkgList
  File "/usr/lib/python3.6/site-packages/AUR/PkgList.py", line 30, in <module>
    import XCGF
ModuleNotFoundError: No module named 'XCGF'

python3-xcgf was installed, I tried to reinstall but same error:

$ yaourt -S python3-xcgf
warning: python3-xcgf-2016.4.5-1 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...

Packages (1) python3-xcgf-2016.4.5-1

Total Installed Size:  0.06 MiB
Net Upgrade Size:      0.00 MiB

:: Proceed with installation? [Y/n] y
(1/1) checking keys in keyring                                                                                                                                     [#####################################################################################################] 100%
(1/1) checking package integrity                                                                                                                                   [#####################################################################################################] 100%
(1/1) loading package files                                                                                                                                        [#####################################################################################################] 100%
(1/1) checking for file conflicts                                                                                                                                  [#####################################################################################################] 100%
(1/1) checking available disk space                                                                                                                                [#####################################################################################################] 100%
:: Processing package changes...
(1/1) reinstalling python3-xcgf                                                                                                                                    [#####################################################################################################] 100%

So okay, trying AUR with yaourt to see if it works, but...

# gpg --recv-keys 1D1F0DC78F173680
gpg: key 1D1F0DC78F173680: "Xyne. (key #3) <xyne@archlinux.ca>" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1

==> Building and installing package
==> Making package: python3-aur 2017.1-1 (Wed Jan 18 17:40:23 CET 2017)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
  -> Found python3-aur-2017.1.tar.xz
  -> Found python3-aur-2017.1.tar.xz.sig
==> Validating source files with md5sums...
    python3-aur-2017.1.tar.xz ... Passed
    python3-aur-2017.1.tar.xz.sig ... Passed
==> Validating source files with sha512sums...
    python3-aur-2017.1.tar.xz ... Passed
    python3-aur-2017.1.tar.xz.sig ... Passed
==> Verifying source file signatures with gpg...
    python3-aur-2017.1.tar.xz ... FAILED (unknown public key 1D1F0DC78F173680)
==> ERROR: One or more PGP signatures could not be verified!
==> ERROR: Makepkg was unable to build python3-aur.
==> Restart building python3-aur ? [y/N]

I did the following and same error happens..

sudo pacman-key --init; sudo pacman-key --populate archlinux; sudo pacman-key --refresh-keys

Please, how to fix it? Maybe it got explained before, but I can't find it...

Last edited by timofonic (2017-01-18 16:54:29)

Slithery · 2017-01-18 16:56:35

Building AUR packages makes use of the user keyring, not pacmans.

https://wiki.archlinux.org/index.php/Ma … e_checking

timofonic · 2017-01-18 17:12:18

slithery wrote:

Building AUR packages makes use of the user keyring, not pacmans.
https://wiki.archlinux.org/index.php/Ma … e_checking

I did read that link before and a identical reply from Xyne on AUR packages comments, but I'm still confused. Sorry, the documentation and your explanation isn't enough clear to me.

Could you please ellaborate a bit more? I don't get what to do to solve it.

EDIT: I was doing things correctly, but used sudo or su. makepkg needs the keys in my user, not root. Okay, sorry...

Last edited by timofonic (2017-01-18 17:32:23)

timofonic · 2017-01-18 17:36:38

I installed it over AUR, then got the SAME error

    $ bauerbill
    Traceback (most recent call last):
      File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
        "__main__", mod_spec)
      File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 38, in <module>
        import AUR.PkgList
      File "/usr/lib/python3.6/site-packages/AUR/PkgList.py", line 30, in <module>
        import XCGF
    ModuleNotFoundError: No module named 'XCGF'

eschwartz · 2017-01-18 18:28:18

As I said on IRC, you will need to rebuild those dependencies since they were built against python35 somehow.

I will note that your repos don't appear to have been synced, or at least not all of them -- since Xyne published a new (rebuilt) version of that package and several others which you don't seem to have available in your local copy of his repo.

Last edited by eschwartz (2017-01-18 18:30:37)

Xyne · 2017-01-18 20:44:54

The ModuleNotFoundErrors mean that you have done a partial upgrade (or downgrade) on your system. Whatever version of Python you are using, you must use packages that were built with that version. The current version of Python in the official repos is 3.6. All of the packages in my repo are built with Python 3.6. If you have enabled my repo, then a simple "pacman -Syu" should make everything work. If not, check that your main mirror is up-to-date.

carbolymer · 2017-01-19 17:09:28

@Xyne, how about this error:

ERROR: PowerpillError: aria2c exited with 19

This makes bauerbill completely unusable for me. (log with description)

Xyne · 2017-01-19 17:59:04

carbolymer wrote:

@Xyne, how about this error:
ERROR: PowerpillError: aria2c exited with 19
This makes bauerbill completely unusable for me. (log with description)

aria2c wrote:

[AbstractCommand.cc:792] errorCode=19 CUID#10 - Name resolution for mirror.aardvark.it failed:Domain name not found

The error message indicates that mirror.aardvark.it doesn't exist. ping, host and everything else confirms this. Use a mirror that exists

The recommended way to use powerpill/bauerbill is to use a single reliable mirror in your mirrorlist and then use the reflector options to automatically use e.g. the 50 most recently synchronized mirrors when downloading packages. That will avoid such errors and also ensure database consistency (i.e. that all databases are downloaded from the same mirror, although I think there's an option for that even with multiple mirrors in the mirrorlist).

Alad · 2017-02-03 12:33:08

I'm having some issue with resolving large dependency trees with bauerbill:

time bauerbill -S --aur ros-indigo-desktop-full
[2017-02-03 13:29:04] ERROR: Unresolved dependencies: sdformat
^C
real    4m15,551s # interrupted with Ctrl+C at this point, no further output
user    3m44,937s
sys     0m0,787s

Cf. aurchain which I mention here because it pretty much stole the dependency resolution from bauerbill ca. april last year (and me being unsure how to install older bauerbill versions, including dependencies)

[archie@thinkpad ~]$ time aurchain ros-indigo-desktop-full | wc -l
247

real    0m1,342s
user    0m0,567s
sys     0m0,063s

Last edited by Alad (2017-02-03 12:35:28)

Spyhawk · 2017-02-03 13:35:08

@Alad: That dependency tree is unsolvable. The "gazebo" package requires "sdformat>=4.2.0", and only "sdformat 4.1.0" is available in the AUR.
Bauerbill shouldn't hang but exit nicely though.

Alad · 2017-02-03 13:53:06

Another victim of versioned dependencies...

zerophase · 2017-02-07 01:17:07

I noticed trying to add commands for editing PKGBUILDS on the prebuild hook adds a space between each letter.

Xyne · 2017-02-10 04:00:20

zerophase wrote:

I noticed trying to add commands for editing PKGBUILDS on the prebuild hook adds a space between each letter.

Are you trying to use a single string as a command? Bauerbill expects each command to be a JSON list, e.g. ["cmd", "--foo", "--bar", "arg1", "arg2"].

@Alad & Spyhawk
The dependency graph resolution algorithm was too inefficient to handle the behemoth that is ros-indigo-desktop-full in a reasonable amount of time. I have tweaked the algorithm to avoid redundant recursion and it should be very fast now (update to python3-xcpf-2017.2.10). Bauerbill may still appear to hang while retrieving data from the AUR but that is due to the number of packages that it has to query. You can use "--debug" if you want to monitor those queries.

Let me know if I borked the build queue ordering. I only tested with some simple packages. There's no way I'm going to fully test ros-indigo-desktop-full

And someone tell the ros-indigo-desktop maintainer(s) that packages in the depends array should not be included in the makedepends array. There seems to be a lot of dupes.

Yah · 2017-02-26 14:18:50

Hi There!
First thanks to Xyne for these greate tools!

Because of the fact that I am using multiple AUR-Packages, I have to update very often.
During updating process I regularly have to face the problem be using the build.sh script successfully fails by the missing command "git pull".

f.ex.

####################### Get PKGBUILDS and related files. #######################
/usr/bin/pbget --aur-only gstreamer0.10

pids=()

#################### Query trust before starting downloads. ####################
/usr/bin/bb-query_trust -l ./gstreamer0.10 AUR/gstreamer0.10 1487978122 nicepack || exit 1

############### Download sources for package base gstreamer0.10. ###############
# Maintainer(s): nicepack
# Last modified: 2017-02-25 00:15:22 CET
# Repository: AUR
# Packages: gstreamer0.10
pushd gstreamer0.10
makepkg --verifysource --nodeps &
pids+=($!)
echo gstreamer0.10 ": $!"
popd

Before running makepkg wouldn't it be a line like this very nice:

test -d .git && git pull

Is this practical and useful?

Right now I using this little script of mine to bypass this problem for myself

#!/bin/bash
#
# Author: Yah(al)
#
# Script to run git pull in all founded AUR main dirs with a .git subfolder
#
#
temp=$PWD
if [ "$#" -eq 1 ]


        then
#                       printf "\$1 %s\n\n", $1

                        while IFS= read line
                        do
                                printf "Run 'git pull' in %s\n", ${line%.git}
                                cd ${line%.git}
                                git pull
                                cd $temp

                        done < <(find ./build -maxdepth 2 -name .git -type d -prune)


        else

                printf "Usage of %s: \n", $0
                printf "%s <folder> - scans dirs for .git subfolders and runnig 'git pull' to maintaine\n", $0


fi

Xyne · 2017-02-27 20:34:09

Yah wrote:

During updating process I regularly have to face the problem be using the build.sh script successfully fails by the missing command "git pull".
f.ex.
####################### Get PKGBUILDS and related files. #######################
/usr/bin/pbget --aur-only gstreamer0.10

pids=()

#################### Query trust before starting downloads. ####################
/usr/bin/bb-query_trust -l ./gstreamer0.10 AUR/gstreamer0.10 1487978122 nicepack || exit 1

############### Download sources for package base gstreamer0.10. ###############
# Maintainer(s): nicepack
# Last modified: 2017-02-25 00:15:22 CET
# Repository: AUR
# Packages: gstreamer0.10
pushd gstreamer0.10
makepkg --verifysource --nodeps &
pids+=($!)
echo gstreamer0.10 ": $!"
popd
Before running makepkg wouldn't it be a line like this very nice:
test -d .git && git pull
...
Script to run git pull in all founded AUR main dirs with a .git subfolder

If I have understood your post, you want to fetch and merge (pull) the changes from the AUR's Git repositories for your PKGBUILDs. Is that correct? The AUR files are download with pbget so any changes would need to be made in that script, but I do not think that it should automatically merge changes.

With the current version of Bauerbill, you can use the hooks (e.g. the "prebuild" hook) via bauerbill.json to run commands to merge your changes before building automatically. When I finally get around to officially releasing the pacboy backend there will be much more flexibility for e.g. running commands before any packages are built.

If you are actually talking about the way that makepkg is downloading the package source files (not the PKGBUILD archive), then that would need to be addressed by the pacman devs.

Btw, successfully fails should be Allan's official tagline

steventheevilz · 2017-03-09 01:09:49

New error on latest version as of posting:

 
[steventheevil@Steven-PC ~]$ sudo bb-wrapper -Syu --aur
[sudo] password for steventheevil: 

03/08 18:59:19 [NOTICE] Downloading 4 item(s)

03/08 18:59:19 [NOTICE] GID#13f2af17d0acda40 - Download has already completed: /var/lib/pacman/sync/core.db

03/08 18:59:19 [NOTICE] Download complete: /var/lib/pacman/sync/core.db

03/08 18:59:19 [NOTICE] GID#fa87206430436017 - Download has already completed: /var/lib/pacman/sync/extra.db

03/08 18:59:19 [NOTICE] Download complete: /var/lib/pacman/sync/extra.db

03/08 18:59:19 [NOTICE] GID#7868fc242f67a49d - Download has already completed: /var/lib/pacman/sync/community.db

03/08 18:59:19 [NOTICE] Download complete: /var/lib/pacman/sync/community.db

03/08 18:59:20 [NOTICE] GID#39fc63b538dd1444 - Download has already completed: /var/lib/pacman/sync/multilib.db

03/08 18:59:20 [NOTICE] Download complete: /var/lib/pacman/sync/multilib.db

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
13f2af|OK  |       0B/s|/var/lib/pacman/sync/core.db
fa8720|OK  |       0B/s|/var/lib/pacman/sync/extra.db
7868fc|OK  |       0B/s|/var/lib/pacman/sync/community.db
39fc63|OK  |       0B/s|/var/lib/pacman/sync/multilib.db

Status Legend:
(OK):download completed.
[2017-03-08 18:59:20] ERROR: LockError(/var/lib/pacman/db.lck): failed to unlock database [[Errno 2] No such file or directory: '/var/lib/pacman/db.lck']
/home/steventheevil/build /home/steventheevil
/home/steventheevil

How to fix:

ERROR: LockError(/var/lib/pacman/db.lck): failed to unlock database [[Errno 2] No such file or directory: '/var/lib/pacman/db.lck']

steventheevilz · 2017-03-09 23:48:13

New error after restart:

[steventheevil@Steven-PC ~]$ bb-wrapper -Syu --aur --build-all

03/09 17:47:45 [NOTICE] Downloading 4 item(s)

03/09 17:47:45 [NOTICE] GID#628e4f95e57f1109 - Download has already completed: /var/lib/pacman/sync/core.db

03/09 17:47:45 [NOTICE] Download complete: /var/lib/pacman/sync/core.db

03/09 17:47:45 [NOTICE] GID#a13063c3f57a698c - Download has already completed: /var/lib/pacman/sync/community.db

03/09 17:47:45 [NOTICE] Download complete: /var/lib/pacman/sync/community.db

03/09 17:47:45 [NOTICE] GID#a0f14b61aaab1f2f - Download has already completed: /var/lib/pacman/sync/extra.db

03/09 17:47:45 [NOTICE] Download complete: /var/lib/pacman/sync/extra.db

03/09 17:47:45 [NOTICE] GID#1116dcb9ca61fd5b - Download has already completed: /var/lib/pacman/sync/multilib.db

03/09 17:47:45 [NOTICE] Download complete: /var/lib/pacman/sync/multilib.db

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
628e4f|OK  |       0B/s|/var/lib/pacman/sync/core.db
a13063|OK  |       0B/s|/var/lib/pacman/sync/community.db
a0f14b|OK  |       0B/s|/var/lib/pacman/sync/extra.db
1116dc|OK  |       0B/s|/var/lib/pacman/sync/multilib.db

Status Legend:
(OK):download completed.
[2017-03-09 17:47:46] WARNING: Installed orphans: ckb-git keepassxc-keepasshttp
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/MemoizeDB.py", line 216, in db_insert
    c.execute(query, args)
sqlite3.OperationalError: attempt to write a readonly database

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 1133, in <module>
    run_main()
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 1116, in run_main
    return main(args)
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 1101, in main
    bb.generate_build_scripts(build_pkgs, build_deps)
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 630, in generate_build_scripts
    scripts['download'] = self.download_script(build_pkgs)
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 652, in download_script
    pbs = XCPF.ArchPkg.collect_pkgbases(build_pkgs)
  File "/usr/lib/python3.6/site-packages/XCPF/ArchPkg.py", line 216, in collect_pkgbases
    pkgbases[pkg.pkgbase()].append(pkg)
  File "/usr/lib/python3.6/site-packages/XCPF/ArchPkg.py", line 481, in pkgbase
    return self.get_pkginfo()['pkgbase']
  File "/usr/lib/python3.6/site-packages/XCPF/ArchPkg.py", line 441, in get_pkginfo
    self.pkg.db.name, self.pkg.arch, self.pkg.name
  File "/usr/lib/python3.6/site-packages/XCPF/common.py", line 159, in memoized_archlinux_org_pkg_info
    txt = mdb.get_one('request', url)
  File "/usr/lib/python3.6/site-packages/MemoizeDB.py", line 542, in get_one
    return self.get_nth_field(table, key, n=n)
  File "/usr/lib/python3.6/site-packages/MemoizeDB.py", line 550, in get_nth_field
    vs = self.get(table, key)
  File "/usr/lib/python3.6/site-packages/MemoizeDB.py", line 511, in get
    self.db_insert(table, key, values)
  File "/usr/lib/python3.6/site-packages/MemoizeDB.py", line 225, in db_insert
    raise MDBError('row insertion(s) failed', error=e)
MemoizeDB.MDBError: MDBError: row insertion(s) failed [attempt to write a readonly database]

Arch Linux

#126 2017-01-07 12:24:15

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#127 2017-01-07 12:34:19

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#128 2017-01-07 12:54:53

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#129 2017-01-07 13:09:08

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#130 2017-01-07 13:58:40

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#131 2017-01-07 16:18:44

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#132 2017-01-16 22:48:57

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#133 2017-01-16 23:46:53

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#134 2017-01-18 16:23:44

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#135 2017-01-18 16:56:35

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#136 2017-01-18 17:12:18

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#137 2017-01-18 17:36:38

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#138 2017-01-18 18:28:18

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#139 2017-01-18 20:44:54

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#140 2017-01-19 17:09:28

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#141 2017-01-19 17:59:04

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#142 2017-02-03 12:33:08

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#143 2017-02-03 13:35:08

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#144 2017-02-03 13:53:06

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#145 2017-02-07 01:17:07

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#146 2017-02-10 04:00:20

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#147 2017-02-26 14:18:50

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#148 2017-02-27 20:34:09

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#149 2017-03-09 01:09:49

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#150 2017-03-09 23:48:13

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

Board footer