Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

ibrokemypie · 2017-05-10 08:46:02

Is there any option to use --aur by default?

Xyne · 2017-05-10 10:13:08

ibrokemypie wrote:

Is there any option to use --aur by default?

Not in the current version. Just use an alias, e.g.

alias bb='/usr/bin/bauerbill --aur'
alias bbw='/usr/bin/bb-wrapper --aur'

SolarAquarion · 2017-05-14 03:49:49

bauerbill has some issues with downloading packages if the pkgbase is not a pkgname

Xyne · 2017-05-14 19:59:14

SolarAquarion wrote:

bauerbill has some issues with downloading packages if the pkgbase is not a pkgname

How am I supposed to do anything about it if you provide no information or examples? What are the "issues"? What's the console output or unexpected behavior? Which packages cause these issues?

SolarAquarion · 2017-05-15 12:55:15

Xyne wrote:

SolarAquarion wrote:
bauerbill has some issues with downloading packages if the pkgbase is not a pkgname
How am I supposed to do anything about it if you provide no information or examples? What are the "issues"? What's the console output or unexpected behavior? Which packages cause these issues?

Deluge-split, which actual names are deluge-common among others

Xyne · 2017-05-15 15:21:15

Fixed. Please update to pbget-2017 and python3-aur-2017.5 and try again.

SolarAquarion · 2017-07-03 14:30:41

For some reason when i do ./download.sh the PKGBUILD's aren't automatically updated so i have to do manual interventions to make it work

Xyne · 2017-07-04 10:48:31

SolarAquarion wrote:

For some reason when i do ./download.sh the PKGBUILD's aren't automatically updated so i have to do manual interventions to make it work

This will be more configurable in a future version. For now, it allows the user to modify the PKGBUILD prior to download (e.g. to download different sources). In the meantime, just write a wrapper script to remove the PKGBUILDs before running the download script.

SolarAquarion · 2017-07-04 11:47:45

No, the git based PKGBUILD's as in the AUR aren't automatically pulled. Like Bauerbill says that chromium-dev needs to be updated and i know it does, but the script didn't pull all the new changes appropriately. So I just need to write a script to remove the PKGBUILD's prior to the download script? huh

Last edited by SolarAquarion (2017-07-04 11:58:41)

Xyne · 2017-07-04 12:15:47

SolarAquarion wrote:

No, the git based PKGBUILD's as in the AUR aren't automatically pulled. Like Bauerbill says that chromium-dev needs to be updated and i know it does, but the script didn't pull all the new changes appropriately. So I just need to write a script to remove the PKGBUILD's prior to the download script? huh

Pbget doesn't update the working directory but it should fetch the changes. cd into the directory and check "git status".

If this is really an issue, I may add options to force an update of the working directory.

SolarAquarion · 2017-07-04 23:57:31

solaraquarion@apollo   ~/build/ts-qtplatform   (git)-[master]  git status
On branch master
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
Untracked files:
(use "git add <file>..." to include in what will be committed)

I'm having a constant issue with the git thing

Xyne · 2017-07-05 12:06:43

SolarAquarion wrote:

I'm having a constant issue with the git thing

If you didn't change anything, why would you expect it to change?

I've added a pull option to pbget. Update pbget, python3-xcpf and python3-aur, then change the "pbget" command in bauerbill.json to

"pbget" : ["/usr/bin/pbget", "--pull"]

Please let me know if it works as expected.

GSF1200S · 2017-08-15 04:47:28

bb -S --build youtube-dl
created /tmp/bbscripts/download.sh
created /tmp/bbscripts/build.sh
/tmp/bbscripts ~
[2017-08-15 04:46:57] INFO: searching ABS Git interface
error: unknown option `get'
usage: git pull [<options>] [<repository> [<refspec>...]]

    -v, --verbose         be more verbose
    -q, --quiet           be more quiet
    --progress            force progress reporting
    --recurse-submodules[=<on-demand>]
                          control for recursive fetching of submodules

Options related to merging
    -r, --rebase[=<false|true|preserve|interactive>]
                          incorporate changes by rebasing rather than merging
    -n                    do not show a diffstat at the end of the merge
    --stat                show a diffstat at the end of the merge
    --log[=<n>]           add (at most <n>) entries from shortlog to merge commit message
    --squash              create a single commit instead of doing a merge
    --commit              perform a commit if the merge succeeds (default)
    --edit                edit message before committing
    --ff                  allow fast-forward
    --ff-only             abort if fast-forward is not possible
    --verify-signatures   verify that the named commit has a valid GPG signature
    --autostash           automatically stash/stash pop before and after rebase
    -s, --strategy <strategy>
                          merge strategy to use
    -X, --strategy-option <option=value>
                          option for selected merge strategy
    -S, --gpg-sign[=<key-id>]
                          GPG sign commit
    --allow-unrelated-histories
                          allow merging unrelated histories

Options related to fetching
    --all                 fetch from all remotes
    -a, --append          append to .git/FETCH_HEAD instead of overwriting
    --upload-pack <path>  path to upload pack on remote end
    -f, --force           force overwrite of local branch
    -t, --tags            fetch all tags and associated objects
    -p, --prune           prune remote-tracking branches no longer on remote
    -j, --jobs[=<n>]      number of submodules pulled in parallel
    --dry-run             dry run
    -k, --keep            keep downloaded pack
    --depth <depth>       deepen history of shallow clone
    --unshallow           convert to a complete repository
    --update-shallow      accept refs that update .git/shallow
    --refmap <refmap>     specify fetch refmap

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.6/site-packages/pbget.py", line 265, in <module>
    main()
  File "/usr/lib/python3.6/site-packages/pbget.py", line 237, in main
    for pkgname, url in opi.retrieve_abs_via_git(output_dir, pkgnames, pull=pargs.pull):
  File "/usr/lib/python3.6/site-packages/XCPF/common.py", line 361, in retrieve_abs_via_git
    url = retrieve_abs_via_git(output_dir, pkgbase, repo=repo, ttl=self.ttl, pull=pull)
  File "/usr/lib/python3.6/site-packages/XCPF/common.py", line 571, in retrieve_abs_via_git
    gr_url = subprocess.check_output(cmd + ['config', '--get', 'remote.origin.url']).decode().strip()
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['git', '-C', '/home/poeticrpm/.cache/XCPF/abs_git/youtube-dl', 'pull', 'config', '--get', 'remote.origin.url']' returned non-zero exit status 129.

Dunno if I need to change bauerbill.json or not, but ABS is no go. Maybe its something only on my end? System fully up-to-date.

Last edited by GSF1200S (2017-08-15 04:49:01)

GSF1200S · 2017-08-23 02:24:58

As it turns out, bauerbill will build packages fine using bb-wrapper -S --build <packagename>. But if the package is added to build patterns in bauerbill.json and an update is available, it will fail.

[GSF1200S@codething ~]$ bb-wrapper -S --build youtube-dl
created build/download.sh
created build/build.sh
~/build ~
[2017-08-23 02:07:28] INFO: searching ABS Git interface
[2017-08-23 02:07:28] INFO: found youtube-dl at https://git.archlinux.org/svntogit/community.git [packages/youtube-dl]
~/build/youtube-dl ~/build
youtube-dl : 3952
~/build
==> Making package: youtube-dl 2017.08.18-1 (Wed Aug 23 02:07:39 UTC 2017)
==> WARNING: Skipping dependency checks.
==> Retrieving sources...
  -> Downloading youtube-dl-2017.08.18.tar.gz...

And then it builds fine.

With bb-wrapper -Syu (and youtube-dl in build patterns) I get:

[2017-08-23 02:06:14] INFO: searching ABS Git interface
[2017-08-23 02:06:14] INFO: found youtube-dl at https://git.archlinux.org/svntogit/community.git [packages/youtube-dl]
error: unknown option `get'
...
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.6/site-packages/pbget.py", line 265, in <module>
    main()
  File "/usr/lib/python3.6/site-packages/pbget.py", line 237, in main
    for pkgname, url in opi.retrieve_abs_via_git(output_dir, pkgnames, pull=pargs.pull):
  File "/usr/lib/python3.6/site-packages/XCPF/common.py", line 361, in retrieve_abs_via_git
    url = retrieve_abs_via_git(output_dir, pkgbase, repo=repo, ttl=self.ttl, pull=pull)
  File "/usr/lib/python3.6/site-packages/XCPF/common.py", line 571, in retrieve_abs_via_git
    gr_url = subprocess.check_output(cmd + ['config', '--get', 'remote.origin.url']).decode().strip()
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['git', '-C', '/home/gsf1200s/.cache/XCPF/abs_git/udisks2', 'pull', 'config', '--get', 'remote.origin.url']' returned non-zero exit status 129.

Apparently last time, it didnt work just using -S --build either. At this point I dont know if its a bug with bauerbill or something wrong with my system.

Last edited by GSF1200S (2017-08-23 02:28:12)

Xyne · 2017-08-24 13:28:18

GSF1200S wrote:

At this point I dont know if its a bug with bauerbill or something wrong with my system.

It's a bug

It should be fixed in python3-xcpf-2017.8. Sorry that it took so long. Thanks for reporting it.

GSF1200S · 2017-08-24 20:13:31

Xyne wrote:

GSF1200S wrote:
At this point I dont know if its a bug with bauerbill or something wrong with my system.
It's a bug
It should be fixed in python3-xcpf-2017.8. Sorry that it took so long. Thanks for reporting it.

No worries- life gets busy Thanks for the fix!

GSF1200S · 2017-08-31 17:09:13

Had a bunch of updates 4 of which I had in build-patterns. udisks2 failed and afterwards bauerbill exited rather than try to build the rest of the packages. There was no meaningful output as it just exited. Once I manually upgraded udisk2 and then re-initiated bauerbill, it worked fine building the rest in succession without issue. If you need all the output reguardless, Ill go back to an old snapshot and try again.

No worries and no hurry!

Alad · 2017-09-06 12:59:57

If you're interested I wrote a replacement for makepkg --pkg that doesn't require shipping a patched makepkg with bauerbill:

https://bbs.archlinux.org/viewtopic.php … 4#p1734644

Xyne · 2017-09-06 16:51:12

@GSF1200S
If you can find a reproducible example then I will try to debug it, but without one it will have to wait. I tried building udisks2 with bauerbill but encountered no error.

@Alad
Thanks but I prefer to use the patched makepkg. It preserves installation reasons and guarantees correct interpretation of all command-line arguments. My original approach was similar to your script but I quickly ran into problems with it (in particular, it would require more extensive argument parsing to preserve common arguments between makepkg and pacman -U... the patch is ultimately simpler and the code already exists in makepkg).

The best option is to try to get the pacman devs to restore the functionality. It was only removed because they decided that all packages in a split package should be built (which makes sense), but they didn't consider the use case of selective installation with makepkg -i.

Btw, you should quote the output of the mktemp command in case TMPDIR includes whitespace.

Alad · 2017-09-06 17:31:49

The best option is to try to get the pacman devs to restore the functionality. It was only removed because they decided that all packages in a split package should be built (which makes sense), but they didn't consider the use case of selective installation with makepkg -i.

Or you just use all benefits already available from pacman itself by using a local repository. I guess this is where pacboy (will?) come in.

Btw, you should quote the output of the mktemp command in case TMPDIR includes whitespace.

Sure, but it's not expanded on a variable assignment:

[user@enotty ~]$ foo='bar baz'
[user@enotty ~]$ bing=$(echo "$foo")
[user@enotty ~]$ echo "$bing"
bar baz

Last edited by Alad (2017-09-06 17:33:51)

Xyne · 2017-09-06 17:48:16

Alad wrote:

Btw, you should quote the output of the mktemp command in case TMPDIR includes whitespace.
Sure, but it's not expanded on a variable assignment:
[user@enotty ~]$ foo='bar baz'
[user@enotty ~]$ bing=$(echo "$foo")
[user@enotty ~]$ echo "$bing"
bar baz

You're right, sorry. I didn't consider that all uses of the assigned variable are quoted.

As for pacboy, it will be possible to add built packages to a repo and then install from that afterwards but it wasn't my intention. The flexibility is there though.

Spyhawk · 2017-09-07 11:15:58

Xyne wrote:

The best option is to try to get the pacman devs to restore the functionality. It was only removed because they decided that all packages in a split package should be built (which makes sense), but they didn't consider the use case of selective installation with makepkg -i.

Your question on the mailing list seemingly stayed unanswered, it might be worth a shot asking it again. It is unclear to me if the removal of the installation part was actually desired, but I don't think there is any downside to have it back.

SolarAquarion · 2017-10-02 03:50:11

solaraquarion@apollo   ~  sudo bauerbill -Su --build-vcs --build-all --aur                                                                                                             1 ↵  3015  18:15:34 
[sudo] password for solaraquarion: 
Traceback (most recent call last):
  File "/usr/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/usr/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/usr/lib/python3.6/http/client.py", line 1392, in connect
    super().connect()
  File "/usr/lib/python3.6/http/client.py", line 936, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/usr/lib/python3.6/socket.py", line 704, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/usr/lib/python3.6/socket.py", line 743, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 1150, in <module>
    run_main()
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 1132, in run_main
    return main(args)
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 1103, in main
    sync_pkgs, sync_deps, build_pkgs, build_deps = bb.get_sync_and_build_targets()
  File "/usr/lib/python3.6/site-packages/Bauerbill.py", line 534, in get_sync_and_build_targets
    pm2ml_pargs,
  File "/usr/lib/python3.6/site-packages/pm2ml.py", line 608, in resolve_targets_from_arguments
    only_needed=pargs.needed,
  File "/usr/lib/python3.6/site-packages/pm2ml.py", line 550, in resolve_targets
    ignoredgroups=ignoredgroups,
  File "/usr/lib/python3.6/site-packages/pm2ml.py", line 454, in resolve_aur_dependencies
    deps = set(iterate(self.buildable_pkgs(pkgs)))
  File "/usr/lib/python3.6/site-packages/pm2ml.py", line 424, in iterate_over_all_uninstalled_deps
    for d in iterate_over_all_deps(pkgs):
  File "/usr/lib/python3.6/site-packages/pm2ml.py", line 107, in iterate_over_all_deps
    for d in pkg.alldeps():
  File "/usr/lib/python3.6/site-packages/XCPF/ArchPkg.py", line 363, in alldeps
    for d in self.deps():
  File "/usr/lib/python3.6/site-packages/AUR/AurPkg.py", line 113, in with_arch_deps
    srcinfo = self.get_srcinfo()
  File "/usr/lib/python3.6/site-packages/AUR/AurPkg.py", line 77, in get_srcinfo
    for si in self.asi.get_pkginfo(pkgbases_and_pkgnames):
  File "/usr/lib/python3.6/site-packages/AUR/SRCINFO.py", line 209, in get_pkginfo
    for pkgname, srcinfo in zip(pkgnames, self.get(pkgbases)):
  File "/usr/lib/python3.6/site-packages/AUR/SRCINFO.py", line 195, in get
    for srcinfo in self.mdb.get_nth_field_many(self.SRCINFO_TABLE, args):
  File "/usr/lib/python3.6/site-packages/MemoizeDB.py", line 579, in get_nth_field_many
    for vs in self.get_many(table, keys):
  File "/usr/lib/python3.6/site-packages/MemoizeDB.py", line 549, in get_many
    for key, values in func(uncached_keys):
  File "/usr/lib/python3.6/site-packages/AUR/SRCINFO.py", line 171, in f
    yield pkgname, (XCGF.text_from_url(url),)
  File "/usr/lib/python3.6/site-packages/XCGF.py", line 51, in text_from_url
    with urllib.request.urlopen(url) as f:
  File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/usr/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/usr/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>

modedit: inserted code tags, please use them when posting content in the future /xyne

Last edited by Xyne (2017-10-03 20:07:14)

Xyne · 2017-10-03 20:49:40

SolarAquarion wrote:

...
socket.gaierror: [Errno -2] Name or service not known
...
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>

There seems to be a defunct URL in one of the target AUR packages. I have updated Bauerbill to handle the error but the problem is in the target package. You should be able to use the "--debug" option to determine which package/URL is causing the error.

GSF1200S · 2017-10-17 02:37:33

Sorry Xyne- never got around to it Bauerbill has been working fine though and with Arch changing their default package hardening (everything is FullRELRO/PIE now) I only rebuild 2-3 packages from ABS now. If I have an issue again I'll let you know. Thanks for the time/help to stay on top of something like this...

Arch Linux

#176 2017-05-10 08:46:02

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#177 2017-05-10 10:13:08

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#178 2017-05-14 03:49:49

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#179 2017-05-14 19:59:14

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#180 2017-05-15 12:55:15

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#181 2017-05-15 15:21:15

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#182 2017-07-03 14:30:41

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#183 2017-07-04 10:48:31

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#184 2017-07-04 11:47:45

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#185 2017-07-04 12:15:47

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#186 2017-07-04 23:57:31

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#187 2017-07-05 12:06:43

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#188 2017-08-15 04:47:28

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#189 2017-08-23 02:24:58

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#190 2017-08-24 13:28:18

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#191 2017-08-24 20:13:31

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#192 2017-08-31 17:09:13

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#193 2017-09-06 12:59:57

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#194 2017-09-06 16:51:12

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#195 2017-09-06 17:31:49

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#196 2017-09-06 17:48:16

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#197 2017-09-07 11:15:58

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#198 2017-10-02 03:50:11

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#199 2017-10-03 20:49:40

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

#200 2017-10-17 02:37:33

Re: Bauerbill reborn: pacman/powerpill+AUR+ABS+configurability

Board footer