[BETA] Pacboy - extensible Pacman wrapper: testers and devs needed :)

Xyne · 2016-07-10 20:37:19

Pacboy is a refactored backend for Powerpill and Bauerbill with the following features:

Fast downloads (packages, PKGBUILDs, package sources) via Aria2c.
Generalized API to support arbitrary repo extensions. You can add build support to pacman sync repos or create build-only third-party repos such as the AUR, CPAN, etc.
Associate packages with repos to always update those packages from those repos regardless of repo order.
Configure which packages should always be built and which should never be built.
Rebuild all rebuildable installed VCS packages.
Download PKGBUILDs and source files for all supported packages (official repos, AUR, and any others supported by an extension). These downloads will be run in parallel (including the source file downloads, if the option is passed).
Generated build scripts are fully customizable via per-package code snippets so users can apply patches, use custom build locations, etc., for any package.

A snapshot archive for testing and development is available here (parent page with last mod times). This will be updated regularly so grab the latest copy before reporting a bug. The updates will be pushed to all of my packages after testing and giving people some time to get ready for the change. The new backend is not fully backwards compatible with the current implementation of powerpill and bauerbill (but all features have been ported, I think).

Testers
The archive provides updated versions of powerpill, bauerbill and pbget along with pacboy. Run these with the "testrun.sh" script in the archive. Check the README for the latest details. Let me know what doesn't work or if any features are missing. Ask here if you need help porting your configuration files.

Developers
Pacboy supports extensions to add build support to existing repos (BuildExtender class) or create entirely new build-only repos (RepoWrapper class). These can be provided as third-party Python 3 modules that implement up to 3 subclasses for either. See the provided pacboy-extension man page for details. There are 2 reference implementation for build extenders (my own repos and the official repos) and 1 for repo wrappers (the AUR).

Build Extenders
If you already provide a repo to the community, the easiest way to add a build extension to Pacboy is to provide downloadable SRCINFO files and archives of PKGBUILDs and related files. This is what I have done on xyne.archlinux.ca. It is then a simple matter of subclassing XacBuildExtender and overriding the two URL formatting methods.
Repo Wrappers
There are a number of developers who have provided ways of automatically generating packages from third-party sources (Hackage, CPAN, PyPI, Ruby Gems, RPM packages, etc.). If you can generate a PKGBUILD then it should be possible to write an extension. Repo wrappers are a bit more difficult than build extenders but I hope that the documentation and reference implementation will be enough to get started. If this gains critical mall then we may end up with a very powerful pacman wrapper.

If anything is unclear or if you want some help, ask here. I still have very little free time at the moment but I will try to reply as soon as possible.

antony · 2016-07-14 03:48:22

I'd be happy to turn pypi2pkgbuild into a repo wrapper but I think I'll need more documentation about the actual API; I don't really want to have to figure out the API from the examples

ybham6 · 2016-07-14 05:15:06

Just a note, you need

python3-colorsysplus

for pacboy to run, otherwise it gives a traceback.

Xyne · 2016-07-14 11:00:43

@antony
Great
I'll try to put together a separate a step-by-step guide with some figures to clarify relations between objects and their internal structures (the grouped downloads object is confusing at first). If you see something in the main modules that is unclearly documented, please let me know.

@ybham6
Thanks. I have added the colorsysplus module to the testing archive.

edit
I've been thinking about how to write a guide but as the author it's difficult to know what exactly needs clarification. I have tried to improve the docstrings and comments and simplify some things but I think if I start writing a guide on my own it may not be as helpful as I would hope.

Maybe we can try to do this semi-interactively.

To start, you will need a subclass of XCPF.Pkg.Pkg. You can probably use the existing XCPF.Pkg.SrcinfoPkg by either

Generating a PKGBUILD with PyPI2PKGBUILD (internally or through subprocess) for each package and then using makepkg or mksrcinfo to generate a SRCINFO file. You can then use XCPF.SRCINFO.Srcinfo to load it and initialize a SrcinfoPkg.
Adapting the code in PyPI2PKGBUILD to dump the metadata directly into a SRCINFO file and then load as above.

Otherwise you could subclass XCPF.Pkg.Pkg and use the code in PyPI2PKGBUILD to provide the metadata directly through Pkg's methods.

Once you have a Pkg subclass, you will need to subclass XCPF.RepoWrapper.RepoWrapper. That will need to encapsulate an interface to PyPI with the ability to query and list packages, generate Pkg objects of the type above, etc. The code here depends entirely on the interface with PyPI: you will need to wrap their online APIs or determine what needs to be downloaded to get access to their packages.

Last edited by Xyne (2016-07-14 20:31:48)

Arch Linux

#1 2016-07-10 20:37:19

[BETA] Pacboy - extensible Pacman wrapper: testers and devs needed :)

#2 2016-07-14 03:48:22

Re: [BETA] Pacboy - extensible Pacman wrapper: testers and devs needed :)

#3 2016-07-14 05:15:06

Re: [BETA] Pacboy - extensible Pacman wrapper: testers and devs needed :)

#4 2016-07-14 11:00:43

Re: [BETA] Pacboy - extensible Pacman wrapper: testers and devs needed :)

Board footer