remakepkg — rebuild pacman packages with changes

ayekat · 2018-02-28 17:59:31

Introducing repkg and remakepkg, tools for applying changes to pacman package (.pkg.tar.xz) files.

Description

repkg can modify metadata (add/remove hard/optional dependencies, provisions, conflicts, replacements, backup files and groups; change the URL and the architecture), create/rename/delete files, change file permissions, and patch text files.
remakepkg is a script that wraps around repkg, since the invocation is a bit cumbersome (see below).

It has been written for people who would like to modify official Arch Linux packages without having to adopt the full-blown package maintenance work (maintaining a PKGBUILD, compiling software, getting frustrated by not being able to sanely keep track of reported issues on bugs.archlinux.org, ...).

Usage: repkg

repkg is the script that does the heavy lifting. It takes a rules file (that contains operations to be applied to a package) and applies it to a package file, producing a new, "repkg'd" package file. Here is an example rules file that adds the dependency `foo` and removes the optional dependency `bar`:

add-depend foo
remove-optdepend bar

To apply these changes to a package, we can invoke repkg as follows:

$ repkg -i /path/to/foobar-1.0-1-any.pkg.tar.xz -r /path/to/rulesfile

This will generate a new package file in the current working directory, and we may run e.g. `pacman -Qpi` on it to inspect it. Note that repkg will error out if either `foo` is already a dependency, or `bar` is not an optional dependency.

All available rules/commands are described in the COMMANDS section in repkg(1).

... but let's admit it, using repkg directly like this is cumbersome; we'd have to download the package file and run repkg with all the command line options. So...

Usage: remakepkg

remakepkg uses a file called REPKGBUILD. A REPKGBUILD file is more or less a rules file, but with an additional meta-info (the name of the package we'd like to repkg). Here is a (more real-life) example PKGBUILD file that fixes some open bugs in the pass package:

# This declares that the rules in this REPKGBUILD file apply to community/pass
# (remakepkg requires this to know which package to download from a mirror):
repkg community/pass

# These rules turn xclip into an optional dependency (FS#55059):
remove-depend xclip
add-optdepend xclip: for X clipboard usage

# These rules add a missing optional dependency (FS#55504):
add-optdepend xdotool: for passmenu

To create a repkg'd package file with this REPKGBUILD, we can simply invoke (in the same directory as the REPKGBUILD file):

$ remakepkg

remakepkg uses the first rule (`repkg ...`) to know which package to repkg. It then downloads it from the package mirror, generates a rules file from the remaining lines, and invokes repkg.

Rationale

Although it was initially written to just fix ignored/rejected packaging bugs, it has turned out to be useful for doing other stuff as well (such as having dash provide /bin/sh instead of bash, or letting pacman optionally depend on a base-devel metapackage, ...).

However, repkg/remakepkg is not a replacement for maintaining a proper PKGBUILD if your goal is to rebuild binaries with alternative compile flags. It is mainly here for fixing package metadata.

To get an idea, the full list of REPKGBUILDs I currently use can be found here.

For a more subjective comment from my side, I have followed the commonly given advice to "get a blog", and it can be read here.

Installation

repkg and remakepkg are part of the pacman-hacks project, a small collection of mediocre little scripts of mine. There is a PKGBUILD available here to build an installable package with all the scripts and man pages from the latest stable release of pacman-hacks.

If you don't want the full load of mediocre ayekat-scripts, there is also remakepkg in the AUR; it only installs remakepkg + the scripts it requires.

Limitations/Bugs

See the list of open issues for repkg and remakepkg, and feel free to add bug reports and feature requests.

Last edited by ayekat (2019-03-14 14:59:09)

ayekat · 2018-03-10 12:48:52

I have added a wrapper script for repkg called remakepkg (which uses REPKGBUILD files (you see what I did there )), so I have updated the post above.

Last edited by ayekat (2018-09-18 22:21:37)

Stefan Husmann · 2018-04-07 15:44:25

I did not have a closer look at your scripts, but do you know the customizepkg flavours in AUR?

ayekat · 2018-04-08 09:31:13

Hey, thanks for the reply!

If I understood customizepkg correctly, it is not quite the same thing. But it does seem to tackle a limitation in makepkg/PKGBUILDs that I have also encountered: there is no elegant way of maintaining custom changes (like custom compilation flags) to PKGBUILDs; one cannot load in a "snippet" or use some sort of "overlay"¹, so customizepkg appears to provide that functionality. It does require you to build/compile the whole package, though.

My goal here is a different one: I only want to modify the packages' metadata (check the list of packages I currently "maintain" through REPKGBUILDs). Given that the metadata can be modified in the built package, I can avoid rebuilding it entirely (it's basically as if I had already built a package with makepkg, then I tweak the PKGBUILD and run `makepkg --repackage`).

I guess if at some point I'm interested in using packages with custom compile flags, something like customizepkg would be a viable option. But I'm not sure if I might not just as well start using something like Gentoo at that point

____
¹ Not the "overlay" in Gentoo parlance.

Last edited by ayekat (2018-04-08 09:35:15)

Lone_Wolf · 2018-04-08 15:55:54

Interesting.

Could this be used to remove systemd dependency (or make it an optdep) from mkinitcpio ?

( without systemd majority of mkinitcpio still works, just some systemd-specific functionality doesn't work )

eschwartz · 2018-04-08 18:07:28

If mkinitcpio does not need systemd, have you considered filing a bugreport asking for it to be removed from the dependencies???

(ISTR other distros used it actually, so that makes sense I guess. It doesn't seem to explicitly invoke it ever, except for sd-shutdown/sd-vconsole.)

ayekat · 2018-04-08 18:43:52

Lone_Wolf wrote:

Could this be used to remove systemd dependency (or make it an optdep) from mkinitcpio ?

REPKGBUILD:

repkg core/mkinitcpio
remove-depend systemd
add-optdepend systemd: for building initcpio with systemd

Output:

$ remakepkg
>>> Parsing REPKGBUILD
>>> Recreating package for core/mkinitcpio
>>> Downloading package file from mirror
>>> Downloading package from http://pkg.adfinis-sygroup.ch/archlinux/core/os/x86_64/mkinitcpio-24-2-any.pkg.tar.xz
>>> Downloading signature from http://pkg.adfinis-sygroup.ch/archlinux/core/os/x86_64/mkinitcpio-24-2-any.pkg.tar.xz.sig
==> Checking mkinitcpio-24-2-any.pkg.tar.xz.sig...
gpg: assuming signed data in 'mkinitcpio-24-2-any.pkg.tar.xz'
gpg: Signature made Fri 06 Oct 2017 21:29:55 CEST
gpg:                using RSA key 487EACC08557AD082088DABA1EB2638FF56C0C53
gpg: Note: trustdb not writable
gpg: Good signature from "Dave Reisner <d@falconindy.com>" [full]
gpg:                 aka "Dave Reisner <dreisner@archlinux.org>" [full]
>>> Remaking package
>>> Detected package mkinitcpio
>>> Removing dependency systemd
>>> Adding optional dependency systemd (for building initcpio with systemd)
>>> Generating new package file /home/ayekat/tmp/mkinitcpio-24-2-any.pkg.tar.xz
>>> Successfully created /home/ayekat/tmp/mkinitcpio-24-2-any.pkg.tar.xz
$ pacman -Qpi mkinitcpio-24-2-any.pkg.tar.xz 
Name            : mkinitcpio
Version         : 24-2
Description     : Modular initramfs image creation utility
Architecture    : any
URL             : https://projects.archlinux.org/mkinitcpio.git/
Licenses        : GPL
Groups          : None
Provides        : None
Depends On      : awk  mkinitcpio-busybox>=1.19.4-2  kmod  util-linux>=2.23  libarchive
                  coreutils  bash  findutils  grep  filesystem>=2011.10-1  gzip
Optional Deps   : xz: Use lzma or xz compression for the initramfs image
                  bzip2: Use bzip2 compression for the initramfs image
                  lzop: Use lzo compression for the initramfs image
                  lz4: Use lz4 compression for the initramfs image
                  mkinitcpio-nfs-utils: Support for root filesystem on NFS
                  systemd: for building initcpio with systemd
Conflicts With  : None
Replaces        : None
Compressed Size : 40.63 KiB
Installed Size  : 214.00 KiB
Packager        : Tinu Weber <http://ayekat.ch>
Build Date      : Sun 08 Apr 2018 20:38:08 CEST
Install Script  : Yes
Validated By    : None
Signatures      : None

Lone_Wolf · 2018-04-10 10:55:12

Thank you, ayekat. That clearly shows how your tool works.
I won't be using that particular change, but it gives a template for some changes I do have in mind.

ESchwartz :
When mkinitcpio gained systemd as a dependency, obarun / systemd-free.org started building their own mkinitcpio version.
I looked into the changes, found it still worked fine if systemd was installed but not booted.

I did have the impression systemd was a valid makedepence.
What is needed at runtime depends on which mkinitcpio hooks are used.
udev, usr, resume, mdadm_udev and the 3 sd-* hooks require systemd support , the others don't.

In my opinion that makes systemd a makedepend and an optional depend.
Whether that's a bug is debatable, and stock mkinitcpio works fine on my multi-init systems.
As those that I knew created their own mkinitcpio binary packages did refuse to have systemd installed at all, I saw no benefit in bringing this up and filing a bug report.

ayekat · 2018-09-12 13:39:45

In case anyone is interested (and crazy enough) to try it out, I was so free to add a PKGBUILD for remakepkg/repkg to the AUR: aur/remakepkg.

Some things have also changed since I last commented here; the provided changelog should cover them, though.

Last edited by ayekat (2018-09-18 22:23:15)

ayekat · 2019-06-25 11:43:38

Hey, here's some updates:

As already mentioned above, remakepkg is really just one part of a project called pacman-hacks, which contains all sorts of scripts I've written to complement the available package management functionalities (one other noteworthy tool in there is probably metapkg/makemetapkg, which can be used to conveniently describe and build meta-packages through so-called METAPKGBUILD files; here's an example for how I organise the packages I need for my university courses).

There is a PKGBUILD for pacman-hacks, but given the quality (or general usefulness) of some of the scripts in there, I haven't uploaded it to the AUR (also because the lack of namespaces in the AUR means that the project name is not very descriptive). To allow people to still use (or try out) makemetapkg and/or remakepkg, I wrote PKGBUILDs that just package the required subsets of pacman-hacks, and uploaded those to the AUR.

But now I find myself doing a lot of duplicate work for every single small upstream release I make, as I have to perform more or less the same packaging actions 3 times. Given that I only know of one other person who actually uses this (and they use the original pacman-hacks PKGBUILD directly anyway), this feels like a bit of wasted time to me.

As I am a firm believer in that no package should be maintained by someone who doesn't actually use it themself (and as I don't want to be scared of releasing bugfixes anymore), I have decided to orphan both aur/makemetapkg and aur/remakepkg. If anyone is interested in picking either (or both) of them up, feel free to do so.

Otherwise, I guess the next AUR clean-up day will get rid of them

Arch Linux

#1 2018-02-28 17:59:31