AUR4 & makepkg-template, mksrcinfo, filter-branch, ...

XZS · 2015-07-02 08:46:33

This goes out to everyone who maintains collections of their PKGBUILDs on GitHub or similar things. With the move to AUR4, many users splinter their repositories to be compliant with the new upload system. I saw solutions using git submodules and subtrees. I struggle with both as most of my PKGBUILDs heavily rely on templates.

Additionally, I am reluctant to clutter my repositories with all these .SRCINFO files. In my whole packaging experience, only once did I have to edit one of them manually, so they would in fact be nothing but redundant in my repositories. And I predict that I will forget to update them all the time. To push to AUR4, they also have to be injected into the history, as AUR4 rejects every commit that does not have them. I find this to be quite ironic as AUR4 itself does not allow history rewrites.

I have yet to move most of my PKGBUILD collections to AUR4 and thus would like to read which approach you took. Did you find a solution to push to AUR4 without rewriting your whole history? Is anyone out there even using makepkg-template? Is it possible to generate the .SRCINFOs in a hook, filter or something similar?

progandy · 2015-07-02 09:14:42

With makepkg-template, you'll have to expand the template for the AUR. I suggest you create a "release" script that expands the template on top of your aur branch, generates the srcinfo, and suggests a sqashed commit message. Then the script commits and pushes the aur branch.

XZS · 2015-07-02 20:37:00

I assume this would mean invoking the release procedure manually. Also, I would have to keep separate repositories for each package in addition to the collection repo where the main work is done.

I was hoping for some strategy that lets me move a bunch of (relevant) commits directly, perhaps maintaining the package repositories as branches of the collection. git subtree split can do something like this, even without processing all revisions from the start again on each invocation. I already saw some repositories distribute that way.

Templates seem to complicate this massively. Before the subtree split, everything would have to be run through git filter-branch --tree-filter 'mkpackage-template; mksrcinfo'. Filter-branch generates the same commits on each invocation, but it always starts at the root commit because commit hashes include the parent pointer, so there is no way to generate a modified version of something without recursively generating all its ancestors. This may take some time on huge repositories. And when mksrcinfo then decides to change its output one day, filter-branch would not even throw out the same modified commits as before...

jjacky · 2015-07-03 09:29:44

Yeah, my solution has been to simply keep maintaining my PKGBUILDs in my current repo, as before. As you've seen, there's no real way to make it "compatible" with AUR4. So instead I just created new repos, one per package and all as per AUR4 requirements, that I only use to push to the AUR.

Basically, I now have a ~/aur-git folder where I have, for each package I maintain in the AUR, two things: a file foobar.last-commit & a folder foobar. The later is obviously the git repo to push to the AUR, while the former contains the commit-id of the commit I last "exported" over here from my actual repo (of PKGBUILDs). When I want to push things to the AUR, I have a little script that will get the commit ids (from my repo) from last-commit to HEAD, then for each one apply the patch on the aur-git repo, generate the .SRCINFO, and commit there. Once done, I can simply push that to the AUR.

The script, which must be called from the package folder in my actual repo, goes like this:

ALL_OFF="\e[0m"
BOLD="\e[1m"
BLUE="${BOLD}\e[36m"
GREEN="${BOLD}\e[32m"
RED="${BOLD}\e[31m"
YELLOW="${BOLD}\e[33m"

msg() {
    local mesg=$1; shift
    printf "${GREEN}==>${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2
}

msg2() {
    local mesg=$1; shift
    printf "${BLUE}  ->${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2
}

error() {
    local mesg=$1; shift
    printf "${RED}==> ERROR:${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2
}

warning() {
    local mesg=$1; shift
    printf "${YELLOW}==> WARNING:${ALL_OFF}${BOLD} ${mesg}${ALL_OFF}\n" "$@" >&2
}

err() {
    error "$@"
    exit 1
}

_confirm () {
	echo
	echo -n "$1 [Y/n] "
	read doit
	case $doit in
		("" | [Yy]*) return 1 ;;
		(*) return 0 ;;
	esac
}


# source location
sce=~/abs
# destination (AUR git repo) location
dst=~/aur-git

pkgname=$(pwd)
[[ ${pkgname%/*} = "$sce" ]] || err "Invalid location"
pkgname=${pkgname##*/}

if [[ ! -d "$dst/$pkgname" ]]; then
    msg "New package $pkgname..."
    mkdir "$dst/$pkgname" || err "Failed to create folder $dst/$pkgname"
    msg2 "Cloning git repo from AUR..."
    git -C "$dst/$pkgname" clone -o aur \
        "ssh://aur@aur4.archlinux.org/$pkgname.git" . \
        || err "Failed to clone AUR git repo"
    touch "$dst/$pkgname.last-commit"
    range="HEAD"
else
    [[ -r "$dst/$pkgname.last-commit" ]] || err "File $pkgname.last-commit missing"
    if [[ -s "$dst/$pkgname.last-commit" ]]; then
        range=$(cat "$dst/$pkgname.last-commit")"..HEAD"
    else
        range="HEAD"
    fi
fi

msg "Processing commits..."
git rev-list --reverse "$range" -- . | while read -r commit; do
    msg2 "Commit $commit..."
    git format-patch -1 -k --stdout "$commit" -- .  |\
        git -C "$dst/$pkgname" am -p2 \
        || err "Failed to import commit"
    mksrcinfo -o "$dst/$pkgname/.SRCINFO" "$dst/$pkgname/PKGBUILD" \
        || err "Failed to generate .SRCINFO"
    git -C "$dst/$pkgname" add .SRCINFO \
        || err "Failed to add .SRCINFO"
    git -C "$dst/$pkgname" commit --amend --no-edit \
        || err "Failed to amend commit"
    echo "$commit" > "$dst/$pkgname.last-commit"
done
msg "done"

_confirm "push to AUR?"
[[ $? -ne 1 ]] && exit 0

msg "Pushing to AUR..."
git -C "$dst/$pkgname" push aur master || err "Failed to push to AUR"
msg "done"

On a daily basis I keep doing things as before, and when I want to push to the AUR I simply run that `mkaur` script and it does it all automatically. I don't actually have to deal with the "aur repos" myself, but I could import the whole history there which is what I wanted, and add .SRCINFO in every commit which was required. It probably has limitations (starting with the need for the folder name to be that of the package/repo, and possibly not change ever) but (so far) it works for me.

Edit: Stop being that lazy and added the few functions (borrowed from makepkg).

Last edited by jjacky (2015-07-03 12:43:26)

Arch Linux

#1 2015-07-02 08:46:33

AUR4 & makepkg-template, mksrcinfo, filter-branch, ...

#2 2015-07-02 09:14:42

Re: AUR4 & makepkg-template, mksrcinfo, filter-branch, ...

#3 2015-07-02 20:37:00

Re: AUR4 & makepkg-template, mksrcinfo, filter-branch, ...

#4 2015-07-03 09:29:44

Re: AUR4 & makepkg-template, mksrcinfo, filter-branch, ...

Board footer