PKGBUILD review requests: jose, tang, luksmeta, and clevis

ShaRose · 2017-10-15 04:46:28

I know it's probably odd to have four PKGBUILD requests at the same time, but these are all semi-related and are dependent on each other, so I thought it was just as well include them all.

Jose

First, here's jose. It's mostly just a dependency for the following packages, but might be useful to some.

PKGBUILD

# Maintainer: Shawn Rose <shawnandrewrose@gmail.com>
pkgname=jose
pkgver=10.1.g198f720
pkgrel=1
pkgdesc="A C-language implementation of the Javascript Object Signing and Encryption
standards."
arch=('i686' 'x86_64' 'armv7h')
url="https://github.com/latchset/jose"
license=('APACHE')
depends=('jansson>=2.10' 'openssl>=1.0.2' 'zlib')
makedepends=('git')
provides=('jose')
source=("$pkgname::git+https://github.com/latchset/jose.git#commit=198f7207427ad7f569aa3592ea16e2bb400db040")
sha256sums=('SKIP')

pkgver() {
  cd "$pkgname"
  git describe --tags --long | sed -r 's/^[a-z]*-?//;s/([^-]*-g)/\1/;s/-/./g'
}

build() {
  cd "$pkgname"
  autoreconf -if
  ./configure --prefix=/usr
  make
}

check() {
  cd "$pkgname"
  make check
}

package() {
  cd "$pkgname"
  make DESTDIR="$pkgdir/" install
}

namcap comes up clean on the PKGBUILD, but gives me warnings like jose W: Unused shared library '/usr/lib/libpthread.so.0' by file ('usr/bin/jose') for the package. As far as I can tell, it's not a big deal, but I'd love to see if there's any way to resolve it.

Of note, I'm pulling a specific commit instead of just tag=v10 because the commit right after that fixes a small FILE* leak, but it's not needed by any means.

tang

Next is tang. This is used with clevis (below) to store secrets with a network presence statelessly: the server doesn't store any data on clients, but the clients need to be able to reach that server at the same address with the same key to be able to decrypt the secret it has.

PKGBUILD

# Maintainer: Shawn Rose <shawnandrewrose@gmail.com>
pkgname=tang
pkgver=6.0.ga0e69ce
pkgrel=1
pkgdesc="A server for binding data to network presence."
arch=('i686' 'x86_64' 'armv7h')
url="https://github.com/latchset/tang"
license=('APACHE')
depends=('http-parser' 'jose')
makedepends=('git')
provides=('tang')
source=("$pkgname::git+https://github.com/latchset/tang.git#tag=v6"
        'http_status_map.h')
sha256sums=('SKIP'
            'd1518e0956f4b5da58ec01aa20b35e6d4abc430b0a336e9680be76b9b9542fd3')

pkgver() {
  cd "$pkgname"
  git describe --tags --long | sed -r 's/^[a-z]*-?//;s/([^-]*-g)/\1/;s/-/./g'
}

prepare() {
  cp "$srcdir/http_status_map.h" "$pkgname/src/http_status_map.h"
  sed -i 's/http_status/tang_http_status/; s/HTTP_STATUS_/TANG_HTTP_STATUS_/' "$pkgname/src/http.c"
  sed -i 's/http_status/tang_http_status/; s/HTTP_STATUS_/TANG_HTTP_STATUS_/' "$pkgname/src/http.h"
  sed -i 's/http_status/tang_http_status/; s/HTTP_STATUS_/TANG_HTTP_STATUS_/' "$pkgname/src/tangd.c"
  sed -i '/#include <regex.h>/ a #include "src/http_status_map.h"' "$pkgname/src/http.h"
  sed -i '/AC_CHECK_HEADER(\[http_parser.h\]/,+6 d' "$pkgname/configure.ac"
}

build() {
  cd "$pkgname"
  autoreconf -if
  ./configure --prefix=/usr --libexecdir="/usr/lib/$pkgname/"
  make
}


check() {
  cd "$pkgname"
  make check
}

package() {
  cd "$pkgname"
  make DESTDIR="$pkgdir/" install
}

http_status_map.h

/* Copyright Joyent, Inc. and other Node contributors. All rights reserved.
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to
 * deal in the Software without restriction, including without limitation the
 * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
 * sell copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
 * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
 * IN THE SOFTWARE.
 */

/*
This was cut and pasted from the commit npmccallum made for http-parser, available at
https://github.com/nodejs/http-parser/commit/335850f6b868d3411968cbf5a4d59fe619dee36f
It is only used that commit is not included in the system's http-parser, since
it was commited after the last tagged release (2.7.1) as of October 14, 2017.
*/

#include <stddef.h>

/* Status Codes */
#define TANG_HTTP_STATUS_MAP(XX)                                                 \
  XX(100, CONTINUE,                        Continue)                        \
  XX(101, SWITCHING_PROTOCOLS,             Switching Protocols)             \
  XX(102, PROCESSING,                      Processing)                      \
  XX(200, OK,                              OK)                              \
  XX(201, CREATED,                         Created)                         \
  XX(202, ACCEPTED,                        Accepted)                        \
  XX(203, NON_AUTHORITATIVE_INFORMATION,   Non-Authoritative Information)   \
  XX(204, NO_CONTENT,                      No Content)                      \
  XX(205, RESET_CONTENT,                   Reset Content)                   \
  XX(206, PARTIAL_CONTENT,                 Partial Content)                 \
  XX(207, MULTI_STATUS,                    Multi-Status)                    \
  XX(208, ALREADY_REPORTED,                Already Reported)                \
  XX(226, IM_USED,                         IM Used)                         \
  XX(300, MULTIPLE_CHOICES,                Multiple Choices)                \
  XX(301, MOVED_PERMANENTLY,               Moved Permanently)               \
  XX(302, FOUND,                           Found)                           \
  XX(303, SEE_OTHER,                       See Other)                       \
  XX(304, NOT_MODIFIED,                    Not Modified)                    \
  XX(305, USE_PROXY,                       Use Proxy)                       \
  XX(307, TEMPORARY_REDIRECT,              Temporary Redirect)              \
  XX(308, PERMANENT_REDIRECT,              Permanent Redirect)              \
  XX(400, BAD_REQUEST,                     Bad Request)                     \
  XX(401, UNAUTHORIZED,                    Unauthorized)                    \
  XX(402, PAYMENT_REQUIRED,                Payment Required)                \
  XX(403, FORBIDDEN,                       Forbidden)                       \
  XX(404, NOT_FOUND,                       Not Found)                       \
  XX(405, METHOD_NOT_ALLOWED,              Method Not Allowed)              \
  XX(406, NOT_ACCEPTABLE,                  Not Acceptable)                  \
  XX(407, PROXY_AUTHENTICATION_REQUIRED,   Proxy Authentication Required)   \
  XX(408, REQUEST_TIMEOUT,                 Request Timeout)                 \
  XX(409, CONFLICT,                        Conflict)                        \
  XX(410, GONE,                            Gone)                            \
  XX(411, LENGTH_REQUIRED,                 Length Required)                 \
  XX(412, PRECONDITION_FAILED,             Precondition Failed)             \
  XX(413, PAYLOAD_TOO_LARGE,               Payload Too Large)               \
  XX(414, URI_TOO_LONG,                    URI Too Long)                    \
  XX(415, UNSUPPORTED_MEDIA_TYPE,          Unsupported Media Type)          \
  XX(416, RANGE_NOT_SATISFIABLE,           Range Not Satisfiable)           \
  XX(417, EXPECTATION_FAILED,              Expectation Failed)              \
  XX(421, MISDIRECTED_REQUEST,             Misdirected Request)             \
  XX(422, UNPROCESSABLE_ENTITY,            Unprocessable Entity)            \
  XX(423, LOCKED,                          Locked)                          \
  XX(424, FAILED_DEPENDENCY,               Failed Dependency)               \
  XX(426, UPGRADE_REQUIRED,                Upgrade Required)                \
  XX(428, PRECONDITION_REQUIRED,           Precondition Required)           \
  XX(429, TOO_MANY_REQUESTS,               Too Many Requests)               \
  XX(431, REQUEST_HEADER_FIELDS_TOO_LARGE, Request Header Fields Too Large) \
  XX(451, UNAVAILABLE_FOR_LEGAL_REASONS,   Unavailable For Legal Reasons)   \
  XX(500, INTERNAL_SERVER_ERROR,           Internal Server Error)           \
  XX(501, NOT_IMPLEMENTED,                 Not Implemented)                 \
  XX(502, BAD_GATEWAY,                     Bad Gateway)                     \
  XX(503, SERVICE_UNAVAILABLE,             Service Unavailable)             \
  XX(504, GATEWAY_TIMEOUT,                 Gateway Timeout)                 \
  XX(505, HTTP_VERSION_NOT_SUPPORTED,      HTTP Version Not Supported)      \
  XX(506, VARIANT_ALSO_NEGOTIATES,         Variant Also Negotiates)         \
  XX(507, INSUFFICIENT_STORAGE,            Insufficient Storage)            \
  XX(508, LOOP_DETECTED,                   Loop Detected)                   \
  XX(510, NOT_EXTENDED,                    Not Extended)                    \
  XX(511, NETWORK_AUTHENTICATION_REQUIRED, Network Authentication Required) \

enum tang_http_status
  {
#define XX(num, name, string) TANG_HTTP_STATUS_##name = num,
  TANG_HTTP_STATUS_MAP(XX)
#undef XX
  };

namcap shows perfectly clean on both PKGBUILD and the compiled package.

You will see how there's a massive header file for this one: The reasoning is noted as a commit in said header file. I had to do a bit of renaming so that just in case someone actually had built a copy of http-parser from the nodejs repository it wouldn't crash instantly due to things already being defined.

I HAD debated also flagging the http-parser package as out-of-date, but since node never tagged any newer versions I didn't think it would count as stable.
Note there IS an http-parser-git package on the AUR, but it's also out of date and I wasn't sure if it was deprecated on purpose. It might be a better idea to have someone update that package, and then remove a lot of the stuff I had to do in here.

luksmeta

Moving along to luksmeta. This is a library that allows you to store arbitrary data in a LUKSv1 header using the gap between the slot definitions and the actual encrypted data.

PKGBUILD

# Maintainer: Shawn Rose <shawnandrewrose@gmail.com>
pkgname=luksmeta
pkgver=8.0.geea76e2
pkgrel=1
pkgdesc="A simple library for storing metadata in the LUKSv1 header."
arch=('i686' 'x86_64' 'armv7h')
url="https://github.com/latchset/luksmeta"
license=('LGPL2.1')
depends=('cryptsetup')
makedepends=('git')
provides=('luksmeta')
source=("$pkgname::git+https://github.com/latchset/luksmeta.git#tag=v8")
sha256sums=('SKIP')

pkgver() {
  cd "$pkgname"
  git describe --tags --long | sed -r 's/^[a-z]*-?//;s/([^-]*-g)/\1/;s/-/./g'
}

build() {
  cd "$pkgname"
  autoreconf -if
  ./configure --prefix=/usr
  make
}

check() {
  cd "$pkgname"
  make check
}

package() {
  cd "$pkgname"
  make DESTDIR="$pkgdir/" install
}

Again, no warnings in namcap for this one. I don't think there's really any problem with this one at all: no ugly hacks to be found.

Which is a shame, because I've saved the worst for last.

clevis

The last package I have been working on is clevis. This is a tool automatic decryption using the above tools, including automatic decryption of encrypted root partitions as long as it can reach the right server(s). Think bitlocker network unlock, but you can do stuff like "You need to be able to reach at least 3 of the 6 servers to unlock".

PKGBUILD

# Maintainer: Shawn Rose <shawnandrewrose@gmail.com>
pkgname=clevis
pkgver=6.0.g18a3e15
pkgrel=1
pkgdesc="A plugable framework for automated decryption."
arch=('i686' 'x86_64' 'armv7h')
url="https://github.com/latchset/clevis"
license=('GPL3')
depends=('jose' 'bash' 'curl')
optdepends=('luksmeta: integration with luksmeta to store clevis information in LUKS volumes'
            'udisk2: Unlock removable devices automatically in your desktop session with udisks2'
            'mkinitcpio-utils: enable decryption of LUKS root volumes at boot'
            'mkinitcpio-netconf: used with mkinitcpio-utils for network access decryption'
            'mkinitcpio-ppp: used with mkinitcpio-utils for network access decryption using ppp')
makedepends=('git')
provides=('clevis')
source=("$pkgname::git+https://github.com/latchset/clevis.git#tag=v6"
        "clevis-mkinitcpio-install"
        "clevis-mkinitcpio-hook")
sha256sums=('SKIP'
            'cddb3431aa02e7b3a0a2b9e04c61f52ff0d86c787bd9442f2fbfabe71ea97052'
            '46916a788ea6a2a55d58066fef74c3fa1334a1fa2a8fdfb616d072e8608c8414')

pkgver() {
  cd "$pkgname"
  git describe --tags --long | sed -r 's/^[a-z]*-?//;s/([^-]*-g)/\1/;s/-/./g'
}

prepare() {
  cd "$pkgname"
  # Kill off everything related to dracut
  sed -i '/^PKG_CHECK_MODULES(\[dracut\].*/ d; /AC_ARG_WITH(\[dracutmodulesdir\]/,+4 d' configure.ac
  sed -i '/dracut/ d' src/Makefile.am
  rm -rf src/dracut/

  if ( ! pacman -T udisks2 >/dev/null ); then
    # Removing udisks2 stuff as well
    sed -i '/^PKG_CHECK_MODULES(\[udisks2\].*/ d' configure.ac
    sed -i '/udisks2/ d' src/Makefile.am
    rm -rf src/udisks2/
  fi

  # Patch the check for tang so it uses the right path and make sure it skips without tang installed
  sed -i 's#/usr/libexec/tang#/usr/lib/tang/tang#;
/function on_exit()/ i ( ! pacman -T tang ) && exit 77' tests/pin-tang

  # Patch out the check for luksmeta, it's optional now.
  sed -i '/^PKG_CHECK_MODULES(\[luksmeta\].*/ d' configure.ac

  # Also patch clevis-bind-luks to print an error when you don't have luksmeta and hide from clevis
  sed -i '/SUMMARY=/ i if ( ! pacman -T luksmeta >/dev/null ); then\
    echo "You need luksmeta installed to use clevis bind luks!"\
    exit 1\
fi\
' src/clevis-bind-luks
}

build() {
  cd "$pkgname"
  autoreconf -if
  ./configure --prefix=/usr
  make
}

check() {
  cd "$pkgname"
  make check
}

package() {
  mkdir -p "$pkgdir"/usr/lib/initcpio/{install,hooks}
  cp clevis-mkinitcpio-install "$pkgdir"/usr/lib/initcpio/install/clevis
  cp clevis-mkinitcpio-hook "$pkgdir"/usr/lib/initcpio/hooks
  cd "$pkgname"
  make DESTDIR="$pkgdir/" install
}

clevis-mkinitcpio-install

#!/bin/bash

build() {
    add_binary "clevis-decrypt-http"
    add_binary "clevis-decrypt-tang"
    add_binary "clevis-decrypt-sss"
    add_binary "clevis-decrypt"
    add_binary "luksmeta"
    add_binary "clevis"
    add_binary "jose"
    add_binary "curl"
    add_binary "bash"
    add_runscript
}

help() {
    cat <<HELPEOF
This hook will attempt to unlock LUKS volumes using data stored in the header
by clevis and luksmeta. Use this hook in combination with any early userspace
networking hook, such as mkinitcpio-netconf or mkinitcpio-ppp. It also requires
mkinitcpio-utils for the encryptssh hook.

An example usage would be to have 'netconf clevis encryptssh' added before your
filesystems hook. You also need to configure clevis unlocking by using the
'clevis bind luks' command on your luks partition.
HELPEOF
}

clevis-mkinitcpio-hook

#!/usr/bin/bash

run_hook ()
{
    clevis_loop &
    echo $! > /.clevis.pid
}

clevis_loop()
{
    while ! [ -c /dev/mapper/control -a -f /.cryptdev -a -f /.cryptname -a -f /.cryptargs ];
do
        sleep 0.1
    done

    CRYPTDEV=$(cat /.cryptdev)
    CRYPTNAME=$(cat /.cryptname)
    UUID=cb6e8904-81ff-40da-a84a-07ab9ab5715e
    luksmeta show -d "$CRYPTDEV" | while read -r slot state uuid; do
        [ "$state" != "active" ] && continue
        [ "$uuid" != "$UUID" ] && continue

        if ( luksmeta load -d "$CRYPTDEV" -s $slot -u $UUID | clevis decrypt \
            | cryptsetup luksOpen $(cat /.cryptdev) $CRYPTNAME $(cat /.cryptargs) ); then
            echo > /.done
            echo -e "\n\n$CRYPTNAME sucessfully decrypted via clevis."
            killall cryptsetup
            break
        fi
    done
    rm /.clevis.pid
}

run_cleanuphook ()
{
    if [ -f /.clevis.pid ]; then
        kill $(/.clevis.pid)
        rm /.clevis.pid
    fi
}

namcap doesn't see any issues with this at all.

Which is a shame, because excluding tang (which wouldn't be nearly as bad if http-parser-git could be updated) this one is the hackiest one. The original repo hard-codes in udisks2 and dracut support with no options to turn them off, and the test for tang doesn't check to see if tang is installed as well as setting the paths explicitly to /usr/libexec for the tang binaries.

For this, I leave udisks2 as an option (Some people might actually want to use it!), but dracut support is hacked out with a few sed statements, and then if udisks2 isn't there I do the same for that. I debated using diff, but I felt having two patches for it was just as bad as the sed scripts. Besides, it's set to a specific tag so it's not like the content is going to change.

I also specifically hack out a luksmeta dependency check: it's not actually required to build, and the patch below it allows people to install or remove it without affecting anything but that specific functionality.

I think that's about everything I had questions about: I think I did a pretty good job of hunting down all dependencies needed, but honestly my biggest worry is all the sed hackery I had to use for this. I do hope that these are good enough to get submitted to the AUR though: It's not QUITE been a week since I first downloaded arch, and I think it'd be fun to have four packages submitted in that time.

eschwartz · 2017-10-15 05:16:00

Well, source=("$pkgname::...") is unnecessary when the repository is already "$pkgname.git#commit=..."

`autoreconf -fi` is really something that belongs in prepare()

jose should not provide itself, this was probably due to using a -git PKGBUILD as the base (though you pin a commit so it isn't a -git package).

There is no need to specify a minimum version for depends.

For your pkgver() functions you should either switch to https://github.com/latchset/clevis.git#tag=v$pkgver" and remove the function or simply pull $url/archive/v$pkgver.tar.gz
For the ones with pinned commits, pkgver() functions make it easy to update the pkgver when you bump the pinned commit, and many repo PKGBUILDs do the same so you can leave it in... but please use the pkgver() function in the Wiki: https://wiki.archlinux.org/index.php/VC … elines#Git
This will make jose be at 10.r1 instead of semver 10.1

As I said on IRC I would suggest using http-parser-git as a depends.

For clevis, this pacman -T stuff is the wrong way to do it, as you should be adding it as a depends (I assume it is a runtime depends if compiled with support). You could, however, add a variable _USE_UDISKS at the top, which if nonzero would add depends+=('udisks2'). I would nope upstream offered ./configure --enable/disable-udisks2 or something of that ilk though.

ShaRose · 2017-10-15 05:39:03

Eschwartz wrote:

Well, source=("$pkgname::...") is unnecessary when the repository is already "$pkgname.git#commit=..."

Yeah, I basically found one base and edited it to fit. I didn't even think to remove stuff like that.

Eschwartz wrote:

`autoreconf -fi` is really something that belongs in prepare()

Will do. I actually suspected that, but I had seen an example somewhere that had it all in build.

Eschwartz wrote:

jose should not provide itself, this was probably due to using a -git PKGBUILD as the base (though you pin a commit so it isn't a -git package).

That's exactly it: I decided since I was ending up doing a lot of sed work I should make sure it doesn't break one day.

Eschwartz wrote:

There is no need to specify a minimum version for depends.

Yeah, I probably should have removed that. There are other minimum version checks in the autotools build scripts as well, but that's not really an issue for arch anyways (as apposed to say ubuntu or debian...)

Eschwartz wrote:

For your pkgver() functions you should either switch to https://github.com/latchset/clevis.git#tag=v$pkgver" and remove the function or simply pull $url/archive/v$pkgver.tar.gz
For the ones with pinned commits, pkgver() functions make it easy to update the pkgver when you bump the pinned commit, and many repo PKGBUILDs do the same so you can leave it in... but please use the pkgver() function in the Wiki: https://wiki.archlinux.org/index.php/VC … elines#Git
This will make jose be at 10.r1 instead of semver 10.1

Yeah, I'd actually removed the r because of that issue with checking depends versions: It was causing me issues since I depended on versions of other packages I was making, and I did that as a test. When I decided to stop checking for versions, I forgot to roll back that change.

Eschwartz wrote:

As I said on IRC I would suggest using http-parser-git as a depends.

Flagged it as out of date. The maintainer seems quite active, so hopefully it'll be updated soon and I can remove some of the ugly hacks. I had originally set this up for a private package and only decided I should release it publicly yesterday, so I didn't think to try flagging it.

Eschwartz wrote:

For clevis, this pacman -T stuff is the wrong way to do it, as you should be adding it as a depends (I assume it is a runtime depends if compiled with support). You could, however, add a variable _USE_UDISKS at the top, which if nonzero would add depends+=('udisks2'). I would hope upstream offered ./configure --enable/disable-udisks2 or something of that ilk though.

Didn't even think about that for udisks2. As for the _USE_UDISKS variable, thinking about it the upstream packages for fedora and the like actually separate it out, so the clevis only parts are in the clevis package, then dracut scripts are in clevis-dracut, and udisks2 support in clevis-udisks2. I could probably do the same for mkinitcpio-clevis (keeping with the other mkinitcpio package naming conventions). Is there a way to do something like that for PKGBUILD, preferably without having to make three separate ones that all compile the same thing but only copy some specific files in?

eschwartz · 2017-10-15 05:49:38

Does http-parser-git not build right now? I haven't tested it...

As for the udisks2 etc. stuff, it really depends, if you Fedora can separate it into split packages then I would assume it is implemented as plugins in which case, hopefully, the user should be able to configure them to not be loaded or else it will gracefully fail to load those optional components. That's sort of a best-case scenario if implemented correctly, as the Arch package can add udisks2 etc. as makedepends for building, and then optdepends for the runtime requirement. If the user doesn't want to use that functionality, then they just don't install the optdepends (or uninstall after building with `makepkg -sr`) and the software would simply ignore it.

But yes, we do have split packages. See the "PACKAGE SPLITTING" section in the PKGBUILD(5) manpage. Generally Arch Linux does not split packages the way Debian etc. do, e.g. we don't split out -dev packages containing headers, and usually ship packages configured --with-kitchen-sink when possible. I suppose if clevis can only implement optional components by uninstalling the udisks2 etc. plugins, then split packages would be justified in this case, but then I'd probably file an upstream feature request to do this more intelligently via configurations.

Other instances where we split packages: linux vs. linux-headers, due to the large size of headers which are only needed for building modules e.g. via dkms, python packages which build python2 and python3 libraries from the same source tarball.

Last edited by eschwartz (2017-10-15 05:54:38)

Arch Linux

#1 2017-10-15 04:46:28