You are not logged in.

#1 2018-09-30 03:34:34

mrfaber
Member
Registered: 2016-08-09
Posts: 18

Selectively installing split packages

Let me start by illustrating my problem:

I would like to build acpi_call against linux-ck, specifically linux-ck-skylake. Right now, I am using this PKGBUILD(shortened):

pkgbase=acpi_call-ck
pkgname=(acpi_call-ck-skylake)

pkgver=1.1.0
pkgrel=1
pkgdesc='A linux kernel module that enables calls to ACPI methods through /proc/acpi/call'
depends=('linux-ck>=4.18' 'linux-ck<4.19')
makedepends=('linux-ck-headers>=4.18' 'linux-ck-headers<4.19')
_extramodules=extramodules-${pkgname#acpi_call-}
source=("${pkgname}-${pkgver}.tar.gz::https://github.com/mkottman/acpi_call/archive/v${pkgver}.tar.gz")
sha256sums=('d0d14b42944282724fca76f57d598eed794ef97448f387d1c489d85ad813f2f0')

prepare() {
  cd ${pkgname%%-*}-${pkgver}

  # Fix build with Linux >= 3.17
  sed -i 's|acpi/acpi.h|linux/acpi.h|' acpi_call.c

  # Fix build with Linux >= 4.12
  sed -i 's|asm/uaccess.h|linux/uaccess.h|' acpi_call.c
}

build() {
  cd ${pkgname%%-*}-${pkgver}
  make KVERSION="$(cat /usr/lib/modules/$_extramodules/version)"
}

package-generic() {
  cd ${pkgname%%-*}-${pkgver}
  install -Dt "$pkgdir/usr/lib/modules/$_extramodules" -m644 *.ko
  find "$pkgdir" -name '*.ko' -exec xz {} +

  echo ${pkgname%%-*} | install -Dm644 /dev/stdin "$pkgdir/usr/lib/modules-load.d/${pkgname}.conf"

  mkdir -p "$pkgdir/usr/share/${pkgname}"
  cp -t "$pkgdir/usr/share/${pkgname}" -dr --no-preserve=ownership examples support
}

# This would fail because /usr/lib/modules/extramodules-ck/ is not present
#package_acpi_call-ck() {
#  package-generic
#}

package_acpi_call-ck-skylake() {
  package-generic
}

Now this could be made much more versatile if one could simply add the other linux-ck versions to pkgname and call something like

package_acpi_call-ck-skylake() {
  package-generic
}
package_acpi_call-ck-haswell() {
  package-generic
}
...

As I understand right now, split packages (i.e. those with a PKGBUILD containing both pkgbase and multiple pkgnames) will be built in their entirety by makepkg.

This makes in unfeasible to provide a package base PKGBUILD, since makepkg would attempt to create a package for all targets and fail because the specific linux-ck-* packages are missing.

So I've patched my makepkg to give it an -O or --only flag that accepts a single pkgname:

diff --git a/usr/bin/makepkg b/makepkg-split
index 359ee0e..f4cf145 100755
--- a/usr/bin/makepkg
+++ b/makepkg-split
@@ -85,6 +85,7 @@ SKIPCHECKSUMS=0
 SKIPPGPCHECK=0
 SIGNPKG=''
 SPLITPKG=0
+SPLITONLYPKG=0
 SOURCEONLY=0
 VERIFYSOURCE=0
 
@@ -1179,6 +1180,7 @@ usage() {
 	printf -- "$(gettext "  -p <file>        Use an alternate build script (instead of '%s')")\n" "$BUILDSCRIPT"
 	printf -- "$(gettext "  -r, --rmdeps     Remove installed dependencies after a successful build")\n"
 	printf -- "$(gettext "  -R, --repackage  Repackage contents of the package without rebuilding")\n"
+	printf -- "$(gettext "  -O, --only <pkg> Build only the specified pkgname of a group of split packages")\n"
 	printf -- "$(gettext "  -s, --syncdeps   Install missing dependencies with %s")\n" "pacman"
 	printf -- "$(gettext "  -S, --source     Generate a source-only tarball without downloaded sources")\n"
 	printf -- "$(gettext "  -V, --version    Show version information and exit")\n"
@@ -1234,12 +1236,12 @@ fi
 ARGLIST=("$@")
 
 # Parse Command Line Options.
-OPT_SHORT="AcCdefFghiLmop:rRsSV"
+OPT_SHORT="AcCdefFghiLmoO:p:rRsSV"
 OPT_LONG=('allsource' 'check' 'clean' 'cleanbuild' 'config:' 'force' 'geninteg'
           'help' 'holdver' 'ignorearch' 'install' 'key:' 'log' 'noarchive' 'nobuild'
           'nocolor' 'nocheck' 'nodeps' 'noextract' 'noprepare' 'nosign' 'packagelist'
           'printsrcinfo' 'repackage' 'rmdeps' 'sign' 'skipchecksums' 'skipinteg'
-          'skippgpcheck' 'source' 'syncdeps' 'verifysource' 'version')
+          'skippgpcheck' 'source' 'only:' 'syncdeps' 'verifysource' 'version')
 
 # Pacman Options
 OPT_LONG+=('asdeps' 'noconfirm' 'needed' 'noprogressbar')
@@ -1281,6 +1283,7 @@ while true; do
 		--noprepare)      RUN_PREPARE='n' ;;
 		--nosign)         SIGNPKG='n' ;;
 		-o|--nobuild)     NOBUILD=1 ;;
+		-O|--only)        shift; SPLITONLYPKG=$1 ;;
 		-p)               shift; BUILDFILE=$1 ;;
 		--packagelist)    PACKAGELIST=1 IGNOREARCH=1;;
 		--printsrcinfo)   PRINTSRCINFO=1 IGNOREARCH=1;;
@@ -1321,7 +1324,7 @@ trap 'trap_exit INT "$(gettext "Aborted by user! Exiting...")"' INT
 trap 'trap_exit USR1 "$(gettext "An unknown error has occurred. Exiting...")"' ERR
 
 # preserve environment variables to override makepkg.conf
-restore_envvars=$(declare -p PKGDEST SRCDEST SRCPKGDEST LOGDEST BUILDDIR PKGEXT SRCEXT GPGKEY PACKAGER CARCH 2>/dev/null || true)
+restore_envvars=$(declare -p PKGDEST SRCDEST SRCPKGDEST LOGDEST BUILDDIR PKGEXT SPLITONLYPKG SRCEXT GPGKEY PACKAGER CARCH 2>/dev/null || true)
 
 # default config is makepkg.conf
 MAKEPKG_CONF=${MAKEPKG_CONF:-$confdir/makepkg.conf}
@@ -1494,6 +1497,11 @@ check_software || exit $E_MISSING_MAKEPKG_DEPS
 if (( ${#pkgname[@]} > 1 )); then
 	SPLITPKG=1
 fi
+# only operate on the specified pkgname if instructed to only build one single
+# package of a split package group
+if (( SPLITONLYPKG )); then
+    pkgname=("${SPLITONLYPKG}")
+fi
 
 # test for available PKGBUILD functions
 if have_function prepare; then

Obviously this is something targeted mainly towards the AUR and would make installing something like notmuch much nicer, and AUR helpers could give this flag to makepkg if they detect a split package(I think yay knows how to do this and some others as well).

But I think this should not only be included in makepkg because it eases AUR helper usage(which I know are very much unsupported) but also helps people using their own PKGBUILDs accross machines.

So, tell me, am I onto something here or should I just write and maintain 20 different PKGBUILDS for the linux-ck optimized packages? I know it is already a pain to parse PKGBUILDs and forcing this much polymorphism into the packaging system feels wrong.

Last edited by mrfaber (2018-09-30 03:40:52)

Offline

#2 2018-09-30 04:15:05

eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 2,543

Re: Selectively installing split packages

We removed this feature from makepkg.

...

I'm not really positive why we need any AUR packages for it at all, since the repos have a dkms version that automatically adapts to *all* your installed kernels, and it's not like providing a PKGBUILD to compile it and package the result, is any better than providing a dkms routine to compile it and copy it directly to your modules directory.

compiled module packages are prinicipally useful for creating prebuilt binary repositories.

...

Also see my broadcom-wl PKGBUILD in [community], for what I regard as a very good way to generically make a binary package.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#3 2018-09-30 04:49:26

mrfaber
Member
Registered: 2016-08-09
Posts: 18

Re: Selectively installing split packages

Eschwartz wrote:

We removed this feature from makepkg.

Interesting, any particular reason?

Eschwartz wrote:

I'm not really positive why we need any AUR packages for it at all, since the repos have a dkms version that automatically adapts to *all* your installed kernels, and it's not like providing a PKGBUILD to compile it and package the result, is any better than providing a dkms routine to compile it and copy it directly to your modules directory.

My main concern is being able to bake dkms modules into the initrd, is this feasible? I also remember really bad experiences with vbox dkms stuff.

Eschwartz wrote:

compiled module packages are prinicipally useful for creating prebuilt binary repositories.

Yup.

Eschwartz wrote:

Also see my broadcom-wl PKGBUILD in [community], for what I regard as a very good way to generically make a binary package.

I see, you mean this PKGBUILD. Anyone wishing to compile a custom version simply appends to pkgname. But with this one would still have to write 20 files if one was to provide a binary repo for linux-ck extra packages. Maybe I should ask graysky how he does it.

It's not that I am just too lazy to edit PKGBUILDs by hand, where this proposal shines is with large "package bases" like notmuch where makepkg -s would pull in dependencies for and build *all* sub-packages.

Or, put another way, this is to make makepkg more potent, by allowing to write and having to maintain less files, through the power of inheritance. One PKGBUILD stays valid across many sub-packages.

Last edited by mrfaber (2018-09-30 04:57:27)

Offline

#4 2018-09-30 05:01:56

eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 2,543

Re: Selectively installing split packages

makepkg doesn't really do inheritance, there is no way to have per-split-package makedepends, and you really need to build everything in build() once. The only thing we support is packaging them separately. And we no longer support only running some of the package_*() functions.

I believe graysky does this the sane way, by creating a locally modified PKGBUILD. It would be simple, to use my PKGBUILD as an example, to merely sed the PKGBUILD to append the custom pkgname you want, run makepkg to compile the module, sed it again, rebuild with a different kernel, etc.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#5 2018-09-30 05:13:25

mrfaber
Member
Registered: 2016-08-09
Posts: 18

Re: Selectively installing split packages

Hmm, that does make sense.
I also just noticed that I would have to move the make KVERSION=...* step to package_*() and that is just plain ugly.

So there goes my vision of putting acpi_call-ck* into the AUR with little pain sad

Offline

#6 2018-09-30 05:38:21

mrfaber
Member
Registered: 2016-08-09
Posts: 18

Re: Selectively installing split packages

I've just published the build to the AUR, see acpi_call-ck-generic.

The PKGBUILD is:

# Maintainer: Adam Hirst <adam@aphirst.karoo.co.uk>
# Maintainer: Zuyi Hu <hzy0668808@gmail.com>
# Maintainer: mrfaber <mrfaber@example.com>
# Contributor: Maxime Gauduin <alucryd@archlinux.org>
# Contributor: mortzu <me@mortzu.de>
# Contributor: fnord0 <fnord0@riseup.net>


pkgbase=acpi_call-ck
pkgname=(acpi_call-ck-generic)
# replace -generic with your appropriate optimized ck version
         # acpi_call-ck-atom
         # acpi_call-ck-silvermont
         # acpi_call-ck-core2
         # acpi_call-ck-nehalem
         # acpi_call-ck-sandybridge
         # acpi_call-ck-ivybridge
         # acpi_call-ck-haswell
         # acpi_call-ck-broadwell
         # acpi_call-ck-skylake
         # acpi_call-ck-p4
         # acpi_call-ck-pentm
         # acpi_call-ck-kx
         # acpi_call-ck-k10
         # acpi_call-ck-bobcat
         # acpi_call-ck-bulldozer
         # acpi_call-ck-piledriver
         # acpi_call-ck-zen
pkgver=1.1.0
pkgrel=10
pkgdesc='A linux-ck kernel module that enables calls to ACPI methods through /proc/acpi/call'
url='https://github.com/mkottman/acpi_call'
arch=('x86_64')
license=('GPL')
depends=('linux-ck>=4.18' 'linux-ck<4.19')
makedepends=('linux-ck-headers>=4.18' 'linux-ck-headers<4.19')
provides=('acpi_call')
_extramodules=extramodules-${pkgname#acpi_call-}
source=("${pkgname}-${pkgver}.tar.gz::https://github.com/mkottman/acpi_call/archive/v${pkgver}.tar.gz")
sha256sums=('d0d14b42944282724fca76f57d598eed794ef97448f387d1c489d85ad813f2f0')

prepare() {
  cd ${pkgname%%-*}-${pkgver}

  # Fix build with Linux >= 3.17
  sed -i 's|acpi/acpi.h|linux/acpi.h|' acpi_call.c

  # Fix build with Linux >= 4.12
  sed -i 's|asm/uaccess.h|linux/uaccess.h|' acpi_call.c
}

build() {
  cd ${pkgname%%-*}-${pkgver}
  make KVERSION="$(cat /usr/lib/modules/$_extramodules/version)"
}

package() {
  cd ${pkgname%%-*}-${pkgver}
  install -Dt "$pkgdir/usr/lib/modules/$_extramodules" -m644 *.ko
  find "$pkgdir" -name '*.ko' -exec xz {} +

  echo ${pkgname%%-*} | install -Dm644 /dev/stdin "$pkgdir/usr/lib/modules-load.d/${pkgname}.conf"

  mkdir -p "$pkgdir/usr/share/${pkgname}"
  cp -t "$pkgdir/usr/share/${pkgname}" -dr --no-preserve=ownership examples support
}


# vim:set ts=2 sw=2 et:

This is my first push to the AUR, let me know if there's anything wrong with it.

Offline

#7 2018-09-30 05:46:35

eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 2,543

Re: Selectively installing split packages

Well, I don't much see the point of the -generic name. Also graysky's optimized packages use native optimizations selected by the kernel gcc patch and encoded in the /usr/lib/modules/*/build/.config for each variant -- in order to actually build them optimized, the installed linux-ck{,-headers} needs to specifically be atom, silvermont, core2, or whatever.

You can do that, by autoselecting it from the end of pkgname.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#8 2018-09-30 21:03:03

mrfaber
Member
Registered: 2016-08-09
Posts: 18

Re: Selectively installing split packages

Ok, I've changed it to default to acpi_call-ck instead of -generic.

Eschwartz wrote:

You can do that, by autoselecting it from the end of pkgname.

It did not occur to me that you could use expressions in the depends and makedepends arrays.

Eschwartz wrote:

lso graysky's optimized packages use native optimizations selected by the kernel gcc patch and encoded in the /usr/lib/modules/*/build/.config for each variant

Do you mean I should source the .config file in the PKGBUILD? Or is changing the depends to the specific -ck variant enough?
Right now, the build() function calls

 make KVERSION="$(cat /usr/lib/modules/$_extramodules/version)"

where $_extramodules is the specific -ck variant.

Offline

#9 2018-10-03 13:32:32

eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 2,543

Re: Selectively installing split packages

mrfaber wrote:

It did not occur to me that you could use expressions in the depends and makedepends arrays.

You can use arbitrary bash. wink

Although if you try to do it inside the package() function itself, since makepkg does not execute it but simply greps it in order to do preliminary linting checks or generate .SRCINFO files, there are more restrictions...

Eschwartz wrote:

lso graysky's optimized packages use native optimizations selected by the kernel gcc patch and encoded in the /usr/lib/modules/*/build/.config for each variant

Do you mean I should source the .config file in the PKGBUILD? Or is changing the depends to the specific -ck variant enough?
Right now, the build() function calls

 make KVERSION="$(cat /usr/lib/modules/$_extramodules/version)"

where $_extramodules is the specific -ck variant.

No, I mean the kernel build system, which is used by out-of-tree modules, should already ensure the module is built with the same CFLAGS that the kernel itself was built with. It does this by saving the original .config as part of the /usr/lib/modules/${uname_r}/build/ directory (Kconfig, .config, include/ and more).

So, changing the depends to the specific -ck variant should be enough.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#10 2018-10-03 20:29:47

mrfaber
Member
Registered: 2016-08-09
Posts: 18

Re: Selectively installing split packages

Ok, thank you!

Offline

Board footer

Powered by FluxBB