PKGBUILD review request: swarmkit

timefairy · 2016-11-20 12:01:55

Hey all,
I'm writting the PKGBUILD for https://github.com/docker/swarmkit
The PKGBUILD:

# Maintainer: Mark Huo <markhuomian at gmail dot com>
pkgname=swarmkit
pkgver=1.12.0
pkgrel=3
pkgdesc="A toolkit for orchestrating distributed systems at any scale. It includes primitives for node discovery, raft-based consensus, task scheduling and more."
arch=('x86_64' 'i686')
url="https://github.com/docker/swarmkit"
license=('Apache')
depends=('glibc')
makedepends=('go>=1.6' 'protobuf3-coex')
source=("${pkgname}-${pkgver}.tar.gz::${url}/archive/v${pkgver}.tar.gz")
sha256sums=('40b5b4649b663b0da66db3dadb4c1c058992878d9c074ad48b3fbb3bb5f02c6a')

package() {
  # prepare
  export GOPATH="$startdir"
  local _build_path="$srcdir/github.com/docker/"
  local _project_root="$srcdir/github.com/docker/swarmkit"
  mkdir -p "$_build_path"
  mv "$srcdir/$pkgname-$pkgver" "$_project_root"

  # build & install
  cd "$_project_root"
  make DESTDIR="$pkgdir/usr" install
}

# vim:set ts=2 sw=2 et:

Any suggestion?
-------
Update: https://bbs.archlinux.org/viewtopic.php … 4#p1671244
Jump to lastest version (skip the discussion): https://bbs.archlinux.org/viewtopic.php … 8#p1673538

Last edited by timefairy (2016-12-02 03:57:52)

a821 · 2016-11-20 13:17:30

I would recommend using separated prepare and build functions. https://wiki.archlinux.org/index.php/Cr … _functions. I don't see other obvious issues, so it looks OK to me.

Last edited by a821 (2016-11-20 13:27:49)

mis · 2016-11-20 13:19:57

I would shorten the pkgdesc, e.g. just 'A toolkit for orchestrating distributed systems at any scale'

atomicbeef · 2016-11-20 17:17:36

You can drop the glibc dependency since it's part of the base group.

timefairy · 2016-11-21 01:06:34

Update:

# Maintainer: Mark Huo <markhuomian at gmail dot com>
pkgname=swarmkit
pkgver=1.12.0
pkgrel=1
pkgdesc="A toolkit for orchestrating distributed systems at any scale."
arch=('x86_64' 'i686')
url="https://github.com/docker/swarmkit"
license=('Apache')
makedepends=('go>=1.6' 'protobuf3-coex')
source=("${pkgname}-${pkgver}.tar.gz::${url}/archive/v${pkgver}.tar.gz")
sha256sums=('40b5b4649b663b0da66db3dadb4c1c058992878d9c074ad48b3fbb3bb5f02c6a')

prepare() {
  export GOPATH="$startdir"
  export _build_path="$srcdir/github.com/docker"
  mkdir -p "$_build_path"
  mv "$srcdir/$pkgname-$pkgver" "$_build_path/swarmkit"
}

package() {
  cd "$_build_path/swarmkit"
  make DESTDIR="$pkgdir/usr" install
}

# vim:set ts=2 sw=2 et:

eschwartz · 2016-11-21 03:12:07

Moving half the rightful build function from the prepare function into the prepare function is not called "using separated prepare and build functions". You are still running `make binaries` via an implicit dependency from `make install` in the package function...

Also, you can probably just symlink the source directory instead of moving it.

Last edited by eschwartz (2016-11-21 03:19:33)

timefairy · 2016-11-22 02:09:04

Eschwartz wrote:

Moving half the rightful build function from the prepare function into the prepare function is not called "using separated prepare and build functions". You are still running `make binaries` via an implicit dependency from `make install` in the package function...
Also, you can probably just symlink the source directory instead of moving it.

Thanks, the PKGBUILD:

# Maintainer: Mark Huo <markhuomian at gmail dot com>
pkgname=swarmkit
pkgver=1.12.0
pkgrel=1
pkgdesc="A toolkit for orchestrating distributed systems at any scale."
arch=('x86_64' 'i686')
url="https://github.com/docker/swarmkit"
license=('Apache')
makedepends=('go>=1.6' 'protobuf>=3')
source=("${pkgname}-${pkgver}.tar.gz::${url}/archive/v${pkgver}.tar.gz")
sha256sums=('40b5b4649b663b0da66db3dadb4c1c058992878d9c074ad48b3fbb3bb5f02c6a')

prepare() {
  export GOPATH="$srcdir"
  export _build_path="$GOPATH/src/github.com/docker"
  mkdir -p "$_build_path"
  ln -s "$srcdir/$pkgname-$pkgver" "$_build_path/swarmkit"
}

build() {
  cd "$_build_path/swarmkit"
  make binaries
}

package() {
  cd "$_build_path/swarmkit"
  make DESTDIR="$pkgdir/usr" install
}

# vim:set ts=2 sw=2 et:

timefairy · 2016-11-22 04:49:11

# Maintainer: Mark Huo <markhuomian at gmail dot com>
pkgname=swarmkit
pkgver=1.12.0
pkgrel=2
pkgdesc="A toolkit for orchestrating distributed systems at any scale."
arch=('x86_64' 'i686')
url="https://github.com/docker/swarmkit"
license=('Apache')
makedepends=('go>=1.6' 'protobuf>=3')
BUILDENV+=('!check')
source=("${pkgname}-${pkgver}.tar.gz::${url}/archive/v${pkgver}.tar.gz")
sha256sums=('40b5b4649b663b0da66db3dadb4c1c058992878d9c074ad48b3fbb3bb5f02c6a')

prepare() {
  export GOPATH="$srcdir"
  export PATH=$PATH:$GOPATH/bin
  export _build_path="$GOPATH/src/github.com/docker"

  mkdir -p "$_build_path"
  ln -s "$srcdir/$pkgname-$pkgver" "$_build_path/swarmkit"

  cd "$_build_path/swarmkit"
  make setup
}

check() {
  cd "$_build_path/swarmkit"
  make all
}

build() {
  cd "$_build_path/swarmkit"

  ## Uncomment to regenerating protobuf bindings
  # make generate
  make binaries
}

package() {
  cd "$_build_path/swarmkit"
  make DESTDIR="$pkgdir/usr" install
}

# vim:set ts=2 sw=2 et:

1. The check() function will fail at stage lint, so I add BUILDENV+=('!check').

? lint
agent/exec/controller_test.go:347:1: context.Context should be the first parameter of a function

2. About the "regenerating protobuf bindings", with/without `make generate` will output different binaries. I think that is because of the difference between the generated protobuf files. May I leave it like that? Any suggestions? Thanks!

without make generate

57357e61221a7321d4e1c35be35ef39f3fd91aac  pkg/swarmkit/usr/bin/protoc-gen-gogoswarm
892c8e9904b6f4fa7181d4598b68cf3b8ccd90fd  pkg/swarmkit/usr/bin/swarm-bench
278bc23c905a29a4fef86a11952b987f6d004edb  pkg/swarmkit/usr/bin/swarmctl
86d9ddc8b2512834b0ee97257191a87e73666e25  pkg/swarmkit/usr/bin/swarmd

with make generate

b61bfb5ea42175e21dd35114c0db2783f479f87e  pkg/swarmkit/usr/bin/protoc-gen-gogoswarm
88249e600bfce5004bd152330ac0f8e96b64600c  pkg/swarmkit/usr/bin/swarm-bench
2997a5380ca1736851219fea283fb62b22426215  pkg/swarmkit/usr/bin/swarmctl
7422e5107675d7a1cffac02ea8680d837bc8c0cd  pkg/swarmkit/usr/bin/swarmd

Last edited by timefairy (2016-11-22 04:50:50)

a821 · 2016-11-22 10:36:16

Respect to your PKGBUILD, I think you've got it wrong. And I don't think what Eschwartz said was meant to be taken literally (make binaries). When you run "make install" the binaries are built if they haven't already been (implicit dependency), so you need to run "make" (to build the binaries) and then "make install" (to install them)
Roughly, in prepare you patch and make the symlinks; in build "./configure and make"; in check "make check" or equivalent if available; and package "make install". The wiki has more details.

Edit: wrong statement

Last edited by a821 (2016-11-23 09:15:55)

eschwartz · 2016-11-22 19:14:00

Actually, I did mean that literally. Because I looked at the Makefile for the upstream source and it seemed like that was what `make install` depended on.
And checking again, the "all" target actually runs the "binaries" target... and does the tests as well (which is bad style).

Nevertheless, it was only an example, albeit the best example I could give.
The build() step is responsible for "build"ing the software, and package() is responsible for copying the resulting files into "$pkgdir" and compressing them into a pkg.tar.xz -- this has ramifications if, say, using `makepkg --repackage`.

If the check() stage shouldn't be run, don't disable it via backdoor modifications of BUILDENV. Just comment out that function, to indicate it doesn't really work but preserve it for future reference and your own tinkering attempts.

If the "generate" target performs pre-compile actions that upstream considers to be important for any end-user deliverable software installation, then, really, they should make sure their Makefile says so.
If you think it is better for users to use binaries with regenerated protobuf bindings, then run that as well. It is your call.

a821 · 2016-11-23 09:17:24

Eschwartz wrote:

Actually, I did mean that literally. Because I looked at the Makefile for the upstream source and it seemed like that was what `make install` depended on. .

Thanks, I should've checked the sources, though first time I heard of a binaries target in a Makefile... learnt something today :-)

atomicbeef · 2016-11-23 15:06:46

Makefiles can have arbitrary target names. Most people stick to standard names though, and that's why it's rare to see things like a "binaries" target.

timefairy · 2016-11-24 05:33:04

Thanks, updated the PKGBUILD:

# Maintainer: Mark Huo <markhuomian at gmail dot com>
pkgname=swarmkit
pkgver=1.12.0
pkgrel=2
pkgdesc="A toolkit for orchestrating distributed systems at any scale."
arch=('x86_64' 'i686')
url="https://github.com/docker/swarmkit"
license=('Apache')
makedepends=('go>=1.6' 'protobuf>=3')
source=("${pkgname}-${pkgver}.tar.gz::${url}/archive/v${pkgver}.tar.gz")
sha256sums=('40b5b4649b663b0da66db3dadb4c1c058992878d9c074ad48b3fbb3bb5f02c6a')

prepare() {
  export GOPATH="$srcdir"
  export PATH=$PATH:$GOPATH/bin
  export _build_path="$GOPATH/src/github.com/docker"

  mkdir -p "$_build_path"
  ln -s "$srcdir/$pkgname-$pkgver" "$_build_path/swarmkit"
}

build() {
  cd "$_build_path/swarmkit"

  make setup generate binaries
}

# check() {
#   cd "$_build_path/swarmkit"
#   make test integration
# }

package() {
  export GOPATH="$srcdir"
  export PATH=$PATH:$GOPATH/bin
  export _build_path="$GOPATH/src/github.com/docker"

  cd "$_build_path/swarmkit"
  make DESTDIR="$pkgdir/usr" install
}

# vim:set ts=2 sw=2 et:

Lone_Wolf · 2016-11-24 12:35:16

Why do both prepare() and package() export the same things ?

timefairy · 2016-11-25 00:07:31

Lone_Wolf wrote:

Why do both prepare() and package() export the same things ?

The `export` is prepared for command `makepkg --repackage`.

Lone_Wolf · 2016-11-25 09:53:15

timefairy wrote:

Lone_Wolf wrote:
Why do both prepare() and package() export the same things ?
The `export` is prepared for command `makepkg --repackage`.

Nice to see you thought about that, but it does raise another question.

Should those 3 assignments even be in prepare() function ?

putting them in the body of the PKGBUILD would make more sense i think.

GOPATH="$srcdir"
PATH=$PATH:$GOPATH/bin
_build_path="$GOPATH/src/github.com/docker"

timefairy · 2016-11-27 01:02:19

Lone_Wolf wrote:

timefairy wrote:
Lone_Wolf wrote:
Why do both prepare() and package() export the same things ?
The `export` is prepared for command `makepkg --repackage`.
Nice to see you thought about that, but it does raise another question.
Should those 3 assignments even be in prepare() function ?
putting them in the body of the PKGBUILD would make more sense i think.
GOPATH="$srcdir"
PATH=$PATH:$GOPATH/bin
_build_path="$GOPATH/src/github.com/docker"

Hi, I've tried something like

pkgname=swarmkit
pkgver=1.12.0
pkgrel=2
pkgdesc="A toolkit for orchestrating distributed systems at any scale."
arch=('x86_64' 'i686')
url="https://github.com/docker/swarmkit"
license=('Apache')
makedepends=('go>=1.6' 'protobuf>=3')
source=("${pkgname}-${pkgver}.tar.gz::${url}/archive/v${pkgver}.tar.gz")
sha256sums=('40b5b4649b663b0da66db3dadb4c1c058992878d9c074ad48b3fbb3bb5f02c6a')
export GOPATH="$srcdir"
export PATH=$PATH:$GOPATH/bin
export _build_path="$GOPATH/src/github.com/docker"

and

pkgname=swarmkit
pkgver=1.12.0
pkgrel=2
pkgdesc="A toolkit for orchestrating distributed systems at any scale."
arch=('x86_64' 'i686')
url="https://github.com/docker/swarmkit"
license=('Apache')
makedepends=('go>=1.6' 'protobuf>=3')
source=("${pkgname}-${pkgver}.tar.gz::${url}/archive/v${pkgver}.tar.gz")
sha256sums=('40b5b4649b663b0da66db3dadb4c1c058992878d9c074ad48b3fbb3bb5f02c6a')
GOPATH="$srcdir"
PATH=$PATH:$GOPATH/bin
_build_path="$GOPATH/src/github.com/docker"

It seems that the "$srcdir" is missing outside of `prepare`, `build`, `check`, `package` functions, and the echo "$GOPATH" in those functions gives nothing.

Lone_Wolf · 2016-11-27 18:17:57

That does make sense as $srcdir is set/created by makepkg at some point.

I suggest you put them back in prepare() & package() functions, with a short comment why there are present.

timefairy · 2016-11-28 04:46:04

# Maintainer: Mark Huo <markhuomian at gmail dot com>
pkgname=swarmkit
pkgver=1.12.0
pkgrel=2
pkgdesc="A toolkit for orchestrating distributed systems at any scale."
arch=('x86_64' 'i686')
url="https://github.com/docker/swarmkit"
license=('Apache')
makedepends=('go>=1.6' 'protobuf>=3')
source=("${pkgname}-${pkgver}.tar.gz::${url}/archive/v${pkgver}.tar.gz")
sha256sums=('40b5b4649b663b0da66db3dadb4c1c058992878d9c074ad48b3fbb3bb5f02c6a')

prepare() {
  export GOPATH="$srcdir"
  export PATH=$PATH:$GOPATH/bin
  export _build_path="$GOPATH/src/github.com/docker"

  mkdir -p "$_build_path"
  ln -s "$srcdir/$pkgname-$pkgver" "$_build_path/swarmkit"
}

build() {
  cd "$_build_path/swarmkit"

  make setup generate binaries
}

# check() {
#   cd "$_build_path/swarmkit"
#   make test integration
# }

package() {
  ## Uncomment following `export`s for `makepkg --repackage`
  # export GOPATH="$srcdir"
  # export PATH=$PATH:$GOPATH/bin
  # export _build_path="$GOPATH/src/github.com/docker"

  cd "$_build_path/swarmkit"
  make DESTDIR="$pkgdir/usr" install
}

# vim:set ts=2 sw=2 et:

atomicbeef · 2016-11-29 21:43:55

I think what Lone_Wolf meant was to leave the actual exports uncommented and place a comment explaining why the exports are on a line above them. There's no point in commenting them out, so you should just leave them uncommented.

timefairy · 2016-12-01 00:13:42

Thanks. The PKGBUILD:
1. Add comments for `export`s.
2. Use `ln -si` in prepare() to fix the "soft link already exists" error when you `makepkg` multiple times. (btw, the `makepkg --cleanbuild` can avoid the error too, but I think the -i option is better.)

# Maintainer: Mark Huo <markhuomian at gmail dot com>
pkgname=swarmkit
pkgver=1.12.0
pkgrel=2
pkgdesc="A toolkit for orchestrating distributed systems at any scale."
arch=('x86_64' 'i686')
url="https://github.com/docker/swarmkit"
license=('Apache')
makedepends=('go>=1.6' 'protobuf>=3')
source=("${pkgname}-${pkgver}.tar.gz::${url}/archive/v${pkgver}.tar.gz")
sha256sums=('40b5b4649b663b0da66db3dadb4c1c058992878d9c074ad48b3fbb3bb5f02c6a')

prepare() {
  ## Set a working golang environment
  ## see https://golang.org/doc/code.html
  export GOPATH="$srcdir"
  export PATH=$PATH:$GOPATH/bin
  export _build_path="$GOPATH/src/github.com/docker"

  mkdir -p "$_build_path"
  ln -si "$srcdir/$pkgname-$pkgver" "$_build_path/swarmkit"
}

build() {
  cd "$_build_path/swarmkit"

  make setup generate binaries
}

# check() {
#   cd "$_build_path/swarmkit"
#   make test integration
# }

package() {
  ## Reserved for `makepkg --repackage`
  export GOPATH="$srcdir"
  export PATH=$PATH:$GOPATH/bin
  export _build_path="$GOPATH/src/github.com/docker"

  cd "$_build_path/swarmkit"
  make DESTDIR="$pkgdir/usr" install
}

# vim:set ts=2 sw=2 et:

Last edited by timefairy (2016-12-01 00:30:34)

JohnBobSmith · 2016-12-01 01:32:46

# check() {
#   cd "$_build_path/swarmkit"
#   make test integration
# }

If the above is going to be commented out, would it not be better to simply remove it? What is it's purpose? Your addition of comments describing why the previously mentioned lines of code in package() and prepare() are required is great! Perhaps adding a similar comment for this function would work too? Especially if the user must intervene/use this code block at some point.

Last edited by JohnBobSmith (2016-12-01 01:35:49)

eschwartz · 2016-12-01 05:53:55

Eschwartz wrote:

If the check() stage shouldn't be run, don't disable it via backdoor modifications of BUILDENV. Just comment out that function, to indicate it doesn't really work but preserve it for future reference and your own tinkering attempts.

But yeah, a comment saying "currently errors during lint" or something would be good.

JohnBobSmith · 2016-12-01 21:13:04

Doh! I didn't read far enough back, thanks Eschwartz. XD

timefairy · 2016-12-02 03:55:58

Thanks.

# Maintainer: Mark Huo <markhuomian at gmail dot com>
pkgname=swarmkit
pkgver=1.12.0
pkgrel=2
pkgdesc="A toolkit for orchestrating distributed systems at any scale."
arch=('x86_64' 'i686')
url="https://github.com/docker/swarmkit"
license=('Apache')
makedepends=('go>=1.6' 'protobuf>=3')
source=("${pkgname}-${pkgver}.tar.gz::${url}/archive/v${pkgver}.tar.gz")
sha256sums=('40b5b4649b663b0da66db3dadb4c1c058992878d9c074ad48b3fbb3bb5f02c6a')

prepare() {
  ## Set a working golang environment
  ## see https://golang.org/doc/code.html
  export GOPATH="$srcdir"
  export PATH=$PATH:$GOPATH/bin
  export _build_path="$GOPATH/src/github.com/docker"

  mkdir -p "$_build_path"
  ln -si "$srcdir/$pkgname-$pkgver" "$_build_path/swarmkit"
}

build() {
  cd "$_build_path/swarmkit"

  ## After regenerating protobuf bindings, the `test` failed,
  ## reserved for future investigation
  # make setup generate
  make binaries
}

check() {
  cd "$_build_path/swarmkit"
  make test
  ## Reserved for lastest git version
  # make test integration
}

package() {
  ## Reserved for `makepkg --repackage`
  export GOPATH="$srcdir"
  export PATH=$PATH:$GOPATH/bin
  export _build_path="$GOPATH/src/github.com/docker"

  cd "$_build_path/swarmkit"
  make DESTDIR="$pkgdir/usr" install
}

# vim:set ts=2 sw=2 et:

Last edited by timefairy (2016-12-02 03:58:49)

Arch Linux

#1 2016-11-20 12:01:55

PKGBUILD review request: swarmkit

#2 2016-11-20 13:17:30

Re: PKGBUILD review request: swarmkit

#3 2016-11-20 13:19:57

Re: PKGBUILD review request: swarmkit

#4 2016-11-20 17:17:36

Re: PKGBUILD review request: swarmkit

#5 2016-11-21 01:06:34

Re: PKGBUILD review request: swarmkit

#6 2016-11-21 03:12:07

Re: PKGBUILD review request: swarmkit

#7 2016-11-22 02:09:04

Re: PKGBUILD review request: swarmkit

#8 2016-11-22 04:49:11

Re: PKGBUILD review request: swarmkit

#9 2016-11-22 10:36:16

Re: PKGBUILD review request: swarmkit

#10 2016-11-22 19:14:00

Re: PKGBUILD review request: swarmkit

#11 2016-11-23 09:17:24

Re: PKGBUILD review request: swarmkit

#12 2016-11-23 15:06:46

Re: PKGBUILD review request: swarmkit

#13 2016-11-24 05:33:04

Re: PKGBUILD review request: swarmkit

#14 2016-11-24 12:35:16

Re: PKGBUILD review request: swarmkit

#15 2016-11-25 00:07:31

Re: PKGBUILD review request: swarmkit

#16 2016-11-25 09:53:15

Re: PKGBUILD review request: swarmkit

#17 2016-11-27 01:02:19

Re: PKGBUILD review request: swarmkit

#18 2016-11-27 18:17:57

Re: PKGBUILD review request: swarmkit

#19 2016-11-28 04:46:04

Re: PKGBUILD review request: swarmkit

#20 2016-11-29 21:43:55

Re: PKGBUILD review request: swarmkit

#21 2016-12-01 00:13:42

Re: PKGBUILD review request: swarmkit

#22 2016-12-01 01:32:46

Re: PKGBUILD review request: swarmkit

#23 2016-12-01 05:53:55

Re: PKGBUILD review request: swarmkit

#24 2016-12-01 21:13:04

Re: PKGBUILD review request: swarmkit

#25 2016-12-02 03:55:58

Re: PKGBUILD review request: swarmkit

Board footer