[SOLVED] Git submodules from git release

nierro · 2018-06-14 19:22:41

Hi everyone!
I need some advice regarding a PKGBUILD.
I'm trying to correctly build a package from a git release; it needs 2 submodules to be inited though.
The normal way, ie: downloading the .tar.gz release archive and in prepare() function calling "git submodule update --init" (https://aur.archlinux.org/cgit/aur.git/ … 6e12f89502) gives

fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

I then tried to fix the issue by downloading the full git repo, move to needed tag and initing the submodules (https://aur.archlinux.org/cgit/aur.git/ … =libulfius). Again, submodules are not built and consequently not installed.
What am I missing?

Thank you very much!

Last edited by nierro (2018-06-16 07:42:49)

loqs · 2018-06-14 19:35:45

pacman -Qlp libulfius-2.3.6-3-x86_64.pkg.tar.xz 
libulfius /usr/
libulfius /usr/include/
libulfius /usr/include/orcania.h
libulfius /usr/include/ulfius.h
libulfius /usr/include/yder.h
libulfius /usr/lib/
libulfius /usr/lib/liborcania.so
libulfius /usr/lib/liborcania.so.1.2
libulfius /usr/lib/liborcania.so.1.2.3
libulfius /usr/lib/libulfius.so
libulfius /usr/lib/libulfius.so.2.3
libulfius /usr/lib/libulfius.so.2.3.6
libulfius /usr/lib/libyder.so
libulfius /usr/lib/libyder.so.1.3
libulfius /usr/lib/libyder.so.1.3.3
...

I would recomend you follow VCS_package_guidelines#Git_Submodules but with your current PKGBUILD the submodules are built and packaged.

nierro · 2018-06-14 19:47:21

Thanks for the feedback; but for me (through 'yay') it is not working...
I tried to follow that wiki page too; but as you can see it refers to git builds (ie: -git packages).
Here the story is quite different: i'd like to init submodules from a git release tarball, and i don't see any way to achieve that.

progandy · 2018-06-14 19:52:56

If you want to use the tarball, then you have to manually download all archives and copy or symlink them in the right place during prepare.

https://github.com/babelouest/ulfius/archive/v2.3.6.tar.gz
https://github.com/babelouest/orcania/archive/a5659295a99e3a91177b56df01e1547d54acea51.tar.gz
https://github.com/babelouest/yder/archive/7bb11f77ee9629ce9adae9e5cfbdcf2e01690fcb.tar.gz

nierro · 2018-06-14 19:56:09

progandy wrote:

If you want to use the tarball, then you have to manually download all archives and copy or symlink them in the right place during prepare.

That's a nice idea! I hoped there was some more support for git submodules in PKGBUILDs and/or something i was missing.
I'll report back as soon as i got time to fix the PKGBUILD.
Thank you!

loqs · 2018-06-14 19:59:49

So to be clear you know the current PKGBUILD is using a git repository not a tarball and it does work for me but does not work for you and you do not want to use a git repository
however upstream ships a tarball that without the submodules then fetch the sources for the submodules as tarballs as well.

nierro · 2018-06-14 20:06:56

loqs wrote:

So to be clear you know the current PKGBUILD is using a git repository not a tarball

Yes i know.

loqs wrote:

and you do not want to use a git repository

Right!

loqs wrote:

however upstream ships a tarball that without the submodules then fetch the sources for the submodules as tarballs as well.

The issue is that it seems the release tarball is not able to fetch the submodules; or i am still missing something else here...

EDIT: nevermind, i may have found a better solution:

pkgname=libulfius
_gitname=ulfius
pkgver=2.3.6
pkgrel=4
pkgdesc="HTTP Framework for REST API in C, using JSON, with websockets and streaming data"
arch=(x86_64)
url="https://github.com/babelouest/${_gitname}"
license=(GPL)
depends=('libmicrohttpd' 'jansson' 'curl' 'gnutls' 'libgcrypt')
makedepends=(git cmake)
source=("${_gitname}-${pkgver}.tar.gz::${url}/archive/v${pkgver}.tar.gz")
sha256sums=("dbf1c4f32768d41b6e45b844f32927b9ae8dbccfa2cc8c6607755a6ee105d9a6")

prepare() {
    cd "${srcdir}/${_gitname}-${pkgver}"
    git init
    git remote add origin https://github.com/babelouest/ulfius.git
    git submodule update --init
    mkdir -p build
}

This should work and fix the issue.

Last edited by nierro (2018-06-14 20:08:20)

loqs · 2018-06-14 20:15:13

The tarball does not contain a git repo so you can not use git commands such as git submodule.
GIT_DISCOVERY_ACROSS_FILESYSTEM=1 allowed git to find a git repo at some higher level in the filesystem perhaps the checkout from AUR.
You now have a tarball in the sources but converting it into a git repo to use git submodules none of which are now in the sources.
How is that better than using all tarballs or all git repos in the sources array?
Edit:
overlong line length

# Maintainer: Federico Di Pierro <nierro92@gmail.com>

pkgname=libulfius
_gitname=ulfius
pkgver=2.3.6
pkgrel=3
pkgdesc="HTTP Framework for REST API in C, using JSON, with websockets and streaming data"
arch=(x86_64)
url="https://github.com/babelouest/${_gitname}"
license=(GPL)
depends=('libmicrohttpd' 'jansson' 'curl' 'gnutls' 'libgcrypt')
makedepends=(git cmake)
source=("git+https://github.com/babelouest/${_gitname}.git#tag=v${pkgver}"
        "git+https://github.com/babelouest/orcania.git"
        "git+https://github.com/babelouest/yder.git")
sha256sums=('SKIP'
            'SKIP'
            'SKIP')

prepare() {
    cd "${srcdir}/${_gitname}"
    git submodule init
    git config submodule.orcania.git.url $srcdir/orcania
    git config submodule.yder.git.url $srcdir/yder
    git submodule update
    mkdir -p build
}

build() {
    cd "${srcdir}/${_gitname}"/build
    cmake \
        -G "Unix Makefiles" \
        -DCMAKE_INSTALL_PREFIX=/usr \
        -DCMAKE_INSTALL_LIBDIR=lib \
        -DCMAKE_BUILD_TYPE="Release" \
        ../
    make
}

package() {
    cd "${srcdir}/${_gitname}"/build
    make DESTDIR="$pkgdir" install
}

Last edited by loqs (2018-06-14 20:22:01)

nierro · 2018-06-14 20:23:47

It feels cleaner to me as it won't download all repository but only the release tarball.
I think it's just a matter of taste at this point: to each his own

EDIT: your example will surely work though!

Last edited by nierro (2018-06-14 20:25:18)

loqs · 2018-06-14 20:35:47

# Maintainer: Federico Di Pierro <nierro92@gmail.com>

pkgname=libulfius
_gitname=ulfius
pkgver=2.3.6
pkgrel=3
pkgdesc="HTTP Framework for REST API in C, using JSON, with websockets and streaming data"
arch=(x86_64)
url="https://github.com/babelouest/${_gitname}"
license=(GPL)
depends=('libmicrohttpd' 'jansson' 'curl' 'gnutls' 'libgcrypt')
makedepends=(cmake)
source=("${_gitname}-${pkgver}.tar.gz::${url}/archive/v${pkgver}.tar.gz"
        "https://github.com/babelouest/orcania/archive/a5659295a99e3a91177b56df01e1547d54acea51.tar.gz"
        "https://github.com/babelouest/yder/archive/7bb11f77ee9629ce9adae9e5cfbdcf2e01690fcb.tar.gz")
sha256sums=('dbf1c4f32768d41b6e45b844f32927b9ae8dbccfa2cc8c6607755a6ee105d9a6'
            '1a59008c1de3045668d3cd5a57c4487b5c6edc6f68068678354131aff0614263'
            'd780d4ca961eea6d00a085cc4b5940c02219a11cd00ff6677d8c3bfb3be14413')

prepare() {
    cd "${srcdir}/${_gitname}-${pkgver}"
    ln -s lib/orcania "${srcdir}"/orcania
    ln -s lib/yder "${srcdir}"/yder
    mkdir -p build
}

build() {
    cd "${srcdir}/${_gitname}-${pkgver}"/build
    cmake \
        -G "Unix Makefiles" \
        -DCMAKE_INSTALL_PREFIX=/usr \
        -DCMAKE_INSTALL_LIBDIR=lib \
        -DCMAKE_BUILD_TYPE="Release" \
        ../
    make
}

package() {
    cd "${srcdir}/${_gitname}-${pkgver}"/build
    make DESTDIR="$pkgdir" install
}

eschwartz · 2018-06-14 22:02:04

As has been mentioned already, you'll need to decide whether it is cleaner to download lots of git history but get free handling of git sbumodule (which will even checkout the right commit for you), or use tarballs, which, since tarballs are not git clones, do not handle git submodules either and require manually tracking commits and moving them into place.

There's nothing else to discuss here. Tarballs fundamentally don't support git submodules. If upstream had provided their own release tarballs, then those would contain the submodule files too... but they didn't, they only provide built packages for a couple distros plus the basic Github Releases source archives. Those source archives are, as you've noticed, incomplete.

nierro · 2018-06-16 07:42:35

Just a small heads up: after a conversation with libulfius developer, i decided to completely split the 2 submodules in their own aur packages, as developer said that he uses these 2 libraries in other projects.
It seems a very clean way to fix this issue, to me.

Arch Linux

#1 2018-06-14 19:22:41

[SOLVED] Git submodules from git release

#2 2018-06-14 19:35:45

Re: [SOLVED] Git submodules from git release

#3 2018-06-14 19:47:21

Re: [SOLVED] Git submodules from git release

#4 2018-06-14 19:52:56

Re: [SOLVED] Git submodules from git release

#5 2018-06-14 19:56:09

Re: [SOLVED] Git submodules from git release

#6 2018-06-14 19:59:49

Re: [SOLVED] Git submodules from git release

#7 2018-06-14 20:06:56

Re: [SOLVED] Git submodules from git release

#8 2018-06-14 20:15:13

Re: [SOLVED] Git submodules from git release

#9 2018-06-14 20:23:47

Re: [SOLVED] Git submodules from git release

#10 2018-06-14 20:35:47

Re: [SOLVED] Git submodules from git release

#11 2018-06-14 22:02:04

Re: [SOLVED] Git submodules from git release

#12 2018-06-16 07:42:35

Re: [SOLVED] Git submodules from git release

Board footer