You are not logged in.

#1 2018-06-14 19:22:41

nierro
Member
From: Milan, Italy
Registered: 2011-09-02
Posts: 849

[SOLVED] Git submodules from git release

Hi everyone!
I need some advice regarding a PKGBUILD.
I'm trying to correctly build a package from a git release; it needs 2 submodules to be inited though.
The normal way, ie: downloading the .tar.gz release archive  and in prepare() function calling "git submodule update --init" (https://aur.archlinux.org/cgit/aur.git/ … 6e12f89502) gives

fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

I then tried to fix the issue by downloading the full git repo, move to needed tag and initing the submodules (https://aur.archlinux.org/cgit/aur.git/ … =libulfius). Again, submodules are not built and consequently not installed.
What am I missing?

Thank you very much!

Last edited by nierro (2018-06-16 07:42:49)

Offline

#2 2018-06-14 19:35:45

loqs
Member
Registered: 2014-03-06
Posts: 17,195

Re: [SOLVED] Git submodules from git release

pacman -Qlp libulfius-2.3.6-3-x86_64.pkg.tar.xz 
libulfius /usr/
libulfius /usr/include/
libulfius /usr/include/orcania.h
libulfius /usr/include/ulfius.h
libulfius /usr/include/yder.h
libulfius /usr/lib/
libulfius /usr/lib/liborcania.so
libulfius /usr/lib/liborcania.so.1.2
libulfius /usr/lib/liborcania.so.1.2.3
libulfius /usr/lib/libulfius.so
libulfius /usr/lib/libulfius.so.2.3
libulfius /usr/lib/libulfius.so.2.3.6
libulfius /usr/lib/libyder.so
libulfius /usr/lib/libyder.so.1.3
libulfius /usr/lib/libyder.so.1.3.3
...

I would recomend you follow VCS_package_guidelines#Git_Submodules but with your current PKGBUILD the submodules are built and packaged.

Offline

#3 2018-06-14 19:47:21

nierro
Member
From: Milan, Italy
Registered: 2011-09-02
Posts: 849

Re: [SOLVED] Git submodules from git release

Thanks for the feedback; but for me (through 'yay') it is not working...
I tried to follow that wiki page too; but as you can see it refers to git builds (ie: -git packages).
Here the story is quite different: i'd like to init submodules from a git release tarball, and i don't see any way to achieve that.

Offline

#4 2018-06-14 19:52:56

progandy
Member
Registered: 2012-05-17
Posts: 5,184

Re: [SOLVED] Git submodules from git release

If you want to use the tarball, then you have to manually download all archives and copy or symlink them in the right place during prepare.

https://github.com/babelouest/ulfius/archive/v2.3.6.tar.gz
https://github.com/babelouest/orcania/archive/a5659295a99e3a91177b56df01e1547d54acea51.tar.gz
https://github.com/babelouest/yder/archive/7bb11f77ee9629ce9adae9e5cfbdcf2e01690fcb.tar.gz

| alias CUTF='LANG=en_XX.UTF-8@POSIX ' |

Offline

#5 2018-06-14 19:56:09

nierro
Member
From: Milan, Italy
Registered: 2011-09-02
Posts: 849

Re: [SOLVED] Git submodules from git release

progandy wrote:

If you want to use the tarball, then you have to manually download all archives and copy or symlink them in the right place during prepare.

That's a nice idea! I hoped there was some more support for git submodules in PKGBUILDs and/or something i was missing.
I'll report back as soon as i got time to fix the PKGBUILD.
Thank you!

Offline

#6 2018-06-14 19:59:49

loqs
Member
Registered: 2014-03-06
Posts: 17,195

Re: [SOLVED] Git submodules from git release

So to be clear you know the current PKGBUILD is using a git repository not a tarball and it does work for me but does not work for you and you do not want to use a git repository
however upstream ships a tarball that without the submodules then fetch the sources for the submodules as tarballs as well.

Offline

#7 2018-06-14 20:06:56

nierro
Member
From: Milan, Italy
Registered: 2011-09-02
Posts: 849

Re: [SOLVED] Git submodules from git release

loqs wrote:

So to be clear you know the current PKGBUILD is using a git repository not a tarball

Yes i know.

loqs wrote:

and you do not want to use a git repository

Right!

loqs wrote:

however upstream ships a tarball that without the submodules then fetch the sources for the submodules as tarballs as well.

The issue is that it seems the release tarball is not able to fetch the submodules; or i am still missing something else here...

EDIT: nevermind, i may have found a better solution:

pkgname=libulfius
_gitname=ulfius
pkgver=2.3.6
pkgrel=4
pkgdesc="HTTP Framework for REST API in C, using JSON, with websockets and streaming data"
arch=(x86_64)
url="https://github.com/babelouest/${_gitname}"
license=(GPL)
depends=('libmicrohttpd' 'jansson' 'curl' 'gnutls' 'libgcrypt')
makedepends=(git cmake)
source=("${_gitname}-${pkgver}.tar.gz::${url}/archive/v${pkgver}.tar.gz")
sha256sums=("dbf1c4f32768d41b6e45b844f32927b9ae8dbccfa2cc8c6607755a6ee105d9a6")

prepare() {
    cd "${srcdir}/${_gitname}-${pkgver}"
    git init
    git remote add origin https://github.com/babelouest/ulfius.git
    git submodule update --init
    mkdir -p build
}

This should work and fix the issue.

Last edited by nierro (2018-06-14 20:08:20)

Offline

#8 2018-06-14 20:15:13

loqs
Member
Registered: 2014-03-06
Posts: 17,195

Re: [SOLVED] Git submodules from git release

The tarball does not contain a git repo so you can not use git commands such as git submodule.
GIT_DISCOVERY_ACROSS_FILESYSTEM=1 allowed git to find a git repo at some higher level in the filesystem perhaps the checkout from AUR.
You now have a tarball in the sources but converting it into a git repo to use git submodules none of which are now in the sources.
How is that better than using all tarballs or all git repos in the sources array?
Edit:
overlong line length

# Maintainer: Federico Di Pierro <nierro92@gmail.com>

pkgname=libulfius
_gitname=ulfius
pkgver=2.3.6
pkgrel=3
pkgdesc="HTTP Framework for REST API in C, using JSON, with websockets and streaming data"
arch=(x86_64)
url="https://github.com/babelouest/${_gitname}"
license=(GPL)
depends=('libmicrohttpd' 'jansson' 'curl' 'gnutls' 'libgcrypt')
makedepends=(git cmake)
source=("git+https://github.com/babelouest/${_gitname}.git#tag=v${pkgver}"
        "git+https://github.com/babelouest/orcania.git"
        "git+https://github.com/babelouest/yder.git")
sha256sums=('SKIP'
            'SKIP'
            'SKIP')

prepare() {
    cd "${srcdir}/${_gitname}"
    git submodule init
    git config submodule.orcania.git.url $srcdir/orcania
    git config submodule.yder.git.url $srcdir/yder
    git submodule update
    mkdir -p build
}

build() {
    cd "${srcdir}/${_gitname}"/build
    cmake \
        -G "Unix Makefiles" \
        -DCMAKE_INSTALL_PREFIX=/usr \
        -DCMAKE_INSTALL_LIBDIR=lib \
        -DCMAKE_BUILD_TYPE="Release" \
        ../
    make
}

package() {
    cd "${srcdir}/${_gitname}"/build
    make DESTDIR="$pkgdir" install
}

Last edited by loqs (2018-06-14 20:22:01)

Offline

#9 2018-06-14 20:23:47

nierro
Member
From: Milan, Italy
Registered: 2011-09-02
Posts: 849

Re: [SOLVED] Git submodules from git release

It feels cleaner to me as it won't download all repository but only the release tarball.
I think it's just a matter of taste at this point: to each his own smile

EDIT: your example will surely work though!

Last edited by nierro (2018-06-14 20:25:18)

Offline

#10 2018-06-14 20:35:47

loqs
Member
Registered: 2014-03-06
Posts: 17,195

Re: [SOLVED] Git submodules from git release

# Maintainer: Federico Di Pierro <nierro92@gmail.com>

pkgname=libulfius
_gitname=ulfius
pkgver=2.3.6
pkgrel=3
pkgdesc="HTTP Framework for REST API in C, using JSON, with websockets and streaming data"
arch=(x86_64)
url="https://github.com/babelouest/${_gitname}"
license=(GPL)
depends=('libmicrohttpd' 'jansson' 'curl' 'gnutls' 'libgcrypt')
makedepends=(cmake)
source=("${_gitname}-${pkgver}.tar.gz::${url}/archive/v${pkgver}.tar.gz"
        "https://github.com/babelouest/orcania/archive/a5659295a99e3a91177b56df01e1547d54acea51.tar.gz"
        "https://github.com/babelouest/yder/archive/7bb11f77ee9629ce9adae9e5cfbdcf2e01690fcb.tar.gz")
sha256sums=('dbf1c4f32768d41b6e45b844f32927b9ae8dbccfa2cc8c6607755a6ee105d9a6'
            '1a59008c1de3045668d3cd5a57c4487b5c6edc6f68068678354131aff0614263'
            'd780d4ca961eea6d00a085cc4b5940c02219a11cd00ff6677d8c3bfb3be14413')

prepare() {
    cd "${srcdir}/${_gitname}-${pkgver}"
    ln -s lib/orcania "${srcdir}"/orcania
    ln -s lib/yder "${srcdir}"/yder
    mkdir -p build
}

build() {
    cd "${srcdir}/${_gitname}-${pkgver}"/build
    cmake \
        -G "Unix Makefiles" \
        -DCMAKE_INSTALL_PREFIX=/usr \
        -DCMAKE_INSTALL_LIBDIR=lib \
        -DCMAKE_BUILD_TYPE="Release" \
        ../
    make
}

package() {
    cd "${srcdir}/${_gitname}-${pkgver}"/build
    make DESTDIR="$pkgdir" install
}

Offline

#11 2018-06-14 22:02:04

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: [SOLVED] Git submodules from git release

As has been mentioned already, you'll need to decide whether it is cleaner to download lots of git history but get free handling of git sbumodule (which will even checkout the right commit for you), or use tarballs, which, since tarballs are not git clones, do not handle git submodules either and require manually tracking commits and moving them into place.

There's nothing else to discuss here. Tarballs fundamentally don't support git submodules. If upstream had provided their own release tarballs, then those would contain the submodule files too... but they didn't, they only provide built packages for a couple distros plus the basic Github Releases source archives. Those source archives are, as you've noticed, incomplete.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#12 2018-06-16 07:42:35

nierro
Member
From: Milan, Italy
Registered: 2011-09-02
Posts: 849

Re: [SOLVED] Git submodules from git release

Just a small heads up: after a conversation with libulfius developer, i decided to completely split the 2 submodules in their own aur packages, as developer said that he uses these 2 libraries in other projects.
It seems a very clean way to fix this issue, to me.

Offline

Board footer

Powered by FluxBB