"git submodule update --force --init --recursive" inside PKGBUILD

ElMastro · 2025-02-17 02:32:19

Hello,
I am sorry for opening a post on this PKGBUILD; to install OpenModelica, the instructions are

git clone https://github.com/OpenModelica/OpenModelica

cd OpenModelica
git checkout master
git pull
# To checkout the latest master on each submodule run
# you will need to merge each submodule, but your changes will remain
git submodule foreach --recursive "git checkout master && git pull"

# Running master on all submodules might lead to build errors
# so use this to make sure you force all submodules to the commits
# from the OpenModelica glue project which are properly tested
git submodule update --force --init --recursive

cd OpenModelica
# Configure CMake, create Makefiles in build_cmake
cmake -S . -B build_cmake
# Compile with generated Makefiles
cmake --build build_cmake --parallel <Nr. of cores> --target install

This works inside bash, without problem. But, if I try it inside a PKGBUILD, like

pkgname=openmodelica-git
pkgver=1.25.0.dev.r401.ge33883cf4e
pkgrel=1
pkgdesc="The Open Source Modelica Suite"
arch=('i686' 'x86_64')
url="https://openmodelica.org"
license=('OSMC-PL')
source=("git+https://github.com/OpenModelica/OpenModelica")
md5sums=('SKIP')
depends=('java-environment' 'lapack' #'lpsolve' 'openscenegraph' 'python' 'ruby' 'suitesparse'
         'boost-libs' 'qt5-webkit' 'qt5-xmlpatterns' 'qt5-svg' 'qt5-tools' # 'fmilib'
         )
makedepends=('git' 'gcc-fortran' 'boost' 'clang' 'cmake' 'jre-openjdk' 'expat' 'ncurses' 'readline') # 'antlr4-runtime'
optdepends=('zip: requested when exporting to FMU'
			'sundials: adds more numerical solvers to the simulation runtime')
provides=('openmodelica')
conflicts=('openmodelica' 'openmodelica-svn' 'openmodelica-dev')

#prepare() {

#}
 
pkgver() {
    #cd $pkgname
    cd OpenModelica
    git describe --long --tags | sed 's/^v//;s/\([^-]*-g\)/r\1/;s/-/./g'
}
 
build() {
    #cd $pkgname
	cd OpenModelica
	# To checkout the latest master on each submodule run
	# you will need to merge each submodule, but your changes will remain
	git submodule foreach --recursive "git checkout master && git pull"

	# Running master on all submodules might lead to build errors
	# so use this to make sure you force all submodules to the commits
	# from the OpenModelica glue project which are properly tested
	git submodule update --force --init --recursive
	
	#All the following is because "git submodule update --force --init --recursive" doesn't work inside PKGBUILD script
	#mkdir -p OpenModelica/OMOptim/common/
	#mkdir -p OpenModelica/OMSens_Qt/common/
	#cp -r {OMSimulator,OMSens_Qt} OpenModelica/
	#cp -r OMCompiler-3rdParty/* OpenModelica/OMCompiler/3rdParty/
	#cp -r OMSimulator-3rdParty/* OpenModelica/OMSimulator/3rdParty/
	#cp -r OpenModelica-common/* OpenModelica/OMSens_Qt/common/
	#cp -r OpenModelica-common/* OpenModelica/OMOptim/common/
}
 
package() {  
    #cd $pkgname
	cd OpenModelica
	# Configure CMake, create Makefiles in build_cmake
	cmake -S . -B build_cmake
	# Compile with generated Makefiles
	cmake --build build_cmake --parallel 8 --target install
}

,

	git submodule update --force --init --recursive

comes out with

fatal: repository '/tmp/openmodelica/OMOptim.git' does not exist
fatal: clone of '/tmp/openmodelica/OMOptim.git' into submodule path '/tmp/openmodelica/src/OpenModelica/OMOptim' failed
Failed to clone 'OMOptim'. Retry scheduled
fatal: repository '/tmp/openmodelica/OMSens.git' does not exist
fatal: clone of '/tmp/openmodelica/OMSens.git' into submodule path '/tmp/openmodelica/src/OpenModelica/OMSens' failed
Failed to clone 'OMSens'. Retry scheduled
fatal: repository '/tmp/openmodelica/OMSens_Qt.git' does not exist
fatal: clone of '/tmp/openmodelica/OMSens_Qt.git' into submodule path '/tmp/openmodelica/src/OpenModelica/OMSens_Qt' failed
Failed to clone 'OMSens_Qt'. Retry scheduled
fatal: repository '/tmp/openmodelica/OMSimulator.git' does not exist
fatal: clone of '/tmp/openmodelica/OMSimulator.git' into submodule path '/tmp/openmodelica/src/OpenModelica/OMSimulator' failed
Failed to clone 'OMSimulator'. Retry scheduled
fatal: repository '/tmp/openmodelica/OMOptim.git' does not exist
fatal: clone of '/tmp/openmodelica/OMOptim.git' into submodule path '/tmp/openmodelica/src/OpenModelica/OMOptim' failed
Failed to clone 'OMOptim' a second time, aborting

.
I have been trying for three days to fix it, but nothing seems to work?
Does someon have a suggestion?
Thank you for your time.

dimich · 2025-02-17 03:35:29

ElMastro wrote:

the instructions are

Weird instructions. Why checkout master for all submodules recursively, then update them back to superproject reference?

git clone --branch master --recurse-submodules https://github.com/OpenModelica/OpenModelica

should give the same result.

Lone_Wolf · 2025-02-17 11:37:27

You have tried to use https://aur.archlinux.org/packages/openmodelica-git ?

ElMastro · 2025-02-18 01:52:19

dimich wrote:

git clone --branch master --recurse-submodules https://github.com/OpenModelica/OpenModelica

Unfortunately, not. I don't know the reason for doing this, it comes from https://github.com/OpenModelica/OpenMod … /README.md

CMake Error at cmake/omc_utils.cmake:16 (add_subdirectory):
  The source directory

    /tmp/openmodelica/src/OpenModelica/OMSens_Qt

  does not contain a CMakeLists.txt file.
Call Stack (most recent call first):
  CMakeLists.txt:173 (omc_add_subdirectory)

is the result.
@Lone_Wolf : I am a bit ashamed to say this, I must confess that the mantainer of openmodelica-git is.....me.
Now, both using the old PKGBUILD (the one in AUR) or the one I posted, the compilation fails at 97% on OMNotebook. with

error: ‘oms_tlm_interpolation_t’ has not been declared

. Running the single commands instead works. It's almost a week I can't understand why.

Only clue I got is that inside OpenModelica folder there are these submodules folder that must be copied/updated (OMSimulator,OMSens_Qt, 3rd-Party parts).

	cp -r {OMSimulator,OMSens_Qt} OpenModelica/
	cp -r OMCompiler-3rdParty/* OpenModelica/OMCompiler/3rdParty/
	cp -r OMSimulator-3rdParty/* OpenModelica/OMSimulator/3rdParty/

solves a part of these problem, but it's dirty and there are other modules missing into it. "git submodule update --force --init --recursive" solves this someway, but I can't understand how to make it work inside the PKGBUILD

loqs · 2025-02-18 12:08:17

ElMastro wrote:

@Lone_Wolf : I am a bit ashamed to say this, I must confess that the mantainer of openmodelica-git is.....me.
Now, both using the old PKGBUILD (the one in AUR) or the one I posted, the compilation fails at 97% on OMNotebook. with

The one in AUR fails to resolve depends for me between jre-openjdk in makedepends and java-environment in depends. After fixing that it fails in prepare() due to missing `protocol.file.allow=always` for submodule update. After fixing that it fails in build due to `-Werror=format-security`.

Lone_Wolf · 2025-02-18 12:11:17

When trying to build in a clean chroot this pops up :

looking for conflicting packages...
:: jre-openjdk-23.0.2.u7-1 and jdk-openjdk-23.0.2.u7-1 are in conflict. Remove jdk-openjdk? [y/N] 
error: unresolvable package conflicts detected
error: failed to prepare transaction (conflicting dependencies)
:: jre-openjdk-23.0.2.u7-1 and jdk-openjdk-23.0.2.u7-1 are in conflict

removing java-environment form depends does make it continue, but it fails hard in prepare() .

makepkg requires special handling to deal with git submodules.
I advise you to check https://wiki.archlinux.org/title/VCS_pa … submodules and implement that before doing any other troubleshooting.

ElMastro · 2025-02-19 02:06:16

Great news! It now works (but almost I don't know why xD).
@loqs: version on AUR was uploaded about two and half a year ago. On February the university (Liu) released on Github this new version. It's completely new, and overall it works on cmake instead of the old version that used another method.
@Lone_Wolf:
On Github it's stated "Java JRE (JDK is option; compiles the Java CORBA interface)" between the requirements. I have

  47 extra/java-environment-common 3-6 [2.72 KiB 0 B] [Installato]
    Common files for Java Development Kits
--
  52 extra/java-runtime-common 3-6 [5.02 KiB 5.26 KiB] [Installato]
    Common files for Java Runtime Environments
--
  57 extra/jdk17-openjdk 17.0.14.u7-1 [255.78 MiB 400.20 MiB] [Installato]
    OpenJDK Java 17 development kit
--
  65 extra/jre-openjdk 23.0.2.u7-1 [29.93 MiB 148.06 MiB] [Installato]
    OpenJDK Java 23 full runtime environment

. I remember Java has always been a nuisance with this package, together with Fortran.
Thank you for the link, it's really clear. However I couldn't make that, I started from @dimich solution.

ElMastro · 2025-02-19 02:09:16

Thank you all for the help, there is no possible way we could have done it without it. The working PKGBUILD shall be something like:

pkgname=openmodelica-git
pkgver=1.25.0.dev.r406.gf9e29c1120
pkgrel=1
pkgdesc="The Open Source Modelica Suite"
arch=('i686' 'x86_64')
url="https://openmodelica.org"
license=('OSMC-PL')
source=("git+https://github.com/OpenModelica/OpenModelica")
md5sums=('SKIP')
depends=('java-environment' 'lapack' #'lpsolve' 'openscenegraph' 'python' 'ruby' 'suitesparse'
         'boost-libs' 'qt5-webkit' 'qt5-xmlpatterns' 'qt5-svg' 'qt5-tools' # 'fmilib'
         )
makedepends=('git' 'gcc-fortran' 'boost' 'clang' 'cmake' 'jre-openjdk' 'expat' 'ncurses' 'readline') # 'antlr4-runtime'
optdepends=('zip: requested when exporting to FMU'
			'sundials: adds more numerical solvers to the simulation runtime')
provides=('openmodelica')
conflicts=('openmodelica' 'openmodelica-svn' 'openmodelica-dev')

#prepare() {

#}
 
pkgver() {
    cd OpenModelica #cd $pkgname
    git describe --long --tags | sed 's/^v//;s/\([^-]*-g\)/r\1/;s/-/./g'
}
 
build() {
	cd OpenModelica #cd $pkgname
	#The following comments are for who comes from github, to have a reference about where we are
	
	# To checkout the latest master on each submodule run
	# you will need to merge each submodule, but your changes will remain
	#git submodule foreach --recursive "git checkout master && git pull"

	# Running master on all submodules might lead to build errors
	# so use this to make sure you force all submodules to the commits
	# from the OpenModelica glue project which are properly tested
	#git config --global protocol.file.allow always #protocol.file.allow=always

	git clone --branch master --recurse-submodules https://github.com/OpenModelica/OpenModelica
}
 
package() {  
	cd OpenModelica/OpenModelica #cd $pkgname
	
	# Configure CMake, create Makefiles in build_cmake
	cmake -S . -B build_cmake
	# Compile with generated Makefiles
	cmake --build build_cmake --parallel 8 --target install
}

It's still a bit ugly (bear with it, after all I am still learning), if we confirm it works and see the suggestions to make it better, I can upload it on AUR

Lone_Wolf · 2025-02-19 10:53:39

Unfortunately it still fails hard in a clean chroot (see below)

You may have figured out a way to build this on your system, but aur users build on their own system.

I suggest you setup clean chroot building and try with the method from vcs packaging guidelines .

.

==> Starting prepare()...
Submodule 'OMCompiler/3rdParty' (/srcdest/OMCompiler-3rdParty) registered for path 'OMCompiler/3rdParty'
Submodule 'OMOptim' (/srcdest/OMOptim.git) registered for path 'OMOptim'
Submodule 'OMSens' (/srcdest/OMSens.git) registered for path 'OMSens'
Submodule 'OMSens_Qt' (/srcdest/OMSens_Qt.git) registered for path 'OMSens_Qt'
Submodule 'OMSimulator' (/srcdest/OMSimulator.git) registered for path 'OMSimulator'
Cloning into '/build/openmodelica-git/src/openmodelica-git/OMCompiler/3rdParty'...
fatal: transport 'file' not allowed
fatal: clone of '/srcdest/OMCompiler-3rdParty' into submodule path '/build/openmodelica-git/src/openmodelica-git/OMCompiler/3rdParty' failed
Failed to clone 'OMCompiler/3rdParty'. Retry scheduled
Cloning into '/build/openmodelica-git/src/openmodelica-git/OMOptim'...
fatal: transport 'file' not allowed
fatal: clone of '/build/openmodelica-git/src/OMOptim' into submodule path '/build/openmodelica-git/src/openmodelica-git/OMOptim' failed
Failed to clone 'OMOptim'. Retry scheduled
Cloning into '/build/openmodelica-git/src/openmodelica-git/OMSens'...
fatal: transport 'file' not allowed
fatal: clone of '/build/openmodelica-git/src/OMSens' into submodule path '/build/openmodelica-git/src/openmodelica-git/OMSens' failed
Failed to clone 'OMSens'. Retry scheduled
Cloning into '/build/openmodelica-git/src/openmodelica-git/OMSens_Qt'...
fatal: transport 'file' not allowed
fatal: clone of '/build/openmodelica-git/src/OMSens_Qt' into submodule path '/build/openmodelica-git/src/openmodelica-git/OMSens_Qt' failed
Failed to clone 'OMSens_Qt'. Retry scheduled
Cloning into '/build/openmodelica-git/src/openmodelica-git/OMSimulator'...
fatal: transport 'file' not allowed
fatal: clone of '/build/openmodelica-git/src/OMSimulator' into submodule path '/build/openmodelica-git/src/openmodelica-git/OMSimulator' failed
Failed to clone 'OMSimulator'. Retry scheduled
Cloning into '/build/openmodelica-git/src/openmodelica-git/OMCompiler/3rdParty'...
fatal: transport 'file' not allowed
fatal: clone of '/srcdest/OMCompiler-3rdParty' into submodule path '/build/openmodelica-git/src/openmodelica-git/OMCompiler/3rdParty' failed
Failed to clone 'OMCompiler/3rdParty' a second time, aborting
==> ERROR: A failure occurred in prepare().
    Aborting...
==> ERROR: Build failed, check /var/lib/archbuild/extra-x86_64/panoramix/build
$

loqs · 2025-02-19 11:46:13

@Lone_Wolf the code in post #8 does not use the source array or git submodule update as instead it uses the following:

	git clone --branch master --recurse-submodules https://github.com/OpenModelica/OpenModelica

So I get a different failure due to missing ccache then after fixing that it fails due to openscenegraph being commented out in the depends array after fixing that it then fails in the build with:

/usr/src/debug/openmodelica-git/OpenModelica/OMSimulator/3rdParty/minizip/src/minizip.c: In function ‘minizip’:
/usr/src/debug/openmodelica-git/OpenModelica/OMSimulator/3rdParty/minizip/src/minizip.c:502:20: error: ‘fin’ may be used uninitialized [-Werror=maybe-uninitialized]
/usr/src/debug/openmodelica-git/OpenModelica/OMSimulator/3rdParty/minizip/src/minizip.c:405:24: note: ‘fin’ was declared here
lto1: some warnings being treated as errors

The current PKGBUILD in AUR fails to build but at least it violate less packaging guidelines.

Last edited by loqs (2025-02-19 11:47:21)

ElMastro · 2025-02-19 23:52:06

I am trying with the clean chroot; wat happens is:

==> Installing missing dependencies...
resolving dependencies...
looking for conflicting packages...
:: jre-openjdk-23.0.2.u7-1 and jdk-openjdk-23.0.2.u7-1 are in conflict. Remove jdk-openjdk? [y/N] 
error: unresolvable package conflicts detected
error: failed to prepare transaction (conflicting dependencies)
:: jre-openjdk-23.0.2.u7-1 and jdk-openjdk-23.0.2.u7-1 are in conflict
==> ERROR: 'pacman' failed to install missing dependencies.
==> Missing dependencies:
  -> git
  -> gcc-fortran
  -> boost
  -> clang
  -> cmake
  -> jre-openjdk
==> ERROR: Could not resolve all dependencies.

Now;

Lone_Wolf wrote:

Unfortunately it still fails hard in a clean chroot (see below)

Cloning into '/build/openmodelica-git/src/openmodelica-git/OMCompiler/3rdParty'...
fatal: transport 'file' not allowed
fatal: clone of '/srcdest/OMCompiler-3rdParty' into submodule path '/build/openmodelica-git/src/openmodelica-git/OMCompiler/3rdParty' failed
Failed to clone 'OMCompiler/3rdParty'. Retry scheduled
Cloning into '/build/openmodelica-git/src/openmodelica-git/OMOptim'...
fatal: transport 'file' not allowed
fatal: clone of '/build/openmodelica-git/src/OMOptim' into submodule path '/build/openmodelica-git/src/openmodelica-git/OMOptim' failed
Failed to clone 'OMOptim'. Retry scheduled
Cloning into '/build/openmodelica-git/src/openmodelica-git/OMSens'...
fatal: transport 'file' not allowed

Maybe adding "git config --global protocol.file.allow always" in the PKGBUILD before " git clone --branch master --recurse-submodules https://github.com/OpenModelica/OpenModelica" improve it (just to see if is this the problem)?
@loqs:

Lone_Wolf wrote:

So I get a different failure due to missing ccache then after fixing that it fails due to openscenegraph being commented out in the depends array after fixing that it then fails in the build with:

Openscenegraph should be optional, it's required because of OMEdit; honestly I don't know why it doesn't work for ccache. I will add opescenegraph as a dependency.

Lone_Wolf wrote:

/usr/src/debug/openmodelica-git/OpenModelica/OMSimulator/3rdParty/minizip/src/minizip.c: In function ‘minizip’:
/usr/src/debug/openmodelica-git/OpenModelica/OMSimulator/3rdParty/minizip/src/minizip.c:502:20: error: ‘fin’ may be used uninitialized [-Werror=maybe-uninitialized]
/usr/src/debug/openmodelica-git/OpenModelica/OMSimulator/3rdParty/minizip/src/minizip.c:405:24: note: ‘fin’ was declared here
lto1: some warnings being treated as errors

First time I see something like this. Would you mind seeing if using the commands in the script (first post) it does it, please?

Of course dependencies need a check, I am on it

Lone_Wolf · 2025-02-20 08:59:41

I had to install cmake & gcc-fortran to get somewhere, but stopped when cmake couldn't find BLAS .

http://0x0.st/8ctd.txt

loqs wrote:

/usr/src/debug/openmodelica-git/OpenModelica/OMSimulator/3rdParty/minizip/src/minizip.c: In function ‘minizip’:
/usr/src/debug/openmodelica-git/OpenModelica/OMSimulator/3rdParty/minizip/src/minizip.c:502:20: error: ‘fin’ may be used uninitialized [-Werror=maybe-uninitialized]
/usr/src/debug/openmodelica-git/OpenModelica/OMSimulator/3rdParty/minizip/src/minizip.c:405:24: note: ‘fin’ was declared here
lto1: some warnings being treated as errors

For now try adding options=(!debug) to the PKGBUILD . Once things build correctly debug stuff can be looked into.

loqs · 2025-02-20 11:02:31

qt5-webkit fails to build after icu has been updated to 76.
Edit:
@Lone_Wolf with options=(!debug):

/build/openmodelica-git/src/OpenModelica/OpenModelica/OMSimulator/3rdParty/minizip/src/minizip.c: In function ‘minizip’:
/build/openmodelica-git/src/OpenModelica/OpenModelica/OMSimulator/3rdParty/minizip/src/minizip.c:502:20: error: ‘fin’ may be used uninitialized [-Werror=maybe-uninitialized]
  502 |                 if (fin)
      |                    ^
/build/openmodelica-git/src/OpenModelica/OpenModelica/OMSimulator/3rdParty/minizip/src/minizip.c:405:24: note: ‘fin’ was declared here
  405 |                 FILE * fin;
      |                        ^
lto1: some warnings being treated as errors
make[3]: *** [/tmp/ccD55GdQ.mk:104: /tmp/cciTH8Ne.ltrans34.ltrans.o] Error

Upstream closed issue https://github.com/OpenModelica/OpenMod … 2115369623

@ElMastro please stop cloning git repositories in build() and performing the build in package().

Last edited by loqs (2025-02-20 18:27:12)

ElMastro · 2025-02-21 00:54:47

@loqs:
Sorry, I apologize. I will move the

cmake -S . -B build_cmake

in build. About the cloning it was not my intention, I have still to understand what to do with submodules (I don't know what submodules are needed).
@Lone_Wolf, I don't know if this can help you, I have

  1 extra/blas 3.12.1-2 [0 B 756.42 KiB] [Installato]
  5 extra/cblas 3.12.1-2 [56.98 KiB 343.55 KiB] [Installato]
 13 extra/liblas 1.8.1.r128+gded46373-12 [715.51 KiB 3.35 MiB] [Installato]

installed. I don't know for sure, it seems #1300 and #1301 issue o github were closed on Oct 28, 2024. Let me know how I can help

Arch Linux

#1 2025-02-17 02:32:19

"git submodule update --force --init --recursive" inside PKGBUILD

#2 2025-02-17 03:35:29

Re: "git submodule update --force --init --recursive" inside PKGBUILD

#3 2025-02-17 11:37:27

Re: "git submodule update --force --init --recursive" inside PKGBUILD

#4 2025-02-18 01:52:19

Re: "git submodule update --force --init --recursive" inside PKGBUILD

#5 2025-02-18 12:08:17

Re: "git submodule update --force --init --recursive" inside PKGBUILD

#6 2025-02-18 12:11:17

Re: "git submodule update --force --init --recursive" inside PKGBUILD

#7 2025-02-19 02:06:16

Re: "git submodule update --force --init --recursive" inside PKGBUILD

#8 2025-02-19 02:09:16

Re: "git submodule update --force --init --recursive" inside PKGBUILD

#9 2025-02-19 10:53:39

Re: "git submodule update --force --init --recursive" inside PKGBUILD

#10 2025-02-19 11:46:13

Re: "git submodule update --force --init --recursive" inside PKGBUILD

#11 2025-02-19 23:52:06

Re: "git submodule update --force --init --recursive" inside PKGBUILD

#12 2025-02-20 08:59:41

Re: "git submodule update --force --init --recursive" inside PKGBUILD

#13 2025-02-20 11:02:31

Re: "git submodule update --force --init --recursive" inside PKGBUILD

#14 2025-02-21 00:54:47

Re: "git submodule update --force --init --recursive" inside PKGBUILD

Board footer