[SOLVED] Attempting my first PKGBUILD, error on undefined cblas_dgemm

feinedsquirrel · 2021-05-31 22:39:35

I am trying to create a PKGBUILD for photoview. Instructions by the original project can be found in the photoview docs pages.

My PKGBUILD is the following:

# Maintainer: name < username at domain dot tld >

# instructions by photoview author:
# https://photoview.github.io/docs/installation-manual/

pkgname=photoview
pkgver=2.3.4
pkgrel=1
pkgdesc="Photo gallery for self-hosted personal servers"
arch=('x86_64')
url="https://github.com/photoview/photoview"
license=('AGPL-3.0')
# openblas -> cblas -> blas
# lapacke -> lapack
# lapack
# openblas
# openblas-lapack (aur) -- fixes "cblas_dgemm" issue, but I am hoping to find a solution using official repos openblas
depends=('nodejs'
         'libjpeg-turbo'
         'libheif'
         'openblas'
         'dlib')
# dlib from aur -- alter pkgbuild prior to installation, replacing blas dep with openblas
optdepends=('mariadb: for complete database support'
            'sqlite: for sqlite support'
            'darktable: for RAW image support'
            'ffmpeg: for video file playback in the browser')
makedepends=('git' 'go' 'npm')
# source=("https://github.com/${pkgname}/${pkgname}/archive/refs/tags/v${pkgver}.tar.gz")
# sha256sums=('c3b9f09e65af1d8116730d9b88b2430c71ba01e4cd8294d205056d81d822f5d0')
source=("git+https://github.com/${pkgname}/${pkgname}.git#tag=v${pkgver}")
sha256sums=('SKIP')

build() {
    # web user-interface
    cd "${srcdir}/${pkgname}/ui"
    npm install
    npm run build

    # api backend
    cd "${srcdir}/${pkgname}/api"
    # cd ./api
    go build -v -o photoview .
}

package() {
    # copy needed files
    cd "${pkgdir}"
    mkdir -p "output"
    cp -r "${srcdir}/${pkgname}/ui/dist/" "${pkgdir}/ouput/ui/"
    cp "${srcdir}/${pkgname}/api/photoview" "${pkgdir}/output/photoview"
    cp -r "${srcdir}/${pkgname}/api/data/" "${pkgdir}/output/data/"
    cp "${srcdir}/${pkgname}/api/example.env" "${pkgdir}/output/.env"

    # eventually, output/ui should be placed in <system>/webapps
    # and output/.env should be placed in /etc
}

I get the following error when I build in a clean chroot, (I pass a pre-built dlib into the chroot via the

-I

parameter):

# github.com/Kagami/go-face
/usr/bin/ld: $WORK/b149/_x005.o: undefined reference to symbol 'cblas_dgemm'
/usr/bin/ld: /usr/lib/libcblas.so.3: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
==> ERROR: A failure occurred in build().
    Aborting...
==> ERROR: Build failed, check /home/myuser/chroot/myuser/build

I don't get this error when I use openblas-lapack from the AUR, but as I mention in my notes-to-self in the PKGBUILD, I hope to avoid another dependency from the AUR.

The photoview project depends on dlib from the AUR, which itself relies on blas. I have found several similar problems on various sites:

https://unix.stackexchange.com/a/417365
https://forum.dlang.org/thread/yurqxupq … .dlang.org
https://stackoverflow.com/a/37866377
https://www.linuxquestions.org/question … as-822851/
http://icl.cs.utk.edu/lapack-forum/view … =12&t=5121

I also read another question/answer that was more clear, but I cannot find it right now. The gist was that I need to link directly to blas (or cblas?) with something like

-lblas

.
I thought maybe I would need to figure out how to link

go-face

with cblas, or something, but since it works with

openblas-lapack

I'm wondering if there is something else that I can do.

Any ideas on how to resolve this?

And while I'm here, the final copy commands fail, saying there isn't a directory for e.g.

"${pkgdir}/ouput/ui/"

. I expected it to create the output/ui directory with contents from ui/dist, as it works that way when I manually complete those last few lines from the command line. Is there something wrong with my syntax?

Thanks in advance

Last edited by feinedsquirrel (2021-06-03 18:13:55)

Lone_Wolf · 2021-06-01 13:45:46

Photoview instructions mention go but building is done with npm , the nodejs guidelines[1] are probably most relevant .

DSO missing from command line

Been a while since I last saw that error.
https://stackoverflow.com/questions/199 … mmand-line gives several good answers.
The one that i feel is the clearest of those is https://stackoverflow.com/a/55086637 .

When DSO missing was common, the typical buildsystem was autotools / gnu make . no clue how to apply that to go sourcecode build with npm .

How is your chroot setup and can you post full makepkg / makekchrootpkg logs as well as the exact command ?
Those logs could be huge, consider using a pastebin client

[1] https://wiki.archlinux.org/title/Node.j … guidelines

Last edited by Lone_Wolf (2021-06-01 13:46:27)

arojas · 2021-06-01 16:13:45

Pretty sure someone somewhere is assuming that libblas provides cblas symbols, which is a Debian specific thing. But it's hard to know where the problem is exactly within the go dependency hell.

As a side note: it is not a good idea to make packages depend on openblas (or any other specific blas implementation for that matter), unless strictly required by upstream. This way you're preventing users from using any other blas implementation. Depend on simple blas, and then users can choose to install any implementation.

Last edited by arojas (2021-06-01 16:16:01)

feinedsquirrel · 2021-06-02 07:26:20

Alright, busy day, finally getting to this.

The ui is built with npm, and the api is built with go. Not sure if that makes a difference.

Per recommendation of arojas (thanks for the tip, makes a lot of sense), I changed the dependency back to blas.

My chroot setup follows the "classic" setup instructions, as I wanted to learn and understand the underpinnings, as opposed to relying on the "convenience way".
I followed the instructions
here (I fully updated my system and rebooted just before, to avoid library mis-matches, and to know the chroot is fully up to date and matched everything on my system):

mkdir ~/chroot
CHROOT=$HOME/chroot
mkarchroot $CHROOT/root base-devel

I did not alter the "makepkg.conf" file, nor did I use a custom "pacman.conf".

Using the default "blas" dependency for "dlib", the output of "makepkg -s" is here:
https://paste.rs/mja

The output of "mkarchroot $CHROOT/root base-devel" is here:
https://paste.rs/S9f

The output of
"makechrootpkg -c -r $CHROOT -I /home/myuser/aursw/dlib_blas/dlib-19.22-1-x86_64.pkg.tar.zst":
https://paste.rs/BD2

For completeness, here (again) is the PKGBUILD I used, with the deps updated to just "blas" and "lapack":
https://paste.rs/8QC

I've heard similar sentiment about go and its dependency situation, as I've started tinkering with hugo recently.

And yes, my biggest worry when I was attempting some of those fixes my first night working on it was that since the photoview author explicitly states "ubuntu" in his manual install instructions, that there would be some annoying small difference in one (or more) of the dependencies that would end up causing issues.

I'm thinking my first angle of attack will probably be to somehow find which go dependency relies on that implicit cblas symbols via libblas. Thank you for the info on that, by the way. That will probably take me a while as I do not know go. (I code scientific algorithms daily in Python, and I used to be mediocre at c++, but it has been years. --- these are to say, I'm not the greatest programmer out there.)

Meanwhile, if either of you see anything in the pasted logs that might help point me in a better direction, let me know. Thanks!

Oh, and also, if it would be helpful to see the logs with it built using "openblas-lapack" from the AUR, where this error doesn't show up, let me know, and I can run those commands and get some more pasted logs.

arojas · 2021-06-02 07:58:30

My guess is that -lcblas is missing from https://github.com/Kagami/go-face/blob/ … face.go#L4

Now, how to "patch" that from the PKGBUILD, I have no idea, since it is one of the dependencies downloaded at build time.

Bevan · 2021-06-02 07:59:45

I have only skimmed over this thread but shouldn't the dependency be cblas instead of blas? Your software requires /usr/lib/libcblas.so.3 which is part of cblas.

The comments in your PKGBUILD suggest that you assume cblas being a dependency of openblas - that is not the case. openblas-lapack from AUR provides cblas which is why it solves your issue as well.

feinedsquirrel · 2021-06-02 16:38:41

Maybe I would have to "patch" photoview to download a fork of go-face that I maintain, with the fix in it? If that indeed fixes the issue, submit a pull request to upstream the fix, perhaps? From the stackoverflow links, I was putting together that I would need to look for -lcblas within go-face, so I'm happy that you mentioned this, so I am more confident that is a good path to look into.

Bevan, even when I attempt with the dependency changed to openblas, which provides blas and cblas (I know cblas is not a dependency, but it does provide it--do I misunderstand something there?Shoot, I must have scrambled something, it doesn't provide cblas), I get the same error. Which makesmade me lean towards a linking error, such as are discussed in several of the stackoverflow link (and other forum links in my original post). ----

--- I just attempted a quick build with cblas included in the dependency list. and the same error appeared. But thank you for pointing that out, because I had somehow thought openblas provided cblas. Must have been the late hour at which I was first attempting all of this. I will need 'cblas' in there for a linking issue (if it is the true cause) to fix it properly.

Bevan · 2021-06-02 16:43:50

feinedsquirrel wrote:

--- I just attempted a quick build with cblas included in the dependency list. and the same error appeared. But thank you for pointing that out, because I had somehow thought openblas provided cblas. Must have been the late hour at which I was first attempting all of this. I will need 'cblas' in there for a linking issue (if it is the true cause) to fix it properly.

Ah, OK. Then it is indeed a more complex linking issue. openblas has cblas in its checkdepends, so maybe that's where the impression of the dependency was coming from.

feinedsquirrel · 2021-06-02 17:51:28

Well, I appreciate you giving me the benefit of the doubt.

I just found this inside of dlib:

dlib/dlib/external/cblas/README wrote:

This folder contains a copy of CBLAS (from http://www.netlib.org/blas/) which
has been setup so you can compile it with CMake. It also only compiles the
part of CBLAS needed by dlib.
Most BLAS libraries come with CBLAS, however, some don't. In particular, if
you are using the BLAS that comes with MATLAB then you will need this CBLAS
code linked into your own to get dlib working with MATLAB's built in BLAS.

Given this, am I correct to think that it is best not to link to that specific version of cblas inside of dlib, but instead to link to cblas available in the official repos (i.e. via pacman)?

So I would need to figure out how to link go-face to use system cblas? Looking at the makefile for go-face, it isn't immediately clear to me how go projects can link to system libraries. If this is correct, and if anyone knows off the top of their head how to do so, I'd appreciate the guidance. Otherwise, I'll be searching the internet (and learning a bit of go[lang]) along the way.

feinedsquirrel · 2021-06-03 11:24:18

https://github.com/Kagami/go-face/blob/master/face.go is indeed missing a "-lcblas" for it to work on Arch. I had to turn this line:

// #cgo LDFLAGS: -ldlib -lblas -llapack -ljpeg

into this line:

// #cgo LDFLAGS: -ldlib -lblas -lcblas -llapack -ljpeg

In order to do so, I forked both go-face and photoview, and added all the rest of the supporting changes everywhere so the PKGBUILD could pull my fork and have everything reference my change. (And yes, I am convinced that the go module dependency system is terrible.)

I mentioned at the end of my first post that I also have a paths issue. I am still running into it, that my copy commands are not working. It still isn't finding the "output/ui" directory:

==> Entering fakeroot environment...
==> Starting package()...
cp: cannot create directory '/build/photoview/pkg/photoview/ouput/ui/': No such file or directory
==> ERROR: A failure occurred in package().
    Aborting...
==> ERROR: Build failed, check /home/myuser/chroot/myuser/build

I tested those copy commands manually, inside the chroot and they seemed to work. I used the expanded values of "pkgdir" and "srcdir" that I echoed in the "build()" function. Any tip on how to resolve that?

arojas · 2021-06-03 11:53:21

feinedsquirrel wrote:

cp -r "${srcdir}/${pkgname}/ui/dist/" "${pkgdir}/ouput/ui/"

You didn't create the output/ui/ subdir. cp doesn't automatically create the target dir tree for you.

Lone_Wolf · 2021-06-03 12:02:16

ouput

Looks like you have a typo in the command .

Last edited by Lone_Wolf (2021-06-03 12:02:43)

feinedsquirrel · 2021-06-03 18:13:34

arojas, I thought that may be what was happening, too, but it is just copying a source dir to a target dir, so as long as "output" exists, it works.

And that was the key, all along. "ouput" didn't exist, but "output" did. Major :facepalm:
Thank you for catching that, Lone_Wolf. Sometimes you just need a fresh pair of eyes. I don't know how many times I re-read that line.

Arch Linux

#1 2021-05-31 22:39:35

[SOLVED] Attempting my first PKGBUILD, error on undefined cblas_dgemm

#2 2021-06-01 13:45:46

Re: [SOLVED] Attempting my first PKGBUILD, error on undefined cblas_dgemm

#3 2021-06-01 16:13:45

Re: [SOLVED] Attempting my first PKGBUILD, error on undefined cblas_dgemm

#4 2021-06-02 07:26:20

Re: [SOLVED] Attempting my first PKGBUILD, error on undefined cblas_dgemm

#5 2021-06-02 07:58:30

Re: [SOLVED] Attempting my first PKGBUILD, error on undefined cblas_dgemm

#6 2021-06-02 07:59:45

Re: [SOLVED] Attempting my first PKGBUILD, error on undefined cblas_dgemm

#7 2021-06-02 16:38:41

Re: [SOLVED] Attempting my first PKGBUILD, error on undefined cblas_dgemm

#8 2021-06-02 16:43:50

Re: [SOLVED] Attempting my first PKGBUILD, error on undefined cblas_dgemm

#9 2021-06-02 17:51:28

Re: [SOLVED] Attempting my first PKGBUILD, error on undefined cblas_dgemm

#10 2021-06-03 11:24:18

Re: [SOLVED] Attempting my first PKGBUILD, error on undefined cblas_dgemm

#11 2021-06-03 11:53:21

Re: [SOLVED] Attempting my first PKGBUILD, error on undefined cblas_dgemm

#12 2021-06-03 12:02:16

Re: [SOLVED] Attempting my first PKGBUILD, error on undefined cblas_dgemm

#13 2021-06-03 18:13:34

Re: [SOLVED] Attempting my first PKGBUILD, error on undefined cblas_dgemm

Board footer