Python not detecting packages installed in the system

evorster · 2025-01-04 08:32:20

Hi there!

I'm trying to make a package for a new dependency of kdenlive-git, which I maintain.

Here is the contents of the PKGBUILD so far:

 This is an example PKGBUILD file. Use this as a start to creating your own,
# and remove these comments. For more information, see 'man PKGBUILD'.
# NOTE: Please fill out the license field for your package! If it is unknown,
# then please put 'unknown'.

# Maintainer: Your Name <superchief@evertvorster.com>
pkgname=python-sam2-git
pkgver=r76.2b90b9f
pkgrel=1
pkgdesc="Facebook Research AI to Segment Anything in Images and Videos"
arch=('x86_64')
url="https://github.com/facebookresearch/sam2"
license=('Apache-2.0')
groups=()
depends=()
makedepends=('git' 'python-fsspec')
provides=("${pkgname%-git}")
conflicts=("${pkgname%-git}")
replaces=()
backup=()
options=()
install=
source=('git+https://github.com/facebookresearch/sam2.git')
noextract=()
sha256sums=('SKIP')

# Please refer to the 'USING VCS SOURCES' section of the PKGBUILD man page for
# a description of each element in the source array.

pkgver() {
        cd "$srcdir/sam2"

# Git, tags available
#       printf "%s" "$(git describe --long | sed 's/\([^-]*-\)g/r\1/;s/-/./g')"

# Git, no tags available
        printf "r%s.%s" "$(git rev-list --count HEAD)" "$(git rev-parse --short HEAD)"
}

prepare() {
        cd "$srcdir/sam2"
#       patch -p1 -i "$srcdir/${pkgname%-VCS}.patch"
}

build() {
echo "Building phase"
        cd "$srcdir/sam2"
        python -m build --wheel --no-isolation
}

package() {
        cd "$srcdir/sam2"
        python -m installer --destdir="$pkgdir" dist/*.whl
}

When makepkg on this PKGBUILD is run, this is the output:

[evert@Evert python-sam2-git]$ makepkg
==> Making package: python-sam2-git r76.2b90b9f-1 (2025-01-04T10:22:57 CAT)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
  -> Updating sam2 git repo...
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (6/6), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 9 (delta 2), reused 2 (delta 2), pack-reused 3 (from 1)
Unpacking objects: 100% (9/9), 17.35 KiB | 986.00 KiB/s, done.
From https://github.com/facebookresearch/sam2
 + a5b6a9a...e853b74 refs/pull/268/merge -> refs/pull/268/merge  (forced update)
==> Validating source files with sha256sums...
    sam2 ... Skipped
==> Extracting sources...
  -> Creating working copy of sam2 git repo...
Reset branch 'makepkg'
==> Starting prepare()...
==> Starting pkgver()...
==> Removing existing $pkgdir/ directory...
==> Starting build()...
Building phase
* Getting build dependencies for wheel...
running egg_info
writing SAM_2.egg-info/PKG-INFO
writing dependency_links to SAM_2.egg-info/dependency_links.txt
writing requirements to SAM_2.egg-info/requires.txt
writing top-level names to SAM_2.egg-info/top_level.txt
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
adding license file 'LICENSE_cctorch'
writing manifest file 'SAM_2.egg-info/SOURCES.txt'

ERROR Missing dependencies:
        torch>=2.5.1
        sympy==1.13.1
==> ERROR: A failure occurred in build().
    Aborting...

I have both python-torchvision-cuda and python-pytorch-cuda installed, as well as python-sympy.
To me it looks like the build script of this package is not detecting the installed packages, but it is detecting others.

How do I make this work?

gromit · 2025-01-04 09:38:43

Which versions of torch and sympy do you have installed? Also note that it asks for torch and not for torchvision

evorster · 2025-01-04 09:50:41

Hi there, Gromit.
Thanks for taking a look!
I'm assuming python-pytorch provides torch.

evert@Evert ~]$ yay -Ss python-pytorch-cuda
extra/python-pytorch-opt-cuda 2.5.1-7 (530.7 MiB 2.1 GiB) 
    Tensors and Dynamic neural networks in Python with strong GPU acceleration (with CUDA and AVX2 CPU optimizations)
extra/python-pytorch-cuda 2.5.1-7 (529.7 MiB 2.1 GiB) (Installed)
    Tensors and Dynamic neural networks in Python with strong GPU acceleration (with CUDA)

And python-sympy should provide sympy:

evert@Evert ~]$ yay -Ss python-sympy
aur/python-sympy-docs 1.13.3-1 (+1 0.00) 
    Documentation for Python Sympy module.
aur/python-sympy-git 1.6.r1720.ge32abe4b08-1 (+0 0.00) 
    Computer algebra system (CAS) in Python, written in pure Python - git version
extra/python-sympy 1.13.3-2 (13.5 MiB 93.3 MiB) (Installed)
    Symbolic manipulation package (Computer Algebra System), written in pure Python

gromit · 2025-01-04 10:14:27

Did you already try to build your package in a clean chroot? https://wiki.archlinux.org/title/Develo … nience_way

Just to make sure that pip installed dependencies don't play into this ..

evorster · 2025-01-04 11:20:44

I don't have pip installed on my system, so no pip installed dependencies. I prefer to have Arch manage the packages rather than python.

Installing into a clean root is a pain, for me at least, because sometimes a package relies on another package that is also in the AUR, and the tools don't resolve that.
Is there an easy way of pulling in dependencies that are in the AUR?

evorster · 2025-01-04 11:23:27

Small correction, it appears I do have something installed that provides pip.
pip list also shows up both torch and sympy, at the correct versions.

However, since a while ago pip is not able to install anything on the system, because the python packages are managed by pacman/yay

Last edited by evorster (2025-01-04 11:24:34)

evorster · 2025-01-04 11:38:48

Thanks for the response, gromit.

I tried using pkgctl build. I also updated the dependencies inside the PKGBUILD to have python-pytorch and python-sympy.

The failure still looks remarkably similar:

==> Starting build()...
Building phase
* Getting build dependencies for wheel...
CUDA_HOME environment variable is not set. Please set it to your CUDA install root.

Failed to build the SAM 2 CUDA extension due to the error above. You can still use SAM 2 and it's OK to ignore the error above, although some post-processing functionality may be limited (which doesn't affect the results in most cases; (see https://github.com/facebookresearch/sam2/blob/main/INSTALL.md).

running egg_info
creating SAM_2.egg-info
writing SAM_2.egg-info/PKG-INFO
writing dependency_links to SAM_2.egg-info/dependency_links.txt
writing requirements to SAM_2.egg-info/requires.txt
writing top-level names to SAM_2.egg-info/top_level.txt
writing manifest file 'SAM_2.egg-info/SOURCES.txt'
reading manifest file 'SAM_2.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
adding license file 'LICENSE_cctorch'
writing manifest file 'SAM_2.egg-info/SOURCES.txt'

ERROR Missing dependencies:
        torch>=2.5.1
        sympy==1.13.1
==> ERROR: A failure occurred in build().
    Aborting...
==> ERROR: Build failed, check /var/lib/archbuild/extra-x86_64/evert-1/build

gromit · 2025-01-04 12:00:06

Do you have a dependency on cuda set? The build complains about CUDA_HOME being missing!

evorster · 2025-01-04 13:02:55

I set cuda as a dependency, and tried again.

After cuda has installed, it still complains about CUDA_HOME, and still does not find torch or sympy.

Is there a way that the locations of torch and sympy can be specified in the PKGBUILD?

Lone_Wolf · 2025-01-04 13:15:17

According to https://github.com/facebookresearch/sam … n/setup.py there are more dependencies needed .

REQUIRED_PACKAGES = [
    "torch>=2.5.1",
    "torchvision>=0.20.1",
    "numpy>=1.24.4",
    "tqdm>=4.66.1",
    "hydra-core>=1.3.2",
    "iopath>=0.1.10",
    "pillow>=9.4.0",
]

Better add all of them to the PKGBUILD.

evorster · 2025-01-04 13:22:32

And this is where I run into a problem. Both python-hydra-core and python-iopath are AUR packages.
How do I tell pkgctl to use packages from AUR?

Lone_Wolf · 2025-01-04 13:38:11

You have to built them first, then add the binary directly to the pkgctl command using -I / -install-to-chroot

man pkgctl-build wrote:

INSTALL OPTIONS
-I, --install-to-chroot FILE
Install a package to the working copy of the chroot

evorster · 2025-01-04 14:15:55

This little rabbit hole is going down deeper than I thought.
python-iopath requires python-portalocker which is also an AUR package.
Unfortunately, this package has an issue installing into the chroot.

Checking PKGBUILD
Checking python-portalocker-3.0.0-1-any.pkg.tar.zst
python-portalocker W: Referenced python module 'winerror' is an uninstalled dependency (needed in files ['usr/lib/python3.13/site-packages/portalocker/portalocker.py'])
python-portalocker W: Referenced python module 'win32con' is an uninstalled dependency (needed in files ['usr/lib/python3.13/site-packages/portalocker/portalocker.py'])
python-portalocker W: Referenced python module 'win32file' is an uninstalled dependency (needed in files ['usr/lib/python3.13/site-packages/portalocker/portalocker.py'])
python-portalocker W: Referenced python module 'msvcrt' is an uninstalled dependency (needed in files ['usr/lib/python3.13/site-packages/portalocker/portalocker.py', 'usr/lib/python3.13/site-packages/portalocker/constants.py'])
python-portalocker W: Referenced python module 'pywintypes' is an uninstalled dependency (needed in files ['usr/lib/python3.13/site-packages/portalocker/portalocker.py'])
python-portalocker W: Dependency python-redis detected but optional (python modules ['redis'] needed in files ['usr/lib/python3.13/site-packages/portalocker/redis.py'])

I have tried again outside of the chroot with the new dependencies installed.
Unfortunately, it still fails to find torch and sympy

Lone_Wolf · 2025-01-05 10:38:05

The setup.py looks very different from the pyproject.toml file, there's a possibilty upstream support for PEP 517 standards is not working as intended.

Try building with setup.py , see https://wiki.archlinux.org/title/Python … _distutils .

Arch Linux

#1 2025-01-04 08:32:20

Python not detecting packages installed in the system

#2 2025-01-04 09:38:43

Re: Python not detecting packages installed in the system

#3 2025-01-04 09:50:41

Re: Python not detecting packages installed in the system

#4 2025-01-04 10:14:27

Re: Python not detecting packages installed in the system

#5 2025-01-04 11:20:44

Re: Python not detecting packages installed in the system

#6 2025-01-04 11:23:27

Re: Python not detecting packages installed in the system

#7 2025-01-04 11:38:48

Re: Python not detecting packages installed in the system

#8 2025-01-04 12:00:06

Re: Python not detecting packages installed in the system

#9 2025-01-04 13:02:55

Re: Python not detecting packages installed in the system

#10 2025-01-04 13:15:17

Re: Python not detecting packages installed in the system

#11 2025-01-04 13:22:32

Re: Python not detecting packages installed in the system

#12 2025-01-04 13:38:11

Re: Python not detecting packages installed in the system

#13 2025-01-04 14:15:55

Re: Python not detecting packages installed in the system

#14 2025-01-05 10:38:05

Re: Python not detecting packages installed in the system

Board footer