You are not logged in.

#1 2022-06-17 21:49:03

newsboost
Member
Registered: 2016-07-24
Posts: 157

How to fix "setuptools_scm" + importlib.metadata reports version 0.0.0

Hi all,

I've been struggling many hours now with attempting to fix the issue here (at least locally) with a python-package: https://aur.archlinux.org/packages/fortls - my problem:
I have installed this package on 2 pc's - both fully updodate Arch Linux pc's. On one pc, the package installs correctly and the version number is correctly written out (2.9.0). On the other pc - a newer one - the printed version becomes 0.0.0. This is a obviously problem when other programs query for the version number and it reports back 0.0.0... I've struggled for some hours with this and I'm also the one who wrote about this problem here: https://aur.archlinux.org/packages/fortls (today), so there's a slightly different explanation of this issue (and with a bit more screen output than here). It's beyond my package-managing skills to solve myself, so I need to ask and figured I would like to ask here. What I can conclude is that the pyproject.toml tells that setuptools_scm is a dependency - but I cannot install the package with that dependency due to a the following python-exception "LookupError: setuptools-scm was unable to detect version". If I remove "setuptools_scm" then I can install it but on one computer the version is correct, on the other, the newer computer the version becomes 0.0.0 when asked from the commandline: "fortls --version" or using:

 >>> from importlib.metadata import version
 >>> print(version('fortls'))

Since I'm learning about PKGBUILD I attempted to fix the issue locally and I hoped it was enough to insert the missing "fortls/_version.py" (which I think would've been generated by setuptools_scm if that even worked)

diff PKGBUILD_ PKGBUILD
17a18,22
>    echo '# coding: utf-8
> # file generated by setuptools_scm
> # don't change, don't track in version control
> version = '2.9.0'
> version_tuple = (2, 9, 0)' > fortls/_version.py

You can see how this fits into the original PKGBUILD at https://aur.archlinux.org/cgit/aur.git/ … D?h=fortls but I'll also just explain it quickly here: I just inserted these lines in the "build() { ... }" section, just after cd "$pkgname-$pkgver" and just before python setup.py build - hoping that would be enough so I could install the package locally and be happy... But it didn't make any difference... The source code for fortls-2.9.0-1-any.pkg.tar.zst is very small (193 kb) so it's a small project that is quick to compile/test.

I can add that I also tried something different and perhaps a bit crazy (but if it worked, I would be happy):

1) I took the fortls-2.9.0-1-any.pkg.tar.zst and extracted it to a TEMP-folder using "tar --use-compress-program=unzstd -xvf fortls-2.9.0-1-any.pkg.tar.zst"
2) Inside the subdirectory of the TEMP-folder: "TEMP/usr/lib/python3.10/site-packages/" I renamed the folder: "fortls-0.0.0-py3.10.egg-info" to "fortls-2.9.0-py3.10.egg-info", so the version number is correct.
3) Inside the file: TEMP/usr/lib/python3.10/site-packages/fortls-2.9.0-py3.10.egg-info/PKG-INFO I replaced "Version: 0.0.0" with the correct "Version: 2.9.0".
4) Then I removed the original .zst package and created a new one: "tar --use-compress-program zstd -cf /tmp/_fortls-2.9.0-1-any.pkg.tar.zst . "
5) And then I tried installing that new package instead: "sudo pacman -U fortls-2.9.0-1-any.pkg.tar.zst"
6) Unfortunately - that also didn't work:

loading packages...
error: missing package metadata in fortls-2.9.0-1-any.pkg.tar.zst
error: 'fortls-2.9.0-1-any.pkg.tar.zst': invalid or corrupted package

I should also mention that there's an open ticket: https://github.com/gnikit/fortls/issues/117 - but I don't need the "right solution" - any hack that'll do it is fine for me, just to learn a bit... The alternative - and perhaps even better solution - although typically not recommended for Arch Linux AFAIK - is to just use "pip install"... I would be interesting to learn something new here so I hope some of you think this is also interesting. In order to fix the PKGBUILD file it's necessary to first understand the problem and why the version number for the same package is different for 2 different machines both running up-to-date Arch Linux. Anyone has any input, ideas, suggestions etc to this?

Last edited by newsboost (2022-06-17 22:16:09)

Offline

#2 2022-06-18 17:08:14

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 11,868

Re: How to fix "setuptools_scm" + importlib.metadata reports version 0.0.0

From this post and the aur comments it looks like all build attempts were with yay ?

Have you tried building with makepkg or  in a clean chroot to verify whether yay is part of the problem ?


Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.


(A works at time B)  && (time C > time B ) ≠  (A works at time C)

Offline

#3 2022-06-18 17:20:51

WorMzy
Forum Moderator
From: Scotland
Registered: 2010-06-16
Posts: 11,783
Website

Re: How to fix "setuptools_scm" + importlib.metadata reports version 0.0.0

Mod note: moving to AUR Issues


Sakura:-
Mobo: MSI MAG X570S TORPEDO MAX // Processor: AMD Ryzen 9 5950X @4.9GHz // GFX: AMD Radeon RX 5700 XT // RAM: 32GB (4x 8GB) Corsair DDR4 (@ 3000MHz) // Storage: 1x 3TB HDD, 6x 1TB SSD, 2x 120GB SSD, 1x 275GB M2 SSD

Making lemonade from lemons since 2015.

Online

#4 2022-06-18 19:25:11

newsboost
Member
Registered: 2016-07-24
Posts: 157

Re: How to fix "setuptools_scm" + importlib.metadata reports version 0.0.0

Lone_Wolf wrote:

From this post and the aur comments it looks like all build attempts were with yay ?

Have you tried building with makepkg or  in a clean chroot to verify whether yay is part of the problem ?

Yes, I tried mostly with "makepkg" - same problem. I was not aware of https://wiki.archlinux.org/title/Develo … ean_chroot so I actually also just tried that - but it again resulted in the same issue, that the python executable will report that the version number is 0.0.0. At this point, having thought some more about this, it is not critical for me if I figure out this issue - because I think this answer makes really good sense too: https://github.com/gnikit/fortls/issues … 1130776015 - but it contradicts the usual way of thinking that we should not use "pip install" (been there, done that and also got messed up my system earlier so now I *really* try hard to avoid pip install - but "pip install --user" sounds like an acceptable compromise, if I don't come up with anything else...

Offline

#5 2022-06-19 12:31:20

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 11,868

Re: How to fix "setuptools_scm" + importlib.metadata reports version 0.0.0

I looked further into this and noticed the wheel dependency mentioned in https://github.com/gnikit/fortls/issues … 1130559525 looked familiar .

I checked Python package guidelines and found wheel in the 'Standards based (PEP 517)' section .

Although fortls documentation doesn't appear to state it explicitly, fortls does follow PEP 517 .
Possible solution : adjust the PKGBUILD to follow PEP 517 .

Below is a PKGBUILD that does that.
It does build in a clean chroot without errors, but keep in mind it's  a quick & dirty version and needs testing.


# Maintainer: Colin Woodbury <colin@fosskers.ca>

pkgname=fortls
pkgver=2.9.0
pkgrel=1
pkgdesc="A modern Language Server for Fortran."
arch=(any)
url="https://github.com/gnikit/fortls"
license=("MIT")
makedepends=(python-build python-installer python-wheel python-setuptools-scm python-setuptools-scm-git-archive)
depends=(python)
provides=("fortran-language-server")
conflicts=("fortran-language-server")
source=("https://files.pythonhosted.org/packages/source/${pkgname::1}/$pkgname/$pkgname-$pkgver.tar.gz")
# source=("$pkgname-$pkgver::https://github.com/gnikit/$pkgname/archive/refs/tags/v$pkgver.tar.gz")
sha256sums=('c17aa80de02f19ccb8c00d23dfb72965a5ba0c8db8dec34ee1049449836415f6')

build() {
    cd "$pkgname-$pkgver"
    python -m build --wheel --no-isolation
}

#	cd "$pkgname-$pkgver"
#	python setup.py build
#}

package() {
    cd "$pkgname-$pkgver"
    python -m installer --destdir="$pkgdir" dist/*.whl

#	cd "$pkgname-$pkgver"
#	python setup.py install --root="$pkgdir/" --optimize=1 --skip-build

	# License
	install -Dm644 LICENSE "$pkgdir"/usr/share/licenses/$pkgname/LICENSE
}

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.


(A works at time B)  && (time C > time B ) ≠  (A works at time C)

Offline

#6 2022-06-19 14:35:30

newsboost
Member
Registered: 2016-07-24
Posts: 157

Re: How to fix "setuptools_scm" + importlib.metadata reports version 0.0.0

Lone_Wolf wrote:

I looked further into this and noticed the wheel dependency mentioned in https://github.com/gnikit/fortls/issues … 1130559525 looked familiar .

I checked Python package guidelines and found wheel in the 'Standards based (PEP 517)' section .

Although fortls documentation doesn't appear to state it explicitly, fortls does follow PEP 517 .
Possible solution : adjust the PKGBUILD to follow PEP 517 .

Sorry, I don't understand this: Doesn't "fortls does follow PEP 517" contradict with "Below is a PKGBUILD that does that" (i.e. follow PEP 517) ? Maybe/probably you meant that "fortls does *NOT* follow PEP 517", but below is a PKGBUILD that does ?

Lone_Wolf wrote:

Below is a PKGBUILD that does that.
It does build in a clean chroot without errors, but keep in mind it's  a quick & dirty version and needs testing.

It does work - at least on one of my pc's thanks! You also replaced the source, which is exactly what I thought was needed. And then you're building using "python -m build --wheel --no-isolation" instead of "python setup.py build", that's interesting (I'm not into the details of these things but I'm learning...). I noticed that there already exists a wheel-file https://files.pythonhosted.org/packages … ne-any.whl which I/we probably could also have used, directly, isn't that right? You also modified the package() function so it's a bit different... Anyway, I'll experiment with this and study this for a while and learn new thanks, so thanks a lot for showing a "working" PKGBUILD-file which I can learn from (this is exactly what I wanted)!

Offline

#7 2022-06-19 15:02:50

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 11,868

Re: How to fix "setuptools_scm" + importlib.metadata reports version 0.0.0

Although fortls documentation doesn't appear to state it explicitly, fortls does follow PEP 517 .
Possible solution : adjust the PKGBUILD to follow PEP 517 .

I should have phrased that differently, trying again.

"Fortls documentation doesn't mention PEP 517 afaik, but the contents of its pyproject.toml are compliant with the PEP 517 standard ."


PEP 517 standard requires certain metadata from directories used by git , and github source tarballs don't include that.
Tarballs on pythonhosted.org do include that metadata.

The wheel file could have been used directly, but there's a big red warning on the python guidelines page that should only be used if there's no source file available .

Also I very much prefer packages that build from source.

Last edited by Lone_Wolf (2022-06-19 15:04:40)


Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.


(A works at time B)  && (time C > time B ) ≠  (A works at time C)

Offline

#8 2022-06-19 18:37:14

newsboost
Member
Registered: 2016-07-24
Posts: 157

Re: How to fix "setuptools_scm" + importlib.metadata reports version 0.0.0

Lone_Wolf wrote:

PEP 517 standard requires certain metadata from directories used by git , and github source tarballs don't include that.
Tarballs on pythonhosted.org do include that metadata.

Yes, it is that metadata that is the problem, that was also my conclusion. Still, I have a pc where it works - and another pc where it doesn't work. So it's really a conundrum in my opinion: Non-deterministic, probably not random - but I don't understand why it works on one of my machines, but not the other... I'll probably never figure it out (and it's also not that important, but I would be happy to see if anyone has such great insights that they understand the reason, probably some old database-/metadata-stuff somewhere?)...

Lone_Wolf wrote:

The wheel file could have been used directly, but there's a big red warning on the python guidelines page that should only be used if there's no source file available .

Also I very much prefer packages that build from source.

I was (am?) in the exact opposite boat. Can I ask, where do you see that big red warning, do you have a link? The reason I'm asking is because I came across something similar to this (not sure if this is the original source I read a while ago, but at least it was similar): https://www.activestate.com/resources/q … an%20sdist. I quote from that page:

.... According to the Python Packaging Authority (PyPA), wheels are the preferred way that pip installs Python modules from the Python Package Index (PyPI) because they’re smaller, faster to install, and more efficient than building the package from the source code contained in an sdist.

I suspect there are similar guidelines somewhere on the internet that contradict building from source, although I'm not 100% sure and am relatively new to building packages, so I must ask: Am I wrong (I'm happy to learn, this is just the opposite of what I thought was "recommended", so I need to ask)?

Offline

#9 2022-06-20 09:57:28

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 11,868

Re: How to fix "setuptools_scm" + importlib.metadata reports version 0.0.0

There are multiple ways to use python stuff and there's no "one size fits everyone" solution .
Also usecase, environment and personal preferences play a big role.

The activestate.com page you quoted only mentions pip and none of the other available methods.

Archlinux wiki[1] has a short overview of available methods to install python stuff on archlinux and some usage notes.

quoting from it :

When installing packages using pip, it is recommended to use a virtual environment to prevent conflicts with system packages in /usr. Alternatively, pip install --user can be used to install packages into the user scheme instead of /usr. pipx and Conda integrate environment management into their workflows.

The activestate.com linked page doesn't mention anything about potential conflicts with system packages.
That strongly suggests they assume there is no systemwide python installation on systems used by their target audience.

There is at least one OS where that assumption is very likely to be true.
On archlinux (and probably all linux distros) however that assumption is false.
This indicates the activestate.com advise may be applicable for archlinux python users that have decided to use pip.

This thread is about issues with an archlinux aur package and best practices for those are found in archlinux wiki .

General packaging guidelines are at [2], and there are lots of pages dealing with specific types of packages.
The relevant page for archlinux python pacakges is at [3] (most of the things I posted in this thread are based on that page).

The first red box on [3] deals with .whl files or building from source.

TL;DR :

If an archlinux user decides pip is the way to go, the advise on the activestate.com page may be valid .
For an AUR package it's not.


[1] https://wiki.archlinux.org/title/Python … management
[2] https://wiki.archlinux.org/title/Arch_p … guidelines
[3] https://wiki.archlinux.org/title/Python … guidelines

Last edited by Lone_Wolf (2022-06-20 10:00:01)


Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.


(A works at time B)  && (time C > time B ) ≠  (A works at time C)

Offline

#10 2022-06-20 19:28:35

newsboost
Member
Registered: 2016-07-24
Posts: 157

Re: How to fix "setuptools_scm" + importlib.metadata reports version 0.0.0

Lone_Wolf wrote:

There are multiple ways to use python stuff and there's no "one size fits everyone" solution .

Ok, thanks a lot for all the great links you provided - I just created some PDFs and will read them closer another day (mostly it's in the weekends I got time). I think you/we misunderstood each other earlier, I'll explain about that in the bottom of this answer...

Lone_Wolf wrote:

General packaging guidelines are at [2], and there are lots of pages dealing with specific types of packages.
The relevant page for archlinux python pacakges is at [3] (most of the things I posted in this thread are based on that page).

The first red box on [3] deals with .whl files or building from source.

TL;DR :

If an archlinux user decides pip is the way to go, the advise on the activestate.com page may be valid .
For an AUR package it's not.


[1] https://wiki.archlinux.org/title/Python … management
[2] https://wiki.archlinux.org/title/Arch_p … guidelines
[3] https://wiki.archlinux.org/title/Python … guidelines

Right. And here's where I think the misunderstanding is. The quote I was referring to, when I wrote that I preferred wheels from pypi is:

.... According to the Python Packaging Authority (PyPA), wheels are the preferred way that pip installs Python modules from the Python Package Index (PyPI) because they’re smaller, faster to install, and more efficient than building the package from the source code contained in an sdist.

I'm not talking about using a virtual environment or not. I'm talking about distributing a PKGBUILD which (inside a virtual environment) either builds everything from source - or uses the wheel as source. Since wheels are (I quote) "the preferred way that pip installs Python modules from the Python Package Index (PyPI) because they’re smaller, faster to install, and more efficient than building the package from the source code", I don't see why people shouldn't use wheel instead of building from source, when people make their PKGBUILD-files. In fact I did it myself, you can see it here: https://aur.archlinux.org/cgit/aur.git/ … -stego-lsb (my first and only package so far). There are no "pip" conflicts with conflicting system files as there would/could be if you just ran "pip install (package)" - the makepkg system should take care of not overwriting files and also clean everything up, when uninstalling the package (which can either be based on a built from source code or from wheel)... I think you misunderstood it because you didn't think of the situation where "pip install" runs inside the virtual environment to provide the normal securities (and exactly the same, we get as when building from source) from the package system.

You make it sound like the difference is to either:
1) use PKGBUILD and then build from source (inside virtual env) or
2) use "pip install" (outside virtual env)

That's not what I'm talking about. I'm talking about PKGBUILD the whole way and either build from source or use wheels, inside the virtual environment ("makepkg")... I prefer the latter... Anyway, I have not (yet) read all the links you provided, but will take a much deeper look into that later so thanks a lot for posting them, I can see there are lot's of things to learn for me still.

Offline

Board footer

Powered by FluxBB