[SOLVED] Update to Python 3.9 and aurutils repos

sshaikh · 2020-12-02 20:55:26

But I'm hoping this is different (and so, interesting) enough to warrant a new thread.

keepmenu (AUR: https://aur.archlinux.org/packages/python-keepmenu-git) returns the following:

❯ keepmenu
Traceback (most recent call last):
  File "/sbin/keepmenu", line 27, in <module>
    from pynput import keyboard
ModuleNotFoundError: No module named 'pynput'

Using tips at https://bbs.archlinux.org/viewtopic.php?id=248548 I see the following:

❯ pacman -Qs pynput
local/python-pynput 1.7.1-1
    Python library to monitor and control user input devices

❯ python -c "import sys; print(sys.path)"
['', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '/usr/lib/python3.9/site-packages']

❯ ls /usr/lib/python3.9/site-packages | grep pynput

❯ pacman -Qkk 1>/dev/null

#no files missing, no reference to pynput or python

I have done two major things since the last time keepmenu was working - switched to using aurutils and just now what seemed to be a big pacman upgrade that included python 3.9. I suspect it's the latter which has done this?

Last edited by sshaikh (2020-12-03 12:48:59)

sshaikh · 2020-12-02 20:59:06

While writing this post I got an alert for the following on reddit: https://www.reddit.com/r/archlinux/comm … on_python/

Rebuilding my aurutils pynput package did the trick and keepmenu works now.

Since nothing had changed with the PKGBUILD or version of pynput, I did this by increasing the pkgrel of the PKGBUILD. Is that the correct way to handle this?

I presume a rebuild with the AUR PKGBUILD would have been forced reinstalled so no pkgrel needed, so it seems this is a particular issue with using a local repo setup?

EDIT: this might be better moved to another subforum?

Last edited by sshaikh (2020-12-02 21:00:20)

Trilby · 2020-12-02 21:05:06

sshaikh wrote:

Since nothing had changed with the PKGBUILD or version of pynput, I did this by increasing the pkgrel of the PKGBUILD. Is that the correct way to handle this?

No. Just rebuild. Although by changing the pkgrel you circularly justified changing the pkgrel. The pkgrel should only be incremented when the PKGBUILD has changed.

Now changing the pkgrel on your own is harmless in the short term - and it's quite possible it will remain harmless. But it's also possible that the maintainer will make a real change in the PKGBUILD, then increment the pkgrel, but as you've already incremented the pkgrel, any automation tools you use would not detect this as a new version.

In the grand scheme of things it's pretty harmless either way, but ideally you should have just rebuilt the package without any change to the pkgrel.

EDIT: actually, I'm rethinking my answer. In the main repos, pkgrel is incremented for this type of rebuild. But in these cases, the maintainer increments it, so any future changes will futher increment. In this case, my concern was you doing this yourself if the AUR maintainer wasn't also intending to do so.

Last edited by Trilby (2020-12-02 21:07:12)

sshaikh · 2020-12-02 21:18:10

Yes - I had the same thoughts as you did. I'm not the owner of the PKGBUILD, but I do own a repo and I suppose technically I should maintain a history of binaries that I've built and used - it also allows pacman to see them as a genuine upgrade, rather than having to force a reinstall.

My understanding is that the pkgrel changes if the PKGBUILD change would result in a new binary. Of course that isn't the case here, but the binary does change.

The issue is that this is a third version that needs maintaining (after that of the PKGBUILD and pynput library). My guess is that given that Arch doesn't support partial upgrades that the maintainers of Python PKGBUILDs should "force" a new binary each time Python is updated, and the pkgrel is a (the only?) way to do that, and I kinda "emulated" that ahead of the maintainer. I suppose technically I should be forking the PKGBUILD by that point but that makes my head hurt.

OTOH it seems odd that the maintainer should have to do it, so I guess this is actually another special case of PKGBUILD like VCS's.

For anyone else who has found their way here I used the following to track what I needed to rebuild:

pacman -Qoq /usr/lib/python3.8

Last edited by sshaikh (2020-12-02 21:19:01)

progandy · 2020-12-02 22:48:59

If you have to / want to increment the pkgrel of a package someone else maintains you could add a second component to the pkgrel, e.g. modified upstream v1-1 becomes local v1-1.1, modified v1-2 becomes v1-2.1.
That is a scheme used by e.g. Manjaro and some others do it in their private repositories (e.g. eschwartz)

sshaikh · 2020-12-03 12:48:40

That's a good idea. I didn't realise the pkgrel could be composite. I'm guessing then that "new" is defined by some kind of sort?

Just out of interest, is there a case of making the python version the major of the library? So for example https://www.archlinux.org/packages/comm … -pykeepass at 3.2.1-3 (for 3.8) becomes 3.9-3.2.1-1 instead of 3.2.1-4

Then the PKGBUILD would specifically install the version that would work with the python specified.

Come to think of it, how does this even work in the official repos? My understanding is that pacman doesn't guarantee order of installation, so what happens if a bumped library package is installed (at 3.8, say) before python is upgraded (to 3.9)?

Last edited by sshaikh (2020-12-03 13:08:30)

Trilby · 2020-12-03 13:18:03

sshaikh wrote:

Come to think of it, how does this even work in the official repos? My understanding is that pacman doesn't guarantee order of installation, so what happens if a bumped library package is installed (at 3.8, say) before python is upgraded (to 3.9)?

Doesn't it? There are occasional warnings for circular dependencies indicating that X will be installed before it's Y dependency. This is harmless as long as it's in the same transaction, and also implies that in the absence of circular dependencies, pacman DOES sort the installation order according to dependencies.

But it's mostly a non-issue anyways as all hooks are run after all packages are installed (I'm less sure of post-install scripts, but this may be a reason to favor hooks over post-install scripts). In the case of library dependencies, though, and in the absence of post-install scripts, the installation order really is irrelevant: the dependent package will not be able to run properly without the dependency, but it can be installed just fine.

Last edited by Trilby (2020-12-03 13:22:45)

sshaikh · 2020-12-03 16:46:40

To be honest when I wrote the above I was conflating the ABS with the official repos (and not the first time). After getting into the habit of building my own packages I keep thinking that's what happens with pacman too. But of course the problem I'm describing is exactly what happens with the AUR.

eschwartz · 2020-12-04 05:00:11

sshaikh wrote:

That's a good idea. I didn't realise the pkgrel could be composite. I'm guessing then that "new" is defined by some kind of sort?

Per man 5 PKGBUILD:

pkgrel
This is the release number specific to the distribution. This allows package maintainers to make updates to the package’s configure flags, for example. This is typically set to 1 for each new upstream software release and incremented for intermediate PKGBUILD updates. The variable is a positive integer, with an optional subrelease level specified by adding another positive integer separated by a period (i.e. in the form x.y).

We deliberately include a hard error in makepkg if you try to use a pkgrel that does not match the regular expression "[0-9]+(\.[0-9]+)?", though pacman will accept it if you produce a package using something other than makepkg. And some other distros, like Parabola, patch pacman to permit pkgrel="1.parabola1" to denote who did the build.

Trilby wrote:

sshaikh wrote:
Come to think of it, how does this even work in the official repos? My understanding is that pacman doesn't guarantee order of installation, so what happens if a bumped library package is installed (at 3.8, say) before python is upgraded (to 3.9)?
Doesn't it? There are occasional warnings for circular dependencies indicating that X will be installed before it's Y dependency. This is harmless as long as it's in the same transaction, and also implies that in the absence of circular dependencies, pacman DOES sort the installation order according to dependencies.

Right, this is even needed (and relied on) for install scriptlets which run immediately before/after the per-package transaction.

Trilby wrote:

But it's mostly a non-issue anyways as all hooks are run after all packages are installed (I'm less sure of post-install scripts, but this may be a reason to favor hooks over post-install scripts). In the case of library dependencies, though, and in the absence of post-install scripts, the installation order really is irrelevant: the dependent package will not be able to run properly without the dependency, but it can be installed just fine.

Hooks are "intended" for anything that gets triggered by multiple packages. However, it could be used to circumvent circular deps, since they run at the end, not per-package. Generally, ordering should be fine though. And hooks might require installing a custom script onto the filesystem unless they're extremely simple and can fit into an execve.

Arch Linux

#1 2020-12-02 20:55:26

[SOLVED] Update to Python 3.9 and aurutils repos

#2 2020-12-02 20:59:06

Re: [SOLVED] Update to Python 3.9 and aurutils repos

#3 2020-12-02 21:05:06

Re: [SOLVED] Update to Python 3.9 and aurutils repos

#4 2020-12-02 21:18:10

Re: [SOLVED] Update to Python 3.9 and aurutils repos

#5 2020-12-02 22:48:59

Re: [SOLVED] Update to Python 3.9 and aurutils repos

#6 2020-12-03 12:48:40

Re: [SOLVED] Update to Python 3.9 and aurutils repos

#7 2020-12-03 13:18:03

Re: [SOLVED] Update to Python 3.9 and aurutils repos

#8 2020-12-03 16:46:40

Re: [SOLVED] Update to Python 3.9 and aurutils repos

#9 2020-12-04 05:00:11

Re: [SOLVED] Update to Python 3.9 and aurutils repos

Board footer