Tool to automate a minimal pip install and utilize Arch packages?

moscow-hunt · 2021-12-16 21:02:12

I understand that pip and pacman are competitors and normally you use one or the other. I've been using Arch for a while, but I have never used pip yet. However, today I am faced with installing a python package that is not in AUR. I looked thru its dependencies and some of them are available as official Arch packages.

From what little I know of pip (having never used it), I believe that if I install this new package with pip, it will pull in all dependencies from pip. I would like to avoid that when some dependencies can be gotten from Arch packages.

I suppose I could always check all dependencies and before running "pip install package" run "pacman -S package" followed by the pip command. Would that work?

My real question is, has anyone created a tool similar to yay to automate this process?

I don't even know if that's possible or a good idea and I'm not a developer, but for those situations where an AUR package is not available and no dev wants to make a PKGBUILD, would a tool like this be a 2nd best option? Does it already exist?

loqs · 2021-12-17 00:04:15

Out of interest what is the package in question?

moscow-hunt · 2021-12-17 01:15:31

loqs wrote:

Out of interest what is the package in question?

It is more of a general question, especially for packages that have a lot of dependencies. Would a tool like what I'm asking about be possible? I'm imaging something like yay or paru that is extended to lastly do a pip install if the package is not available from the official repositories or AUR.

However, to answer your question, this is the package I am interested in installing now:

autotranscode · PyPI.

afaik, it will install Arch Linux - python-tqdm 4.62.3-3 (any) from pip rather than from the official Arch repos.

Trilby · 2021-12-17 01:28:27

moscow-hunt wrote:

I understand that pip and pacman are competitors ...

No, they aren't. They might be described as "competing" with one another if you use pip completely incorrectly. But that simply shouldn't be done.

moscow-hunt wrote:

normally you use one or the other.

Ah ... not sure what this means. You'll definitely use pacman if you are on an arch system. You can additionally use pip to install local python content for a regular user, but not system-wide as root.

moscow-hunt wrote:

However, today I am faced with installing a python package that is not in AUR.

Then make a PKGBUILD for it.

moscow-hunt wrote:

My real question is, has anyone created a tool similar to yay to automate this process?

Probably not for the process you describe, as it's not a very good idea. But I believe I once saw a tool that would auto-generate a PKGBUILD for a cheese-shop package. However, while reasonable, this seems a bit silly as the PKGBUILD would be trivial to write anyways. Just fill out the variables appropriately at the top of a PKGBUILD, then the functions should likely only need to be the following:

build() {
	python setup.py build
}

package() {
	python setup.py install --root="$pkgdir" --optimize=1
}

Last edited by Trilby (2021-12-17 01:32:02)

lmn · 2021-12-17 01:28:48

moscow-hunt wrote:

but for those situations where an AUR package is not available and no dev wants to make a PKGBUILD

There is always the possibility to package it yourself following the python packaging guidlines.

Another way is to use the --user flag of pip to install it into your user scheme. This has the advantage of automatic dependency resolution but comes with the drawback of possibly shadowing a system install.

Would a tool like what I'm asking about be possible?

There are attempts to do this for example pip2pkgbuild, but I cannot speak to its quality.
The problem stems at least from the fragmentation of the python build requirements, PEP518 and PEP517 seem to try to rectify this but in general it is a little bit under standardized.

Edit: fixed links and got sniped by a avionic gumshoe

Last edited by lmn (2021-12-17 01:33:07)

loqs · 2021-12-17 01:40:29

build tested only

# This is an example PKGBUILD file. Use this as a start to creating your own,
# and remove these comments. For more information, see 'man PKGBUILD'.
# NOTE: Please fill out the license field for your package! If it is unknown,
# then please put 'unknown'.

# See http://wiki.archlinux.org/index.php/Python_Package_Guidelines for more
# information on Python packaging.

# Maintainer: Your Name <youremail@domain.com>
pkgname=python-autotranscode
_name=${pkgname#python-}
pkgver=0.9.8
pkgrel=1
pkgdesc="A simple, fast and reliable music library transcoder"
arch=('any')
url="https://pypi.org/project/autotranscode/"
license=('GPL3')
depends=('python' 'python-tqdm' 'ffmpeg')
makedepends=('python-setuptools')
source=(https://files.pythonhosted.org/packages/source/${_name::1}/$_name/$_name-$pkgver.tar.gz)
sha256sums=('23fba3ed040532b212ae188e6e91adef091dbcad27d54b63595b6ed13beee17c')

build() {
  cd $_name-$pkgver
  python setup.py build
}

package() {
  cd $_name-$pkgver
  python setup.py install --root="$pkgdir" --optimize=1 --skip-build
}

Edit:
Added ffmpeg to depends

Last edited by loqs (2021-12-17 01:47:25)

lmn · 2021-12-17 01:46:30

As far as I can see https://pypi.org/project/autotranscode/ depends on ffmpeg, so your package should do that too.

loqs · 2021-12-17 01:48:55

Thank you for spotting that. Added ffmpeg to depends.

bulletmark · 2021-12-17 01:53:57

You can do what you want just using a virtual env:

$ mkdir autotranscode & cd autotranscode
$ python -m venv --system-site-packages venv
$ venv/bin/pip install autotranscode

Then just run it with:

venv/bin/autotranscode -h

Note the venv --system-site-packages option uses python-tqdm and any other dependencies available from what you have installed in the standard Arch packages and the AUR.

Last edited by bulletmark (2021-12-17 02:52:53)

moscow-hunt · 2021-12-17 02:09:59

I will test out the PKGBUILD @loqs and @lmn. This is very helpful.

BTW, the reason I said "I understand that pip and pacman are competitors" is because I had just read eschwartz's post where he said that.

Problem with dependencies on a python package / AUR Issues, Discussion & PKGBUILD Requests / Arch Linux Forums

The only thing which is allowed to handle deps in a pacman package is *pacman*. Only one tool can be in charge. This is EITHER pacman OR pip.
pip is a package manager. It is a competitor of pacman.

I thought that was probably relevant to a question about a tool that would be able to deftly manage both official packages and pip packages. Or maybe I misunderstood eschwartz. Anyway, thank you both for the PKGBUILD! That's the right solution in this case.

moscow-hunt · 2021-12-17 02:11:16

bulletmark wrote:

Note the venv --system-site-packages option uses python-tqdm and any other dependencies available from what you have installed from the AUR.

Good to know!

Arch Linux

#1 2021-12-16 21:02:12

Tool to automate a minimal pip install and utilize Arch packages?

#2 2021-12-17 00:04:15

Re: Tool to automate a minimal pip install and utilize Arch packages?

#3 2021-12-17 01:15:31

Re: Tool to automate a minimal pip install and utilize Arch packages?

#4 2021-12-17 01:28:27

Re: Tool to automate a minimal pip install and utilize Arch packages?

#5 2021-12-17 01:28:48

Re: Tool to automate a minimal pip install and utilize Arch packages?

#6 2021-12-17 01:40:29

Re: Tool to automate a minimal pip install and utilize Arch packages?

#7 2021-12-17 01:46:30

Re: Tool to automate a minimal pip install and utilize Arch packages?

#8 2021-12-17 01:48:55

Re: Tool to automate a minimal pip install and utilize Arch packages?

#9 2021-12-17 01:53:57

Re: Tool to automate a minimal pip install and utilize Arch packages?

#10 2021-12-17 02:09:59

Re: Tool to automate a minimal pip install and utilize Arch packages?

#11 2021-12-17 02:11:16

Re: Tool to automate a minimal pip install and utilize Arch packages?

Board footer