You are not logged in.

#1 2020-08-21 16:19:08

galvez_65
Member
Registered: 2018-06-29
Posts: 20

packing python software with a virtualenv

Hi all, I'm looking for guidance, suggestions, best practices for creating packages for programs written in python, specifically around the use of virtualenvs.  I've looked over the python packaging best practices  and they make perfect sense for pure python packages and for packages where all the dependencies are already in the repos or would be safe to install globally.  I'm writing a program which is not a pure python program, I'm not sure I want all the dependencies globally installed, and and I would like to lock down some of the dependency versions. This is exactly the kind of use case which calls for a virtualenv. What I ended up doing was in the postinstall hook of the $pkgname.instal file to create a virtualenv and then pip install my dependencies from an requirement.txt file.

What I'm looking for 1) does this sound reasonable? and 2) are they any best practices around using a python virtualenv in building an arch package. 

thank in advance for any advise

Offline

#2 2020-08-21 20:15:23

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: packing python software with a virtualenv

Please do NOT do this.

It's utterly forbidden to pip install in the postinstall hook of the $pkgname.install as that doesn't even constitute a package, pacman must be able to track the files and you cannot YOLO download and compile source code during pacman -Syu or pacman -U ./path/to/foo.pkg.tar.zst

It's strongly discouraged to create virtualenvs even in the package() function (installing to "$pkgdir"), and there's really no reason the software should not work with current versions of the dependent global packages -- if it doesn't work, the package should be patched to work and those patches sent upstream, or backported if upstream already fixed it.

See e.g.

https://wiki.gentoo.org/wiki/Why_not_bu … pendencies
https://fedoraproject.org/wiki/Bundled_Libraries
https://www.debian.org/doc/debian-polic … eddedfiles

Last edited by eschwartz (2020-08-21 20:17:30)


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#3 2020-08-22 03:17:27

galvez_65
Member
Registered: 2018-06-29
Posts: 20

Re: packing python software with a virtualenv

Thanks for the explanation, I didn't realize the use of virtualenv in a package would be that problematic. I'll refactor my package to not use a virtualenv. shouldn't hard to do really, just need to manage the dependencies better

Offline

#4 2020-08-22 22:16:27

galvez_65
Member
Registered: 2018-06-29
Posts: 20

Re: packing python software with a virtualenv

Again thanks for the links and for the advise. I took the time to completely refactor my program to install as a standard python package. It actually really simplified the entire build process.

Offline

#5 2020-08-23 02:41:17

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: packing python software with a virtualenv

Great, happy to help. smile


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#6 2021-01-26 22:38:15

skrat
Member
Registered: 2013-05-10
Posts: 39

Re: packing python software with a virtualenv

I'll just necrobump this, it's not THAT old. So essentially I'm here for the same answer. @eschwartz has a little too pursitic point of view. I'll just take the reasonable part not to pip install anything in postinstall, that's awful indeed.

I don't see any problem at all with boostrapping a virtualenv and package() and then using that env from your app, it's perfectly reasonable. You just have to undestand that this is not about developing any upstream package, but the use case here is deploying your application using pacman to presumably arch based server. In that case why wouldn't I DTFIW? So say I have a django web app with its own deps, and of course most of them are neither in arch's repos or AUR, and even then, it's much simpler to just package everything up and be done with the deployment, almost like using docker, but even more lightweight, no container, nada.

Offline

Board footer

Powered by FluxBB