Programmatic solution to Python version problem?

rallyemax · 2013-05-17 15:37:47

To my knowledge, a sane solution to the Python version problem has not yet been implemented. If I am wrong, please correct me and that will be the end of that.

I currently have both python2 and python3 installed, and by forcing the "/usr/bin/python" symlink to point to python2, about 99% of my problems are solved. That leaves the 1%. As developers gradually migrate to python3, I believe that number will rise. Thus far I have solved any problems that arise in an ad-hoc fashion. I point all my Python scripts to python2 in the hashbang, and when a third-party script is not working, I edit its hashtag as appropriate. This works, but is not a sane, systemic solution to the problem.

I propose a solution where /usr/bin/python points to a script that determines whether the invoked script is python2 or python3 compatible, and then runs the appropriate Python version automagically. To save time on subsequent invocations, the script could maintain a database (loosely speaking) of scripts that have been run (this could cause problems if an already-known third-party python2 script subsequently migrates to python3; if this is rare, the user could manually edit the historical database each time it happens).

The purpose of this post is to gauge interest. I have no code to propose yet, and frankly haven't even checked to see if there are existing tools to verify compatibility of a script with python2 or python3 (because I assume, rightly or wrongly, that such a tool would be trivial to write).

I don't propose this to become part of any standard installation of python2 or python3, but it would be a useful tool for those of us who need to have both versions installed.

Frankly, I think this should have been dealt with at the level of the Python dev team, especially given their explicit recognition of the fact that python2 is currently the standard and their concurrent push to make python3 the standard in the near future.

Please leave a comment with your thoughts. If there is sufficient interest, I would be happy to take a stab at this, though my coding is seldom pythonic (and my shell scripting is even more informal/sloppy) and it would be nice to have one or more experienced developers either directly involved or at least serving in an advisory capacity.

Thanks for reading.

drcouzelis · 2013-05-17 16:36:38

rallyemax wrote:

To my knowledge, a sane solution to the Python version problem has not yet been implemented. If I am wrong, please correct me and that will be the end of that.

I could have sworn I read that there's a new guideline that all Python applications should use either "/usr/bin/python2" or "/usr/bin/python3" and not "/usr/bin/python", but I can't find anything. Anyway, I kind of like the idea, because it means being explicit instead of implicit about which version of Python you want to use.

I currently have both python2 and python3 installed, and by forcing the "/usr/bin/python" symlink to point to python2, about 99% of my problems are solved.

What problems are solved? I have python2 and python3 installed, and everything on my computer that's related to Python has been installed via pacman and I've never had any problems.

I propose a solution where /usr/bin/python points to a script that determines whether the invoked script is python2 or python3 compatible, and then runs the appropriate Python version automagically.

How would the script make the distinction?

karol · 2013-05-17 16:42:51

What scripts are you talking about? If they're installed via a PKGBUILD, they should conform to Arch Python Package Guidelines:
https://wiki.archlinux.org/index.php/Py … Guidelines
https://wiki.archlinux.org/index.php/Py … ld_scripts

Installing via pacman should be hassle-free and python2 libraries should be marked as such, so no need to guess which version pf python do they go with.

falconindy · 2013-05-17 17:31:11

You're looking for PEP 394 which is what the proper solution is.

rallyemax · 2013-05-17 17:45:40

karol wrote:

What scripts are you talking about? If they're installed via a PKGBUILD, they should conform to Arch Python Package Guidelines:
Installing via pacman should be hassle-free and python2 libraries should be marked as such, so no need to guess which version pf python do they go with.

In theory, yes. And since I can't think of any examples in the binary repos, perhaps there the problem is indeed nonexistent. The story is different in AUR, where I've encountered the problem several times (most recently today with byobu). Probably the answer is I should be less lazy and propose changes to any packages that have the non-canonical hashbang to /usr/bin/Python (or its env equivalent).

Indeed, maybe the solution is for me to point /use/bin/python to a simple shell script that notifies me every time it is invoked and throws a notification/exception so that I can fix any non compliant code. It would be less ad hoc if I forced myself to do this every time, I suppose, and I can force myself to do just that by pointing /usr/bin/python only to the notification/exception and not to Python itself.

I guess that's more of an Arch way to do it - explicit and simple, rather than implicit and masking complexity.

I have a tendency to overcomplicate things. Thanks for letting me view this from a better perspective.

drcouzelis · 2013-05-17 17:46:01

falconindy wrote:

You're looking for PEP 394 which is what the proper solution is.

You found it!

PEP 394 wrote:

Arch Linux announcement that their "python" link now refers Python 3

It even includes a shoutout to Arch Linux.

sidneyk · 2013-05-18 05:02:36

I had to use this from the Python wiki page after my Android build system stopped working one day:

/usr/local/bin/python

#!/bin/bash
script=`readlink -f -- "$1"`
case "$script" in
/path/to/project1/*|/path/to/project2/*|/path/to/project3*)
    exec python2 "$@"
    ;;
esac

exec python3 "$@"

Where /path/to/project1/*|/path/to/project2/*|/path/to/project3* is a list of patterns separated by | matching all project trees.

Don't forget to make it executable:

# chmod +x /usr/local/bin/python

Afterwards scripts within the specified project trees will be run with Python 2.

Of course, this requires you to determine your python2 projects and to add their directories to the script. Once done, it intercepts the call to python and if your directory is in the list in the script then it will execute python2, otherwise python3.

Mr.Elendig · 2013-05-18 11:29:32

It really isn't hard to deal with python being python3. (and messing with the symlink isn't really the right answer)
1. use distutils/distribute/similar, it will set the shebang of your scripts for you.
2. Don't hardcode calls to "/usr/bin/python" or whatever in your code. Make it a build flag / env.var / config option / whatever.
3. virtualenv/venv/tox

The bonus point is that doing this will solve all the issues you will have on systems where eg python is 2.4, and your code requires >= 2.6 or whatever too. *coughRHELcough*

Arch Linux

#1 2013-05-17 15:37:47

Programmatic solution to Python version problem?

#2 2013-05-17 16:36:38

Re: Programmatic solution to Python version problem?

#3 2013-05-17 16:42:51

Re: Programmatic solution to Python version problem?

#4 2013-05-17 17:31:11

Re: Programmatic solution to Python version problem?

#5 2013-05-17 17:45:40

Re: Programmatic solution to Python version problem?

#6 2013-05-17 17:46:01

Re: Programmatic solution to Python version problem?

#7 2013-05-18 05:02:36

Re: Programmatic solution to Python version problem?

#8 2013-05-18 11:29:32

Re: Programmatic solution to Python version problem?

Board footer