Most scientific python packages broken

sidc · 2008-11-03 03:50:48

After the update to 2.6, packages such as numpy, scipy, matplotlib, etc are not working anymore. I get the following error in ipython when trying to import any one of them.

In [1]: import numpy
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)

/home/msid/<ipython console> in <module>()

/usr/lib/python2.5/site-packages/numpy/__init__.py in <module>()
    123         return loader(*packages, **options)
    124 
--> 125     import add_newdocs
    126     __all__ = ['add_newdocs']
    127 

/usr/lib/python2.5/site-packages/numpy/add_newdocs.py in <module>()
      7 #       core/fromnumeric.py, core/defmatrix.py up-to-date.

      8 
----> 9 from lib import add_newdoc
     10 
     11 add_newdoc('numpy.core', 'dtype',

/usr/lib/python2.5/site-packages/numpy/lib/__init__.py in <module>()
      2 from numpy.version import version as __version__
      3 
----> 4 from type_check import *
      5 from index_tricks import *
      6 from function_base import *

/usr/lib/python2.5/site-packages/numpy/lib/type_check.py in <module>()
      6            'common_type']
      7 
----> 8 import numpy.core.numeric as _nx
      9 from numpy.core.numeric import asarray, asanyarray, array, isnan, \
     10                 obj2sctype, zeros

/usr/lib/python2.5/site-packages/numpy/core/__init__.py in <module>()
      3 from numpy.version import version as __version__
      4 
----> 5 import multiarray
      6 import umath
      7 import _internal # for freeze programs

ImportError: libpython2.5.so.1.0: cannot open shared object file: No such file or directory

I even tried reinstalling them, but the packages always install in the python2.5 directory . Is a recompilation needed in the repositories?

Allan · 2008-11-03 03:53:39

There are a few [community] packages that still need rebuilt. We were working on it when the server went down. They should get done soon but file big reports for the packages you notice.

BTW, a simple rebuild using ABS should get you by for the time being...

sidc · 2008-11-03 03:58:20

Yes, I know that rebuilding using ABS should get me by, but I am behind my college proxy, and ABS does not go through proxy( which I discovered after several attempts ).

Allan · 2008-11-03 04:00:31

New feature in abs-2.3.... Try "abs -t"

sidc · 2008-11-03 04:06:06

Wow thx !!!! *bows down* .Its these little things that just make my day a lot brighter

sidc · 2008-11-03 17:02:54

Oh damn, apparently numpy is not compatible with python 2.6. Is there anyway to make it use python 2.5?

Profjim · 2008-11-04 02:28:52

What do you mean, make it use python2.5? It only gets loaded by an already-running Python interpreter, there's no way that numpy can change the version of the interpreter that's loading it.

What you can control is: (1) keep a copy of python2.5 on your machine, and (2) install numpy in the /usr/lib/python2.5/site-packages folder. (2) is easy: just tweak the numpy PKGBUILD to make sure that the python interpreter that's doing the install is itself 2.5. (1) is of course doable but I think will be more work---that is, if you also want to upgrade to python2.6. You're going to have to rewrite the python PKGBUILDs to enable you to have different versions side-by-side. Or, you could just let pacman upgrade to python2.6, and install python2.5 from source in /usr/local/bin (without using pacman).

When someone has gotten (debugged) PKGBUILD tweaks that install different versions of Python beside each other, please share the love.

Allan · 2008-11-04 02:41:31

Profjim wrote:

When someone has gotten (debugged) PKGBUILD tweaks that install different versions of Python beside each other, please share the love.

Well, there is python and python24 in the official repos and python3 in the AUR that can guide you.

Can you please file a bug report about numpy to remind the maintainer to fix it for python2.6.

sidc · 2008-11-04 07:59:52

Profjim wrote:

What do you mean, make it use python2.5? It only gets loaded by an already-running Python interpreter, there's no way that numpy can change the version of the interpreter that's loading it.
What you can control is: (1) keep a copy of python2.5 on your machine, and (2) install numpy in the /usr/lib/python2.5/site-packages folder. (2) is easy: just tweak the numpy PKGBUILD to make sure that the python interpreter that's doing the install is itself 2.5. (1) is of course doable but I think will be more work---that is, if you also want to upgrade to python2.6. You're going to have to rewrite the python PKGBUILDs to enable you to have different versions side-by-side. Or, you could just let pacman upgrade to python2.6, and install python2.5 from source in /usr/local/bin (without using pacman).
When someone has gotten (debugged) PKGBUILD tweaks that install different versions of Python beside each other, please share the love.

Numpy is already installed in python2.5/site-packages, but when I try to import it in ipython, I get the error

ImportError: libpython2.5.so.1.0: cannot open shared object file: No such file or directory

And since there is no python2.5 in the repos, only 2.4, I guess I will have to install that.

Last edited by sidc (2008-11-04 08:01:48)

kazuo · 2008-11-04 10:55:55

Bugs reports:

Numpy: http://bugs.archlinux.org/task/11987
Scipy: http://bugs.archlinux.org/task/11988
Matplotlib: http://bugs.archlinux.org/task/11989

Profjim · 2008-11-04 13:14:35

ImportError: libpython2.5.so.1.0: cannot open shared object file: No such file or directory

That's because you no longer have python2.5 (fully) installed. Having numpy where python2.5 will look for it is only part of the story.

What you say about ipython puzzles me. For, since you don't have python2.5 (fully) installed, I'm expecting that your ipython instance is using the python2.6 interpreter. On the other hand, when you try to import numpy from within ipython, it knows to get it from the python2.5 site-packages directory. Why would it do that if the running interpreter is python2.6?

Perhaps there's one of those python aliases (what are they called? site.pth or something like that?) in the python2.6 site-packages directory that adds the older site-packages directory to the path that python2.6 uses to search for modules.

In any event, if you want to use ipython and numpy both with python2.5, you'll need to make sure that you have a copy of ipython installed where python2.5 can find it, and a way to start ipython using the python2.5 interpreter. This will most likely involve tweaking the ipython PKGBUILD too, to let you have multiple versions of ipython installed side-by-side, one for each version of python.

Language upgrades are a PITA.

Last edited by Profjim (2008-11-04 13:22:40)

sidc · 2008-11-04 14:25:38

Oh man....I just think I'll end up doing some dirty makeshift thing if I try to repair this. I think I'll wait for the packages to be rebuilt in the repos and then I'll do something about this.

R00KIE · 2008-11-04 18:06:47

How about making a link to libpython2.6.so.1.0 ?
I did that to make medit work and it complained about the exact same lib. Now medit got updated and it works without the link.

sidc · 2008-11-04 19:05:54

Lol, thats exactly the temporary fix I was talking about. I did just that, but somehow I feel its no more than a dirty quick fix

Allan · 2008-11-04 23:22:33

Don't symlink libraries. They have difference so-names for a reason. This may seem to fix a problem but creates others that are very difficult to figure out..

prakti · 2008-11-07 14:12:36

Why has the upgrade to 2.6 been done at all. Is there some sort of automated build-system that checks compatibility?

prakti · 2008-11-07 15:08:51

The scipy-problem with python-2.6 seems to be corrected in the svn-version. I'm trying to whip up an AUR-package.

prakti · 2008-11-07 15:29:54

The AUR-package is here: http://aur.archlinux.org/packages.php?ID=21287

byte · 2008-11-07 16:28:14

prakti wrote:

Why has the upgrade to 2.6 been done at all. Is there some sort of automated build-system that checks compatibility?

Not really. Devs check which packages depend on python (and which packages depend on packages that depend on python...) and rebuild them. It's a bit tricky with interpreted languages, because you really have to run and test the application; even if there was some sort of successful binary compilation involved, you can't really be sure that it'll run the same as before with the old libraries.
There are some tools like 'lddd' or the regular "integrity check" reports you find on the arch-dev-public mailing list, but that's not really suited for stuff like Python/Perl/Ruby.

As to why the upgrade was made altogether, that's easy: It's the latest stable upstream version and Arch is Rolling Release ;]
Upgrading important 'glue' packages like this one, or GCC, Xorg, Berkeley DB etc., will always cause some amount of trouble for a while, but by reporting such bugs you'll not only help other Archers with your experience, but also improve the integration between upstream projects.

The_ouroboros · 2008-12-11 10:56:47

I have python 2.6 installed...will scipy and numpy work?

dcrabs · 2008-12-11 11:30:25

The_ouroboros wrote:

I have python 2.6 installed...will scipy and numpy work?

yes, it works fine here. the only thing I see is this:

DeprecationWarning: os.popen4 is deprecated.

The_ouroboros · 2008-12-11 13:36:20

so I just install python-scipy, python-numpy, python-matplotlib and they'll work fine?

Last edited by The_ouroboros (2008-12-11 13:38:21)

The_ouroboros · 2008-12-11 19:00:52

up

byte · 2008-12-11 21:26:29

Just install them and try it?!

xaiviax · 2008-12-12 01:42:49

I use numpy and matplotlib and they work fine.

Arch Linux

#1 2008-11-03 03:50:48

Most scientific python packages broken

#2 2008-11-03 03:53:39

Re: Most scientific python packages broken

#3 2008-11-03 03:58:20

Re: Most scientific python packages broken

#4 2008-11-03 04:00:31

Re: Most scientific python packages broken

#5 2008-11-03 04:06:06

Re: Most scientific python packages broken

#6 2008-11-03 17:02:54

Re: Most scientific python packages broken

#7 2008-11-04 02:28:52

Re: Most scientific python packages broken

#8 2008-11-04 02:41:31

Re: Most scientific python packages broken

#9 2008-11-04 07:59:52

Re: Most scientific python packages broken

#10 2008-11-04 10:55:55

Re: Most scientific python packages broken

#11 2008-11-04 13:14:35

Re: Most scientific python packages broken

#12 2008-11-04 14:25:38

Re: Most scientific python packages broken

#13 2008-11-04 18:06:47

Re: Most scientific python packages broken

#14 2008-11-04 19:05:54

Re: Most scientific python packages broken

#15 2008-11-04 23:22:33

Re: Most scientific python packages broken

#16 2008-11-07 14:12:36

Re: Most scientific python packages broken

#17 2008-11-07 15:08:51

Re: Most scientific python packages broken

#18 2008-11-07 15:29:54

Re: Most scientific python packages broken

#19 2008-11-07 16:28:14

Re: Most scientific python packages broken

#20 2008-12-11 10:56:47

Re: Most scientific python packages broken

#21 2008-12-11 11:30:25

Re: Most scientific python packages broken

#22 2008-12-11 13:36:20

Re: Most scientific python packages broken

#23 2008-12-11 19:00:52

Re: Most scientific python packages broken

#24 2008-12-11 21:26:29

Re: Most scientific python packages broken

#25 2008-12-12 01:42:49

Re: Most scientific python packages broken

Board footer