You are not logged in.

#1 2018-01-29 17:33:15

andalenavals
Member
Registered: 2017-10-30
Posts: 39

Pixmappy PKGBUILD issues and sugestions

Hi guys,
I trying to contribute in AUR with a famous package of cosmology, however some predefined tests were not passed, apparently there is a problem related with python-numpy version. Then I did the downgrade from  1.14.0-1 to 1.11.3-1, everything was apparently fine, until I realize that I needed to rebuild python-galsim-git, python-astropy, python-lmfit and other packages. So that

after

trizen -Rs python-lmfit 

I did

trizen -Syu python-lmfit

But it did not pass the check because somewhere still it is looking for the most recent version (I guess).
Here is the output of the error

==> Starting check()...
RuntimeError: module compiled against API version 0xb but this version of numpy is 0xa
EEEEEEEEEEEEEEEEEEE
======================================================================
ERROR: Failure: ImportError (numpy.core.multiarray failed to import)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/tmp/trizen-andres/python2-lmfit/src/lmfit-py-0.9.3/tests/lmfit_testutils.py", line 1, in <module>
    from lmfit import Parameter
  File "/tmp/trizen-andres/python2-lmfit/src/lmfit-py-0.9.3/lmfit/__init__.py", line 37, in <module>
    from .minimizer import minimize, Minimizer, MinimizerException
  File "/tmp/trizen-andres/python2-lmfit/src/lmfit-py-0.9.3/lmfit/minimizer.py", line 23, in <module>
    from scipy.optimize import leastsq as scipy_leastsq
  File "/usr/lib/python2.7/site-packages/scipy/optimize/__init__.py", line 240, in <module>
    from .optimize import *
  File "/usr/lib/python2.7/site-packages/scipy/optimize/optimize.py", line 37, in <module>
    from .linesearch import (line_search_wolfe1, line_search_wolfe2,
  File "/usr/lib/python2.7/site-packages/scipy/optimize/linesearch.py", line 18, in <module>
    from scipy.optimize import minpack2
ImportError: numpy.core.multiarray failed to import

And when I check numpy vesion doing

import numpy
numpy.version.version

The output is just fine.

Thanks in advance for your help

Last edited by andalenavals (2018-01-31 17:57:17)

Offline

#2 2018-01-29 18:55:49

Slithery
Administrator
From: Norfolk, UK
Registered: 2013-12-01
Posts: 5,776

Re: Pixmappy PKGBUILD issues and sugestions

Moving to 'AUR Issues'.


No, it didn't "fix" anything. It just shifted the brokeness one space to the right. - jasonwryan
Closing -- for deletion; Banning -- for muppetry. - jasonwryan

aur - dotfiles

Offline

#3 2018-01-29 22:38:09

andalenavals
Member
Registered: 2017-10-30
Posts: 39

Re: Pixmappy PKGBUILD issues and sugestions

I forgot to mention, that I also tried to rebuild scipy. But It is still not passing the check().

Offline

#4 2018-01-30 11:32:30

a821
Member
Registered: 2012-10-31
Posts: 381

Re: Pixmappy PKGBUILD issues and sugestions

andalenavals wrote:

Hi guys,
I trying to contribute in AUR with a famous package of cosmology, however some predefined tests were not passed, apparently there is a problem related with python-numpy version. Then I did the downgrade from  1.14.0-1 to 1.11.3-1, everything was apparently fine, until I realize that I needed to rebuild python-galsim-git, python-astropy, python-lmfit and other packages. So that

Are you stuck in a classic X-Y problem? You want to install a "famous package of cosmology" (which one?). If that doesn't work, then post full errors, commands involved, PKGBUILD, &c. Downgrading packages will lead to disaster and very likely it's the wrong path.

Also, don't use AUR helpers when debugging, just makepkg.

Last edited by a821 (2018-01-30 11:44:58)

Offline

#5 2018-01-30 11:40:10

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,449
Website

Re: Pixmappy PKGBUILD issues and sugestions

The problem is (should be) self-evident.  Cosmology uses an old version of numpy.  The solution is definitely not to downgrade numpy and every other package that depends on it; the solution is to update/fix cosmology to work with the current numpy.

We can help you with that, but not with the above noted X-Y problem.  You showed some output, but none of it relevant to the actual problem which was described only as "some predefined tests were not passed".  Which ones, how did they not pass, what errors did you get?

Last edited by Trilby (2018-01-30 11:42:03)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#6 2018-01-31 02:14:45

andalenavals
Member
Registered: 2017-10-30
Posts: 39

Re: Pixmappy PKGBUILD issues and sugestions

Sorry guys for not being accurate.
The name of the package is pixmappy. This is the PKGBUILD I have written.

_pkgname=python-pixmappy
pkgbase=python-pixmappy-git
pkgname=("python-pixmappy-git" "python2-pixmappy-git")
pkgver=r50.814558e
pkgrel=1
pkgdesc="Python interface to gbdes pixel map (astrometry) solutions "
arch=('any')
url="https://github.com/gbernstein/pixmappy"
license=('BSD')
makedepends=('git' 'python-astropy' 'python-future' 'python-numpy' 'python-scipy' 'python-yaml' 'python2-astropy' 'python2-future' 'python2-numpy' 'python2-scipy' 'python2-yaml')
optdepends=('python-scikit-learn' 'python2-scikit-learn' 'python-lmfit' 'python2-lmfit')
checkdepends=('python-setuptools' 'python2-setuptools')
source=("${_pkgname}::git+${url}")
md5sums=('SKIP')

pkgver() {
  	 cd $_pkgname
  	 printf "r%s.%s" "$(git rev-list --count HEAD)" "$(git rev-parse --short HEAD)"
}

prepare() {
	  cp -a $_pkgname{,-py2}
}

build() {
	cd "$srcdir"/$_pkgname
	python setup.py build

	cd  "$srcdir"/$_pkgname-py2
	python2 setup.py build
}

check() {
	cd "$srcdir"/$_pkgname/tests
	nosetests -v || warning 'Tests failed'

	cd "$srcdir"/$_pkgname-py2/tests
	nosetests2 -v || warning 'Tests failed'
}

package_python-pixmappy-git() {
			      depends=('python')
			      cd $_pkgname
			      python setup.py install --root=${pkgdir} --prefix=/usr --optimize=1
			      install -Dm644 Piff_LICENSE $pkgdir/usr/share/licenses/$pkgname/LICENSE	      
}

package_python2-pixmappy-git() {
			      depends=('python')
			      cd $_pkgname
			      python2 setup.py install --root=${pkgdir} --prefix=/usr --optimize=1
			      install -Dm644 Piff_LICENSE $pkgdir/usr/share/licenses/$pkgname/LICENSE	      
}

And here are the predefined test outputs.

[andres@PAN tests]$ nosetests -v
test_galsim.test_basic ... ok
test_galsim.test_tpv ... /usr/lib/python3.6/site-packages/galsim/deprecated/__init__.py:53: GalSimDeprecationWarning: (GalSimDeprecationWarning(...), 'angle.rad() has been deprecated since GalSim version 1.5.  Use angle.rad instead.  rad is now a property rather than a function.  Although note that the return type is not exactly a float (so you can get this message), but acts in most ways list one and is convertible into a real float with float(angle.rad) if needed.')
  warnings.warn(s, GalSimDeprecationWarning)
ok
test_galsim.test_complex ... ok
test_galsim.test_cache ... ERROR
test_galsim.test_sky ... /usr/lib/python3.6/site-packages/galsim/deprecated/__init__.py:53: GalSimDeprecationWarning: (GalSimDeprecationWarning(...), 'image.center() has been deprecated since GalSim version 1.5.  Use image.center instead.  center is now a property rather than a function.  Although note that the return type is not a PositionI (so you can get this message), but acts in most ways like a PositionI and is convertible into one using image.center.pos if needed.')
  warnings.warn(s, GalSimDeprecationWarning)
ok
test_galsim.test_repr ... ok
test_nogalsim.test_tpv ... ERROR
test_nogalsim.test_complex ... ERROR
test_reference.test_tpv ... ERROR
test_reference.test_complex ... ERROR

======================================================================
ERROR: test_galsim.test_cache
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_galsim.py", line 144, in test_cache
    wcs = MockGalSimWCS('input/test.astro', wcs_name='D231890/N1')
  File "/usr/lib/python3.6/site-packages/pixmappy/GalSimWCS.py", line 59, in __init__
    pmc = PixelMapCollection(file_name)
  File "/usr/lib/python3.6/site-packages/pixmappy/PixelMapCollection.py", line 491, in __init__
    self.root = pickle.load(f)
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

======================================================================
ERROR: test_nogalsim.test_tpv
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_nogalsim.py", line 21, in test_tpv
    test_tpv()
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_reference.py", line 13, in test_tpv
    pmc = pixmappy.PixelMapCollection(os.path.join('input', 'tpv.yaml'))
  File "/usr/lib/python3.6/site-packages/pixmappy/PixelMapCollection.py", line 491, in __init__
    self.root = pickle.load(f)
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

======================================================================
ERROR: test_nogalsim.test_complex
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_nogalsim.py", line 25, in test_complex
    test_complex()
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_reference.py", line 46, in test_complex
    pmc = pixmappy.PixelMapCollection(os.path.join('input', 'complex_wcs.astro'))
  File "/usr/lib/python3.6/site-packages/pixmappy/PixelMapCollection.py", line 491, in __init__
    self.root = pickle.load(f)
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

======================================================================
ERROR: test_reference.test_tpv
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_reference.py", line 13, in test_tpv
    pmc = pixmappy.PixelMapCollection(os.path.join('input', 'tpv.yaml'))
  File "/usr/lib/python3.6/site-packages/pixmappy/PixelMapCollection.py", line 491, in __init__
    self.root = pickle.load(f)
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

======================================================================
ERROR: test_reference.test_complex
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_reference.py", line 46, in test_complex
    pmc = pixmappy.PixelMapCollection(os.path.join('input', 'complex_wcs.astro'))
  File "/usr/lib/python3.6/site-packages/pixmappy/PixelMapCollection.py", line 491, in __init__
    self.root = pickle.load(f)
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

----------------------------------------------------------------------
Ran 10 tests in 19.108s

FAILED (errors=5)
[andres@PAN tests]$ nosetests2 -v
test_galsim.test_basic ... ERROR
test_galsim.test_tpv ... ERROR
test_galsim.test_complex ... ERROR
test_galsim.test_cache ... ERROR
test_galsim.test_sky ... ERROR
test_galsim.test_repr ... ERROR
test_nogalsim.test_tpv ... ERROR
test_nogalsim.test_complex ... ERROR
test_reference.test_tpv ... ERROR
test_reference.test_complex ... ERROR

======================================================================
ERROR: test_galsim.test_basic
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_galsim.py", line 20, in test_basic
    wcs = pixmappy.GalSimWCS(file_name, dir=input_dir, exp=exp, ccdnum=ccdnum)
  File "/usr/lib/python2.7/site-packages/pixmappy/GalSimWCS.py", line 59, in __init__
    pmc = PixelMapCollection(file_name)
  File "/usr/lib/python2.7/site-packages/pixmappy/PixelMapCollection.py", line 491, in __init__
    self.root = pickle.load(f)
ValueError: unsupported pickle protocol: 3

======================================================================
ERROR: test_galsim.test_tpv
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_galsim.py", line 63, in test_tpv
    wcs1 = pixmappy.GalSimWCS(yaml_name, wcs_name='testwcs')
  File "/usr/lib/python2.7/site-packages/pixmappy/GalSimWCS.py", line 59, in __init__
    pmc = PixelMapCollection(file_name)
  File "/usr/lib/python2.7/site-packages/pixmappy/PixelMapCollection.py", line 491, in __init__
    self.root = pickle.load(f)
ValueError: unsupported pickle protocol: 3

======================================================================
ERROR: test_galsim.test_complex
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_galsim.py", line 108, in test_complex
    wcs = pixmappy.GalSimWCS(os.path.join('input', 'complex_wcs.astro'), wcs_name='TEST/N1')
  File "/usr/lib/python2.7/site-packages/pixmappy/GalSimWCS.py", line 59, in __init__
    pmc = PixelMapCollection(file_name)
  File "/usr/lib/python2.7/site-packages/pixmappy/PixelMapCollection.py", line 491, in __init__
    self.root = pickle.load(f)
ValueError: unsupported pickle protocol: 3

======================================================================
ERROR: test_galsim.test_cache
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_galsim.py", line 144, in test_cache
    wcs = MockGalSimWCS('input/test.astro', wcs_name='D231890/N1')
  File "/usr/lib/python2.7/site-packages/pixmappy/GalSimWCS.py", line 59, in __init__
    pmc = PixelMapCollection(file_name)
  File "/usr/lib/python2.7/site-packages/pixmappy/PixelMapCollection.py", line 491, in __init__
    self.root = pickle.load(f)
ValueError: unsupported pickle protocol: 3

======================================================================
ERROR: test_galsim.test_sky
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_galsim.py", line 171, in test_sky
    wcs = pixmappy.GalSimWCS(dir='input', file_name='test.astro', exp=375294, ccdnum=14)
  File "/usr/lib/python2.7/site-packages/pixmappy/GalSimWCS.py", line 59, in __init__
    pmc = PixelMapCollection(file_name)
  File "/usr/lib/python2.7/site-packages/pixmappy/PixelMapCollection.py", line 491, in __init__
    self.root = pickle.load(f)
ValueError: unsupported pickle protocol: 3

======================================================================
ERROR: test_galsim.test_repr
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_galsim.py", line 215, in test_repr
    wcs = pixmappy.GalSimWCS(dir='input', file_name='test.astro', exp=375294, ccdnum=14)
  File "/usr/lib/python2.7/site-packages/pixmappy/GalSimWCS.py", line 59, in __init__
    pmc = PixelMapCollection(file_name)
  File "/usr/lib/python2.7/site-packages/pixmappy/PixelMapCollection.py", line 491, in __init__
    self.root = pickle.load(f)
ValueError: unsupported pickle protocol: 3

======================================================================
ERROR: test_nogalsim.test_tpv
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_nogalsim.py", line 21, in test_tpv
    test_tpv()
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_reference.py", line 13, in test_tpv
    pmc = pixmappy.PixelMapCollection(os.path.join('input', 'tpv.yaml'))
  File "/usr/lib/python2.7/site-packages/pixmappy/PixelMapCollection.py", line 491, in __init__
    self.root = pickle.load(f)
ValueError: unsupported pickle protocol: 3

======================================================================
ERROR: test_nogalsim.test_complex
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_nogalsim.py", line 25, in test_complex
    test_complex()
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_reference.py", line 46, in test_complex
    pmc = pixmappy.PixelMapCollection(os.path.join('input', 'complex_wcs.astro'))
  File "/usr/lib/python2.7/site-packages/pixmappy/PixelMapCollection.py", line 491, in __init__
    self.root = pickle.load(f)
ValueError: unsupported pickle protocol: 3

======================================================================
ERROR: test_reference.test_tpv
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_reference.py", line 13, in test_tpv
    pmc = pixmappy.PixelMapCollection(os.path.join('input', 'tpv.yaml'))
  File "/usr/lib/python2.7/site-packages/pixmappy/PixelMapCollection.py", line 491, in __init__
    self.root = pickle.load(f)
ValueError: unsupported pickle protocol: 3

======================================================================
ERROR: test_reference.test_complex
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy/tests/test_reference.py", line 46, in test_complex
    pmc = pixmappy.PixelMapCollection(os.path.join('input', 'complex_wcs.astro'))
  File "/usr/lib/python2.7/site-packages/pixmappy/PixelMapCollection.py", line 491, in __init__
    self.root = pickle.load(f)
ValueError: unsupported pickle protocol: 3

----------------------------------------------------------------------
Ran 10 tests in 0.835s

FAILED (errors=10)

Do you thing I am right when I believe that this is related with numpy version?

Offline

#7 2018-01-31 02:53:43

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,449
Website

Re: Pixmappy PKGBUILD issues and sugestions

andalenavals wrote:

Do you thing I am right when I believe that this is related with numpy version?

No.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#8 2018-01-31 10:25:57

Morn
Member
Registered: 2012-09-02
Posts: 886

Re: Pixmappy PKGBUILD issues and sugestions

andalenavals wrote:
ValueError: unsupported pickle protocol: 3

https://docs.python.org/2/library/pickle.html

There are currently 3 different protocols which can be used for pickling.

Protocol version 0 is the original ASCII protocol and is backwards compatible with earlier versions of Python.
Protocol version 1 is the old binary format which is also compatible with earlier versions of Python.
Protocol version 2 was introduced in Python 2.3. It provides much more efficient pickling of new-style classes.

https://docs.python.org/3.6/library/pickle.html

There are currently 5 different protocols which can be used for pickling. The higher the protocol used, the more recent the version of Python needed to read the pickle produced.

Protocol version 0 is the original “human-readable” protocol and is backwards compatible with earlier versions of Python.
Protocol version 1 is an old binary format which is also compatible with earlier versions of Python.
Protocol version 2 was introduced in Python 2.3. It provides much more efficient pickling of new-style classes. Refer to PEP 307 for information about improvements brought by protocol 2.
Protocol version 3 was added in Python 3.0. It has explicit support for bytes objects and cannot be unpickled by Python 2.x. This is the default protocol, and the recommended protocol when compatibility with other Python 3 versions is required.
Protocol version 4 was added in Python 3.4. It adds support for very large objects, pickling more kinds of objects, and some data format optimizations. Refer to PEP 3154 for information about improvements brought by protocol 4.

I think you need to run this with Python 3, not Python 2.

Last edited by Morn (2018-01-31 10:28:50)

Offline

#9 2018-01-31 11:50:55

andalenavals
Member
Registered: 2017-10-30
Posts: 39

Re: Pixmappy PKGBUILD issues and sugestions

Morn wrote:

I think you need to run this with Python 3, not Python 2.

Python version should not be a issue. I got not troubles when I installed piff in my ubuntu machine. And there it is running python2

Offline

#10 2018-01-31 11:58:10

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,449
Website

Re: Pixmappy PKGBUILD issues and sugestions

Nothing in this thread suggests that the built package would fail to work either.  All you've shown us are that the checks are failing.  And they fail with error messages that are very clear and readily googlable (but which were completely hidden from us for almost all of this thread where you were instead giving a completely bogus XY theory about numpy).

There are any number of reasons for false-positives for checks failure.  Do the checks fail once the package is installed?  To the checks fail on Ubuntu?  Do you run the checks in ubuntu in a staged root directory from a PKGBUILD like is being done here?

Last edited by Trilby (2018-01-31 12:01:26)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#11 2018-01-31 11:58:31

progandy
Member
Registered: 2012-05-17
Posts: 5,184

Re: Pixmappy PKGBUILD issues and sugestions

andalenavals wrote:
Morn wrote:

I think you need to run this with Python 3, not Python 2.

Python version should not be a issue. I got not troubles when I installed piff in my ubuntu machine. And there it is running python2

The tests seem to rely on pickled objects that were created with python3, so you can run the tests only with python3. The library itself may still work fine with python2.

Last edited by progandy (2018-01-31 11:59:39)


| alias CUTF='LANG=en_XX.UTF-8@POSIX ' |

Offline

#12 2018-01-31 13:28:44

Morn
Member
Registered: 2012-09-02
Posts: 886

Re: Pixmappy PKGBUILD issues and sugestions

andalenavals wrote:
Morn wrote:

I think you need to run this with Python 3, not Python 2.

Python version should not be a issue. I got not troubles when I installed piff in my ubuntu machine. And there it is running python2

Are you sure this is exactly the same version of the program? The author might have introduced some changes that are Python 3-only since then.

Certainly the broken unit tests suggest that the author is using Python 3 now, so I would use that too if there are problems.

Last edited by Morn (2018-01-31 13:29:02)

Offline

#13 2018-01-31 13:34:59

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,449
Website

Re: Pixmappy PKGBUILD issues and sugestions

I don't know much anything about `nose` but all the output from the tests suggest that it is using already installed components, not those under $pkgdir.

Last edited by Trilby (2018-01-31 13:35:23)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#14 2018-01-31 15:04:37

andalenavals
Member
Registered: 2017-10-30
Posts: 39

Re: Pixmappy PKGBUILD issues and sugestions

Sorry for making this threat a mess.

Trilby wrote:

There are any number of reasons for false-positives for checks failure.  Do the checks fail once the package is installed?  To the checks fail on Ubuntu?  Do you run the checks in ubuntu in a staged root directory from a PKGBUILD like is being done here?

The outputs are after installing the package i. e makepkg -s and pacman -U snapshot.xz. I am running the tests going to src/python-pixmappy-git/tests and then using nosetests. In a similar way I do the same in ubuntu, the diference is that there I am only using python2 . Here the ouput of the test in ubuntu.

andres@desksobreira:~/psfwork/pixmappy/tests$ nosetests -v
Test basic operation of the GalSimWCS class ... ok
Test that reading a tpv file is equivalent to a regular TPV FITS wcs ... ok
Test a complex PMC file against some reference values ... ok
Test the caching features of GalSimWCS ... ok
Test using the GalSimWCS to fill an image with constant surface brightness from the sky ... ok
Test some things like repr, pickling, etc. to make sure they are reasonable. ... ok
test_nogalsim.test_tpv ... ok
test_nogalsim.test_complex ... ok
Test that reading a tpv file is equivalent to a regular TPV FITS wcs ... ok
Test a complex PMC file against some reference values ... ok

----------------------------------------------------------------------
Ran 10 tests in 13.148s

OK

The root directory there is /usr/local that was your question?

Are you sure this is exactly the same version of the program? The author might have introduced some changes that are Python 3-only since then.

I have checked it. It is the same version

Last edited by andalenavals (2018-01-31 19:16:52)

Offline

#15 2018-01-31 21:21:52

andalenavals
Member
Registered: 2017-10-30
Posts: 39

Re: Pixmappy PKGBUILD issues and sugestions

Now I remember that when I installed pixmappy in ubuntu I had to modify the setup.py file. I did

#from distutils.core import setup
from setuptools import setup

What is the most reasonable way to do this?. I have already tried to include as makedepends python-setuptools. But it still not working.
I do not thing that create athoer source with a new setup.py to be a good idea.

Offline

#16 2018-01-31 21:32:21

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,449
Website

Re: Pixmappy PKGBUILD issues and sugestions

`man sed`


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#17 2018-01-31 22:59:58

andalenavals
Member
Registered: 2017-10-30
Posts: 39

Re: Pixmappy PKGBUILD issues and sugestions

I was expecting this new PKGBUILD, will solve the problem. But I am feeling that I am walking in circles again. Look this is the current PKGBUILD

pkgbase=python-pixmappy-git
pkgname=("python-pixmappy-git" "python2-pixmappy-git")
pkgver=r39.b7c163f
pkgrel=1
pkgdesc="Python interface to gbdes pixel map (astrometry) solutions "
arch=('any')
url="https://github.com/gbernstein/pixmappy"
license=('BSD')
makedepends=('git' 'python-astropy' 'python-future' 'python-numpy' 'python-scipy' 'python-yaml' 'python2-astropy' 'python2-future' 'python2-numpy' 'python2-scipy' 'python2-yaml' 'python-setuptools' 'python2-setuptools')
optdepends=('python-scikit-learn' 'python2-scikit-learn' 'python-lmfit' 'python2-lmfit')
checkdepends=('python-setuptools' 'python2-setuptools')
source=("${pkgbase}::git+${url}#branch=pickle")
md5sums=('SKIP')

pkgver() {
  	 cd "${pkgbase}"
  	 printf "r%s.%s" "$(git rev-list --count HEAD)" "$(git rev-parse --short HEAD)"
}

prepare() {
	  cp -a $pkgbase{,-py2}
}

build() {
	cd "$srcdir"/$pkgbase
	sed -i 's_distutils.core_setuptools_g' setup.py 
	python setup.py build

	cd  "$srcdir"/$pkgbase-py2
	sed -i 's_distutils.core_setuptools_g' setup.py 
	python2 setup.py build
}

check() {
	cd "$srcdir"/$pkgbase/tests
	nosetests -v || warning 'Tests failed'

	cd "$srcdir"/$pkgbase-py2/tests
	nosetests2 -v || warning 'Tests failed'
}

package_python-pixmappy-git() {
			      depends=('')
			      cd $pkgbase
			      #sed -i 's_distutils.core_setuptools_g' setup.py
			      python setup.py install --root=${pkgdir} --prefix=/usr --optimize=1
			      install -Dm644 Piff_LICENSE $pkgdir/usr/share/licenses/$pkgname/LICENSE	      
}

package_python2-pixmappy-git() {
			      depends=('')
			      cd $pkgbase-py2
			      #sed -i 's_distutils.core_setuptools_g' setup.py 
			      python2 setup.py install --root=${pkgdir} --prefix=/usr --optimize=1
			      install -Dm644 Piff_LICENSE $pkgdir/usr/share/licenses/$pkgname/LICENSE	      
}

After installing the packages these are the outputs of the tests

[andres@PAN tests]$ nosetests -v
test_galsim.test_basic ... ERROR
test_galsim.test_tpv ... ERROR
test_galsim.test_complex ... ERROR
test_galsim.test_cache ... ERROR
test_galsim.test_sky ... ERROR
test_galsim.test_repr ... ERROR
test_nogalsim.test_tpv ... ERROR
test_nogalsim.test_complex ... ERROR
test_reference.test_tpv ... ERROR
test_reference.test_complex ... ERROR

======================================================================
ERROR: test_galsim.test_basic
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy-git-py2/tests/test_galsim.py", line 20, in test_basic
    wcs = pixmappy.GalSimWCS(file_name, dir=input_dir, exp=exp, ccdnum=ccdnum)
  File "/usr/lib/python3.6/site-packages/pixmappy/GalSimWCS.py", line 59, in __init__
    pmc = PixelMapCollection(file_name)
  File "/usr/lib/python3.6/site-packages/pixmappy/PixelMapCollection.py", line 486, in __init__
    self.root = pickle.load(f)
TypeError: a bytes-like object is required, not 'str'

======================================================================
ERROR: test_galsim.test_tpv
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy-git-py2/tests/test_galsim.py", line 63, in test_tpv
    wcs1 = pixmappy.GalSimWCS(yaml_name, wcs_name='testwcs')
  File "/usr/lib/python3.6/site-packages/pixmappy/GalSimWCS.py", line 59, in __init__
    pmc = PixelMapCollection(file_name)
  File "/usr/lib/python3.6/site-packages/pixmappy/PixelMapCollection.py", line 486, in __init__
    self.root = pickle.load(f)
TypeError: a bytes-like object is required, not 'str'

======================================================================
ERROR: test_galsim.test_complex
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy-git-py2/tests/test_galsim.py", line 108, in test_complex
    wcs = pixmappy.GalSimWCS(os.path.join('input', 'complex_wcs.astro'), wcs_name='TEST/N1')
  File "/usr/lib/python3.6/site-packages/pixmappy/GalSimWCS.py", line 59, in __init__
    pmc = PixelMapCollection(file_name)
  File "/usr/lib/python3.6/site-packages/pixmappy/PixelMapCollection.py", line 486, in __init__
    self.root = pickle.load(f)
TypeError: a bytes-like object is required, not 'str'

======================================================================
ERROR: test_galsim.test_cache
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy-git-py2/tests/test_galsim.py", line 144, in test_cache
    wcs = MockGalSimWCS('input/test.astro', wcs_name='D231890/N1')
  File "/usr/lib/python3.6/site-packages/pixmappy/GalSimWCS.py", line 59, in __init__
    pmc = PixelMapCollection(file_name)
  File "/usr/lib/python3.6/site-packages/pixmappy/PixelMapCollection.py", line 486, in __init__
    self.root = pickle.load(f)
TypeError: a bytes-like object is required, not 'str'

======================================================================
ERROR: test_galsim.test_sky
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy-git-py2/tests/test_galsim.py", line 171, in test_sky
    wcs = pixmappy.GalSimWCS(dir='input', file_name='test.astro', exp=375294, ccdnum=14)
  File "/usr/lib/python3.6/site-packages/pixmappy/GalSimWCS.py", line 59, in __init__
    pmc = PixelMapCollection(file_name)
  File "/usr/lib/python3.6/site-packages/pixmappy/PixelMapCollection.py", line 486, in __init__
    self.root = pickle.load(f)
TypeError: a bytes-like object is required, not 'str'

======================================================================
ERROR: test_galsim.test_repr
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy-git-py2/tests/test_galsim.py", line 215, in test_repr
    wcs = pixmappy.GalSimWCS(dir='input', file_name='test.astro', exp=375294, ccdnum=14)
  File "/usr/lib/python3.6/site-packages/pixmappy/GalSimWCS.py", line 59, in __init__
    pmc = PixelMapCollection(file_name)
  File "/usr/lib/python3.6/site-packages/pixmappy/PixelMapCollection.py", line 486, in __init__
    self.root = pickle.load(f)
TypeError: a bytes-like object is required, not 'str'

======================================================================
ERROR: test_nogalsim.test_tpv
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy-git-py2/tests/test_nogalsim.py", line 21, in test_tpv
    test_tpv()
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy-git-py2/tests/test_reference.py", line 13, in test_tpv
    pmc = pixmappy.PixelMapCollection(os.path.join('input', 'tpv.yaml'))
  File "/usr/lib/python3.6/site-packages/pixmappy/PixelMapCollection.py", line 486, in __init__
    self.root = pickle.load(f)
TypeError: a bytes-like object is required, not 'str'

======================================================================
ERROR: test_nogalsim.test_complex
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy-git-py2/tests/test_nogalsim.py", line 25, in test_complex
    test_complex()
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy-git-py2/tests/test_reference.py", line 46, in test_complex
    pmc = pixmappy.PixelMapCollection(os.path.join('input', 'complex_wcs.astro'))
  File "/usr/lib/python3.6/site-packages/pixmappy/PixelMapCollection.py", line 486, in __init__
    self.root = pickle.load(f)
TypeError: a bytes-like object is required, not 'str'

======================================================================
ERROR: test_reference.test_tpv
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy-git-py2/tests/test_reference.py", line 13, in test_tpv
    pmc = pixmappy.PixelMapCollection(os.path.join('input', 'tpv.yaml'))
  File "/usr/lib/python3.6/site-packages/pixmappy/PixelMapCollection.py", line 486, in __init__
    self.root = pickle.load(f)
TypeError: a bytes-like object is required, not 'str'

======================================================================
ERROR: test_reference.test_complex
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy-git-py2/tests/test_reference.py", line 46, in test_complex
    pmc = pixmappy.PixelMapCollection(os.path.join('input', 'complex_wcs.astro'))
  File "/usr/lib/python3.6/site-packages/pixmappy/PixelMapCollection.py", line 486, in __init__
    self.root = pickle.load(f)
TypeError: a bytes-like object is required, not 'str'

----------------------------------------------------------------------
Ran 10 tests in 0.923s

FAILED (errors=10)
[andres@PAN tests]$ nosetests2 -v
test_galsim.test_basic ... ok
test_galsim.test_tpv ... /usr/lib/python2.7/site-packages/galsim/deprecated/__init__.py:53: GalSimDeprecationWarning: (GalSimDeprecationWarning(...), 'angle.rad() has been deprecated since GalSim version 1.5.  Use angle.rad instead.  rad is now a property rather than a function.  Although note that the return type is not exactly a float (so you can get this message), but acts in most ways list one and is convertible into a real float with float(angle.rad) if needed.')
  warnings.warn(s, GalSimDeprecationWarning)
ok
test_galsim.test_complex ... ok
test_galsim.test_cache ... ok
test_galsim.test_sky ... /usr/lib/python2.7/site-packages/galsim/deprecated/__init__.py:53: GalSimDeprecationWarning: (GalSimDeprecationWarning(...), 'image.center() has been deprecated since GalSim version 1.5.  Use image.center instead.  center is now a property rather than a function.  Although note that the return type is not a PositionI (so you can get this message), but acts in most ways like a PositionI and is convertible into one using image.center.pos if needed.')
  warnings.warn(s, GalSimDeprecationWarning)
ok
test_galsim.test_repr ... ok
test_nogalsim.test_tpv ... ERROR
test_nogalsim.test_complex ... ok
test_reference.test_tpv ... ERROR
test_reference.test_complex ... ok

======================================================================
ERROR: test_nogalsim.test_tpv
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy-git-py2/tests/test_nogalsim.py", line 21, in test_tpv
    test_tpv()
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy-git-py2/tests/test_reference.py", line 37, in test_tpv
    ra, dec = wcs1.toSky(x,y)
  File "/usr/lib/python2.7/site-packages/pixmappy/PixelMapCollection.py", line 432, in toSky
    return self.projection.toSky(xw, yw)
  File "/usr/lib/python2.7/site-packages/pixmappy/PixelMapCollection.py", line 658, in toSky
    return coord.icrs.ra.deg, coord.icrs.dec.deg
  File "/usr/lib/python2.7/site-packages/astropy/coordinates/sky_coordinate.py", line 517, in __getattr__
    return self.transform_to(attr)
  File "/usr/lib/python2.7/site-packages/astropy/coordinates/sky_coordinate.py", line 482, in transform_to
    new_coord = trans(self.frame, generic_frame)
  File "/usr/lib/python2.7/site-packages/astropy/coordinates/transformations.py", line 1314, in __call__
    curr_coord = t(curr_coord, curr_toframe)
  File "/usr/lib/python2.7/site-packages/astropy/coordinates/transformations.py", line 1232, in __call__
    newrep = self._apply_transform(fromcoord, M, None)
  File "/usr/lib/python2.7/site-packages/astropy/coordinates/transformations.py", line 1005, in _apply_transform
    rep = rep.transform(matrix)
  File "/usr/lib/python2.7/site-packages/astropy/coordinates/representation.py", line 1115, in transform
    newxyz = np.einsum('...ij,j...->i...', matrix, oldxyz.value)
  File "/usr/lib/python2.7/site-packages/numpy/core/einsumfunc.py", line 1087, in einsum
    einsum_call=True)
  File "/usr/lib/python2.7/site-packages/numpy/core/einsumfunc.py", line 688, in einsum_path
    input_subscripts, output_subscript, operands = _parse_einsum_input(operands)
  File "/usr/lib/python2.7/site-packages/numpy/core/einsumfunc.py", line 432, in _parse_einsum_input
    raise TypeError("For this input type lists must contain "
TypeError: For this input type lists must contain either int or Ellipsis
-------------------- >> begin captured stdout << ---------------------
coord =  1322.1 857.2
  PMC:  52.6012378091 -28.6654337942
  astropy:  52.601237809079514 -28.66543379422616
coord =  1 1
  PMC:  52.5338184835 -28.7606054233
  astropy:  52.53381848351462 -28.760605423292034
coord =  0 0
  PMC:  52.5337398407 -28.7606758543
  astropy:  52.53373984070186 -28.760675854311447
coord =  943.234 234.943
  PMC:  52.5505071271 -28.6929154246
  astropy:  52.55050712705404 -28.69291542456591
coord =  2048 2048
  PMC:  52.699732852 -28.6115665486
  astropy:  52.69973285203426 -28.611566548584047

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_reference.test_tpv
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/packages/python-pixmappy-git/python-pixmappy-git/src/python-pixmappy-git-py2/tests/test_reference.py", line 37, in test_tpv
    ra, dec = wcs1.toSky(x,y)
  File "/usr/lib/python2.7/site-packages/pixmappy/PixelMapCollection.py", line 432, in toSky
    return self.projection.toSky(xw, yw)
  File "/usr/lib/python2.7/site-packages/pixmappy/PixelMapCollection.py", line 658, in toSky
    return coord.icrs.ra.deg, coord.icrs.dec.deg
  File "/usr/lib/python2.7/site-packages/astropy/coordinates/sky_coordinate.py", line 517, in __getattr__
    return self.transform_to(attr)
  File "/usr/lib/python2.7/site-packages/astropy/coordinates/sky_coordinate.py", line 482, in transform_to
    new_coord = trans(self.frame, generic_frame)
  File "/usr/lib/python2.7/site-packages/astropy/coordinates/transformations.py", line 1314, in __call__
    curr_coord = t(curr_coord, curr_toframe)
  File "/usr/lib/python2.7/site-packages/astropy/coordinates/transformations.py", line 1232, in __call__
    newrep = self._apply_transform(fromcoord, M, None)
  File "/usr/lib/python2.7/site-packages/astropy/coordinates/transformations.py", line 1005, in _apply_transform
    rep = rep.transform(matrix)
  File "/usr/lib/python2.7/site-packages/astropy/coordinates/representation.py", line 1115, in transform
    newxyz = np.einsum('...ij,j...->i...', matrix, oldxyz.value)
  File "/usr/lib/python2.7/site-packages/numpy/core/einsumfunc.py", line 1087, in einsum
    einsum_call=True)
  File "/usr/lib/python2.7/site-packages/numpy/core/einsumfunc.py", line 688, in einsum_path
    input_subscripts, output_subscript, operands = _parse_einsum_input(operands)
  File "/usr/lib/python2.7/site-packages/numpy/core/einsumfunc.py", line 432, in _parse_einsum_input
    raise TypeError("For this input type lists must contain "
TypeError: For this input type lists must contain either int or Ellipsis
-------------------- >> begin captured stdout << ---------------------
coord =  1322.1 857.2
  PMC:  52.6012378091 -28.6654337942
  astropy:  52.601237809079514 -28.66543379422616
coord =  1 1
  PMC:  52.5338184835 -28.7606054233
  astropy:  52.53381848351462 -28.760605423292034
coord =  0 0
  PMC:  52.5337398407 -28.7606758543
  astropy:  52.53373984070186 -28.760675854311447
coord =  943.234 234.943
  PMC:  52.5505071271 -28.6929154246
  astropy:  52.55050712705404 -28.69291542456591
coord =  2048 2048
  PMC:  52.699732852 -28.6115665486
  astropy:  52.69973285203426 -28.611566548584047

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
Ran 10 tests in 11.811s

FAILED (errors=2)

So I accept that we should update cosmology test for python3. And that this is the main reason why none nosetests passess. However, since this package is running perfect in a ubuntu machine with python2 at least I should solve those two nosetests2 that are not passing. And how you can see here  https://github.com/astropy/astropy/issues/7051  the message

TypeError: For this input type lists must contain either int or Ellipsis

is due to the python-numpy version. Since dowgrading the package seems a crazy idea, then How can addapt the test to this new numpy version. I do not see a clear path for this.

Last edited by andalenavals (2018-02-01 00:23:38)

Offline

#18 2018-02-01 09:58:40

Morn
Member
Registered: 2012-09-02
Posts: 886

Re: Pixmappy PKGBUILD issues and sugestions

According to the page you linked to, this bug is fixed in the NumPy git version. So you could also install that for the moment until the next stable NumPy release comes out.

Here is an archived PKGBUILD for numpy-git that might still work: https://github.com/aur-archive/python-n … r/PKGBUILD

Offline

Board footer

Powered by FluxBB