Arch Linux

djscholl

Member

From: Michigan, USA

Registered: 2006-05-24

Posts: 56

hdf4 and pyhdf AUR PKGBUILDS for review

pyhdf is a Python interface to the HDF4 library. HDF4 is a scientific data file format. pyhdf links to the static libs built by the HDF4 package. I decided to split HDF4 out into its own package because someone else might want to link to these libraries, or use the numerous HDF4 utilities that accompany them. There is an hdf5 package in the AUR already, but hdf5 coexists with hdf4, rather than replacing it.

Here is the HDF4 PKGBUILD:

# Contributor: David Scholl <djscholl>
pkgname=hdf4
pkgver=2r1
pkgrel=1
pkgdesc="General purpose library and file format for storing scientific data."
url="http://hdf.ncsa.uiuc.edu/hdf4.html"
license="custom"
depends=('zlib' 'libjpeg')
makedepends=(gcc-fortran)
source=(ftp://ftp.ncsa.uiuc.edu/HDF/HDF/HDF_Current/src/HDF4.$pkgver.tar.gz ftp://ftp.ncsa.uiuc.edu/HDF/HDF/HDF_Current/src/patches/configure)
md5sums=('9082c6fa913b9188452fa6c5217e1573' '845b0e1a20c0a969fca386b8830236ed')

build() {
  cp ./configure $startdir/src/HDF4.$pkgver
  cd $startdir/src/HDF4.$pkgver
  ./configure --prefix=/usr F77=gfortran FFLAGS=-ffixed-line-length-0
  make || return 1
  make prefix=$startdir/pkg/usr install
  mkdir -p $startdir/pkg/usr/share/licenses/hdf4
  cp $startdir/src/HDF4.$pkgver/COPYING $startdir/pkg/usr/share/licenses/hdf4
}

HDF4 was originally built with commercial Fortran 77 compilers, although g77 worked too. Since g77 is gone, I built it with gfortran. I encountered a couple of issues with the "make check" command, which compiles and runs tests of the libraries and utilities. The main package builds without errors using plain gfortran. However, the test code contains lines longer than 72 characters, so I had to add the FFLAGS=-ffixed-line-length-0 option to the configure script to be able to build the tests. When I manually do

cd $startdir/src/HDF4.$pkgver
make check || return 1

all of the tests are built and passed without errors. However, when I put these two lines into the build script, the tests fail as follows:

============================
HDF-SD C interfaces tests
============================
srcdir="." ./hdftest > hdfout.new
make[2]: *** [check] Error 1
make[2]: Leaving directory `/home/dscholl/abs/hdf4check/src/HDF4.2r1/mfhdf/libsrc'
make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory `/home/dscholl/abs/hdf4check/src/HDF4.2r1/mfhdf'
make: *** [check-recursive] Error 1
==> ERROR: Build Failed.  Aborting...

I would like to include the tests in the build, because it's bad when your data gets mangled. However, I don't know how to get the tests to run in the build script. If anyone has any suggestions I would appreciate them.

HDF4 is released under a license which resembles other Open Source licenses I have read, so I treated it as a custom license. The HDF4 package can be built with an optional compression library called szip, which is not an Open Source product. szip is available under non-commercial and commercial licenses, but prospective commercial users must contact the licensing agent. The AUR doesn't have a way to make that clear to prospective users, so I left out szip.

Here is the PKGBUILD for pyhdf:

# Contributor: David Scholl <djscholl>
pkgname=pyhdf
pkgver=0.7.3
pkgrel=1
pkgdesc="Python bindings for the HDF library."
url="http://pysclint.sourceforge.net/pyhdf/"
license="Python"
depends=('python' 'python-numeric' 'zlib' 'libjpeg')
makedepends=(hdf4)
source=(http://dl.sourceforge.net/pysclint/$pkgname-0.7-3.tar.gz)
md5sums=('f55aa19e61cf6501f436e27783098000')

build() {
  cd $startdir/src/$pkgname-0.7-3
  sed -i '/#extra_compile_args=["-DNOSZIP"],/s/#//' setup.py
  sed -i '/libraries = ["mfhdf", "df", "jpeg", "z", "sz"]/s/libraries/#libraries/' setup.py
  sed -i '/#libraries = ["mfhdf", "df", "jpeg", "z"]/s/#//' setup.py
  python setup.py install --root=$startdir/pkg
}

pyhdf assumes the availability of szip by default, so its setup.py must be hacked to build it without szip.  I do this with three sed commands, following the instructions in the INSTALL file and the comments in the setup.py. The sourceforge page for pyhdf describes it as being released under the Python license, but there is no copyright file included with the source distribution, nor did I find one on the project website.

If anyone has corrections or improvements to suggest, please post.

djscholl

Member

From: Michigan, USA

Registered: 2006-05-24

Posts: 56

Re: hdf4 and pyhdf AUR PKGBUILDS for review

I have done some further digging into the "make check" issue, but I don't know enough to solve it. I'm hoping someone will be able to figure out a patch or work-around that I can use.

The error message announces leaving the src/HDF4.2r1/mfhdf/libsrc directory. We can find the first part of the error message in the Makefile:

less -N src/HDF4.2r1/mfhdf/libsrc/Makefile

    729 check:
    730         @echo "============================"
    731         @echo "HDF-SD C interfaces tests"
    732         @echo "============================"
    733         srcdir="$(srcdir)" ./hdftest > hdfout.new
    734         @cmd="$(DIFF) hdfout.new $(srcdir)/hdfout.sav";                 

It appears to me that line 733 bombs when "make check" is run from the build script, and the diff command in 734 doesn't get a chance to run.

The hdfout.new from the build script "make check" ends in

*** UNEXPECTED VALUE from second SDstart is 393216 at line   61 in tsd.c
num_err == 1

The file src/HDF4.2r1/mfhdf/libsrc/hdfout.sav ends in

num_err == 0

less -N src/HDF4.2r1/mfhdf/libsrc/tsd.c
     
     24 #define FILE_NAME     "sdtest.hdf"      /* data file to test ID types */
     
     59     /* Create a protected file */
     60     fid = SDstart(FILE_NAME, DFACC_CREATE);
     61     VERIFY(fid, FAIL, "second SDstart");

Running "make check" manually:

$ ls -l sdtest.hdf
--w------- 1 dscholl dscholl 2503 2006-08-13 09:45 sdtest.hdf

Running "make check" from within the build script:

$ ls -l sdtest.hdf
-rw------- 1 dscholl dscholl 2502 2006-08-13 09:48 sdtest.hdf

I'm not a C programmer, but it appears to me that the hdftest binary is trying to create a read-protected --w------- file, and actually creating a -rw------- file. When it tries to read it, it generates an error message and the Makefile stops running. What is different about the build script environment that would cause this difference in file creation?

iKevin

Member

From: Ann Arbor, MI

Registered: 2003-07-20

Posts: 132

Re: hdf4 and pyhdf AUR PKGBUILDS for review

I sent you email :-)

Kev

djscholl

Member

From: Michigan, USA

Registered: 2006-05-24

Posts: 56

Re: hdf4 and pyhdf AUR PKGBUILDS for review

With help from ikevin, I was able to figure out why the "make check" wouldn't run as root. One of the tests for the netcdf library checks whether the library can write-protect a file by creating a file, restricting its permissions, and verifying that overwriting it fails. Root can always overwrite a write-protected file, so the test fails (the overwrite succeeds) when run as root.

I put in a couple of sed commands to comment out that specific test, and the remainder runs fine. In my opinion, automating the test for numerical correctness is more important than being able to verify that the file system's write protection is functional. I'll go ahead and upload the packages, since there were no other comments.