Installing ObsPy from source on Rocks cluster

Users Mailing List Archive
1

Hi, having difficult installing obspy on rocks cluster.
I’ve performed a multi-user installation of Canopy on a rocks cluster (http://blog.enthought.com/python/installing-and-managing-a-central-python-install-with-enthought-canopy-v1-1/) and all users now have the correct python environment on their path.

easy_install does not work at all (and not only for obspy), it fails with:

[root@esc python]# ls
canopy-1.3.0-full-rh5-64.sh obspy obspy.zip
[root@esc python]# which easy_install
/usr/local/Canopy_64bit/User/bin/easy_install
[root@esc python]# easy_install -U obspy.zip
Traceback (most recent call last):
File “/usr/local/Canopy_64bit/User/bin/easy_install”, line 9, in
load_entry_point(‘distribute’, ‘console_scripts’, ‘easy_install’)()
File “/usr/local/Canopy_64bit/User/lib/python2.7/site-packages/setuptools-2.2-py2.7.egg/pkg_resources.py”, line 353, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File “/usr/local/Canopy_64bit/User/lib/python2.7/site-packages/setuptools-2.2-py2.7.egg/pkg_resources.py”, line 2320, in load_entry_point
raise ImportError(“Entry point %r not found” % ((group,name),))
ImportError: Entry point (‘console_scripts’, ‘easy_install’) not found

…so I’m trying to install from source:

[root@esc ~]# cd /share/apps/src/python/obspy
[root@esc obspy]# ls
CHANGELOG.txt debian misc README.md
CONTRIBUTING.md MANIFEST.in obspy setup.py
[root@esc obspy]# which python
/usr/local/Canopy_64bit/User/bin/python
[root@esc obspy]# su shahar
[shahar@esc obspy]$ which python
~/Enthought/Canopy_64bit/User/bin/python
[shahar@esc obspy]$ exit
exit
[root@esc obspy]# python setup.py develop -N -U --verbose
running develop
running build_scripts
running egg_info
running build_src
build_src
building extension “libgse2_Linux_64bit_py27” sources
building extension “libmseed_Linux_64bit_py27” sources
building extension “libsegy_Linux_64bit_py27” sources
building extension “libsignal_Linux_64bit_py27” sources
building extension “libevresp_Linux_64bit_py27” sources
building extension “libtau_Linux_64bit_py27” sources
f2py options: []
f2py: build/libtau_Linux_64bit_py27.pyf
Reading fortran codes…
Reading file ‘build/libtau_Linux_64bit_py27.pyf’ (format:free)
Post-processing…
Block: libtau_Linux_64bit_py27
Block: ttimes
Post-processing (stage 2)…
Building modules…
Building module “libtau_Linux_64bit_py27”…
Constructing wrapper function “ttimes”…
phcd,tt,toang,dtdd,dtdh,dddp = ttimes(deltain,zs,modnam)
Wrote C/API module “libtau_Linux_64bit_py27” to file “build/libtau_Linux_64bit_py27module.c”
adding ‘build/src.linux-x86_64-2.7/fortranobject.c’ to sources.
adding ‘build/src.linux-x86_64-2.7’ to include_dirs.
creating build/src.linux-x86_64-2.7
copying /usr/local/Canopy_64bit/User/lib/python2.7/site-packages/numpy/f2py/src/fortranobject.c -> build/src.linux-x86_64-2.7
copying /usr/local/Canopy_64bit/User/lib/python2.7/site-packages/numpy/f2py/src/fortranobject.h -> build/src.linux-x86_64-2.7
building data_files sources
build_src: building npy-pkg config files
creating obspy.egg-info
writing requirements to obspy.egg-info/requires.txt
writing obspy.egg-info/PKG-INFO
writing namespace_packages to obspy.egg-info/namespace_packages.txt
writing top-level names to obspy.egg-info/top_level.txt
writing dependency_links to obspy.egg-info/dependency_links.txt
writing entry points to obspy.egg-info/entry_points.txt
writing manifest file ‘obspy.egg-info/SOURCES.txt’
reading manifest file ‘obspy.egg-info/SOURCES.txt’
reading manifest template ‘MANIFEST.in’
no previously-included directories found matching ‘doc’
writing manifest file ‘obspy.egg-info/SOURCES.txt’
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
customize Gnu95FCompiler
Found executable /usr/bin/gfortran
customize Gnu95FCompiler
customize Gnu95FCompiler using build_ext
building ‘libgse2_Linux_64bit_py27’ extension
compiling C sources
C compiler: gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O2 -fPIC

creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/share
creating build/temp.linux-x86_64-2.7/share/apps
creating build/temp.linux-x86_64-2.7/share/apps/src
creating build/temp.linux-x86_64-2.7/share/apps/src/python
creating build/temp.linux-x86_64-2.7/share/apps/src/python/obspy
creating build/temp.linux-x86_64-2.7/share/apps/src/python/obspy/obspy
creating build/temp.linux-x86_64-2.7/share/apps/src/python/obspy/obspy/gse2
creating build/temp.linux-x86_64-2.7/share/apps/src/python/obspy/obspy/gse2/src
creating build/temp.linux-x86_64-2.7/share/apps/src/python/obspy/obspy/gse2/src/GSE_UTI
compile options: ‘-I/usr/local/Canopy_64bit/User/lib/python2.7/site-packages/numpy/core/include -I/root/Canopy/appdata/canopy-1.3.0.1715.rh5-x86_64/include/python2.7 -c’
gcc: /share/apps/src/python/obspy/obspy/gse2/src/GSE_UTI/gse_functions.c
gcc -pthread -shared -Wl,-rpath,’$ORIGIN/…/lib’ -g -L/home/builder/jenkins/workspace/Python_Masters/ARCH/x86_64/label/Linux/core/lib build/temp.linux-x86_64-2.7/share/apps/src/python/obspy/obspy/gse2/src/GSE_UTI/gse_functions.o -L/home/builder/jenkins/workspace/Python_Masters/ARCH/x86_64/label/Linux/core/lib -lpython2.7 -o obspy/lib/libgse2_Linux_64bit_py27.so
/usr/bin/ld: cannot find -lpython2.7
collect2: ld returned 1 exit status
/usr/bin/ld: cannot find -lpython2.7
collect2: ld returned 1 exit status
error: Command “gcc -pthread -shared -Wl,-rpath,’$ORIGIN/…/lib’ -g -L/home/builder/jenkins/workspace/Python_Masters/ARCH/x86_64/label/Linux/core/lib build/temp.linux-x86_64-2.7/share/apps/src/python/obspy/obspy/gse2/src/GSE_UTI/gse_functions.o -L/home/builder/jenkins/workspace/Python_Masters/ARCH/x86_64/label/Linux/core/lib -lpython2.7 -o obspy/lib/libgse2_Linux_64bit_py27.so” failed with exit status 1

Has anyone encountered this issue or has any insight as to why I’m failing at this?

Thanks,

Shahar

2

looks like python 2.7 header files are missing ... using Ubuntu you would run "aptitude install python-dev"

Cheers,
Robert

3

Hey Shahar,

gcc: /share/apps/src/python/obspy/obspy/gse2/src/GSE_UTI/gse_functions.c
gcc -pthread -shared -Wl,-rpath,'$ORIGIN/../lib' -g -L/home/builder/jenkins/workspace/Python_Masters/ARCH/x86_64/label/Linux/core/lib build/temp.linux-x86_64-2.7/share/apps/src/python/obspy/obspy/gse2/src/GSE_UTI/gse_functions.o -L/home/builder/jenkins/workspace/Python_Masters/ARCH/x86_64/label/Linux/core/lib -lpython2.7 -o obspy/lib/libgse2_Linux_64bit_py27.so
/usr/bin/ld: cannot find -lpython2.7
collect2: ld returned 1 exit status
/usr/bin/ld: cannot find -lpython2.7

It cannot find the python library.

$ locate libpython2.7

should give you some hints where it is. Take care to choose the correct one for your Python installation. You can then force the addition of the directory for the library with

LDFLAGS=“-L/path/to/dir” python setup.py develop -N -U —verbose

Hope it helps.

Cheers!

Lion

4

Thanks Lion,
I've made some progress:

[root@esc obspy]# LDFLAGS="-L/usr/local/Canopy_64bit/System/lib/" python setup.py develop -N -U --verbose

long output …

finally, it fails with:
/usr/lib/gcc/x86_64-redhat-linux/4.4.6/libgfortranbegin.a(fmain.o): In function `main':
(.text+0x26): undefined reference to `MAIN__'
collect2: ld returned 1 exit status
/usr/lib/gcc/x86_64-redhat-linux/4.4.6/libgfortranbegin.a(fmain.o): In function `main':
(.text+0x26): undefined reference to `MAIN__'
collect2: ld returned 1 exit status
error: Command "/usr/bin/gfortran -Wall -L/usr/local/Canopy_64bit/System/lib/ build/temp.linux-x86_64-2.7/build/libtau_Linux_64bit_py27module.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/fortranobject.o build/temp.linux-x86_64-2.7/share/apps/src/python/obspy/obspy/taup/src/emdlv.o build/temp.linux-x86_64-2.7/share/apps/src/python/obspy/obspy/taup/src/libtau.o build/temp.linux-x86_64-2.7/share/apps/src/python/obspy/obspy/taup/src/ttimes_subrout.o -L/home/builder/jenkins/workspace/Python_Masters/ARCH/x86_64/label/Linux/core/lib -lpython2.7 -lgfortran -o obspy/lib/libtau_Linux_64bit_py27.so" failed with exit status 1

what now?
Shahar

5

Hi Shahar,

I guess LDFLAGS overwrites the default linker flags. You just have to play with the linker flags a bit to get the desired output. Usually the setup.py should already set all necessary flags and I have no clue why you need to do it manually in the first place - might have to do with it being from Canopy.

Anyway, try this:

$ LDFLAGS="-L/usr/local/Canopy_64bit/System/lib -shared" python setup.py develop -N -U —verbose

or this

$ LD_LIBRARY_PATH="/usr/local/Canopy_64bit/System/lib:$LD_LIBRARY_PATH" python setup.py develop -N -U --verbose

Cheers!

Lion

6

Hi Lion,
so

$ LDFLAGS="-L/usr/local/Canopy_64bit/System/lib -shared" python setup.py develop -N -U —verbose

worked.

Adding obspy 0.9.1-252-g10c72 to easy-install.pth file
Installing obspy-mseed-recordanalyzer script to /usr/local/Canopy_64bit/User/bin
Installing obspy-scan script to /usr/local/Canopy_64bit/User/bin
Installing obspy-dataless2xseed script to /usr/local/Canopy_64bit/User/bin
Installing obspy-reftek-rescue script to /usr/local/Canopy_64bit/User/bin
Installing obspy-xseed2dataless script to /usr/local/Canopy_64bit/User/bin
Installing obspy-indexer script to /usr/local/Canopy_64bit/User/bin
Installing obspy-print script to /usr/local/Canopy_64bit/User/bin
Installing obspy-runtests script to /usr/local/Canopy_64bit/User/bin
Installing obspy-dataless2resp script to /usr/local/Canopy_64bit/User/bin
Installing obspy-plot script to /usr/local/Canopy_64bit/User/bin
Installing obspy-mopad script to /usr/local/Canopy_64bit/User/bin

Installed /share/apps/src/python/obspy

however, installation is not quite… working.

In [1]: from obspy import read

7

Since you run Python 2.7 you can avoid the dependency on "future" by
installing from the "releases" branch (instead of "master" branch).

Just do "git checkout releases" if you have a local git clone of the
repository, or if you downloaded a tarball use this link instead:
https://github.com/obspy/obspy/archive/releases.zip. If you don't depend
on the newest bugfixes you can also go with 0.9.1:

Tobias

8

interesting…
did what you suggested and it worked but when I do:

In [1]: from obspy import read

In [2]: st = read()

i get:

/usr/local/Canopy_64bit/User/lib/python2.7/site-packages/numpy/oldnumeric/__init__.py:11: ModuleDeprecationWarning: The oldnumeric module will be dropped in Numpy 1.9
  warnings.warn(_msg, ModuleDeprecationWarning)

9

sorry, I just realized that it worked, that was just a warning…

10

Hi Tobi,

a bit off-topic, but is there a 0.9.1 release out, I have missed? What's the status?
~petr

11

Hi Shahar,

I agree that the warning is pretty annoying.

You can get rid of it by simply removing the warning at the beginning of this file:

/usr/local/Canopy_64bit/User/lib/python2.7/site-packages/numpy/oldnumeric/__init__.py

It has no further consequences as the module will be gone in NumPy 1.9 in any case.

As Tobi suggested I would also strongly recommend going with the latest release and not one of our development branches as you are installing it on a production system.

Can you also run the `obspy-runtests` script to launch our test suite? I remember we had some issues on red hat systems.

Cheers!

Lion

12

I can confirm that there are still some rough edges with the development branch.

I specifically experienced problems with obspy.db and believe these are mainly due to string handling, probably introduces with Python 3 (and future) support. Not documented yet, due to lack of time for detailed testing. Similar problems might appear else, I guess.

~petr

13

We tagged a release at github but didn't find the time to do all the
releasing around it yet..
In any case we want to have a proper new release out pretty soon. Of
course the will be an announcement on the list.

Tobias