[SOLVED] Make dl work in Python bindings

snack · 2011-02-11 08:53:23

Hi, I'm totally new to Python and bindings to C++ but at work I'm supposed to work with them so I need a little help. I must write Python bindings to a C++ class and currently I'm trying to use Boost::Python to produce an experimental libbindings.so. I then import it in python2 typing "import libbindings" at the prompt, and everything is fine. The problem is that my class constructor dynamically loads a .so library using dlopen:

void *_gcdLibrary = dlopen(geoLib.data(), RTLD_LAZY);

but the library isn't loaded since my standard error message prints this value for dlerror():

undefined symbol: _ZTI27G4VUserDetectorConstruction

That symbol is actually defined in a library linked to libbindings.so, but evidently there's something not working...
I don't know if this kind of error can be ironed out. If yes, and if there is someone willing to help I can provide more info. BTW, I'm not compiling the bindings using bjam as recommended (since I can't make it work) but instead I use cmake; maybe this can be the problem...
Thanks

Last edited by snack (2011-02-14 15:52:18)

tavianator · 2011-02-12 00:40:16

I believe you will need to pass "RTLD_LAZY | RTLD_GLOBAL" in order for other code to resolve the RTTI for that class (_ZTI27G4VUserDetectorConstruction demangles to "typeinfo for G4VUserDetectorConstruction").

snack · 2011-02-12 14:13:30

Thanks for the suggestion, tavianator, but it doesn't work. I double-checked with grep for _ZTI27G4VUserDetectorConstruction in the shared library I try to load and it says that the binary file contains that string (I don't know how a more precise check can be made). Moreover, the code works fine when compiled with gcc and run...
Maybe it's worth pointing out that I wrote the binding only for my class, not for G4VUserDetectorConstruction. In my naive noob-ness, I thought that if my class wraps the handling of dlopen and all the stuff coming from the dynamically loaded library, then to use it from python shell I would have to write bindings only for my class and not for what is used inside its methods... maybe I am wrong?

tavianator · 2011-02-12 16:49:23

No you're right, there's no reason that you'd have to write a python binding for that class. But based on how you seem to have written the binding, I think I now know what's going on. It's quite likely that your code needs to reference something about the class it uses before the body of the constructor is run, so using dlopen() in your constructor is probably too late. Instead, why not just mark that library as an explicit dependency of yours, so that python loads both? You can do this by linking your library itself with -l<whatever>.

EDIT: for the record, a more precise check could have been made with "nm <library> | grep _ZTIwhatever" which would show you the type of symbol and it's location as well.

Last edited by tavianator (2011-02-12 16:50:57)

snack · 2011-02-14 14:35:37

I don't link the library I want to load to the one containing my wrapper class because it's meant to be a plugin library. I want to be able to select at runtime which library should be loaded by my class. I tried also to load it directly from python2, but:

>>> import dl
>>> geolib = dl.open("/home/mori/software/GCDSoftware/lib/libGCDGeometryConstructionGEOV01R00.so")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
dl.error: /home/mori/software/GCDSoftware/lib/libGCDGeometryConstructionGEOV01R00.so: undefined symbol: _ZTI27G4VUserDetectorConstruction

It makes me think that the problem is inside the library itself, and that it is not related to how the wrapper binding is written... Maybe the problem is that G4VUserDetectorConstruction is a virtual class? It has a pure virtual method implemented in a derived class, and this implementation is contained inside the library...
Anyway, checking the library with nm I get:

$ nm $GCD_SYS/lib/libGCDGeometryConstructionGEOV01R00.so | grep _ZTI27G4VUserDetectorConstruction
2243:         U _ZTI27G4VUserDetectorConstruction

snack · 2011-02-14 15:51:35

I think I found the solution: the missing symbol was defined in another library, which was not linked to the one I was trying to dynamically load. It is anyway strange because my wrapper library actually linked it. To be a bit more clear: my wrapper class is in a library, call it libWrapper. The library I try to load is libPlugin, and the missing symbol is in libSymbol. libWrapper is linked to libSymbol, but libPlugin is not, and this generates the error under the python shell when I try to invoke wrapper's constructor (which calls dlopen("libPlugin")). If I link libPlugin against libsymbol, then everything is fine and wrapper's constructor is executed without problems. I find it strange because when constructor is called, libWrapper has also been loaded and also libSymbol, so _ZTI27G4VUserDetectorConstruction should be available, but dlopen gets angry anyway.
But I'm not an expert, so maybe this is correct... anyway, it seems to work now, so thanks again tavianator!

Arch Linux

#1 2011-02-11 08:53:23

[SOLVED] Make dl work in Python bindings

#2 2011-02-12 00:40:16

Re: [SOLVED] Make dl work in Python bindings

#3 2011-02-12 14:13:30

Re: [SOLVED] Make dl work in Python bindings

#4 2011-02-12 16:49:23

Re: [SOLVED] Make dl work in Python bindings

#5 2011-02-14 14:35:37

Re: [SOLVED] Make dl work in Python bindings

#6 2011-02-14 15:51:35

Re: [SOLVED] Make dl work in Python bindings

Board footer