[Python] Building Extensions for Python in C

dragonhawk360 · 2010-11-05 01:05:29

I'm learning some of the more advanced features of Python (using 2.7 from repos), and one feature I want to learn is building extensions in C, but not many reliable tutorials exist online. They are either outdated, or irrelevant (serving as ads for books). The official documentation is very confusing as well; read it a few times and am still lost. Is Python.h installed already with Arch, or is there a specifically needed package? Must I do a custom build of python?

Yannick_LM · 2010-11-05 01:40:09

Just to be sure, by "official", you mean this doc:

http://docs.python.org/extending/extending.html ?

I personally found it quite clear

You can still fall back to using swig if you don't mind trusting auto-generated code:

http://www.swig.org/tutorial.html

>> Is Python.h installed already with Arch ?

Yes:

/usr/include/python3.1/Python.h

>> Must I do a custom build of python?

I don't think that's necessary.

dragonhawk360 · 2010-11-05 02:01:06

Yannick_LM wrote:

Just to be sure, by "official", you mean this doc:
http://docs.python.org/extending/extending.html ?
I personally found it quite clear :)
You can still fall back to using swig if you don't mind trusting auto-generated code:
http://www.swig.org/tutorial.html

Exactly the documentation I was talking about. I considered it official as it was hosted on their site along with all of the documentation for Python. I'll go back and reread it a little more carefully, maybe give it a few days to "soak in." Also, with SWIG (haven't looked just yet), is this a recommended option for an extension development system? Auto-generated code always seems to be a big hit-or-miss, and I don't want to spend time learning how to use it if it's buggy.

>> Is Python.h installed already with Arch ?
Yes:
/usr/include/python3.1/Python.h

I am running Python 2.7, but after a quick look, I see it's in the same location.

>> Must I do a custom build of python?
I don't think that's necessary.

Thank goodness. I was afraid of the time it might have taken to do this. Thank you for your help so far, Yannick; it really pushed me forward.

Foucault · 2010-11-05 15:56:16

Although my C skills are limited I find easy to use Pyrex to interface python to C. You can either write C extensions indirectly by using a specialized python dialect that translates to C code, or write a generic C library and interface it via Pyrex to python code. Unless you have some specialized needs, I think it is easier (albeit less agile) to use Pyrex than plunge into python internals. It, also, seems cleaner and more "pythonic" to me than SWIG.

dragonhawk360 · 2010-11-05 20:15:01

Pyrex is a very nice interface, but the fatal fault is it cannot load and use *.h files, or preprocessor statements. It also is limited in the creation of class objects. Documentation strings are limited. And a few others that don't matter to my just messing with it, but if I ever decide to make a serious extension, I probably wouldn't use it.

kachelaqa · 2010-11-06 17:49:38

dragonhawk360 wrote:

Pyrex is a very nice interface, but the fatal fault is it cannot load and use *.h files, or preprocessor statements. It also is limited in the creation of class objects. Documentation strings are limited. And a few others that don't matter to my just messing with it, but if I ever decide to make a serious extension, I probably wouldn't use it.

you may find cython to be much more to your liking: http://cython.org/

it's a good entry point for developing python extensions, because it is very easy to get started (even if you don't know much C).

there's a tutorial here: http://conference.scipy.org/proceedings … 9/paper_1/

dragonhawk360 · 2010-11-07 14:37:23

Just like to point out I do understand C and the Standard Library quite well, since I did not make that clear on the first post.

@kachelaqa, Cython reads to me much like SWIG did. Same way to define c-types, functions, and even building it. Difference is in the documentation. But it is definitely something else to look at.

cactus · 2010-11-07 22:11:57

I honestly found cython to be a bit confusing and difficult to use. Could just be the lack of concise examples for wrapping C libraries with anything remotely complicated going on (like using structs and pointers to structs). I found writing the c-module directly to be easier.

If you are doing something simple (calling a C library function once or twice), then something like ctypes is useful. There is alot of function call overhead though, so I wouldn't recommend using it if you will be calling the C code frequently -- then your best bet is indeed writing a c-module.

kachelaqa · 2010-11-08 20:15:44

dragonhawk360 wrote:

Just like to point out I do understand C and the Standard Library quite well, since I did not make that clear on the first post.
@kachelaqa, Cython reads to me much like SWIG did. Same way to define c-types, functions, and even building it. Difference is in the documentation. But it is definitely something else to look at.

if you didn't find it already, there is also useful page on the cython wiki which compares various ways to wrap c/c++ to python:

http://wiki.cython.org/WrappingCorCpp

dragonhawk360 · 2010-11-09 21:38:22

cactus wrote:

I honestly found cython to be a bit confusing and difficult to use. Could just be the lack of concise examples for wrapping C libraries with anything remotely complicated going on (like using structs and pointers to structs). I found writing the c-module directly to be easier.
If you are doing something simple (calling a C library function once or twice), then something like ctypes is useful. There is alot of function call overhead though, so I wouldn't recommend using it if you will be calling the C code frequently -- then your best bet is indeed writing a c-module.

This is what I've decided upon. Other than the documentation on the python site, does the community have any other nice resources or tutorials?

cactus · 2010-11-14 05:24:02

Some links I found that seem reasonable upon cursory review.

http://starship.python.net/crew/arcege/ … pyext.html
http://starship.python.net/crew/mwh/toext/
http://www.dalkescientific.com/writings … sions.html
http://docs.python.org/extending/extending.html

dragonhawk360 · 2010-11-14 18:17:29

cactus wrote:

Some links I found that seem reasonable upon cursory review.
http://starship.python.net/crew/arcege/ … pyext.html
http://starship.python.net/crew/mwh/toext/
http://www.dalkescientific.com/writings … sions.html
http://docs.python.org/extending/extending.html

I will have to read all of these then. Thank you very much.

Arch Linux

#1 2010-11-05 01:05:29

[Python] Building Extensions for Python in C

#2 2010-11-05 01:40:09

Re: [Python] Building Extensions for Python in C

#3 2010-11-05 02:01:06

Re: [Python] Building Extensions for Python in C

#4 2010-11-05 15:56:16

Re: [Python] Building Extensions for Python in C

#5 2010-11-05 20:15:01

Re: [Python] Building Extensions for Python in C

#6 2010-11-06 17:49:38

Re: [Python] Building Extensions for Python in C

#7 2010-11-07 14:37:23

Re: [Python] Building Extensions for Python in C

#8 2010-11-07 22:11:57

Re: [Python] Building Extensions for Python in C

#9 2010-11-08 20:15:44

Re: [Python] Building Extensions for Python in C

#10 2010-11-09 21:38:22

Re: [Python] Building Extensions for Python in C

#11 2010-11-14 05:24:02

Re: [Python] Building Extensions for Python in C

#12 2010-11-14 18:17:29

Re: [Python] Building Extensions for Python in C

Board footer