GNU Info

(python2.1-ext.info)Providing a C API for an Extension Module

---

Prev: Writing Extensions in C++ Up: Extending Python with C or C++

---

Enter node , (file) or (file)node

Providing a C API for an Extension Module
=========================================

This manual section was written by Konrad Hinsen
<hinsen@cnrs-orleans.fr>.
Many extension modules just provide new functions and types to be used
from Python, but sometimes the code in an extension module can be
useful for other extension modules. For example, an extension module
could implement a type "collection" which works like lists without
order. Just like the standard Python list type has a C API which
permits extension modules to create and manipulate lists, this new
collection type should have a set of C functions for direct
manipulation from other extension modules.

At first sight this seems easy: just write the functions (without
declaring them `static', of course), provide an appropriate header
file, and document the C API. And in fact this would work if all
extension modules were always linked statically with the Python
interpreter. When modules are used as shared libraries, however, the
symbols defined in one module may not be visible to another module.
The details of visibility depend on the operating system; some systems
use one global namespace for the Python interpreter and all extension
modules (e.g. Windows), whereas others require an explicit list of
imported symbols at module link time (e.g. AIX), or offer a choice of
different strategies (most Unices). And even if symbols are globally
visible, the module whose functions one wishes to call might not have
been loaded yet!

Portability therefore requires not to make any assumptions about symbol
visibility. This means that all symbols in extension modules should be
declared `static', except for the module's initialization function, in
order to avoid name clashes with other extension modules (as discussed
in section~Note: Module's Method Table and Initialization Function).
And it means that symbols that _should_ be accessible from other
extension modules must be exported in a different way.

Python provides a special mechanism to pass C-level information (i.e.
pointers) from one extension module to another one: CObjects.  A
CObject is a Python data type which stores a pointer (`void *').
CObjects can only be created and accessed via their C API, but they can
be passed around like any other Python object. In particular, they can
be assigned to a name in an extension module's namespace.  Other
extension modules can then import this module, retrieve the value of
this name, and then retrieve the pointer from the CObject.

There are many ways in which CObjects can be used to export the C API
of an extension module. Each name could get its own CObject, or all C
API pointers could be stored in an array whose address is published in
a CObject. And the various tasks of storing and retrieving the pointers
can be distributed in different ways between the module providing the
code and the client modules.

The following example demonstrates an approach that puts most of the
burden on the writer of the exporting module, which is appropriate for
commonly used library modules. It stores all C API pointers (just one
in the example!) in an array of `void' pointers which becomes the value
of a CObject. The header file corresponding to the module provides a
macro that takes care of importing the module and retrieving its C API
pointers; client modules only have to call this macro before accessing
the C API.

The exporting module is a modification of the `spam' module from
section~Note: A Simple Example. The function `spam.system()' does not
call the C library function `system()' directly, but a function
`PySpam_System()', which would of course do something more complicated
in reality (such as adding "spam" to every command). This function
`PySpam_System()' is also exported to other extension modules.

The function `PySpam_System()' is a plain C function, declared `static'
like everything else:

     static int
     PySpam_System(command)
         char *command;
     {
         return system(command);
     }

The function `spam_system()' is modified in a trivial way:

     static PyObject *
     spam_system(self, args)
         PyObject *self;
         PyObject *args;
     {
         char *command;
         int sts;
     
         if (!PyArg_ParseTuple(args, "s", &command))
             return NULL;
         sts = PySpam_System(command);
         return Py_BuildValue("i", sts);
     }

In the beginning of the module, right after the line

     #include "Python.h"

two more lines must be added:

     #define SPAM_MODULE
     #include "spammodule.h"

The `#define' is used to tell the header file that it is being included
in the exporting module, not a client module. Finally, the module's
initialization function must take care of initializing the C API
pointer array:

     void
     initspam()
     {
         PyObject *m;
         static void *PySpam_API[PySpam_API_pointers];
         PyObject *c_api_object;
     
         m = Py_InitModule("spam", SpamMethods);
     
         /* Initialize the C API pointer array */
         PySpam_API[PySpam_System_NUM] = (void *)PySpam_System;
     
         /* Create a CObject containing the API pointer array's address */
         c_api_object = PyCObject_FromVoidPtr((void *)PySpam_API, NULL);
     
         if (c_api_object != NULL) {
             /* Create a name for this object in the module's namespace */
             PyObject *d = PyModule_GetDict(m);
     
             PyDict_SetItemString(d, "_C_API", c_api_object);
             Py_DECREF(c_api_object);
         }
     }

Note that `PySpam_API' is declared `static'; otherwise the pointer
array would disappear when `initspam' terminates!

The bulk of the work is in the header file `spammodule.h', which looks
like this:

     #ifndef Py_SPAMMODULE_H
     #define Py_SPAMMODULE_H
     #ifdef __cplusplus
     extern "C" {
     #endif
     
     /* Header file for spammodule */
     
     /* C API functions */
     #define PySpam_System_NUM 0
     #define PySpam_System_RETURN int
     #define PySpam_System_PROTO (char *command)
     
     /* Total number of C API pointers */
     #define PySpam_API_pointers 1
     
     #ifdef SPAM_MODULE
     /* This section is used when compiling spammodule.c */
     
     static PySpam_System_RETURN PySpam_System PySpam_System_PROTO;
     
     #else
     /* This section is used in modules that use spammodule's API */
     
     static void **PySpam_API;
     
     #define PySpam_System \
      (*(PySpam_System_RETURN (*)PySpam_System_PROTO) PySpam_API[PySpam_System_NUM])
     
     #define import_spam() \
     { \
       PyObject *module = PyImport_ImportModule("spam"); \
       if (module != NULL) { \
         PyObject *module_dict = PyModule_GetDict(module); \
         PyObject *c_api_object = PyDict_GetItemString(module_dict, "_C_API"); \
         if (PyCObject_Check(c_api_object)) { \
           PySpam_API = (void **)PyCObject_AsVoidPtr(c_api_object); \
         } \
       } \
     }
     
     #endif
     
     #ifdef __cplusplus
     }
     #endif
     
     #endif /* !defined(Py_SPAMMODULE_H */

All that a client module must do in order to have access to the
function `PySpam_System()' is to call the function (or rather macro)
`import_spam()' in its initialization function:

     void
     initclient()
     {
         PyObject *m;
     
         Py_InitModule("client", ClientMethods);
         import_spam();
     }

The main disadvantage of this approach is that the file `spammodule.h'
is rather complicated. However, the basic structure is the same for
each function that is exported, so it has to be learned only once.

Finally it should be mentioned that CObjects offer additional
functionality, which is especially useful for memory allocation and
deallocation of the pointer stored in a CObject. The details are
described in the  in the section "CObjects" and in the implementation
of CObjects (files `Include/cobject.h' and `Objects/cobject.c' in the
Python source code distribution).