Wrapping C code including Python API using SWIG and distutils fails on Mac 10.8 64bits

Question

I have been trying to wrap an existing C code into a Python module for some time now and I keep running into recurrent errors and a failed build... After an extensive look at the available documentation and posts on this forum I would like to know if someone has an idea why it is not working.

I would like to use the stsci.imagemanip.interp2d.expand2d() function which consists of a python script calling a C function which uses the Python.h library. This under Mac 10.8 64bits with Python 2.7.

I am new to wrapping C functions to Python and I decided to use SWIG and distutils.

My input files are the following:

bilinearinterp.i

/* bilinearinterp.i */
%module bilinearinterp

%{
#define SWIG_FILE_WITH_INIT
#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
#include <stdio.h>
#include <Python.h>
#include <math.h>
#include <numpy/arrayobject.h>
#include "bilinearinterp.h"
%}

static PyObject * bilinearinterp(PyObject *obj, PyObject *args);

bilinearinterp.c

/* File: bilinearinterp.c */
#include <Python.h>
#include <stdio.h>
#include <math.h>
#include <numpy/arrayobject.h>
#include "bilinearinterp.h"

static void InterpInfo (float ai, int npts, int *i, float *p, float *q) {

/* arguments:
float ai        i: independent variable in same units as i
int npts        i: size of array within which *i is an index
int *i          o: array index close to ai
float *p, *q    o: weights for linear interpolation
*/

DO SOME CALCULATIONS 
}

int unbin2d (float *a, float *b, int inx, int iny, int onx, int ony) {

/* arguments:
PyArrayObject *a        i: input data
PyArrayObject *b        o: output data
*/

DO SOME CALCULATIONS 
CALL InterpInfo
return (1);
}

static PyObject * bilinearinterp(PyObject *obj, PyObject *args)
{
    PyObject *input, *output;
    PyArrayObject *dataa, *datab;
    int inx, iny, onx, ony;

    int status=0;

    if (!PyArg_ParseTuple(args,"OO:bilinearinterp",&input,&output))
        return NULL;

    dataa = (PyArrayObject *)PyArray_ContiguousFromAny(input, PyArray_FLOAT, 1, 2);
    datab = (PyArrayObject *)PyArray_ContiguousFromAny(output, PyArray_FLOAT, 1, 2);

    inx = PyArray_DIM(input,0);
    iny = PyArray_DIM(input,1);
    onx = PyArray_DIM(output,0);
    ony = PyArray_DIM(output,1);

    status = unbin2d((float *)dataa->data,(float *)datab->data, inx, iny, onx, ony);

    Py_XDECREF(dataa);
    Py_XDECREF(datab);

    return Py_BuildValue("i",status);
}

static PyMethodDef bilinearinterp_methods[] =
{
    {"bilinearinterp",  bilinearinterp, METH_VARARGS, 
        "bilinearinterp(input, output)"},
    {0,            0}                             /* sentinel */
};

void initbilinearinterp(void) {
    Py_InitModule("bilinearinterp", bilinearinterp_methods);
    import_array();
}

bilinearinterp.h

/* File: bilinearinterp.H */
static PyObject * bilinearinterp(PyObject *obj, PyObject *args);

setup.py

#!/usr/bin/env python

from distutils.core import setup, Extension

bilinearinterp_module = Extension('_bilinearinterp',
                                  sources = ['bilinearinterp_wrap.c','bilinearinterp.c'],
# Path to locate numpy/arrayobject.h     
                                  include_dirs=['/Library/Python/2.7/site-packages/numpy-override/numpy/core/include'])

setup (name = 'bilinearinterp',
       version = '1.1',
       author      = "Space Telescope Science Institute - stsci_python",
       description = """bilinear interpretation for 2D array extrapolation""",
       ext_modules = [bilinearinterp_module],
       py_modules = ["bilinearinterp"],
       )

Then I run the following in Terminal:

>>>swig -python bilinearinterp.i
>>>python setup.py build_ext --inplace

The appropriate files are being created: _bilinearinterp.so bilinearinterp.py bilinearinterp.pyc bilinearinterp_wrap.c As well as a build directory containing some files in it.

I get warnings when setup.py is running but it seems to complete. Over the various tests I did two recurrent errors keep coming back:

1. warning: "Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]

   Which may induces several other warnings associated to numpy

2. warning: function 'bilinearinterp' has internal linkage but is not defined [-Wundefined-internal] static PyObject * bilinearinterp(PyObject *obj, PyObject *args); bilinearinterp_wrap.c:2971:24: note: used here result = (PyObject *)bilinearinterp(arg1,arg2);

Then when calling the module in a python script I get the following:

>>> from bilinearinterp import bilinearinterp as lininterp
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "bilinearinterp.py", line 28, in <module>
    _bilinearinterp = swig_import_helper()
  File "bilinearinterp.py", line 24, in swig_import_helper
    _mod = imp.load_module('_bilinearinterp', fp, pathname, description)
ImportError: dlopen(./_bilinearinterp.so, 2): Symbol not found: _bilinearinterp
  Referenced from: ./_bilinearinterp.so
  Expected in: flat namespace
 in ./_bilinearinterp.so

Would someone have an idea from where the error could be coming from?

Is it from my usage of SWIG or from setup.py?

---

Another approach with only a swig interface file and the C code (without including the *.h file) does not work. Based on the swig introduction chapter

bilinearinterp.i

/* bilinearinterp.i */
%module bilinearinterp

%{
static PyObject * bilinearinterp(PyObject *obj, PyObject *args);
%}

static PyObject * bilinearinterp(PyObject *obj, PyObject *args);

And I compile using:

>>>swig -python -o bilinearinterp_wrap.c bilinearinterp.i
>>>gcc -c -fpic bilinearinterp.c bilinearinterp_wrap.c
-I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
-I/System//Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include
bilinearinterp_wrap.c:2955:19: warning: function 'bilinearinterp' has internal
      linkage but is not defined [-Wundefined-internal]
static PyObject * bilinearinterp(PyObject *obj, PyObject *args);
                  ^
bilinearinterp_wrap.c:2971:24: note: used here
  result = (PyObject *)bilinearinterp(arg1,arg2);
                       ^
1 warning generated.
>>>gcc -bundle -flat_namespace -undefined suppress bilinearinterp.o bilinearinterp_wrap.o -o _bilinearinterp.so

Then I still get the same import error in the python interpreter. Could it be due to my numpy installation or my gcc arguments?

Thanks

Answer 1

Well, I don't quite understand what you are trying to achieve, but I see some bad practices. For example, you make bilinearinterp everywhere static, and static functions are not exported and only can be used in the same module where implemented. I think it would be helpful to read a bit about SWIG design ideas. For sure it will take a some time, but you will get a broader perspective of what SWIG is and isn't capable of.