I am wrapping a c++ program via cython and would like to make a vector from this program available in Python.
I have written a getter method for the vector, which works fine except that it returns a copy of the data:
# in cython
from libcpp.vector cimport vector
cimport numpy as np
cdef np.ndarray[double] vector_to_double_arr(vector[double] vec):
cdef double[::1] arr = <double [:vec.size()]>vec.data()
return np.asarray(arr)
def get_variable()
return vector_to_double_arr(my_c_vector) # my_c_vector is a global variable defined elsewhere
That is, in python, an assignment to the returned array has no effect, since the value is assigned to the copy only.
# in python
get_variable()[0] = 4 # has no effect
In the past, I have circumvented this problem by creating numpy arrays first and operating in c++ on the internal data. However, now I have to deal with a pre-existing vector.
I have thought of returning the memoryview directly:
cdef np.ndarray[double] vector_to_double_arr(vector[double] vec):
cdef double[::1] arr = <double [:vec.size()]>vec.data()
return arr
However, I have not figured out how to make the memoryview appear as an array in Python. I would need some object that wraps this memoryview and can write to it in a proper way. I was thinking of creating a numpy array and assigning to the np.ndarray.data
property, but writing to it is bad practice and did not work.
How could I write to the c++ vector from Python? As an extension: how would I do this, if I were not dealing with a vector but a double array?
Edit
I have now implemented the solution suggested by @ead by copying the code:
from libc.stdlib cimport free
from cpython.object cimport PyObject
from cpython.ref cimport Py_INCREF
np.import_array()
cdef class MemoryNanny:
cdef void* ptr # set to NULL by "constructor"
def __dealloc__(self):
print("freeing ptr=", <unsigned long long>(self.ptr)) #just for debugging
free(self.ptr)
@staticmethod
cdef create(void* ptr):
cdef MemoryNanny result = MemoryNanny()
result.ptr = ptr
print("nanny for ptr=", <unsigned long long>(result.ptr)) #just for debugging
return result
cdef extern from "numpy/arrayobject.h":
# a little bit awkward: the reference to obj will be stolen
# using PyObject* to signal that Cython cannot handle it automatically
int PyArray_SetBaseObject(np.ndarray arr, PyObject *obj) except -1 # -1 means there was an error
cdef array_from_ptr(void * ptr, np.npy_intp N, int np_type):
cdef np.ndarray arr = np.PyArray_SimpleNewFromData(1, &N, np_type, ptr)
nanny = MemoryNanny.create(ptr)
Py_INCREF(nanny) # a reference will get stolen, so prepare nanny
PyArray_SetBaseObject(arr, <PyObject*>nanny)
return arr
I access the pointer then as follows:
cdef MyClass()
...
@property
def myArray(self):
return array_from_ptr(myArrayPointer, myArrayPointerLength, np.NPY_FLOAT64)
Accessing the property from Python works well, but as soon as I change any value in the array (which works, too) the program halts.
m = MyClass()
print(m.myArray) # prints what it is supposed to print
m.myArray[0] = 1 # works
print(m.myArray) # prints what is expected, but terminates the program after freeing the freeing the pointer.
...
When I delete the last print statement in the code block above, everything works still. Why? I am puzzled. What could I do to solve the problem?