Data exchange - Python and Fortran

Question

We are developing a scientific application which has the interface in python 2.7 and the computation routines written in Intel Visual Fortran. Reading the source files is done using python, then only the required data for computations has to be passed to standalone Fortran algorithms. Once the computations done, the data has to be read by python once again.

Using formatted text files seems to be taking too long and not efficient. Further, we would like to have a standard intermediate format. There can be about 20 arrays and those are huge (if written to formatted text, the file is about 500 MB).

Q1. In a similar situation where Python and Fortran data exchange is necessary. What would be recommended way of interaction? (e.g.: writing an intermediate data to be read by the other or calling Fortran from within Python or using numpy to create compatible arrays or etc.)

Q2. If writing intermediate structures is recommended, What format is good for data exchange? (We came across CDF, NETCdf, binary streaming, but didn't try any so far.)

Answer 1

The standard way of wrapping Fortran code in Python is with f2py (included in the numpy module).

For the output of intermediary results, a number of formats could work, it really depends on your requirements.

• For simple datasets, from python, just use numpy.save.
• If your datasets become large, HDF5 with, for instance, PyTables in Python and libhdf5 in Fortran could be used.
• Otherwise, if you don't want to link your code to an external library, custom binary files written from Fortran and parsed with numpy could work too.

Answer 2

I would interface directly between Python and Fortran. It is relatively straightforward to allocate memory using Numpy and pass a pointer through to Fortran. You use iso_c_binding to write C-compatible wrapper routines for your Fortran routines and ctypes to load the Fortran .dll and call the wrappers. If you're interested I can throw together a simple example (but I am busy right this moment).