I am trying to use MPI_Op_create() to create my own reduction so I can pass a custom structure type for function MPI_Allreduce(). See link for example: http://www.netlib.org/utk/papers/mpi-book/node118.html

The reduction I define has signiture:

void reduction_op(data_t *in, data_t *inout, int *len, MPI_Datatype * datatype)

Where data_t is the name of my custom structure. If I pass reduction_op to MPI_OP_create() I get the following compiler error:

kmeans_short.cpp:60:5: error: no matching function for call to 'MPI_Op_create'
    MPI_Op_create(reduction_op, 1, &reduc_op);
/usr/local/include/mpi.h:1051:5: note: candidate function not viable: no known conversion from
      'void (data_t *, data_t *, int *, MPI_Datatype *)' (aka 'void (data *, data *, int *, int
      *)') to 'MPI_User_function *' (aka 'void (*)(void *, void *, int *, int *)') for 1st
int MPI_Op_create(MPI_User_function *user_fn, int commute, MPI_Op *op) MPICH_API_PUBLIC;
1 error generated.
make: *** [kmeans] Error 1

See below for a toy example. I compile with mpicxx (I also tried mpicc and mpic++ and got the same error). Any help on resolving the above compilation error would be much appreciated!

#include <iostream>
#include <stdlib.h>
#include <mpi.h>
#include <unistd.h>
#include <float.h>
#define N_DATA 1493

using namespace std;

#define FEATURES 8
typedef struct data{//Custom data structure for reduce operation
    float feat[FEATURES];
    long cluster;
} data_t;

void reduction_op(data_t *in, data_t *inout, int *len, MPI_Datatype * datatype){
    data_t temp;
    for(int i=0; i< *len; i++){
        temp.cluster = in->cluster + inout->cluster;
        for(int j=0; j<FEATURES; j++)
            temp.feat[j] = in->feat[j] + inout->feat[j];
        *inout = temp;

int main(int argc, char * argv[]){

    MPI_Init(&argc, &argv);
    int n_data = 1493;

    int world_size;
    MPI_Comm_size(MPI_COMM_WORLD, &world_size);

    int p_data = n_data/world_size; /*length of data per process*/ 
    int world_rank;
    MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);

    data_t data; data.feat[0] = 0.0; data.cluster = 0;

    //mpi type for data_t
    MPI_Datatype MPI_data_t;
    int structlen = 2;
    int blocklength[structlen];
    MPI_Datatype type[structlen];
    MPI_Aint displacement[structlen];
    blocklength[0] = FEATURES; type[0] = MPI_FLOAT;
    displacement[0] = (size_t)&(data.feat)-(size_t)&data;
    blocklength[1] = 1; type[1] = MPI_LONG;
    displacement[1] = (size_t)&(data.cluster) - (size_t)&data;
    MPI_Type_create_struct(structlen, blocklength, displacement, type, &MPI_data_t);

    MPI_Op reduc_op;
    MPI_Op_create(reduction_op, 1, &reduc_op); //ERROR OCCURS HERE


    return 0;

1 Answers

bearaqua On Best Solutions

You should change your function declaration of reduce_op to

void reduction_op(void *in, void *inout, int *len, int* datatype)

And then recast void* in and void* inout to data_t. Basically, your definition of reduction_op has to explicitly follow the definition of MPI_User_function.