Force MPI_Send to use eager or rendezvouz protocol

Question

I'm doing a little MPI (openmpi) program in C for a workshop at college. Our objective is to observe the time difference between the two main protocols of MPI, eager and rendezvous, regarding the message size.

We haven't worked with MPI before, and we have thought that there may be a way to "select" between the two protocols. Searching on google for information about how to do it I found (somewhere I don't remember) that there is an eager limit. I read that it is set by the MPI implementation, and also, that you can change it somehow.

Any advice on how to choose between the protocols? Are there any relation between the protocols and MPI_Send/MPI_Isend?

I thought that changing the receiver buffer size will break from eager and start using rendezvous. But it's just a hunch.

Here is my code for now:

#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include "mpi.h"


#define KBdata 32000 //openmpi default buffer size
#define ndata KBdata/4 //number of ints that fits in buffer

int main(int argc, char *argv[]) {

    int myid, numprocs;
    int tag,source,destination,count;
    int buffer[ndata];
    MPI_Status status;
    MPI_Request request;

    int iter = 20;

    MPI_Init(&argc,&argv);
    MPI_Comm_size(MPI_COMM_WORLD,&numprocs);
    MPI_Comm_rank(MPI_COMM_WORLD,&myid);

    if (myid == 0 && numprocs == 2) { //to 
        int recvID = 1;
        double acum = 0;
        int i;
        double startT;
        for (i=0;i<iter;++i)
        {
            double startTime = MPI_Wtime();

            MPI_Send(&buffer,ndata,MPI_INT,recvID,0,MPI_COMM_WORLD);

            double endTime = MPI_Wtime();
            double elapsed = endTime - startTime;
            acum += elapsed;
            printf("%d, %f, elapsed: %f\n",i,acum,elapsed);fflush(stdout);
            usleep(500000);
        }
        printf("total: %f\nmean: %f\n", acum, acum/iter);   
    }
    else if (numprocs == 2) {
        int i;
        for(i=0; i<iter; ++i)
        {
            printf("Waiting for receive\n");fflush(stdout);
            MPI_Recv(&buffer,ndata,MPI_INT,0,0,MPI_COMM_WORLD,&status);
            printf("Received %d\n",i);fflush(stdout); 
        }
    }
    else {
        printf("Need only 2 threads\n");
    }

    MPI_Finalize();


    return 0;
}

Thank you in advice.

Answer 1

There is no direct connection between eager/rendezvous and MPI_Send/Isend. However, if you're under the eager limit, your MPI_Send is no longer blocking. If you want it to block regardless, you can use MPI_Ssend.

Regarding eager limits:

MVAPICH2:
  MV2_IBA_EAGER_THRESHOLD= < nbytes >
Intel MPI (depending on version):
  I_MPI_EAGER_THRESHOLD= < nbytes >
  I_MPI_SHM_EAGER_THRESHOLD= < nbytes >
Open MPI:
  --mca_btl_openib_eager_limit < nbytes >
  --mca_btl_openib_rndv_eager_limit < nbytes >
Cray MPICH:
  MPICH_GNI_MAX_EAGER_MSG_SIZE=<value>