I have problems with Scalapack when processor grids do not use all available MPI cores. Say I know the grid to setup Pc*Pr<=NP
When I setup a process grid
Cblacs_get(0, 0, &ctxt);
const char *order = ( column_major ? "Col" : "Row" );
Cblacs_gridinit(&ctxt, order, procrows, proccols);
Cblacs_gridinfo( ctxt, &procrows, &proccols, &myrow, &mycol );
Cblacs_gridinfo
changes the input grid size from 2
x2
to -1
x-1
. In this case I was testing with 11
MPI cores.
This, by itself, does not raise any errors but when I try to setup a descriptor vector
int irsrc = 0, icsrc = 0;
descinit_(descA, &M, &N, &Mb, &Nb,&irsrc,&icsrc,&ctxt, &lda, &info);
On processes with ids 4-10
i get error
{ -1, -1}: On entry to DESCINIT parameter number 6 had an illegal value
QUESTION:
what is the correct way of handling MPI cores which are not on the grid? Shall I skip everything on all cores for which
Cblacs_gridinfo( ctxt, &procrows, &proccols, &myrow, &mycol );
returned myrow=mycol=-1
? Is this a part of API?
Unless someone has a better source, the second post in http://icl.cs.utk.edu/lapack-forum/viewtopic.php?t=139 provides a C example which calls ScaLAPACK functions only for processes where