Blacs: processor grids that do not use all MPI cores

212 views Asked by At

I have problems with Scalapack when processor grids do not use all available MPI cores. Say I know the grid to setup Pc*Pr<=NP

When I setup a process grid

Cblacs_get(0, 0, &ctxt);
const char *order = ( column_major ? "Col" : "Row" );
Cblacs_gridinit(&ctxt, order, procrows, proccols);
Cblacs_gridinfo( ctxt, &procrows, &proccols, &myrow, &mycol );

Cblacs_gridinfo changes the input grid size from 2x2 to -1x-1. In this case I was testing with 11 MPI cores.

This, by itself, does not raise any errors but when I try to setup a descriptor vector

int irsrc = 0, icsrc = 0;
descinit_(descA, &M, &N, &Mb, &Nb,&irsrc,&icsrc,&ctxt, &lda, &info);

On processes with ids 4-10 i get error

{   -1,   -1}:  On entry to DESCINIT parameter number    6 had an illegal value

QUESTION:

what is the correct way of handling MPI cores which are not on the grid? Shall I skip everything on all cores for which

Cblacs_gridinfo( ctxt, &procrows, &proccols, &myrow, &mycol );

returned myrow=mycol=-1? Is this a part of API?

1

There are 1 answers

0
Denis On

Unless someone has a better source, the second post in http://icl.cs.utk.edu/lapack-forum/viewtopic.php?t=139 provides a C example which calls ScaLAPACK functions only for processes where

 if ((myrow>-1)&(mycol>-1)&(myrow<nprow)&(mycol<npcol)) {
    // do the job
    Cblacs_gridexit( 0 );
 }