Passing 3d arrays using MPI_Bcast

Question

I'm trying to pass 3D arrays to all other processes (in FORTRAN 77) using MPI_Bcast. v1 is a common block array. I'm also not sure if I need to broadcast the calculated values of the common array v1 to all other processes or they will be changed in each processes because of being common. The following is the related piece of code:

  parameter (nprocz=48,nzro=1)

    do i=i101,i102
      dist  = 0.015*float(i-iv0)
      adamp = exp(-dist*dist)
      do j = je0, je1-1
      do k = ke0, ke1
        v1(k,j,i) = v1(k,j,i)*adamp
      end do
      end do
    end do

    nmpi01=floor((iv0-ie0-nzro)/(nprocz-1))
    if (mpirank .le. nprocz-2) then
       i101=ie0+(mpirank*nmpi01)
       i102=ie0+(mpirank+1)*nmpi01-1
    else
       i101=ie0+(mpirank*nmpi01)
       i102=iv0-1
    endif

   MPI_Bcast(v1(:,:,i101:i102),(ke1-ke0+1)*(je1-je0)*(i102-i101+1)
 & ,MPI_FLOAT,mpirank,MPI_COMM_WORLD,ierr01)

I get the error message:

PGFTN-S-0081-Matrix/vector v1 illegal as subprogram argument

The sizes of the arrays being passed in are correct. Any comment?

---

I corrected the code and I looped over the ranks and compute all elements of rcount and displs in each rank:

integer :: myscount, myi101

do rank = 0, nprocz-1
  nmpi01=floor((iv0-ie0-nzro)/(nprocz-1))
  if (rank .le. nprocz-2) then
    i101=ie0+(rank*nmpi01)
    i102=ie0+(rank+1)*nmpi01-1
  else
    i101=ie0+(rank*nmpi01)
    i102=iv0-1
  endif
  scount=(i102-i101+1)*(je1-je0)*(ke1-ke0+1)
  rcount(rank+1)=scount
  displs(rank+1)=rank*scount+1
  if (rank .eq. mpirank) then
    myscount = scount
    myi101 = i101
 end if
end do

scount = myscount
i101 = myi101

call mpi_allgatherv(...)

But still wrong results. 1-in my case, results at each part are used for the next part, especially after mpi_allgatherv.so do i need to add mpi_barrier after each mpi_allgatherv? 2-should mpi_in_place be used? consider i have only one 3d array v1 that each sub-array v1(1,1,i) is calculated by some process and i want to put the calculated subarray in the appropriate part of the same array. 3- i guess i should have displs(i) = sum(rcount(1:i-1))+1 for i=>2 considering that always displs(1)=1 in fortran77. so i corrected to this: before the loop displs(1)=1, inside the loop displs(rank+2)=rank*scount+1 and after the loop displs(nprocz+1)=0. am I right?

Answer 1

As I recall, Fortran 77 was more restrictive about array subscripts than Fortran 90, and pgftn is a Fortran 77 compiler. I would try passing v1(1,1,i101) to mpi_bcast, not v1(:,:,i101:i102). (Or use pgf95 with the "-Mfixed" flag.)

If each process needs to see v1, then you do need to communicate it using MPI. No variable is shared between MPI tasks, not even those in a common block. However, if every process is calculating a different part of v1, so every process needs a piece from every other process, you can't use mpi_bcast to do that; use mpi_allgather instead.

Also, as noted above, when you use MPI procedures, you should call them, because they are subroutines.