Skip to content

Commit 8d8cc3d

Browse files
committed
Fixes for ifx compiler: mpi tag overflow in edyn_mpi
The MPI_ibsend calls in edyn_mpi were generating MPI tags by concatenating the source and destination PE numbers which overflows the maximum MPI tag size (~65536) - the original tag numbers were around 10M (e.g., 12341235) Since communicators in mp_mag_jslot is scoped to the cols_comm sub-communicator for a single magnetic longitude column (up to nmagtaskj-1), there is no need for globally unique tags. The code has been updated to use sender rank number which is unique within the sub-communicator (tij) which is isrc at the receiver end.
1 parent 60bc7b0 commit 8d8cc3d

1 file changed

Lines changed: 6 additions & 4 deletions

File tree

src/ionosphere/waccmx/edyn_mpi.F90

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1684,7 +1684,7 @@ subroutine mp_mag_jslot(fin,mlon00,mlon11,mlat00,mlat11, &
16841684
integer :: ier,njneed,i,j,n,nj,idest, &
16851685
icount,len,nlons,isrc,msgid,ifld,sndbuf_cntr
16861686
integer :: tij ! rank in cols_comm (0 to nmagtaskj-1)
1687-
integer :: jhave(mxneed),njhave,wid
1687+
integer :: jhave(mxneed),njhave
16881688
integer :: peersneed(mxneed,0:nmagtaskj-1)
16891689
integer :: jneedall (mxneed,0:nmagtaskj-1)
16901690
real(r8) :: sndbuf(mxmaglon+2,mxneed,nf,sndbuf_cntr_max)
@@ -1730,7 +1730,6 @@ subroutine mp_mag_jslot(fin,mlon00,mlon11,mlat00,mlat11, &
17301730
njhave = njhave+1
17311731
jhave(njhave) = peersneed(j,n)
17321732
idest = n
1733-
wid = itask_table_geo(mytidi,idest)
17341733
endif
17351734
enddo
17361735
if (njhave > 0) then
@@ -1749,7 +1748,9 @@ subroutine mp_mag_jslot(fin,mlon00,mlon11,mlat00,mlat11, &
17491748
enddo
17501749
enddo
17511750
len = nlons*njhave*nf
1752-
msgid = mytid+wid*10000
1751+
! sending tag uniquely identifies sender - tij is comm. rank
1752+
! within this communicator only.
1753+
msgid = tij
17531754
call mpi_ibsend(sndbuf(1:nlons,1:njhave,:,sndbuf_cntr),len,MPI_REAL8, &
17541755
idest,msgid,cols_comm,ibsend_requests(sndbuf_cntr),ier)
17551756
if (ier /= 0) &
@@ -1783,7 +1784,8 @@ subroutine mp_mag_jslot(fin,mlon00,mlon11,mlat00,mlat11, &
17831784
isrc = tasks(n)%magtidj ! task id in cols_comm to recv from
17841785
nlons = mlon11-mlon00+1
17851786
len = nlons*njhave*nf
1786-
msgid = mytid*10000+n
1787+
! receive tag is sender rank (tij from sender == magtidj)
1788+
msgid = isrc
17871789
rcvbuf = 0._r8
17881790
call mpi_recv(rcvbuf(1:nlons,1:njhave,:),len,MPI_REAL8, &
17891791
isrc,msgid,cols_comm,irstat,ier)

0 commit comments

Comments
 (0)