Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable GPU execution of atm_compute_moist_coefficients via OpenACC #1238

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
+23 −2
Open

Enable GPU execution of atm_compute_moist_coefficients via OpenACC #1238

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 23 additions & 2 deletions src/core_atmosphere/dynamics/mpas_atm_time_integration.F
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1730,39 +1730,60 @@ subroutine atm_compute_moist_coefficients( dims, state, diag, mesh, &
call mpas_pool_get_array(diag, 'cqw', cqw)
call mpas_pool_get_array(diag, 'cqu', cqu)

MPAS_ACC_TIMER_START('atm_compute_moist_coefficients [ACC_data_xfer]')
!$acc enter data copyin(qtot, cqw, cqu, scalars)
MPAS_ACC_TIMER_STOP('atm_compute_moist_coefficients [ACC_data_xfer]')


!$acc parallel
!$acc loop
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the loop directives should be more explicit (e.g. loop gang worker here). Just to match with other work we're doing.

! do iCell = cellSolveStart,cellSolveEnd
do iCell = cellStart,cellEnd
qtot(1:nVertLevels,iCell) = 0.0
!$acc loop
do k = 1,nVertLevels
qtot(k,iCell) = 0.0
!$acc loop seq
do iq = moist_start, moist_end
qtot(k,iCell) = qtot(k,iCell) + scalars(iq, k, iCell)
end do
end do
end do
!$acc end parallel


! do iCell = cellSolveStart,cellSolveEnd
!$acc parallel
!$acc loop gang worker
do iCell = cellStart,cellEnd
!$acc loop vector
do k = 2, nVertLevels
qtotal = 0.5*(qtot(k,iCell)+qtot(k-1,iCell))
cqw(k,iCell) = 1.0 / (1.0 + qtotal)
end do
end do
!$acc end parallel

! would need to compute qtot for all cells and an openmp barrier to use qtot below.

!$acc parallel
!$acc loop
do iEdge = edgeStart,edgeEnd
cell1 = cellsOnEdge(1,iEdge)
cell2 = cellsOnEdge(2,iEdge)
if (cell1 <= nCellsSolve .or. cell2 <= nCellsSolve) then
do k = 1, nVertLevels
!$acc loop
do k = 1, nVertLevels
qtotal = 0.0
!$acc loop seq
do iq = moist_start, moist_end
qtotal = qtotal + 0.5 * ( scalars(iq, k, cell1) + scalars(iq, k, cell2) )
end do
cqu(k,iEdge) = 1.0 / (1.0 + qtotal)
end do
end if
end do
!$acc end parallel
!$acc exit data copyout(cqw, cqu, qtot) delete(scalars)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This exit data directive should be surrounded by MPAS_ACC_TIMER_{START,STOP} calls.


end subroutine atm_compute_moist_coefficients

Expand Down