Vector Halo Exchange issue(s)

[FlorianDeconinck]

A list of issue has crept up around the vector halo exchange. The extent of the problems are still to be determined but we are seeing:

• a transparency issue: 1x1 and 3x3 layout seems to not give the same results for vector halo update on ndsl.
• a difference with Fortran: likewise, when comparing 3x3 layout between ndsl and fms differences in data selection shows (ranks seems correct).
• confusion about the C-grid/D-grid position of U and V in the acoustics portion of pyFV3: the last 2 halo exchanges on the n_split need to be clarified versus the reference Fortran.

Overall, the FMS and original Fortran uses the "staircase" projection of the cube-sphere, while ndsl uses the "T" projection. We need to go back and figure out exactly if this change has been carried through the code properly and document the differences to the original model.

DOD:

• Use FMS to generate a failing unit test for ndsl on 3x3 layout for vector exchange
• Fix the vector halo exchange or move roll the fix into the larger non-square layout work
• Document cube-sphere representation for ndsl (vs FMS)
• Investigate the acoustics halo exchange and make sure they are well named and perform the same operation as the reference Fortran

[bensonr]

I'd like to add as part of the documenting of the cubed-sphere representation we should agree on the nomenclature for north, south, east, and west based on the origin of the domain, i.e. which "corner" is (0,0) located at for each tile.