Skip to content

Commit

Permalink
Update enkfgdas_update job resource/env settings (#3047)
Browse files Browse the repository at this point in the history
Updates for the `enkfgdas_update` job from testing by NCO/GDIT and EMC.
They produce a more stable runtime and bitwise identical outputs to ops.

Add `export OMP_PROC_BIND=true` and update `APRUN_ENKF` to use
`--cpu-bind core` instead of `--cpu-bind depth` in WCOSS2.env. 
Also, adjust npe (280) and thread (16) values.

NCO requests that we include these updates in the GFSv16.3.21 package.

Refs #2591
  • Loading branch information
KateFriedman-NOAA authored Oct 30, 2024
1 parent 5ca00b2 commit ab8e453
Show file tree
Hide file tree
Showing 5 changed files with 10 additions and 7 deletions.
4 changes: 3 additions & 1 deletion docs/Release_Notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,9 @@ and are being retired from the WAFS package:
ENVIRONMENT AND RESOURCE CHANGES
--------------------------------

* No changes from GFS v16.3.20
* Updates to improve the enkfgdas_update job runtime stability.
* Run with `--cpu-bind core` instead of `--cpu-bind depth`
* Change tasks to 280 and threads to 16

PRE-IMPLEMENTATION TESTING REQUIREMENTS
---------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion ecf/scripts/enkfgdas/analysis/create/jenkfgdas_update.ecf
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
#PBS -q %QUEUE%
#PBS -A %PROJ%-%PROJENVIR%
#PBS -l walltime=00:35:00
#PBS -l select=35:mpiprocs=9:ompthreads=14:ncpus=126
#PBS -l select=35:mpiprocs=8:ompthreads=16:ncpus=128
#PBS -l place=vscatter:exclhost
#PBS -l debug=true

Expand Down
3 changes: 2 additions & 1 deletion env/WCOSS2.env
Original file line number Diff line number Diff line change
Expand Up @@ -91,12 +91,13 @@ elif [ $step = "eobs" ]; then
elif [ $step = "eupd" ]; then

export OMP_PLACES=cores
export OMP_PROC_BIND=true
export OMP_STACKSIZE=2G
export MPICH_COLL_OPT_OFF=1
export FI_OFI_RXM_SAR_LIMIT=3145728

export NTHREADS_ENKF=$nth_eupd
export APRUN_ENKF="$launcher -n ${npe_enkf:-${npe_eupd:-$PBS_NP}} -ppn $npe_node_eupd --cpu-bind depth --depth $NTHREADS_ENKF"
export APRUN_ENKF="$launcher -n ${npe_enkf:-${npe_eupd:-$PBS_NP}} -ppn $npe_node_eupd --cpu-bind core --depth $NTHREADS_ENKF"

export CFP_MP=${CFP_MP:-"NO"}
export USE_CFP=${USE_CFP:-"YES"}
Expand Down
4 changes: 2 additions & 2 deletions parm/config/config.resources.emc.dyn
Original file line number Diff line number Diff line change
Expand Up @@ -318,8 +318,8 @@ elif [ $step = "eupd" ]; then
export npe_eupd=960
export nth_eupd=7
elif [[ "$machine" = "WCOSS2" ]]; then
export npe_eupd=315
export nth_eupd=14
export npe_eupd=280
export nth_eupd=16
fi
elif [ $CASE = "C384" ]; then
export npe_eupd=270
Expand Down
4 changes: 2 additions & 2 deletions parm/config/config.resources.nco.static
Original file line number Diff line number Diff line change
Expand Up @@ -238,8 +238,8 @@ elif [ $step = "ediag" ]; then
elif [ $step = "eupd" ]; then

export wtime_eupd="00:35:00"
export npe_eupd=315
export nth_eupd=14
export npe_eupd=280
export nth_eupd=16
export npe_node_eupd=$(echo "$npe_node_max / $nth_eupd" | bc)

elif [ $step = "ecen" ]; then
Expand Down

0 comments on commit ab8e453

Please sign in to comment.