You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The detailed steps required for this are outlined in #378. I break it down here as bite size tasks and provide a little more context for R2D2 and adding a suite. This is not an easy task so @cohen-seth, please ask our help if any of this is not clear or if you are stuck.
Create a tarball with BUFR files that need to be converted located in /discover/nobackup/projects/gmao/input/dao_ops/ops/flk/ncep_g5obs/bufr for testing. We need two sets of files for 2021-12-12T00:00:00Z and 2021-12-12T06:00:00Z for x0048 for now. We will then need a new task called get_bufr.py that copies these files to the SWELL cycle directory and extracts them.
Create BUFR yaml files defined in here using GMAO settings in src/swell/configuration/jedi/interfaces/geos_atmosphere/observations. They could reside in a folder under observations called bufr for now.
For the offline converter, we need a new suite. convert_ncdiags is the exact template we should replicate; for our case, at minimum, we need get_bufr.py and bufr_to_ioda.py. bufr_to_ioda.py converts and stores to R2D2 (see below) and will execute bufr2ioda.x.
If bufr_to_ioda.py requires compute nodes it could be separated into separate convert and store tasks. save_obs_diags.py uses store API, @asewnath modified that one, she could answer if there are any questions.
Details:
Using R2D2 for input is not necessary as we will be using obsys*.rc files located in here.
R2D2 utilizes two locations while using fetch (obtaining files from the database) and store (save files from cycle directory to database) methods: Shared:/discover/nobackup/projects/gmao/advda/R2D2DataStore/Shared Local: Defined in experiment.yaml by the r2d2_local_path key. For simplicity a similar structure should be followed, for instance I use: /discover/nobackup/dardag/R2D2DataStore/Local. Shared locations for platforms are defined here.
Adding a new suite is as easy as copying convert_ncdiags to a new folder and manipulating the flow.cylc file. The tricky part is manipulating task_questions.yaml according to required keys that could be utilized for a certain task. For instance get_bufr task will require a key in task_questions.yaml pointing to the BUFR tarball described in step 1.
A similar structure is used for suite_questions.yaml which controls which suites can access which suite configuration inputs.
It is possible to visualize Cylc flow.cylc files using the following command (for 3dvar suite) AFTER succesfully running the swell create command and creating an experiment folder (e.g., 3dvar-suite for the below example): cylc graph -c -o 3dvar_graph.png ./swell-3dvar/3dvar-suite/
The detailed steps required for this are outlined in #378. I break it down here as bite size tasks and provide a little more context for R2D2 and adding a suite. This is not an easy task so @cohen-seth, please ask our help if any of this is not clear or if you are stuck.
Create a tarball with BUFR files that need to be converted located in
/discover/nobackup/projects/gmao/input/dao_ops/ops/flk/ncep_g5obs/bufr
for testing. We need two sets of files for2021-12-12T00:00:00Z
and2021-12-12T06:00:00Z
forx0048
for now. We will then need a new task calledget_bufr.py
that copies these files to the SWELL cycle directory and extracts them.Create BUFR yaml files defined in here using GMAO settings in
src/swell/configuration/jedi/interfaces/geos_atmosphere/observations
. They could reside in a folder underobservations
calledbufr
for now.For the offline converter, we need a new suite.
convert_ncdiags
is the exact template we should replicate; for our case, at minimum, we needget_bufr.py
andbufr_to_ioda.py
.bufr_to_ioda.py
converts and stores to R2D2 (see below) and will executebufr2ioda.x
.If
bufr_to_ioda.py
requires compute nodes it could be separated into separate convert and store tasks.save_obs_diags.py
usesstore
API, @asewnath modified that one, she could answer if there are any questions.Details:
Using R2D2 for input is not necessary as we will be using
obsys*.rc
files located in here.R2D2 utilizes two locations while using
fetch
(obtaining files from the database) andstore
(save files from cycle directory to database) methods:Shared:
/discover/nobackup/projects/gmao/advda/R2D2DataStore/Shared
Local: Defined in
experiment.yaml
by ther2d2_local_path
key. For simplicity a similar structure should be followed, for instance I use:/discover/nobackup/dardag/R2D2DataStore/Local
.Shared
locations for platforms are defined here.Adding a new suite is as easy as copying
convert_ncdiags
to a new folder and manipulating theflow.cylc
file. The tricky part is manipulatingtask_questions.yaml
according to required keys that could be utilized for a certain task. For instanceget_bufr
task will require a key intask_questions.yaml
pointing to the BUFR tarball described in step 1.A similar structure is used for
suite_questions.yaml
which controls which suites can access whichsuite
configuration inputs.It is possible to visualize Cylc
flow.cylc
files using the following command (for3dvar
suite) AFTER succesfully running theswell create
command and creating an experiment folder (e.g.,3dvar-suite
for the below example):cylc graph -c -o 3dvar_graph.png ./swell-3dvar/3dvar-suite/
FYI @ashiklom @gmao-yzhu
The text was updated successfully, but these errors were encountered: