Skip to content

Commit

Permalink
update docs w/changes from PR ufs-community#129
Browse files Browse the repository at this point in the history
  • Loading branch information
gspetro-NOAA committed Aug 5, 2024
1 parent fa0b8b0 commit f56d91c
Show file tree
Hide file tree
Showing 5 changed files with 25 additions and 65 deletions.
3 changes: 2 additions & 1 deletion doc/source/BackgroundInfo/Introduction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ Since the |latestr| release, the following capabilities have been added to the L
* Extended container support (:land-wflow-repo:`PR #85 <pull/85>`)
* Updated directory structure for NCO compliance (:land-wflow-repo:`PR #75 <pull/75>`)
* Removed land driver from CTest (:land-wflow-repo:`PR #123 <pull/123>`)
* Removed land-driver and vector2tile (:land-wflow-repo:`PR #129 <pull/129>`)

The Land DA System citation is as follows and should be used when presenting results based on research conducted with the Land DA System:

Expand Down Expand Up @@ -46,7 +47,7 @@ Customizing the Workflow
=========================

* :numref:`Chapter %s: Available Workflow Configuration Parameters <ConfigWorkflow>` explains all of the user-configurable options currently available in the workflow configuration file (``land_analysis*.yaml``).
* :numref:`Chapter %s: Model <Model>` provides information on input data and configuration parameters in the Noah-MP LSM and its Vector-to-Tile Converter.
* :numref:`Chapter %s: Model <Model>` provides information on input data and configuration parameters in the Noah-MP LSM.
* :numref:`Chapter %s: DA Framework <DASystem>` provides information on the DA system, required data, and configuration parameters.

Reference
Expand Down
16 changes: 0 additions & 16 deletions doc/source/BackgroundInfo/TechnicalOverview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -115,22 +115,10 @@ This :term:`umbrella repository` uses Git submodules and an ``app_build.sh`` fil
- land-apply_jedi_incr
- Contains code that applies the JEDI-generated DA increment to UFS ``sfc_data`` restart
- https://github.com/NOAA-PSL/land-apply_jedi_incr
* - ufsLand.fd
- ufs-land-driver-emc-dev
- Repository for the UFS Land Driver
- https://github.com/NOAA-EPIC/ufs-land-driver-emc-dev
* - *-- ccpp-physics*
- *-- ccpp-physics*
- Repository for the Common Community Physics Package (CCPP)
- https://github.com/ufs-community/ccpp-physics/
* - ufs_model.fd
- ufs-weather-model
- Repository for the UFS Weather Model (WM). This repository contains a number of subrepositories, which are documented :ufs-wm:`in the WM User's <CodeOverview.html>`.
- https://github.com/ufs-community/ufs-weather-model/
* - vector2tile_converter.fd
- land-vector2tile
- Contains code to map between the vector format used by the Noah-MP offline driver, and the tile format used by the UFS atmospheric model.
- https://github.com/NOAA-PSL/land-vector2tile

.. note::
The prerequisite libraries (including NCEP Libraries and external libraries) are not included in the UFS Land DA System repository. The `spack-stack <https://github.com/JCSDA/spack-stack>`_ repository assembles these prerequisite libraries. Spack-stack has already been built on :ref:`preconfigured (Level 1) platforms <LevelsOfSupport>`. However, it must be built on other systems. See the :spack-stack:`spack-stack Documentation <>` for details on installing spack-stack.
Expand Down Expand Up @@ -166,11 +154,7 @@ The ``land-DA_workflow`` is evolving to follow the :term:`NCEP` Central Operatio
│ ├── (conda)
│ ├── test
│ ├── tile2tile_converter.fd
│ ├── ufsLand.fd
│ │ ├── ccpp-physics
│ │ └── driver
│ ├── ufs_model.fd
│ ├── vector2tile_converter.fd
│ ├── CMakeLists.txt
│ └── app_build.sh
├── ush
Expand Down
3 changes: 1 addition & 2 deletions doc/source/BuildingRunningTesting/BuildRunLandDA.rst
Original file line number Diff line number Diff line change
Expand Up @@ -309,8 +309,7 @@ As the experiment progresses, it will generate a number of directories to hold i
│ │ ├── hofx
│ │ └── plot
│ └── output
│ └── logs
│ └── run_<forcing> (<LOGDIR>): Directory containing the log files for the Rocoto workflow
│ └── logs (<LOGDIR>): Directory containing the log files for the Rocoto workflow
└── tmp (<DATAROOT>)
├── <jobid> (<DATA>): Working directory
└── DATA_SHARE
Expand Down
34 changes: 13 additions & 21 deletions doc/source/BuildingRunningTesting/TestingLandDA.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,43 +33,35 @@ If the tests are successful, a message will be printed to the console. For examp
.. code-block:: console
Test project /work/noaa/epic/${USER}/landda/land-DA_workflow/sorc/build
Start 1: test_vector2tile
1/6 Test #1: test_vector2tile ................. Passed 12.01 sec
Start 2: test_create_ens
2/6 Test #2: test_create_ens .................. Passed 13.91 sec
Start 3: test_letkfoi_snowda
3/6 Test #3: test_letkfoi_snowda .............. Passed 67.94 sec
Start 4: test_apply_jediincr
4/6 Test #4: test_apply_jediincr .............. Passed 6.88 sec
Start 5: test_tile2vector
5/6 Test #5: test_tile2vector ................. Passed 15.36 sec
Start 6: test_ufs_datm_land
6/6 Test #6: test_ufs_datm_land ............... Passed 98.56 sec
100% tests passed, 0 tests failed out of 6
Total Test time (real) = 217.06 sec
Start 1: test_create_ens
1/4 Test #1: test_create_ens .................. Passed 13.91 sec
Start 2: test_letkfoi_snowda
2/4 Test #2: test_letkfoi_snowda .............. Passed 67.94 sec
Start 3: test_apply_jediincr
3/4 Test #3: test_apply_jediincr .............. Passed 6.88 sec
Start 4: test_ufs_datm_land
4/4 Test #4: test_ufs_datm_land ............... Passed 98.56 sec
100% tests passed, 0 tests failed out of 4
Total Test time (real) = 187.29 sec
Tests
*******

The CTests test the operability of six major elements of the Land DA System: ``vector2tile``, ``create_ens``, ``letkfoi_snowda``, ``apply_jediincr``, ``tile2vector``, and ``ufs_datm_land``. The tests and their dependencies are listed in the ``land-DA_workflow/test/CMakeLists.txt`` file. Currently, the CTests are only run on Hera and Orion; they cannot yet be run via container.
The CTests test the operability of four major elements of the Land DA System: ``create_ens``, ``letkfoi_snowda``, ``apply_jediincr``, and ``ufs_datm_land``. The tests and their dependencies are listed in the ``land-DA_workflow/test/CMakeLists.txt`` file. Currently, the CTests are only run on Hera and Orion; they cannot yet be run via container.

.. list-table:: *Land DA CTests*
:widths: 20 50
:header-rows: 1

* - Test
- Description
* - ``test_vector2tile``
- Tests the vector-to-tile function for use in JEDI.
* - ``test_create_ens``
- Tests creation of a pseudo-ensemble for use in LETKF-OI.
* - ``test_letkfoi_snowda``
- Tests the use of LETKF-OI to assimilate snow data.
* - ``test_apply_jediincr``
- Tests the ability to add a JEDI increment.
* - ``test_tile2vector``
- Tests the tile-to-vector function for use in ``ufs-land-driver``
* - ``test_ufs_datm_land``
- Tests proper functioning of the UFS land model (``ufs-datm-lnd``)
34 changes: 9 additions & 25 deletions doc/source/CustomizingTheWorkflow/ConfigWorkflow.rst
Original file line number Diff line number Diff line change
Expand Up @@ -75,11 +75,10 @@ Entities are constants that can be referred to throughout the workflow using the
MACHINE: "orion"
SCHED: "slurm"
ACCOUNT: "epic"
EXP_NAME: "LETKF"
EXP_BASEDIR: "/work/noaa/epic/{USER}/landda_test"
JEDI_INSTALL: "/work/noaa/epic/UFS_Land-DA_Dev/jedi_v7_stack1.6"
WARMSTART_DIR: "/work/noaa/epic/UFS_Land-DA_Dev/inputs/DATA_RESTART"
FORCING: "gswp3"
ATMOS_FORC: "gswp3"
RES: "96"
FCSTHR: "24"
NPROCS_ANALYSIS: "6"
Expand All @@ -99,9 +98,8 @@ Entities are constants that can be referred to throughout the workflow using the
COMROOT: "&PTMP;/&envir;/com"
DATAROOT: "&PTMP;/&envir;/tmp"
KEEPDATA: "YES"
LOGDIR: "&COMROOT;/output/logs/run_&FORCING;"
LOGDIR: "&COMROOT;/output/logs;"
LOGFN_SUFFIX: "<cyclestr>_@Y@m@[email protected]</cyclestr>"
PATHRT: "&EXP_BASEDIR;"
PDY: "<cyclestr>@Y@m@d</cyclestr>"
cyc: "<cyclestr>@H</cyclestr>"
DATADEP_FILE1: "<cyclestr>&WARMSTART_DIR;/ufs_land_restart.@Y-@m-@[email protected]</cyclestr>"
Expand All @@ -122,9 +120,6 @@ Entities are constants that can be referred to throughout the workflow using the
``ACCOUNT:`` (Default: "epic")
An account where users can charge their compute resources on the specified ``MACHINE``. To determine an appropriate ``ACCOUNT`` field on a system with a Slurm job scheduler, users may run the ``saccount_params`` command to display account details. On other systems, users may run the ``groups`` command, which will return a list of projects that the user has permissions for. Not all of the listed projects/groups have an HPC allocation, but those that do are potentially valid account names.

``EXP_NAME:`` (Default: "LETKF")
Placeholder --- currently not used in workflow.

``EXP_BASEDIR:`` (Default: "/scratch2/NAGAPE/epic/{USER}/landda_test" or "/work/noaa/epic/{USER}/landda_test")
The full path to the parent directory of ``land-DA_workflow`` (i.e., ``$LANDDAROOT`` in the documentation).

Expand All @@ -134,8 +129,8 @@ Entities are constants that can be referred to throughout the workflow using the
``WARMSTART_DIR:`` (Default: "/scratch2/NAGAPE/epic/UFS_Land-DA_Dev/inputs/DATA_RESTART" or "/work/noaa/epic/UFS_Land-DA_Dev/inputs/DATA_RESTART")
The path to restart files for a warmstart experiment.

``FORCING:`` (Default: "gswp3")
Type of atmospheric forcing data used. Valid values: ``"gswp3"`` | ``"era5"``
``ATMOS_FORC:`` (Default: "gswp3")
Type of atmospheric forcing data used. Valid values: ``"gswp3"``

``RES:`` (Default: "96")
Resolution of FV3 grid. Currently, only C96 resolution is supported.
Expand Down Expand Up @@ -213,15 +208,12 @@ Standard environment variables are defined in the NCEP Central Operations :nco:`
``KEEPDATA:`` (Default: "YES")
Flag to keep data ("YES") or not ("NO") that is copied to the ``$DATAROOT`` directory during the forecast experiment.

``LOGDIR:`` (Default: "&COMROOT;/output/logs/run_&FORCING;")
``LOGDIR:`` (Default: "&COMROOT;/output/logs;")
Path to the directory containing log files for each workflow task.

``LOGFN_SUFFIX:`` (Default: "<cyclestr>_@Y@m@[email protected]</cyclestr>")
The cycle suffix appended to each task's log file. It will be rendered in the form ``_YYYYMMDDHH.log``. For example, the ``prep_obs`` task log file for the Jan. 4, 2000 00z cycle would be named: ``prep_obs_2000010400.log``.

``PATHRT:`` (Default: "&EXP_BASEDIR;")
The path to the ``EXP_BASEDIR`` for regression tests (RTs).

``PDY:`` (Default: "<cyclestr>@Y@m@d</cyclestr>")
Date in YYYYMMDD format.

Expand Down Expand Up @@ -288,7 +280,6 @@ Parameters for a particular task are set in the ``workflow.tasks.task_<name>:``
SCHED: "&SCHED;"
ACCOUNT: "&ACCOUNT;"
EXP_NAME: "&EXP_NAME;"
ATMOS_FORC: "&FORCING;"
RES: "&RES;"
TSTUB: "&TSTUB;"
model_ver: "&model_ver;"
Expand Down Expand Up @@ -334,13 +325,13 @@ The ``attrs:`` section for each task includes the ``cycledefs:`` attribute and t
Task Environment Variables (``envars``)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The ``envars:`` section for each task reuses many of the same variables and values defined as ``entities:`` for the overall workflow. These values are needed for each task, but setting them individually is error-prone. Instead, a specific workflow task can reference workflow entities using the ``&VAR;`` syntax. For example, to set the ``ATMOS_FORC:`` value in ``task_analysis:`` to the value of the workflow ``FORCING`` entity, the following statement can be added to the task's ``envars:`` section:
The ``envars:`` section for each task reuses many of the same variables and values defined as ``entities:`` for the overall workflow. These values are needed for each task, but setting them individually is error-prone. Instead, a specific workflow task can reference workflow entities using the ``&VAR;`` syntax. For example, to set the ``ACCOUNT:`` value in ``task_analysis:`` to the value of the workflow ``ACCOUNT:`` entity, the following statement can be added to the task's ``envars:`` section:

.. code-block:: console
task_analysis:
envars:
ATMOS_FORC: "&FORCING;"
ACCOUNT: "&ACCOUNT;"
For most workflow tasks, whatever value is set in the ``workflow.entities:`` section should be reused/referenced in other tasks. For example, the ``MACHINE`` variable must be defined for each task, and users cannot switch machines mid-workflow. Therefore, users should set the ``MACHINE`` variable in the ``workflow.entities:`` section and reference that definition in each workflow task. For example:

Expand Down Expand Up @@ -477,8 +468,7 @@ Parameters for the observation preparation task are set in the ``task_prep_obs:`
MACHINE: "&MACHINE;"
SCHED: "&SCHED;"
ACCOUNT: "&ACCOUNT;"
EXP_NAME: "&EXP_NAME;"
ATMOS_FORC: "&FORCING;"
ATMOS_FORC: "&ATMOS_FORC;"
model_ver: "&model_ver;"
HOMElandda: "&HOMElandda;"
COMROOT: "&COMROOT;"
Expand Down Expand Up @@ -513,8 +503,6 @@ Parameters for the pre-analysis task are set in the ``task_pre_anal:`` section o
MACHINE: "&MACHINE;"
SCHED: "&SCHED;"
ACCOUNT: "&ACCOUNT;"
EXP_NAME: "&EXP_NAME;"
ATMOS_FORC: "&FORCING;"
RES: "&RES;"
TSTUB: "&TSTUB;"
WARMSTART_DIR: "&WARMSTART_DIR;"
Expand Down Expand Up @@ -579,8 +567,6 @@ Parameters for the post analysis task are set in the ``task_post_anal:`` section
MACHINE: "&MACHINE;"
SCHED: "&SCHED;"
ACCOUNT: "&ACCOUNT;"
EXP_NAME: "&EXP_NAME;"
ATMOS_FORC: "&FORCING;"
RES: "&RES;"
TSTUB: "&TSTUB;"
model_ver: "&model_ver;"
Expand Down Expand Up @@ -623,7 +609,6 @@ Parameters for the plotting task are set in the ``task_plot_stats:`` section of
MACHINE: "&MACHINE;"
SCHED: "&SCHED;"
ACCOUNT: "&ACCOUNT;"
EXP_NAME: "&EXP_NAME;"
model_ver: "&model_ver;"
RUN: "&RUN;"
HOMElandda: "&HOMElandda;"
Expand Down Expand Up @@ -664,8 +649,7 @@ Parameters for the forecast task are set in the ``task_forecast:`` section of th
MACHINE: "&MACHINE;"
SCHED: "&SCHED;"
ACCOUNT: "&ACCOUNT;"
EXP_NAME: "&EXP_NAME;"
ATMOS_FORC: "&FORCING;"
ATMOS_FORC: "&ATMOS_FORC;"
RES: "&RES;"
WARMSTART_DIR: "&WARMSTART_DIR;"
model_ver: "&model_ver;"
Expand Down

0 comments on commit f56d91c

Please sign in to comment.