Merge pull request #4624 from ESMCI/rljacob/docs/tutorial-prep

Update the documentation for the basic usage of CCS.
Define more key concepts. Add table for compset alias one-letter case names.
Make sure files created after create_newcase and ./case.setup match current function.
Change titles of two major sections to be about using CCS and configuring CCS.

.gitignore

@@ -4,6 +4,7 @@ buildnmlc
 buildlib.*c
 CIME.egg-info/
 build/
+_build/
 dist/
 .coverage
 

doc/source/users_guide/cime-dir.rst

@@ -25,25 +25,22 @@ CIME's content is split into several subdirectories. Users should start in the *
    ========================== ==================================================================
    Directory or Filename               Description
    ========================== ==================================================================
+   **CIME/**	              **The main CIME source**
+   CIME/ParamGen              Python tool for generating runtime params
+   CIME/Servers		      Scripts to interact with input data servers
+   CIME/SystemTests           Scripts for create_test tests.
+   CIME/Tools		      Auxillary tools, scripts and functions.
    CMakeLists.txt	      For building with CMake
+   CONTRIBUTING.md            Guide for contributing to CIME
    ChangeLog		      Developer-maintained record of changes to CIME
    ChangeLog_template	      Template for an entry in ChangeLog
    LICENSE.TXT		      The CIME license
-   README		      Brief intro to CIME
+   MANIFEST.in                
    README.md		      README in markdown language
-   README.unit_testing	      Instructions for running unit tests with CIME
-   **config/**		      **Shared and model-specific configuration files**
-   config/cesm/	              CESM-specific configuration options
-   config/e3sm/	              E3SM-specific configuration options
+   conftest.py
+   doc			      Documentation for CIME in rst format
+   docker		      Container for CIME testing
    **scripts/**		      **The CIME user interface**
-   scripts/lib/  	      Infrastructure source code for CIME scripts and functions
-   scripts/Tools/	      Auxiliary tools; scripts and functions
-   **src/**		      **Model source code provided by CIME**
-   src/components/	      CIME-provided components including data and stub models
-   src/drivers/  	      CIME-provided main driver for a climate model
-   src/externals/	      Software provided with CIME for building a climate model
-   src/share/    	      Model source code provided by CIME and used by multiple components
-   **tests/**		      **Tests**
    **tools/**		      **Standalone climate modeling tools**
    utils/		      Some Perl source code needed by some prognostic components
    ========================== ==================================================================

doc/source/users_guide/create-a-case.rst

@@ -63,7 +63,7 @@ In the argument to ``--case``, the case name is taken from the string after the
 The output from create_newcase includes information such as.
 
 - The compset longname is ``2000_DATM%NYF_SLND_DICE%SSMI_DOCN%DOM_DROF%NYF_SGLC_SWAV``
-- The model resolution is ``a%0.9x1.25_l%0.9x1.25_oi%gx1v6_r%r05_m%gx1v6_g%null_w%null``
+- The grid set is ``a%0.9x1.25_l%0.9x1.25_oi%gx1v6_r%r05_m%gx1v6_g%null_w%null``
 
 `create_newcase  <../Tools_user/create_newcase.html>`_ installs files in ``$CASEROOT`` that will build and run the model and to optionally archive the case on the target platform.
 
@@ -74,64 +74,70 @@ Running `create_newcase  <../Tools_user/create_newcase.html>`_ creates the follo
 - `case.build  <../Tools_user/case.build.html>`_
      Script to build component and utility libraries and model executable.
 
+- `case.cmpgen_namelist <../Tools_user/case.submit.html>`_
+     Script to perform namelist baseline operations (compare, generate, or both)."
+
+-  case.qstatus
+    Script to query the queue on any queue system.
+
 - `case.setup  <../Tools_user/case.setup.html>`_
     Script used to set up the case (create the case.run script, Macros file and user_nl_xxx files).
 
-- `case.st_archive <../Tools_user/case.st_archive.html>`_
-     Script to perform short term archiving to disk for your case output. Note that this script is run automatically by the normal CIME workflow.
-
 - `case.submit <../Tools_user/case.submit.html>`_
      Script to submit the case to run using the machine's batch queueing system.
 
-- `case.cmpgen_namelist <../Tools_user/case.submit.html>`_
-     Script to perform namelist baseline operations (compare, generate, or both)."
+- `check_case <../Tools_user/check_case.html>`_
+     Script to verify case is set up correctly.
 
-- `xmlchange <../Tools_user/xmlchange.html>`_
-     Script to modify values in the xml files.
+- `check_input_data <../Tools_user/check_input_data.html>`_
+     Script for checking for various input data sets and moving them into place.
 
-- `xmlquery <../Tools_user/xmlquery.html>`_
-     Script to query values in the xml files.
+- `pelayout <../Tools_user/pelayout.html>`_
+     Script to query and modify the NTASKS, ROOTPE, and NTHRDS for each component model.
 
 - `preview_namelists <../Tools_user/preview_namelists.html>`_
      Script for users to see their component namelists in ``$CASEROOT/CaseDocs`` before running the model.
 
 - `preview_run <../Tools_user/preview_run.html>`_
      Script for users to see batch submit and mpirun command."
 
-- `check_input_data <../Tools_user/check_input_data.html>`_
-     Script for checking for various input data sets and moving them into place.
+- `xmlchange <../Tools_user/xmlchange.html>`_
+     Script to modify values in the xml files.
 
-- `check_case <../Tools_user/check_case.html>`_
-     Script to verify case is set up correctly.
+- `xmlquery <../Tools_user/xmlquery.html>`_
+     Script to query values in the xml files.
 
-- `pelayout <../Tools_user/pelayout.html>`_
-     Script to query and modify the NTASKS, ROOTPE, and NTHRDS for each component model.
-     This a convenience script that can be used in place of `xmlchange <../Tools_user/xmlchange.html>`_ and `xmlquery <../Tools_user/xmlquery.html>`_.
 
 **XML Files**
 
 - env_archive.xml
    Defines patterns of files to be sent to the short-term archive.
    You can edit this file at any time. You **CANNOT** use `xmlchange <../Tools_user/xmlchange.html>`_  to modify variables in this file."
 
-- env_mach_specific.xml
-   Sets a number of machine-specific environment variables for building and/or running.
-   You **CANNOT** use `xmlchange <../Tools_user/xmlchange.html>`_  to modify variables in this file.
+- env_batch.xml
+   Sets batch system settings such as wallclock time and queue name."
 
 - env_build.xml
    Sets model build settings. This includes component resolutions and component compile-time configuration options.
    You must run the case.build command after changing this file.
 
-- env_run.xml
-   Sets runtime settings such as length of run, frequency of restarts, output of coupler diagnostics, and short-term and long-term archiving.
-   This file can be edited at any time before a job starts.
+- env_case.xml
+   Parameters set by create_newcase
 
 - env_mach_pes.xml
    Sets component machine-specific processor layout (see changing pe layout ).
    The settings in this are critical to a well-load-balanced simulation (see :ref:`load balancing <optimizing-processor-layout>`).
 
-- env_batch.xml
-   Sets batch system settings such as wallclock time and queue name."
+- env_mach_specific.xml
+   Sets a number of machine-specific environment variables for building and/or running.
+   You **CANNOT** use `xmlchange <../Tools_user/xmlchange.html>`_  to modify variables in this file.
+
+- env_run.xml
+   Sets runtime settings such as length of run, frequency of restarts, output of coupler diagnostics, and short-term and long-term archiving.
+   This file can be edited at any time before a job starts.
+
+- env_workflow.xml
+   Sets paramateres for the runtime workflow.
 
 **User Source Mods Directory**
 

doc/source/users_guide/index.rst

@@ -6,7 +6,7 @@
 .. _users-guide1:
 
 #######################################
-Case Control System Part 1: Basic Usage
+Using the Case Control System
 #######################################
 
 .. toctree::
@@ -22,12 +22,13 @@ Case Control System Part 1: Basic Usage
    cime-change-namelist.rst
    cime-config.rst
    cime-customize.rst
+   testing.rst
    troubleshooting.rst
 
 .. _users-guide2:
 
 #######################################################################################
-Case Control System Part 2: Configuration, Porting, Testing and Use Cases
+Configuring the Case Control System
 #######################################################################################
 
 .. toctree::
@@ -40,8 +41,6 @@ Case Control System Part 2: Configuration, Porting, Testing and Use Cases
    machine.rst
    pes-threads.rst
    porting-cime.rst
-   timers.rst
-   testing.rst
    unit_testing.rst
    multi-instance.rst
    workflows.rst

doc/source/users_guide/introduction-and-overview.rst

@@ -26,7 +26,7 @@ Other prerequisites:
 
 CIME's commands are Python scripts and require a correct version of
 the Python interpreter to be installed. The Python version must be
-greater than 2.7.  Determine which version you have
+greater than 2.11.  Determine which version you have
 like this:
 ::
 
@@ -37,7 +37,7 @@ Consult your local documentation if you need to update your python version.
 Key Terms and concepts
 ======================
 
-The following key terms and concepts are ingrained in CIME and used frequently in this documentation.
+The following key terms and concepts are ingrained in the CCS and used frequently in this documentation.
 See the :ref:`glossary` for a more complete list of terms.
 
 **components**
@@ -63,26 +63,61 @@ See the :ref:`glossary` for a more complete list of terms.
    *data*: For some climate problems, it is necessary to reduce feedbacks within the system by replacing an active model with a
    version that sends and receives the same variables to and from other models, but with the values read from files rather
    than computed from the equations. The values received are ignored. These active-model substitutes are called *data models*.
-   CIME provides data models for each of the possible components.  You could add your own data model implementation of a component
-   but as for active models only one at a time can be used.
 
-   *stub*: For some configurations, no data model is needed, so CIME provides *stub* versions that simply occupy the
-   required place in the driver and do not send or receive any data.
+   *stub*: For some configurations, no data model is needed and one instead uses a *stub* version that simply occupies the
+   required place in the driver and does not send or receive any data.  For example, if you are setting up an aqua-planet case
+   you would only need a stub for the land model.
 
-**component set** or **compset**:   The particular combination of active, data and stub versions of the 7 components is referred to
+**component set** or **compset**:
+
+   The particular combination of active, data and stub versions of the 7 components is referred to
    as a *component set* or  *compset*.  The Case Control System allows one to define
-   several possible compsets and configure and run them on supported platforms. See :ref:`Component Sets<compsets>` for more information.
+   several possible compsets and configure and run them on supported platforms.
+   Here is an example of a component set *longname* from E3SM for a fully coupled active case:
+::
+
+   1850SOI_EAM%CMIP6_ELM%CNPRDCTCBCTOP_MPASSI_MPASO_MOSART_SGLC_SWAV
+
+See :ref:`Component Sets<compsets>` for more information. "GLC" originally meant "glacier model" and is now an ice-sheet model but the GLC letters have stuck.
+
+**compset alias**:
+
+   Typing a compset longname like the above can be exhausting so the CCS allows defining a shorter *compset alias*
+   which is a short string that substitutes for the longname. In E3SM, the above longname can be reffered to as "WCYCL1850".
+
+.. note:: 
+
+     Long ago, CESM established a convention for the first letter in a compset alias based
+     on the combination of active, data and stub components.
+     If you see mention of "B-case" or "F-case", it comes from these conventions.
+     They pre-date the introduction of a wave model as an option.
+
+    ===  ========================================================================================
+    A    All data models
+    B    All models fully active with stub glc
+    C    Active ocean with data atm, river and sea ice. stub lnd, glc
+    D    Active sea ice with data atm, ocean (slab) and river. stub lnd, glc
+    E    Active atm, lnd, sea-ice, river, data ocean (slab), stub glc
+    F    Active atm, lnd, river, sea-ice (thermodynamics only), data ocean (fixed SST), stub glc
+    G    Active ocean and sea ice, data atmosphere and river, stub lnd and glc 
+    H    Active ocean and sea ice, data atmosphere, stub lnd, river and glc 
+    I    Active land and river model, data atmosphere, stub ocn, sea-ice, glc
+    IG   Active land, river and ice-sheet, data atmosphere, stub ocn, sea-ice
+    S    All stub models (for testing only)
+    X    All x-compsets (2D sine waves for each component except stub glc; for testing only)
+    ===  ========================================================================================
+..
+**grid set**:
 
-**grid** or **model grid**:
    Each active model must solve its equations on a numerical grid. CIME allows models within the system to have
-   different grids. The resulting set of all numerical grids is called the *model grid* or sometimes just the *grid*, where
-   *grid* is a unique name that denotes a set of numerical grids. Sometimes the *resolution* also refers to a specific set
-   of grids.
+   different grids. The resulting set of all numerical grids is called the *grid set* or usually just the *grid*. Like
+   the compset longnamme, the CCS allows one to define an alias to represent a grid set.  This alias is also referred to
+   as the *grid* or sometimes the *resolution*.
 
 **machine and compilers**:
    The *machine* is the computer you are using to run CIME and build and run the climate model. It could be a workstation
    or a national supercomputer. The exact name of  *machine* is typically the UNIX hostname but it could be any string.  A machine
-   may have one more more versions of Fortran, C and C++ *compilers* that are needed to compile the model's source code and CIME
+   may have one more more versions of Fortran, C and C++ *compilers* that are needed to compile the model's source code and CIME.
 
 **case**:
     To build and execute a CIME-enabled climate model, you have to make choices of compset, model grid,
@@ -95,10 +130,10 @@ See the :ref:`glossary` for a more complete list of terms.
    the model source code and version-controlled together, its possible to match supported out-of-the-box cases with specific
    versions of the model source code, promoting reproducibility and provenance.  An out-of-the-box case is also called a *base case*
 
-CIME and your environment
+CCS and your environment
 =========================
 
-Before using any CIME commands, set the ``CIME_MODEL`` environment variable. In bash, use **export** as shown and replace
+Before using any CCS commands, set the ``CIME_MODEL`` environment variable. In bash, use **export** as shown and replace
 **<model>** with the appropriate text. Current possibilities are "e3sm" or "cesm."
 ::
 
@@ -139,17 +174,14 @@ After you submit the case, you can follow the progress of your run by monitoring
 Repeat the command until you see the message ``case.run success``.
 
 
-Discovering available cases with **query_config**
+Discovering available pre-defined compsets with **query_config**
 =================================================
 
-Your CIME-driven model has many more possible cases besides the simple one in the above Quick Start.
+Your CIME-driven model likely has many compset and gridset aliases defined for cases that are widely used by the
+model developers.
 
 Use the utility `query_config <../Tools_user/query_config.html>`_  to see which out-of-the-box compsets, components, grids and machines are available for your model.
 
-If CIME is downloaded in standalone mode, only standalone CIME compsets can be queried.
-
-If CIME is part of a CIME-driven model, `query_config <../Tools_user/query_config.html>`_ will allow you to query all prognostic component compsets.
-
 To see lists of available compsets, components, grids and machines, look at the **help** text::
 
   > query_config --help