Skip to content

Development Workflow

Jump to bottom
Michael Kavulich edited this page Jun 13, 2024 · 19 revisions

THIS PAGE IS UNDER CONSTRUCTION

We are in the process of updating and modernizing these instructions. Please bear with us as we work through these updates.

Table of Contents

  1. Open an Issue
  2. Develop new feature or bug fix
  3. Test! Test! Test!
  4. Submit Pull Request
  5. Code Merged to Main
  6. Post-merge Testing

Open an Issue

All development, whether for a new feature, bug fix, re-engineering task, or documentation or data update, MUST begin with opening an issue. An open issue allows for transparency, developer and community input, assignment of the task, and progress tracking.

To open an issue, follow these steps

Develop new feature or bug fix

Coding should take place in a branch in your GitHub fork. If you have not yet created a fork of ccpp-framework, you can create one at https://github.com/NCAR/ccpp-framework/fork by selecting your username from the "Owner" dropdown menu.

Test! Test! Test!

A number of tests are required to pass prior to a PR being merged, and it is highly recommended that you run as many of these tests as possible prior to opening your PR.

Continuous Integration (CI) tests

CI tests (found in ccpp-framework/.github/workflows/) are set to run automatically once a PR is opened. These are mainly unit tests for exercising various parts of the framework system, but there are also "smoke tests" for certain components, such as the ccpp_track_variables.py script. More testing may be rolled into continuous integration in the future as well.

Tests specific to "prebuild" (ccpp_prebuild.py) are contained in the top-level test_prebuild/ directory. Other tests, including those specific to "capgen" (ccpp_capgen.py), are contained in the top-level test/ directory. To run these tests, navigate to the appropriate directory and run the test scripts; in test_prebuild/ this script is ./run_all_tests.sh, in test/ these scripts are run_fortran_tests.sh and pylint_test.sh. You will need to have all of the framework prerequisites installed (Python 3.8 or higher, a compatible C and Fortran compiler, with MPI enabled) as well as cmake, make, and the python package pytest.

Regression tests

Because the CCPP framework does not generate data on its own except to create software caps, regression testing is done in the context of a host model utilizing the framework. When opening a PR to the develop branch, you should ensure regression testing is passing (or at least, has expected differences with no errors) in one or more host models with your latest code.

SCM

The CCPP Single-Column Model (SCM) has a regression test documented here:

UFS Weather Model

The UFS Weather Model has a regression test documented here:

CESM

The Community Earth System Model has a regression test documented here:

NEMSfv3gfs regression testing on Cheyenne (work in progress)

Prerequisites:

How to set up password-less GitHub access on Cheyenne

  • search the web for git config credential.helper store or git config credential.helper cache
  • make sure to load the git module before making those changes: module load git/2.10.2

Instructions (on Cheyenne):

  • check out the latest version of ccpp-framework from http://www.github.com/NCAR/ccpp-framework
  • go to ccpp-framework/test/nemsfv3gfs and adjust the configuration file regression_test_nemsfv3gfs_cheyenne.cfg
    (see documentation in regression_test_nemsfv3gfs.py)
  • load the git module via
. setenv_cheyenne.sh       # for bash
source setenv_cheyenne.sh  # for csh/tcsh

./regression_test_nemsfv3gfs.py --config=regression_test_nemsfv3gfs_cheyenne.cfg
  • go for lunch or (recommended) a lunchtime run as running the tests will take some time
  • once the tests and the run are done, watch out for a line reading
REGRESSION TEST WAS SUCCESSFUL

or

REGRESSION TEST FAILED
  • the compile logs and the output for the individual regression tests can be found in
$BASEDIR/rt_YYYYMMDDTHHMMSS/tests/log_$MACHINE.$COMPILER

where $BASEDIR is configured in the regression test config file described above, $MACHINE is the name of the system (cheyenne, theia, ...) and $COMPILER the configuration option compiler in the regression test config file.

How to make adjustments and rerun the tests manually?

While most of the wonderful development made by ccpp-framework developers will pass the tests right away, it may occasionally be necessary to make adjustments and rerun the tests manually (instead of rerunning the script regression_test_nemsfv3gfs.py). This is the procedure:

  • locate the work directory $WORKDIR=$BASEDIR/rt_YYYYMMDDTHHMMSS, which is the top-level directory of the NEMSfv3gfs main repository
  • regression test run directories are located in $WORKDIR/FV3_RT/rt_$$ where $$ stands for a unique PID
  • replace/update code as needed in the main repository or the submodules
  • change to $BASEDIR/rt_YYYYMMDDTHHMMSS/tests and execute
ACCNR=$ACCOUNT NEMS_COMPILER=$COMPILER RUNDIR_ROOT=$RUNDIR ./rt.sh -l $RTCONF 2>&1 | tee rt.log

where $ACCOUNT corresponds to the configuration option account in the regression test config file, $COMPILER to , $RTCONF to the configuration option config, and $RUNDIR to $BASEDIR/rt_YYYYMMDDTHHMMSS/FV3_RT, or, for convience, to $PWD/../FV3_RT when executing rt.sh from $BASEDIR/rt_YYYYMMDDTHHMMSS/tests

  • this will create a new regression test directory $WORKDIR/FV3_RT/rt_$$ (see rt.log)
  • it is also possible to manually run the compile command instead of executing rt.sh; however, this is more complicated as the compiler options may depend on which test is failing (contact a GMTB developer in this case)

Submit Pull Request

To submit a pull request, use GitHub's "New pull request" button and set the proper compare (your development) and base (the branch on this repository to which to merge your development).

  • Enter a short description in the title bar
  • In the description text box, enter a longer description with information about the change.
  • Using proper syntax, enter the issues which are resolved by this pull request (e.g., fixes #42). If there is no issue, create one to describe the problem this PR is solving.
  • Add appropriate labels
  • Add an appropriate set of reviewers including, at least, the CODEOWNERS.
  • Assign the PR to yourself or to a member of CODEOWNERS.

If you are assigned as a reviewer of a pull request, please review the code as soon as possible. If you feel that you do not have the time or expertise to review the PR, you may remove your name from the list of reviewers unless you are the last reviewer from your organization. One person from each participating organization must review each PR.

Code Merged to Main

Post-merge Testing

CCPP Framework wiki

Clone this wiki locally