Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add warning for unused v4 input features #104

Merged
merged 1 commit into from
Feb 3, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions online_testing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ Take our MLP baseline model (from the [Stable Machine-Learning Parameterization]

If you want to reproduce the U-Net models from [Stable Machine-Learning Parameterization](https://arxiv.org/abs/2407.00124) paper, run the [create_dataset_example_v4.ipynb](./data_preparation/create_dataset/create_dataset_example_v4.ipynb) notebook to prepare the input/output files for the Unet_v4 model. Or run the [create_dataset_example_v5.ipynb](./data_preparation/create_dataset/create_dataset_example_v5.ipynb) notebook to prepare the input/output files for the Unet_v5 model. 'v4' is the unconstrained U-Net, while 'v5' is the constrained U-Net, please refer to original paper for more details.

**Note:** For both 'v4' and 'v5' input features in the [data_utils.py](../climsim_utils/data_utils.py), the code includes the following variables into the preprocessed input data array: ```'tm_state_ps', 'tm_pbuf_SOLIN', 'tm_pbuf_LHFLX', 'tm_pbuf_SHFLX', 'tm_pbuf_COSZRS', 'icol'```. These features are not yet implemented in the online E3SM code, so during the U-Net training, these input features were set to 0 at the beginning of the forward method of the U-Net model (i.e., they are not used in U-Net training). To be comopatible with the E3SM code, you should exclude these features from the input feature array when you train your own model.

---

## 3. Model training
Expand Down Expand Up @@ -76,6 +78,10 @@ The E3SM MMF-NN-Emulator code expects the NN model to take un-normalized input f

For example, the [v5_nn_wrapper.ipynb](./model_postprocessing/v5_nn_wrapper.ipynb) notebook shows how to create a wrapper for the U-Net model to read raw input features, calculate additional needed input features, normalize the input, clip input values, pass them to the U-Net model, de-normalize the output features, and apply the temperature-based liquid-ice cloud partitioning.

Currently two input configurations are supported in the E3SM MMF-NN Emulator code: 'v2rh' and 'v4' (see the v2_rh_inputs and v4_inputs in [data_utils.py](../climsim_utils/data_utils.py)). If you did not use these two configurations during training, your wrapper function should internally convert the input array first to be compatible with your input features. For example, in the [v5_nn_wrapper.ipynb](./model_postprocessing/v5_nn_wrapper.ipynb) notebook, the wrapper NN function will still take an v4 input array but will internally convert it to a v5 input array before passing it to the saved U-Net model.

Again, note that in the 'v4' input configuration, the input features include the following variables: ```'tm_state_ps', 'tm_pbuf_SOLIN', 'tm_pbuf_LHFLX', 'tm_pbuf_SHFLX', 'tm_pbuf_COSZRS', 'icol'```. These features are not yet implemented in the online E3SM code, you should exclude these features from the input feature array when you train your own model. During our U-Net training, these input features were set to 0 at the beginning of the forward method of the U-Net model (i.e., they are not used in U-Net training)

---

## 5. Run hybrid E3SM MMF-NN-Emulator simulations
Expand Down