diff --git a/docs/config-and-intervention/optimize-intervention-policy.md b/docs/config-and-intervention/optimize-intervention-policy.md index 9bac338..9edcd97 100644 --- a/docs/config-and-intervention/optimize-intervention-policy.md +++ b/docs/config-and-intervention/optimize-intervention-policy.md @@ -20,8 +20,6 @@ Given a model configuration and a proposed intervention policy, you can identify - The [PyCIEMSS `optimize` interface](https://github.com/ciemss/pyciemss/blob/6a41b1a8247dd76f929488a479f4d27671120b36/pyciemss/interfaces.py#L818). - The [PyCIEMSS `optimize` interface notebook](https://github.com/ciemss/pyciemss/blob/main/docs/source/optimize_interface.ipynb). - - ## Create an optimize intervention policy operator Assuming you have a [model configuration](configure-model.md) and an [intervention policy](create-intervention-policy.md) in your workflow, add an Optimize intervention policy operator: diff --git a/docs/config-and-intervention/validate-model-configuration.md b/docs/config-and-intervention/validate-model-configuration.md index 4500ca9..caa53b3 100644 --- a/docs/config-and-intervention/validate-model-configuration.md +++ b/docs/config-and-intervention/validate-model-configuration.md @@ -61,7 +61,7 @@ The Validate configuration operator allows you to assess and refine model config ??? list "Open the Validate configuration operator" - - Make sure you've connected a model configuration to the Validate configuration operator and then click **Review checks**. + - Make sure you've connected a model configuration to the Validate configuration operator and then click **Open**. ### Use the wizard to validate a model configuration diff --git a/docs/datasets/compare-datasets.md b/docs/datasets/compare-datasets.md index 4d3e27d..ef11638 100644 --- a/docs/datasets/compare-datasets.md +++ b/docs/datasets/compare-datasets.md @@ -59,7 +59,7 @@ You can set up your dataset comparison by selecting a baseline and adjusting key ??? list "Define the comparison" - 1. Select **Compare the impact of interventions**. + 1. Select **Compare scenarios**. 2. (Optional) Specify which dataset is the baseline simulation. 3. (Optional) Select **Average treatment effect** to include a summary of the overall impact of interventions in the resulting comparison tables. diff --git a/docs/datasets/transform-dataset.md b/docs/datasets/transform-dataset.md index 7345f11..db2bd83 100644 --- a/docs/datasets/transform-dataset.md +++ b/docs/datasets/transform-dataset.md @@ -96,7 +96,7 @@ Prompts and responses are written to cells where you can preview, edit, and run ??? list "Open the Transform dataset code notebook" - - Make sure you've connected one or more datasets to the Transform dataset operator and then click **Edit**. + - Make sure you've connected one or more datasets to the Transform dataset operator and then click **Open**. ??? list "Rerun a code notebook" @@ -399,7 +399,7 @@ The following sections show examples of how to prompt the Transform dataset AI a Before you combine datasets, make sure they share at least one common column like name, ID, date, or location. You can ask the AI assistant to link them by matching records based on the common data so that information aligns correctly. - 1. Connect the outputs of each dataset to the input of a Transform dataset operator and then click **Edit**. + 1. Connect the outputs of each dataset to the input of a Transform dataset operator and then click **Open**. 2. Ask the assistant to: ```{ .text .wrap } diff --git a/docs/get-started/example-workflow.md b/docs/get-started/example-workflow.md index 75c0859..08b98b5 100644 --- a/docs/get-started/example-workflow.md +++ b/docs/get-started/example-workflow.md @@ -4,273 +4,317 @@ title: "Tutorial: Recreate, modify, and simulate a model" # Recreate, modify, and simulate a model -This workflow tutorial is designed to help you learn how in a few steps—all without needing to code—you can recreate, refine, and analyze models in Terarium. +This tutorial is designed to help you learn how—all without needing to code—you can recreate, refine, and analyze models in Terarium. You can follow along by looking at the shared [Terarium help sample project](https://app.terarium.ai/projects/cc51dd40-f08a-4085-9eb1-ae22d7483839/overview). -You'll start with only a dataset of COVID cases and deaths and a set of LaTeX equations that describe an SEIRHD (1) model. You'll bring these into Terarium to recreate the model, incorporate vaccination information, calibrate the model to LA county data, and create intervention policies. Throughout the workflow, you'll learn how to: -{ .annotate } - -1. Susceptible, Exposed, Infected, Recovered, Hospitalized, and Deceased +The goal of the modeling exercise in the project is to reduce COVID hospitalizations in LA county. Starting with only a dataset of cases and deaths and a few scientific papers describing disease models, it shows how to:
-- **Upload and modify models and data** +- **Upload and modify models and data** --- - - - [Create a project](#create-a-project) - [Upload resources](#upload-resources) + - [Create and compare models from equations](#create-and-compare-models-from-equations) + - [Edit a model](#edit-models) + - [Stratify a model](#stratify-models) to account for dimensions like vaccination status - [Work with data](#work-with-data) - - [Create models from equations](#create-models-from-equations) - - [Edit models](#edit-models) - **Simulate models and explore intervention policies** --- - - [Configure and simulate models](#configure-and-simulate-models) - - [Calibrate model configurations](#calibrate-model-configurations) + - [Configure and calibrate a model](#configure-and-calibrate-a-model) + - [Run a sensitivity analysis](#run-a-sensitivity-analysis) - [Create and simulate intervention policies](#create-and-simulate-intervention-policies) to run what-if scenarios - [Optimize intervention policies](#optimize-intervention-policies) - - [Stratify models](#stratify-models) to account for different dimensions + - [Compare datasets](#compare-datasets)
## Upload and modify models and data -### Create a project +### Upload resources -A project is a space to explore a problem or scenario. It stores your modeling resources (models, datasets, and documents) and various workflows that modify and simulate your models and data. +Begin setting up your project by uploading the models, papers (documents), and datasets you need for your modeling processes. In this case, that includes a dataset of U.S. COVID cases and deaths from 2021 and a set of papers describing different disease models. -??? list "Create a project for modifying the SEIRHD model" +
+![](../img/get-started/resources.png) +
+
-
+??? list "Upload modeling resources" - - ![](../img/get-started/example-home.png) - - ![](../img/get-started/example-project-create.png) + ![](../img/get-started/example-upload-dataset.png) -
+ - Drag the dataset and document files into the Resources panel and then click **Upload**. - 1. On the Home page, click :octicons-plus-24:{ aria-hidden="true" } **New project**. - 2. Give the project the title *SEIRHD model*, select a thumbnail image, and then click **Create**. +### Create and compare models from equations -??? list "Create a workflow to capture your modeling and simulation work" +Terarium can [automatically recreate a model](https://app.terarium.ai/projects/cc51dd40-f08a-4085-9eb1-ae22d7483839/workflow/78688b6d-9932-4d24-b090-16ba03c7d255) from a set of ordinary differential equations. In this case, we create models by extracting equations from the uploaded documents, but you could also get equations from pasted images or manually enter them as LaTeX. - ![](../img/get-started/example-workflow-new.png) +When the extraction and creation is complete, Terarium builds visual representations of the extracted SIR and SEIRHD models that show how people progress between disease states. - 1. Click :octicons-plus-24:{ aria-hidden="true" } **New** in the Workflows section of the Resources panel. - 2. Make sure **Blank Canvas** is selected, call the workflow *SEIRHD model with interventions*, and click **Create**. +
-### Upload resources +- ![](../img/get-started/example-model-from-equations-workflow.png) +- ![](../img/get-started/example-model-from-equations-notebook.png) -Begin setting up your project by gathering resources such as models, datasets, and documents. Upload the following dataset that records COVID cases and deaths in California. +
-
-![SEIRHD equations](../img/get-started/example-dataset.png) -
:material-download:{ aria-hidden="true" } [Download](../assets/ca-counties-2021.csv)
-
+??? list "Recreate a model from a paper" -??? list "Upload the California COVID case data" + 1. Drag each document into the workflow canvas, hover over its output, click Link :octicons-plus-24:{ title="Link" aria-labelledby="link-icon-label" } > **Create model from equations**, and then click **Open**. + 2. Review the equations automatically extracted from the document. To make changes or correct extraction errors, click an equation to edit the LaTeX version. + 3. Select each equation you want to include in the model and then click **Run** > **Mira**. + 4. At the top of the Output panel, click **Save for re-use**, and then enter a unique name. - ![](../img/get-started/example-upload-dataset.png) +To understand the extracted SIR and SEIRHD models better and decide which one to use, we pipe them into a Compare models operator. This uses an AI assistant to create side-by-side model cards for each according to our modeling goal of reducing hospitalizations - 1. Save the CSV to your computer. - 2. In the Resources panel, click **Upload** and drag in or browse to the location of the CSV file. - 3. Click **Upload**. +
+ +- ![](../img/get-started/example-compare-workflow.png) +- ![](../img/get-started/example-compare-wizard.png) + +
+ +??? list "Compare models" + + 1. Hover the output of a Create model from equations operator and click Link :octicons-plus-24:{ title="Link" aria-labelledby="link-icon-label" } > **Compare models**. + 2. Click the output of the other Create model from equations operator and connect it to the new Compare models operator. + 3. Click **Open**. + 4. Enter your goal for making the comparison. In this case: + + > I need to forecast the impacts of transmission rates and vaccination rates on hospitalizations. Which model is best suited for this work? + + 5. Click **Compare** and review the summary tailored to the specified goal. + +The AI-generated summary indicates that the SEIRHD model would be best for forecasting the impacts of transmission and vaccination on hospitalizations. + +### Edit models -??? list "Enrich the case data with information the describes the columns" +Now we want to [update the SEIRHD model to allow people to move from infected to recovered without becoming hospitalized](https://app.terarium.ai/projects/cc51dd40-f08a-4085-9eb1-ae22d7483839/workflow/d0a259d7-55fa-4438-93e0-807b5e2824fc). Even if you don't have any coding experience, you can use Terarium's AI-assisted Edit model notebook. The assistant simplifies the process of changing or building off an existing model—no knowledge of specialized modeling libraries needed! - 1. Open the dataset by clicking its name in the Resources panel. - 2. Add information that helps you understand what the columns contain by clicking **Enrich metadata**. +
+ +- ![](../img/get-started/example-model-edit-workflow.png) +- ![](../img/get-started/example-model-edit-notebook.png) + +
+ +??? list "Add a new transition from infected to recovered" + + 1. Pipe the model into an Edit model operator and then click **Open**. + 2. Add a new parameter for the transition rate law by asking the AI assistant to: + + ```{ .text .wrap } + Add a new parameter λ with the description "recovery rate from infection" + ``` + + 3. Add the new transition by asking the assistant to: + + ```{ .text .wrap } + Add a new transition from a state named I to a state named R with the rate law I * λ + ``` + + 4. Click :material-play-outline:{ aria-hidden="true" } **Run** to apply the changes. Compare the edited model to the previous state by changing the output in the top right. + 5. Click **Save for re-use** and then enter *SEIRHD edited* as the name of the new model. + +### Stratify models + +Now we want to stratify our edited model to account for vaccinated and unvaccinated groups. Terarium's stratification process is an error-proof approach to stratifying along any dimension, such as age, sex, and location. + +
+ +- ![](../img/get-started/example-model-stratify-workflow.png) +- ![](../img/get-started/example-model-stratify-wizard.png) + +
+ +??? list "Stratify the SEIRHD model by age" + + 1. Pipe the model into an Stratify model operator and then click **Open**. + 2. Name the strata "vaccination". + 3. Select to stratify the *S*, *E*, and *I* state variables and the *beta* (infection rate), *gamma* (latent time), *v* (hospitalized rate), and *λ* (recovery rate) parameters. + 4. List the labels for each strata group: + + ``` text + u, v + ``` + + 5. Choose the allowed transitions and interactions between strata: + + - Select **Create new transitions between strata** to allow unvaccinated people to turn into vaccinated people. + - Clear **Allow existing interactions between strata** to prevent vaccinated and unvaccinated people from interacting with and infecting each other. ???+ note - Because you don't have any documents in your project yet, Terarium assumes the dataset relates to epidemiology when it attempts to explain the columns. When you have documents in your project, you can use them to provide context for the columns. + In this case because vaccinated people cannot turn into unvaccinated people, additional settings must be configured in the Notebook view. For more information, see the code inside the Stratify model operator in the [Terarium help sample project](https://app.terarium.ai/projects/cc51dd40-f08a-4085-9eb1-ae22d7483839/workflow/cfdf914b-078e-4e67-ab6f-15bbf5c0ee1a) or see [Stratify a model](../modeling/stratify-model.md#add-or-edit-code). + 7. Click **Stratify**. + 8. Click **Save for re-use** and edit the name of the new model. ### Work with data -The uploaded dataset covers COVID cases and deaths in all of California for the second half of 2021. Use Terarium's data transformation tools to filter down to focus on LA county in the last two months of the year. +The uploaded dataset covers all of the U.S. for 2021. However, we're only interested in LA county. We can use Terarium's data transformation tools to filter down to just what we need. + +We'll also add a timestep column, which we'll need later to calibrate our model to the historical data. + +
+ +- ![](../img/get-started/example-dataset-transform-workflow.png) +- ![](../img/get-started/example-dataset-transform-notebook.png) + +
??? list "Filter the case data to focus on LA county" - 2. Drag the dataset from the Resources panel onto the canvas, hover over its output, click link :octicons-plus-24:{ title="Link" aria-labelledBy="link-icon-label" } > **Transform dataset**, and then click **Edit**. - 3. Preview the data by clicking Run :material-play-outline:{ title="Run" aria-labelledBy="run-icon-label" }. - 4. Ask the assistant to filter the data: - - ``` markdown - Filter the data to LA county from October 31st to December 31st and show me - the new dataset. - ``` - - 5. Ask the assistant to create a timestep column in the new dataset `filtered_d1`: + 1. Drag the dataset from the Resources panel onto the canvas, hover over its output, click link :octicons-plus-24:{ title="Link" aria-labelledBy="link-icon-label" } > **Transform dataset**, and then click **Open**. + 2. Preview the data by clicking Run :material-play-outline:{ title="Run" aria-labelledBy="run-icon-label" }. + 3. Ask the assistant to filter the data: - ``` text - Add a new "timestep" column to filtered_d1. Make the value for the earliest - date equal 0 and add 1 for each subsequent date. Show me the result. + ``` { .text .wrap } + filter the data for LA county from October 28th 2021 to December 28th 2021. Add a new column named timestep with the first value starting at 0 and increasing by n+1. ``` - - 6. Ask the assistant to plot the data over time: + 4. Ask the assistant to plot the data over time: - ``` text - Plot deaths over time for filtered_d1. + ``` { .text .wrap } + plot cases over time for the filtered_df ``` - 7. Inspect the generated code, change the following line to include `COVID` in the title, and then click Run :material-play-outline:{ title="Run" aria-labelledBy="run-icon-label" } to redraw the plot. + 5. Inspect the generated code, change the following line to include `COVID` in the title, and then click Run :material-play-outline:{ title="Run" aria-labelledBy="run-icon-label" } to redraw the plot. ``` { .python .no-copy } - plt.title('Deaths Over Time in Los Angeles County') + plt.title('Number of Cases Over Time') ``` - 8. At the top of the window, select *filter_d1* and click **Save for reuse**. + 6. Show the plot in the workflow by selecting **Display on node thumbnail**. + 7. At the top of the window, select *filter_d1*, click **Save for reuse**, and enter *LA county cases and deaths* as the name of the new dataset. Doing your data transformations in Terarium helps make your modeling process more transparent and reproducible. -### Create models from equations +## Simulate models and explore intervention policies -![SEIRHD equations](../img/get-started/example-equations-SEIRHD.png){ align="right" } +### Configure and calibrate a model -Terarium can automatically recreate a model from a set of ordinary differential equations. You can extract equations from a document or pasted image or manually enter them as LaTeX. +Before you can simulate the modified SEIRHD model, we need to configure it to set the initial values for its states and parameters. To improve its performance, we can also adjust these by calibrating it against the context of the LA county data. -The equations in the image to the right define a COVID SEIRHD model. You can copy and paste it into Terarium to automatically recreate the model with a visual representation that shows how people progress between disease states. +In this example, we'll work with an already existing model configuration, but normally you can manually create configurations based on your expert knowledge or automatically extract them from documents or datasets in your project. -??? list "Create an SEIRHD model from a screenshot of equations" +
- 1. Copy the image above by right-clicking and selecting **Copy image**. - 2. Right-click anywhere on the workflow canvas, select **Modeling** > **Create model from equations**, and then click **Edit**. - 3. Right-click inside the text box and select **Paste**. - 4. When the LaTeX versions of the pasted equations appear, click :octicons-plus-24:{ aria-hidden="true" } **Add** and then click **Run** to create the model. - 5. At the top of the window, click **Save for reuse**. +- ![](../img/get-started/example-model-calibrate-workflow.png) +- ![](../img/get-started/example-model-calibrate-notebook.png) -### Edit models - -To account for vaccination within the SEIRHD model, you can use Terarium's AI-assisted Edit model notebook. The assistant simplifies the process of changing or building off an existing model so you don't need to be familiar with specialized modeling libraries. +
-??? list "Add a new state variable and parameter for vaccination" +??? list "Calibrate the SEIRHD model to the LA county data" - 1. Hover over the output of the Create model from equations operator, click link :octicons-plus-24:{ title="Link" aria-labelledBy="link-icon-label" } > **Edit model**, and then click **Edit**. - 2. Add a new state variable for vaccination to the SEIRHD model by asking the AI assistant to: - - ``` text - Create a new state variable named V and add a transition named T6 from S to V - with the rate law rv*S. - ``` + 1. Pipe the Configure model operator and the transformed dataset into a Calibrate operator and then click **Open**. + 2. Map the model variables to the dataset variables: - 3. Add a new parameter by asking the assistant to: - - ``` text - Add a new parameter rv with the description vaccination rate and the initial - value set to 0. - ``` + - Set the **Dataset: Timeline variable** to *timestep*. + - Map model observables `cumulative cases` and `deceased` to dataset variables `cases` and `deaths` respectively. - 4. Click **Run** to apply the changes. Compare the edited model to the previous state by changing the output in the top right. - 5. Click **Save for reuse**. + 3. Change the **End time** to *150* and click :material-play-outline:{ aria-hidden="true" } **Run**. -## Simulate models and explore intervention policies +When you calibrate a model, you can review the following immediate visual feedback to help you spot issues quickly: -### Configure and simulate models +- A loss chart showing error over time. +- Cases and deaths data over time, with observations from the dataset and the projected number of cases before and after the calibration. -Before you can simulate your new model, you need to set values for its state variables and parameters. +### Run a sensitivity analysis -??? list "Configure the SEIRHDV model" +Next, we'll simulate our model configuration to perform a sensitivity analysis to explore the effects of infection rate (beta) and vaccination rate (r_Sv) on hospitalizations. - 1. Hover over the output of the Edit model operator, click link :octicons-plus-24:{ title="Link" aria-labelledBy="link-icon-label" } > **Configure model**, and then click **Edit**. - 2. Edit the initial values of the state variables and set the parameter values. Set at least one parameter to be a uniform distribution with a min and max value. - 3. Click **Save as**, enter a unique for the configuration, and click **Save**. - -??? list "Simulate the SEIRHDV model" +![](../img/get-started/example-sensitivity-workflow.png) - 1. Hover over the output of the Configure model operator, click link :octicons-plus-24:{ title="Link" aria-labelledBy="link-icon-label" } > **Simulate**, and click **Edit**. +??? list "Run a sensitivity analysis with the Simulate operator" + + 1. Pipe the Configure model operator a Simulate operator and then click **Open**. 2. Change the End time to **150** days and click :material-play-outline:{ aria-hidden="true" } **Run**. -The base SEIRHD model configured against calibrated data shows hospitalizations peaking at ~38,000 at day 150. +The scatterplots below the sensitivity chart the parameters combine to affect hospitalizations shown in the chart above. Generally, high vaccination rates and low infection rates tend to reduce hospitalizations. -### Calibrate model configurations +![](../img/get-started/example-sensitivity-wizard.png) -To improve the performance of the edited model, you can adjust its parameters by calibrating it against the context of the LA county data. When you calibrate a model, Terarium provides immediate visual feedback to help you spot issues quickly. +### Create and simulate intervention policies -??? list "Calibrate the SEIRHD model to the LA county data" +Our sensitivity analysis showed us the infection rates we should aim for to reduce hospitalizations. Now we can create different masking intervention policies to visualize the impact of different what-if masking scenarios that might get us there. For this, we'll use parameter NPI_mult, which is a multiplier for the transmission rate. - 1. Hover over the output of the Configure model operator and then click link :octicons-plus-24:{ title="Link" aria-labelledBy="link-icon-label" } > **Calibrate**. - 2. Click the output of the Transform dataset operator and then click the Dataset input of the Calibrate Operator. - 3. Map the model variables to the dataset variables: - - - Set the **Dataset: Timeline variable** to *timestep*. - - Map model variables `I` and `D` to dataset variables `cases` and `deaths` respectively. - - 4. Change the **End time** to *150* and click :material-play-outline:{ aria-hidden="true" } **Run**. +We'll create two policies: -When the calibration is complete, you can review: +- One that sets NPI_mult to 50% on day 63. +- One that sets to 50% only when hospitalizations cross 8,000. -- A loss chart showing error over time. -- Cases and deaths data over time, with observations from the dataset and the projected number of cases before and after the calibration. +
-### Create and simulate intervention policies +- ![](../img/get-started/example-interventions-static.png) +- ![](../img/get-started/example-interventions-dynamic.png) -You can configure complex intervention policies to create and visualize the impact of different what-if scenarios. +
-The base SEIRHD model configured against calibrated data shows hospitalizations peaking at ~38,000 at day 150. Given the high number of hospitalizations, you want to explore the impacts of different interventions: +??? list "Create and simulate an intervention policy to increase masking" -??? list "Create and simulate a intervention policy to increase the vaccination rate" + 1. Pipe the Configure model operator into two different Create intervention policy operators and then click **Open**. + 2. Set the intervention policies: + 1. On one policy, create a new **Static** intervention starting at day *63* that sets *Paramater* *NPI_mult* to *0.5*. + 2. On the other, create a new **Dynamic** intervention that sets *Paramater* *NPI_mult* to *0.5* when hospitalizations (H) cross the threshold of 8,000. + 3. Save each intervention policy and give them unique names. + 4. Create three new Simulate operators: + 1. One the first, pipe in only the model configuration to get a baseline without interventions. + 2. On the second, pipe in the model configuration and the static intervention. + 3. On the third, pipe in the model configuration and the dynamic intervention. + 5. Open each Simulate operator, change the **End time** to *150* days, and click :material-play-outline:{ aria-hidden="true" } **Run**. + 6. Click **Save for re-use** to save each simulation result as a new dataset. - 1. Hover over the output of the Edit model operator, click link :octicons-plus-24:{ title="Link" aria-labelledBy="link-icon-label" } > **Create intervention policy**, and then click **Open**. - 2. Create a new **Static** intervention policy starting at day *65* that sets *Paramater* *r_Sv* to *70,000*. - 3. Click **Save** and call the new intervention policy *Vaccination Campaign*. - 4. Hover over the output of the Create intervention policy operator and click link :octicons-plus-24:{ title="Link" aria-labelledBy="link-icon-label" } > **Simulate**. - 5. Connnect the output of the Configure model operator to the Model configuration input of the Simulate operator and then click **Edit**. - 6. Change the End time to **150** days and click :material-play-outline:{ aria-hidden="true" } **Run**. - 7. To see the impacts on hospitalizations, add a chart for H and hover over the graph to see that hospitalizations peak around 8,795 at day 126 with vaccination. +Both interventions reduce hospitalizations compared to the baseline, but the static intervention of introducing masking at day 63 is more effective than the dynamic intervention that waits for hospitalizations to reach 8,000. -This contrasts with 38,000 hospitalizations when the vaccination rate is only 10,000 per day. +![](../img/get-started/example-interventions.png) ### Optimize intervention policies -In Terarium, you can optimize interventions to meet specified constraints, allowing you to get answers to key decision maker questions faster. To find how many people may need to be vaccinated per day (minimum increase in daily vaccinations) to prevent hospitalizations from exceeding capacity: +In Terarium, you can optimize interventions to meet specified constraints, allowing you to get answers to key decision maker questions faster. We want to find how effective masking needs to be to prevent hospitalizations from exceeding capacity. -??? list "Optimize the intervention policy to find how many daily vaccinations are needed to reduce hospitalizations" +![](../img/get-started/example-interventions-optimize-settings.png) - 1. Set the daily vaccination rate parameter r_SV to 20,000 per day starting at day 65. - 2. Connect to an Optimize intervention policy operator. - 3. Set the success criteria: +??? list "Optimize the intervention policy to find how effective masking needs to be to reduce hospitalizations" + + 1. Pipe the static intervention and model configuration into an Optimize intervention policy operator and click **Open**. + 2. Set the success criteria: - - Ensure hospitalizations (H) are below 20,000 at all timepoints in 95% of simulated outcomes. + - Ensure hospitalizations (H) are below 10,000 at all timepoints in 95% of simulated outcomes. 4. Specify a new intervention: - - Find a new value for the parameter r_SV (vaccination rate) with the objective being closest to the lower bound (10,000 per day -- no change). - - Set the upper bound to 60,000 per day and the initial guess to 20,000. + - Find a new value for the parameter NPI_mult with the objective being closest to the upper bound of the range from 10–90%. + - Set the initial guess to 50%. 5. Simulate for 150 days and click **Run**. -By simulating the optimized intervention policy, you can see how the estimates of daily vaccinations affect peak hospitalizations: +By simulating the optimized intervention policy, you can see how the estimates of masking compliance affect peak hospitalizations: -- **Initial guess**: 20,000 vaccinations per day, which leads to peak hospitalizations of 31,208. -- **Optimization**: 47,429 vaccinations per day, which leads to peak hospitalizations of 15,569. +- **Initial guess**: NPI_mult of 50%, which leads to peak hospitalizations of 9,602. +- **Optimization**: NPI_mult of 20.7%, which leads to peak hospitalizations of 4,071. -### Stratify models +![](../img/get-started/example-interventions-optimized.png) -In Terarium, you can leverage an error-proof approach to stratify your model along dimensions such as age, sex, and location. +## Compare datasets -??? list "Stratify the SEIRHD model by age" +Finally, you can take the results of your simulations, interventions, and optimizations and compare them to see which works best at reducing hospitalizations. The Compare datasets operator lets you compare scenarios based on the various simulation results you've generated. - 1. Hover over the output of the Edit model operator, click link :octicons-plus-24:{ title="Link" aria-labelledBy="link-icon-label" } > **Stratify model**, and then click **Edit**. - 2. Name the strata *Age*. - 3. Select to stratify the *S*, *E*, *I*, *R*, *H*, and *D* state variables and the *b* (transmission) parameter. - 5. List the labels for each strata group: - - ``` text - Young, Old - ``` +![](../img/get-started/example-compare-settings.png) - 6. Choose the allowed transitions and interactions between strata: +??? list "Optimize the intervention policy to find how effective masking needs to be to reduce hospitalizations" - - Select **Allow existing interactions between strata** to allow young and old people to interact with and infect each other. - - Clear **Create new transitions between strata** to prevent young people from turning into old people and vice versa. + 1. Pipe the datasets you created from simulating your different intervention policies into a Compare datasets operator and click **Open**. + 2. Select Compare scenarios, choose the baseline dataset, and click **Run**. + 3. In the Output settings, select H (hospitalizations) to plot the variable over time for each of the datasets. - 7. Click **Stratify**. +You can see the effectiveness of the different intervention policies on a single plot. Starting masking at day 63 is most effective at reducing hospitalizations. -Looking at the stratified model graph and state variables, you can see that Susceptible is stratified by young and old groups that can interact with each other. +![](../img/get-started/example-compare-datasets.png) ## What's next? -You've completed the sample SEIRHD model workflow! You now have the tools you need to start uploading, transforming, and simulating models and model resources in Terarium. \ No newline at end of file +We've completed the sample SEIRHD model workflow! You now have the tools you need to start uploading, transforming, and simulating models and model resources in Terarium. \ No newline at end of file diff --git a/docs/get-started/using-terarium.md b/docs/get-started/using-terarium.md index f64f0ea..9581c1c 100644 --- a/docs/get-started/using-terarium.md +++ b/docs/get-started/using-terarium.md @@ -126,7 +126,7 @@ Terarium's operators support various ways for you to configure complex scientifi ??? list "Use a Terarium operator" 1. Make sure you've connected all the required inputs. - 1. Click **Open** or **Edit** on the operator node. + 1. Click **Open** on the operator node. 2. Switch to the Wizard or Notebook view depending on your preference. ???+ note diff --git a/docs/img/get-started/example-compare-datasets.png b/docs/img/get-started/example-compare-datasets.png new file mode 100644 index 0000000..04a47f8 Binary files /dev/null and b/docs/img/get-started/example-compare-datasets.png differ diff --git a/docs/img/get-started/example-compare-settings.png b/docs/img/get-started/example-compare-settings.png new file mode 100644 index 0000000..0996f5f Binary files /dev/null and b/docs/img/get-started/example-compare-settings.png differ diff --git a/docs/img/get-started/example-compare-wizard.png b/docs/img/get-started/example-compare-wizard.png new file mode 100644 index 0000000..15f2ffb Binary files /dev/null and b/docs/img/get-started/example-compare-wizard.png differ diff --git a/docs/img/get-started/example-compare-workflow.png b/docs/img/get-started/example-compare-workflow.png new file mode 100644 index 0000000..9ae97f9 Binary files /dev/null and b/docs/img/get-started/example-compare-workflow.png differ diff --git a/docs/img/get-started/example-dataset-transform-notebook.png b/docs/img/get-started/example-dataset-transform-notebook.png new file mode 100644 index 0000000..cfac310 Binary files /dev/null and b/docs/img/get-started/example-dataset-transform-notebook.png differ diff --git a/docs/img/get-started/example-dataset-transform-workflow.png b/docs/img/get-started/example-dataset-transform-workflow.png new file mode 100644 index 0000000..4a2558e Binary files /dev/null and b/docs/img/get-started/example-dataset-transform-workflow.png differ diff --git a/docs/img/get-started/example-dataset.png b/docs/img/get-started/example-dataset.png deleted file mode 100644 index a7305ab..0000000 Binary files a/docs/img/get-started/example-dataset.png and /dev/null differ diff --git a/docs/img/get-started/example-home.png b/docs/img/get-started/example-home.png deleted file mode 100644 index 920eeae..0000000 Binary files a/docs/img/get-started/example-home.png and /dev/null differ diff --git a/docs/img/get-started/example-interventions-dynamic.png b/docs/img/get-started/example-interventions-dynamic.png new file mode 100644 index 0000000..704cd04 Binary files /dev/null and b/docs/img/get-started/example-interventions-dynamic.png differ diff --git a/docs/img/get-started/example-interventions-optimize-settings.png b/docs/img/get-started/example-interventions-optimize-settings.png new file mode 100644 index 0000000..5443b4b Binary files /dev/null and b/docs/img/get-started/example-interventions-optimize-settings.png differ diff --git a/docs/img/get-started/example-interventions-optimized.png b/docs/img/get-started/example-interventions-optimized.png new file mode 100644 index 0000000..7607a83 Binary files /dev/null and b/docs/img/get-started/example-interventions-optimized.png differ diff --git a/docs/img/get-started/example-interventions-static.png b/docs/img/get-started/example-interventions-static.png new file mode 100644 index 0000000..535dcea Binary files /dev/null and b/docs/img/get-started/example-interventions-static.png differ diff --git a/docs/img/get-started/example-interventions.png b/docs/img/get-started/example-interventions.png new file mode 100644 index 0000000..e51ed94 Binary files /dev/null and b/docs/img/get-started/example-interventions.png differ diff --git a/docs/img/get-started/example-model-calibrate-notebook.png b/docs/img/get-started/example-model-calibrate-notebook.png new file mode 100644 index 0000000..29cdf2e Binary files /dev/null and b/docs/img/get-started/example-model-calibrate-notebook.png differ diff --git a/docs/img/get-started/example-model-calibrate-workflow.png b/docs/img/get-started/example-model-calibrate-workflow.png new file mode 100644 index 0000000..47cc765 Binary files /dev/null and b/docs/img/get-started/example-model-calibrate-workflow.png differ diff --git a/docs/img/get-started/example-model-edit-notebook.png b/docs/img/get-started/example-model-edit-notebook.png new file mode 100644 index 0000000..825d776 Binary files /dev/null and b/docs/img/get-started/example-model-edit-notebook.png differ diff --git a/docs/img/get-started/example-model-edit-workflow.png b/docs/img/get-started/example-model-edit-workflow.png new file mode 100644 index 0000000..c93a384 Binary files /dev/null and b/docs/img/get-started/example-model-edit-workflow.png differ diff --git a/docs/img/get-started/example-model-from-equations-notebook.png b/docs/img/get-started/example-model-from-equations-notebook.png new file mode 100644 index 0000000..57a5be1 Binary files /dev/null and b/docs/img/get-started/example-model-from-equations-notebook.png differ diff --git a/docs/img/get-started/example-model-from-equations-workflow.png b/docs/img/get-started/example-model-from-equations-workflow.png new file mode 100644 index 0000000..9d730ff Binary files /dev/null and b/docs/img/get-started/example-model-from-equations-workflow.png differ diff --git a/docs/img/get-started/example-model-stratify-wizard.png b/docs/img/get-started/example-model-stratify-wizard.png new file mode 100644 index 0000000..c9345cb Binary files /dev/null and b/docs/img/get-started/example-model-stratify-wizard.png differ diff --git a/docs/img/get-started/example-model-stratify-workflow.png b/docs/img/get-started/example-model-stratify-workflow.png new file mode 100644 index 0000000..b74415c Binary files /dev/null and b/docs/img/get-started/example-model-stratify-workflow.png differ diff --git a/docs/img/get-started/example-project-create.png b/docs/img/get-started/example-project-create.png deleted file mode 100644 index 88110f6..0000000 Binary files a/docs/img/get-started/example-project-create.png and /dev/null differ diff --git a/docs/img/get-started/example-sensitivity-wizard.png b/docs/img/get-started/example-sensitivity-wizard.png new file mode 100644 index 0000000..bb0890b Binary files /dev/null and b/docs/img/get-started/example-sensitivity-wizard.png differ diff --git a/docs/img/get-started/example-sensitivity-workflow.png b/docs/img/get-started/example-sensitivity-workflow.png new file mode 100644 index 0000000..44bdb4a Binary files /dev/null and b/docs/img/get-started/example-sensitivity-workflow.png differ diff --git a/docs/img/get-started/example-upload-dataset.png b/docs/img/get-started/example-upload-dataset.png index c5f75f4..f805d71 100644 Binary files a/docs/img/get-started/example-upload-dataset.png and b/docs/img/get-started/example-upload-dataset.png differ diff --git a/docs/img/get-started/example-workflow-new.png b/docs/img/get-started/example-workflow-new.png deleted file mode 100644 index 91e110c..0000000 Binary files a/docs/img/get-started/example-workflow-new.png and /dev/null differ diff --git a/docs/img/get-started/resources.png b/docs/img/get-started/resources.png new file mode 100644 index 0000000..c399ccb Binary files /dev/null and b/docs/img/get-started/resources.png differ diff --git a/docs/img/models/model-optimize-operator.png b/docs/img/models/model-optimize-operator.png deleted file mode 100644 index ad2aeeb..0000000 Binary files a/docs/img/models/model-optimize-operator.png and /dev/null differ diff --git a/docs/modeling/create-model-from-equations.md b/docs/modeling/create-model-from-equations.md index 92f5bd8..c46ddf1 100644 --- a/docs/modeling/create-model-from-equations.md +++ b/docs/modeling/create-model-from-equations.md @@ -71,7 +71,7 @@ In some cases you may want to change the LaTeX either to correct or and new deta ??? list "Select equations from a document" - 1. In the workflow, make sure the document is connected to the operator input and then click **Edit**. + 1. In the workflow, make sure the document is connected to the operator input and then click **Open**. 2. In the Input panel, review and select the equations you want to include in the model. ??? list "Edit equations extracted from a document" diff --git a/docs/modeling/edit-model.md b/docs/modeling/edit-model.md index 73a94c6..8fd025f 100644 --- a/docs/modeling/edit-model.md +++ b/docs/modeling/edit-model.md @@ -79,7 +79,7 @@ Generated code appears below your prompts, where you can preview, edit, and run ??? list "Open the Edit model code notebook" - - Make sure you've connected a model or model configuration to the Edit model operator and then click **Edit**. + - Make sure you've connected a model or model configuration to the Edit model operator and then click **Open**. ### Use the AI assistant to edit a model diff --git a/docs/modeling/index.md b/docs/modeling/index.md index 4cf8869..a13ea33 100644 --- a/docs/modeling/index.md +++ b/docs/modeling/index.md @@ -51,7 +51,7 @@ In a workflow graph, a model resource shows its underlying diagram or equations. ??? list "Copy a model" 1. Add the Model operator to a workflow graph and connect it to an Edit model operator. - 2. Click **Edit** on the Edit model operator. + 2. Click **Open** on the Edit model operator. 3. Click **Save for re-use**, enter a name for the copy, and click **Save**. ### What can I do with a model resource? diff --git a/docs/modeling/stratify-model.md b/docs/modeling/stratify-model.md index 804e2ad..df6bd31 100644 --- a/docs/modeling/stratify-model.md +++ b/docs/modeling/stratify-model.md @@ -51,7 +51,7 @@ The Stratify model operator adapts to your level of coding experience. You can s ??? list "Open the Stratify model operator" - - Make sure you've connected a model or model configuration to the Stratify model operator and then click **Edit**. + - Make sure you've connected a model or model configuration to the Stratify model operator and then click **Open**. ### Use the wizard to stratify a model diff --git a/docs/simulation/sensitivity-analysis.md b/docs/simulation/sensitivity-analysis.md index 766c979..70a610f 100644 --- a/docs/simulation/sensitivity-analysis.md +++ b/docs/simulation/sensitivity-analysis.md @@ -57,7 +57,7 @@ Once you've built your workflow, you can edit the Simulate run and output settin ??? list "Conduct a sensitivity analysis" - 1. On the Simulate operator, click **Edit**. + 1. On the Simulate operator, click **Open**. 2. Select the Simulate [run settings](../simulation/simulate-model.md). 3. Click **Run** to start the simulation. 4. Preview the sensitivity analysis results. To add variables or parameters, use the **Output** settings. diff --git a/docs/simulation/simulate-ensemble.md b/docs/simulation/simulate-ensemble.md index 8b39fe8..9e740c6 100644 --- a/docs/simulation/simulate-ensemble.md +++ b/docs/simulation/simulate-ensemble.md @@ -29,9 +29,9 @@ More info coming soon. 1. [Configure two or more models](../config-and-intervention/configure-model.md). 2. Right-click anywhere on the workflow graph and select **Simulation** > **Simulate ensemble**. 3. Connect the Configure model outputs to the Simulate ensemble inputs. - 4. Click **Edit**. - 5. Enter custom weights for the model configurations or click **Set weights to be equal** to treat them as equally important. - 6. Create a mapping between the state variables of the model configurations. + 4. Click **Open**. + 5. Create a mapping between the state variables of the model configurations. + 6. Enter custom weights for the model configurations to specify your confidence in them. 7. Set the time span: - **Start** and **End Step** - **Number of samples** diff --git a/docs/workflows/index.md b/docs/workflows/index.md index 1028408..c45b3f3 100644 --- a/docs/workflows/index.md +++ b/docs/workflows/index.md @@ -356,7 +356,7 @@ Resources and operators in the workflow graph summarize the data and inputs/outp Perform one of the following actions: - - Click **Open** or **Edit**. + - Click **Open**. - Click Menu :fontawesome-solid-ellipsis-vertical:{ title="Menu" aria-labelledBy="menu-icon-label" } > :fontawesome-solid-up-right-from-square:{ aria-hidden="true" } **Open in new window**. ??? list "Duplicate a resource or operator"