Skip to content

Commit

Permalink
Merge pull request #45 from DARPA-ASKEM/simulate
Browse files Browse the repository at this point in the history
Update Simulate and add Simulate intervention policy
  • Loading branch information
mecrouch authored Jan 22, 2025
2 parents 3859f7c + 563440b commit 67f7ba6
Show file tree
Hide file tree
Showing 15 changed files with 238 additions and 39 deletions.
2 changes: 1 addition & 1 deletion docs/config-and-intervention/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,6 @@ With Terarium's configuration and intervention operators, you can:

---

More info coming soon.
Assess the impact of your intervention on a variable of interest.

</div>
54 changes: 53 additions & 1 deletion docs/config-and-intervention/simulate-intervention-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,56 @@ title: Simulate an intervention policy

# Simulate an intervention policy

More info coming soon.
You can simulate an intervention policy to assess the impact of your intervention on a variable of interest. For example, you can see how increasing vaccination rate affects hospitalizations.

## Build an intervention policy simulation workflow

You can build a workflow to simulate an intervention policy manually.

<figure markdown>![](../img/config-and-intervention/intervention-simulate/workflow.png)<figcaption markdown></figcaption></figure>

Before you get started, you'll need a:

- [Model](../modeling/index.md).
- [Model configuration](../config-and-intervention/configure-model.md).

Once you've set up and run the workflow, you'll have a set of intervention policy simulation results, which you can use as a dataset.

??? list "Build an intervention policy simulation workflow"

1. Add your model to a workflow, hover over its output, and click <span class="sr-only" id="link-icon-label">Link</span> :octicons-plus-24:{ title="Link" aria-labelledby="link-icon-label" } > **Configure model**. Click **Open** on the Configure model operator to [select or create a new configuration](configure-model.md).
2. Hover over the Model output again and click <span class="sr-only" id="link-icon-label">Link</span> :octicons-plus-24:{ title="Link" aria-labelledby="link-icon-label" } > **Create intervention policy**. Click **Open** on the Create intervention policy operator to [select or create a new intervention policy](create-intervention-policy.md).
3. Hover over the Create intervention policy output and click <span class="sr-only" id="link-icon-label">Link</span> :octicons-plus-24:{ title="Link" aria-labelledby="link-icon-label" } > **Simulate**.
4. Click the output of the Configure model operator and connect it to the Simulate model configuration input.
5. Connect the model configuration output to the simulate input.

## Run a simulation

Once you've built your workflow, you can edit the Simulate run and output settings to generate the intervention policy simulation.

??? list "Run an intervention policy simulation"

1. Click **Open** on the Simulate operator.
2. Select the Simulate [run settings](../simulation/simulate-model.md).
3. Click **Run** to start the simulation.
4. Preview the intervention policy results. To add variables or parameters, use the **Output** settings.

## View and save simulation results with interventions

[Simulation results](../simulation/simulate-model.md) are summarized on the operator in the workflow and in the operator details. When you simulate with an intervention you can see the impacts of the intervention. There are visual representations of:

- The parameter you changed in your intervention.

<figure markdown>![](../img/config-and-intervention/intervention-simulate/interventions-over-time.png)<figcaption markdown>At day 2, vaccination rate increases from 10,000 people per day to 25,000 people per day.</figcaption></figure>

- Simulation results with the intervention shown in green and results for the baseline (no intervention) in gray.

<figure markdown>![](../img/config-and-intervention/intervention-simulate/hospitalized.png)<figcaption markdown>Increasing vaccination rate from 10,000 people per day to 25,000 people per day decreases hospitalization rate from ~13,000 to ~10,000.</figcaption></figure>

??? list "Save the simulation results as a new dataset"

- On the Output panel, click **Save for re-use**.

???+ tip

Using a descriptive naming convention for your datasets will help you keep track of them when you want to compare multiple scenarios.
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ The Validate configuration operator takes a model configuration and an optional

- Do one of the following actions:

- On a operator that outputs a model configuration, click <span class="sr-only" id="link-icon-label">Link</span> :octicons-plus-24:{ title="Link" aria-labelledby="link-icon-label" } > **Validate configuration**.
- On an operator that outputs a model configuration, click <span class="sr-only" id="link-icon-label">Link</span> :octicons-plus-24:{ title="Link" aria-labelledby="link-icon-label" } > **Validate configuration**.
- Right-click anywhere on the workflow graph, select **Config & Intervention** > **Validate configuration**, and then connect a model to the Validate configuration input.

## Validate a model configuration
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs/img/models/simulate-pyciemss-operator.png
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
6 changes: 3 additions & 3 deletions docs/simulation/calibrate-ensemble.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Once you've completed the calibration, the thumbnail preview shows the calibrate

- Do one of the following actions:

- On a operator that outputs a model configuration, click <span class="sr-only" id="link-icon-label">Link</span> :octicons-plus-24:{ title="Link" aria-labelledby="link-icon-label" } > **Calibrate ensemble**.
- On an operator that outputs a model configuration, click <span class="sr-only" id="link-icon-label">Link</span> :octicons-plus-24:{ title="Link" aria-labelledby="link-icon-label" } > **Calibrate ensemble**.
- Right-click anywhere on the workflow graph, select **Simulation** > **Calibrate ensemble**, and then connect two or more model configurations and a dataset to the Calibrate ensemble input.

## Calibrate ensemble
Expand All @@ -44,7 +44,7 @@ The Calibrate ensemble operator allows you to define how to:

- [Map your model configurations and dataset](#map-dataset-columns-and-model-variables).
- [Assign your level of confidence in each model](#assign-model-weights).
- [Choose how to run the validation](#configure-the-run-settings).
- [Choose how to run the calibration](#configure-the-run-settings).

??? list "Open a Calibrate ensemble operator"

Expand Down Expand Up @@ -86,7 +86,7 @@ By tuning weights, you can better match the ensemble's output to your expectatio

### Configure the run settings

The Calibrate ensemble run settings allow you to fine-tune the time frame, solver behavior, and inference process. By adjusting these parameters, you can balance performance and precision.
The Calibrate ensemble run settings allow you to fine-tune the time frame, solver behavior, and inference process. By adjusting these settings, you can balance performance and precision.

![](../img/simulation/calibrate-ensemble/other-settings.png)

Expand Down
7 changes: 3 additions & 4 deletions docs/simulation/sensitivity-analysis.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,8 @@ Once you've built your workflow, you can edit the Simulate run and output settin
??? list "Conduct a sensitivity analysis"

1. On the Simulate operator, click **Edit**.
2. Select the Simulate [run settings](../simulation/simulate-model.md)
3. Run the simulation.
2. Select the Simulate [run settings](../simulation/simulate-model.md).
3. Click **Run** to start the simulation.
4. Preview the sensitivity analysis results. To add variables or parameters, use the **Output** settings.

## Understand sensitivity analysis results
Expand All @@ -68,8 +68,7 @@ The results of each analysis are represented as a sensitivity analysis graph and

In the sensitivity analysis graph, color coding represents variations in the output variable (such as hospitalizations or case counts) across different parameter values or combinations. Cool colors represent lower values of the output variable, while warm colors represent higher values.


<figure markdown>![](../img/simulation/sensitivity/graph.png)<figcaption markdown>Sensitivity analysis graph for males infected with Mpox.</figcaption></figure>
<figure markdown>![](../img/simulation/sensitivity/graph.png)<figcaption markdown>Sensitivity analysis graph for men infected with Mpox.</figcaption></figure>

In the parameter comparisons:

Expand Down
Loading

0 comments on commit 67f7ba6

Please sign in to comment.