Skip to content

Commit

Permalink
Processing container details
Browse files Browse the repository at this point in the history
  • Loading branch information
jangevaare committed Apr 26, 2024
1 parent bff8380 commit 17ee75f
Showing 1 changed file with 40 additions and 1 deletion.
41 changes: 40 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,48 @@ ArcGIS Online item url. Changes with addition/removal of features to dataset req
**Example**: `https://services6.arcgis.com/ghjer345tert/arcgis/rest/services/PROD_PHU_Base_Aggregated/FeatureServer/0/query`

**2. `output`**
The filename where the output will be written.
The filename where the output in CSV format will be written.
**Example**: `wsi.csv`

## Processing Container
This container takes the CSV output from the retrieval container, and performs standardization and trend analysis on the data. There are disease target-specific outputs at both the sewershed and region-level. Sewershed weighting is required in order to perform region-level analyses. The container uses the following arguments:

**1. `input`**
CSV file containing at minimum columns: sampleDate, siteName, mN1, mN2, mFluA, mFluB, and mBiomarker. Intention is to use the file that is output from the retrieval container for this.
**Example**: `wsi.csv`

**2. `weights`**
CSV file with columns: Site, and Weight. The site column corresponds to siteName values in the `input`. Weights represents factor used for combing site-specific trends into a single regional trend. Weights are decimal numbers and should sum to 1. The weights may be set to be equal, or correspond to population weighting, sampling frequency, or any other user-determined criteria
**Example**: `weights.csv`

**3. `patch`**
Optional CSV file with columns: Date, Site, and one or more of mN1, mN2, mFluA, mFluB, mBiomarker. Values in the patch file will add or overide any existing values in the primary input file. Useful, for adding historical data not present in WSI, or fixing erroneous data.
**Example**: `patch.csv`

**4. `output_region_covid`**
Optional output location for CSV file containing regional summary for SARS-CoV-2. No output will be generated if left blank.
**Example**: `output_region_covid.csv`

**5. `output_region_flu_a`**
Optional output location for CSV file containing regional summary for Influenza A. No output will be generated if left blank.
**Example**: `output_region_flu_a.csv`

**6. `output_region_flu_b`**
Optional output location for CSV file containing regional summary for Influenza B. No output will be generated if left blank.
**Example**: `output_region_flu_b.csv`

**7. `output_covid`**
Optional output location for CSV file containing site-specific SARS-CoV-2 data. No output will be generated if left blank.
**Example**: `output_covid.csv`

**8. `output_flu_a`**
Optional output location for CSV file containing site-specific Influenza A data. No output will be generated if left blank.
**Example**: `output_flu_a.csv`

**9. `output_flu_b`**
Optional output location for CSV file containing site-specific Influenza B data. No output will be generated if left blank.
**Example**: `output_flu_b.csv`

## Pipeline Orchestration
This data pipeline can be orchestrated by a variety of tools that support containerized components, but has been developed and tested with [Kubeflow Pipelines](https://www.kubeflow.org/), which is based on [Argo Workflows](https://argoproj.github.io/argo-workflows/).

Expand Down

0 comments on commit 17ee75f

Please sign in to comment.