Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify multinomial #7

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Clarify multinomial #7

wants to merge 1 commit into from

Conversation

teganmaharaj
Copy link
Collaborator

No description provided.

@teganmaharaj teganmaharaj requested a review from mweiss17 April 28, 2020 18:50
Copy link
Contributor

@mnqu mnqu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some comments to the evaluation metrics.

- (*N_e*) binary array 0/1 for each encounter, where the target is 1 if Alice was infected by that Bob encounter and 0 if they are were not
- Binary variable 0/1 for the special "null" encounter, which indicates whether Alice has been infected by any of the *N_e* encounters; 0 if not, 1 if so

* Regression of personal infectiousness: (14) array of floats of Alice's infectiousness for each of the past 14 days

**Metrics:**

* **P**: Precision is of the top 1% of highest-risk people, what % are correctly identified as being infected
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my understanding, "Precision is of the top 1% of highest-risk people identified by the ML model, what % are truly infected". I am not sure whether they have the same meaning. Similar for P-A and P-UA.

* Regression of personal infectiousness: (14) array of floats of Alice's infectiousness for each of the past 14 days

**Metrics:**

* **P**: Precision is of the top 1% of highest-risk people, what % are correctly identified as being infected
* **P-U**: Precision-Untested is of the top 1% of highest-risk people, excluding those who have a positive test, what % are correctly identified as being infected
* **P-A**: Precision-Asymptomatic is of the top 1% of highest-risk people, excluding those who have a positive test and those who have symptoms, what % are correctly identified as being infected
* **P-UA**: Precision-Asymptomatic is of the top 1% of highest-risk people, excluding those who have a positive test and those who have symptoms, what % are correctly identified as being infected
* **R**: Recall is what % of those infected are correctly identified as being infected
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recall here is also recall at top 1%, i.e., "Recall is what % of those infected are identified as top 1% highest-risk people by the ML model". Similar for R-U and R-UA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants