Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify multinomial #7

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Clarify multinomial #7

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,21 +76,23 @@ Inputs and targets are described for 1 data example (1 person), from the point o
- `day` [0-13] day of the encounter, 0 being today and 13 being 14 days ago

**Targets:**
* Classification of infection status:
- Binary variable 0/1 of whether Alice is infected
* Classification of infection status: *N_e* + 1 array which is a multinomial of all encounters + a special "null" encounter, structured as follows:
- (*N_e*) binary array 0/1 for each encounter, where the target is 1 if Alice was infected by that Bob encounter and 0 if they are were not
- Binary variable 0/1 for the special "null" encounter, which indicates whether Alice has been infected by any of the *N_e* encounters; 0 if not, 1 if so

* Regression of personal infectiousness: (14) array of floats of Alice's infectiousness for each of the past 14 days

**Metrics:**

* **P**: Precision is of the top 1% of highest-risk people, what % are correctly identified as being infected
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my understanding, "Precision is of the top 1% of highest-risk people identified by the ML model, what % are truly infected". I am not sure whether they have the same meaning. Similar for P-A and P-UA.

* **P-U**: Precision-Untested is of the top 1% of highest-risk people, excluding those who have a positive test, what % are correctly identified as being infected
* **P-A**: Precision-Asymptomatic is of the top 1% of highest-risk people, excluding those who have a positive test and those who have symptoms, what % are correctly identified as being infected
* **P-UA**: Precision-Asymptomatic is of the top 1% of highest-risk people, excluding those who have a positive test and those who have symptoms, what % are correctly identified as being infected
* **R**: Recall is what % of those infected are correctly identified as being infected
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recall here is also recall at top 1%, i.e., "Recall is what % of those infected are identified as top 1% highest-risk people by the ML model". Similar for R-U and R-UA.

* **R-U**: Recall-Untested is what % of those infected are correctly identified as being infected, among people who have not been tested
* **R-A** Recall-Asymptomatic is what % of those infected are correctly identified as being infected, among people who are asymptomatic
* **MSE**: is Mean Squared Error, between the target risk and the prediction for the infectiousness of each person. (Possibly N/A for non-ML methods)
* **MRR**: is Mean Reciprocal Rank TODO
* **R-UA**: Recall-Asymptomatic is what % of those infected are correctly identified as being infected, excluding those who have symptoms and those who have a positive test
* **EIE**: Expected Infectiousness Error: difference between the expected infectiousness target infectiousnes
* **MSE**: is Mean Squared Error, between the target infectiousness and the prediction for the infectiousness of each Alice. (Possibly N/A for non-ML methods).
* **MRR**: is Mean Reciprocal Rank, where the predictions of being infected in each encounter (including the special "null" encounter) are ranked. The target is to put the infection encounter top in the ranking, or put the null encounter top if none of the encounters resulted in infection. (Possibly N/A for non-ML methods).


## Results
Expand Down