feat: flag and filter credible sets #879
Conversation
| @@ -45,6 +45,8 @@ def __init__( | |||
| .annotate_study_type(study_index) # Add study type to study locus | |||
| .qc_redundant_top_hits_from_PICS() # Flagging top hits from studies with PICS summary statistics | |||
| .qc_explained_by_SuSiE() # Flagging credible sets in regions explained by SuSiE | |||
| # Flagging credible sets with PIP > 1 or PIP < 0.99 | |||
| .qc_abnormal_pips(sum_pips_lower_threshold=0.99,sum_pips_upper_threshold=1) | |||
There was a problem hiding this comment.
I guess that we are not dropping credible sets due to the lower bound, only the upper?
There was a problem hiding this comment.
Both below lower or above the upper bound are dropped, in theory - in practice I'm not so sure
|
Do you have an idea of how many CS have this issue? |
When I last looked at gwas catalog susie I didn't find any credible sets with the issue. I guess @addramir would know more about where this has been a problem in the past |
|
I would be happy if 0 CSs are having this issue. If not - we have a problem. |
…unt for floating point errors
|
Implemented @ireneisdoomed's suggestion and after cross-checking vs gwas catalog susie I tweaked the threshold to be 1.0001 rather than 1 as there were some sums creeping through likely due to floating point error. |
src/gentropy/dataset/study_locus.py
Outdated
| # Flagging loci with failed studies: | ||
| .withColumn( | ||
| "qualityControls", | ||
| self.update_quality_flag( | ||
| qc_select_expression, | ||
| f.col("sumPosteriorProbability").isNotNull(), | ||
| f.col("pipOutOfRange") == "outside", |
There was a problem hiding this comment.
nit: this could be a boolean
✨ Context
This PR adds a study locus QC flag for when the sum of posterior inclusion probabilities are not in the expected range [0.99,1] addresses issues/#3566
🛠 What does this PR implement
🙈 Missing
🚦 Before submitting
devbranch?make test)?poetry run pre-commit run --all-files)?