Skip to content

Commit

Permalink
Merge pull request #133 from USEPA/package-updates
Browse files Browse the repository at this point in the history 
remove test files
  • Loading branch information
@cristinamullin
cristinamullin authored Jan 3, 2024
2 parents 8a5649f + 91b3848 commit 6edb886
Show file tree
Hide file tree
Showing 5 changed files with 45 additions and 435 deletions.
33 changes: 10 additions & 23 deletions R/utils_flag_functions.R
View file
Original file line number Diff line number Diff line change
Expand Up @@ -110,9 +110,8 @@ applyFlags <- function(in_table, orgs) {
if ("ProjectFileUrl" %in% names(out)) {
out <- TADA::TADA_FindQAPPDoc(out, clean = FALSE)
}
# Dataset includes depth profile data - no function for this? How is this one
# supposed to work?
out <- out
# Dataset includes depth profile data - no function for this yet
# out <- out

# Aggregated continuous data
out <- TADA::TADA_FindContinuousData(out, clean = FALSE)
Expand All @@ -123,26 +122,14 @@ applyFlags <- function(in_table, orgs) {
# Below WQX Lower Threshold
out <- TADA::TADA_FlagBelowThreshold(out, clean = FALSE)

# Convert depth height units - THIS ONE ONLY GETS RUN WHEN USER RUNS THE CLEANING
# FILTER AFTER MAKING ALL DECISIONS, AND SUMMARY COUNTS BASED ON UNIQUE UNITS IN
# DEPTH HEIGHT COLUMNS
# CM - this is already done via TADA autoclean 9/12/23, simply add note somewhere for users?
# out <-
# TADA::TADA_ConvertDepthUnits(out, unit = 'ft', transform = TRUE) # input$depthunit is dummy variable that would connect to the drop down

# Convert time zones - no flag function to run beforehand. This one might be
# tricky to implement - acts on ActivityStartTime.Time?
# CM - this is already done via dataRetrieval 9/12/23, simply add note somewhere for users?
# out = out

# Invalid coordinates - not included in mock up page?
# out <-
# TADA::TADA_FlagCoordinates(
# out,
# clean_outsideUSA = "no",
# clean_imprecise = FALSE,
# errorsonly = FALSE
# )
# Invalid coordinates
out <-
TADA::TADA_FlagCoordinates(
out,
clean_outsideUSA = "no",
clean_imprecise = FALSE,
flaggedonly = FALSE
)


return(out)
Expand Down
1 change: 1 addition & 0 deletions inst/flag_prompts.csv
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,4 @@ Order,Level,Prompt,flagType
14,Optional,"Metadata indicates result(s) are aggregate, high-frequency continuous sensor results, not discrete samples","Metadata indicates result(s) are aggregate, high-frequency continuous sensor results, not discrete samples"
15,Optional,"Result value(s) outside of the national upper range for a given characteristic, possibly indicating non-sensical value(s)","Result value(s) outside of the national upper range for a given characteristic, possibly indicating non-sensical value(s)"
16,Optional,"Result value(s) outside of the national lower range for a given characteristic, possibly indicating non-sensical value(s)","Result value(s) outside of the national lower range for a given characteristic, possibly indicating non-sensical value(s)"
17,Optional,Coordinates are outside of the United States,TADA.InvalidCoordinates.Flag
64 changes: 34 additions & 30 deletions inst/flag_tests.csv
View file
Original file line number Diff line number Diff line change
@@ -1,37 +1,41 @@
flagType,columnName,flagValue,keep,remove
"Result value(s) outside of the national upper range for a given characteristic, possibly indicating non-sensical value(s)",TADA.ResultValueAboveUpperThreshold.Flag,Y,0,1
"Result value(s) outside of the national upper range for a given characteristic, possibly indicating non-sensical value(s)",TADA.ResultValueAboveUpperThreshold.Flag,Not Reviewed,0,0
"QAPP approval column is ""N"" (indicating not approved) or NA",QAPPApprovedIndicator,NA,0,1
"QAPP approval column is ""N"" (indicating not approved) or NA",QAPPApprovedIndicator,Y,0,0
"QAPP approval column is ""N"" (indicating not approved) or NA",QAPPApprovedIndicator,N,0,1
Sample media is not water,TADA.ActivityMediaName,WATER,0,1
"Measurement activity type code indicates it is a QC replicate, duplicate, or blank",TADA.ActivityType.Flag,Non_QC,0,0
"Measurement activity type code indicates it is a QC replicate, duplicate, or blank",TADA.ActivityType.Flag,QC_replicate,0,1
"Measurement activity type code indicates it is a QC replicate, duplicate, or blank",TADA.ActivityType.Flag,QC_duplicate,0,1
"Measurement activity type code indicates it is a QC replicate, duplicate, or blank",TADA.ActivityType.Flag,QC_calibration,0,1
"Measurement activity type code indicates it is a QC replicate, duplicate, or blank",TADA.ActivityType.Flag,QC_blank,0,1
"Measurement activity type code indicates it is a QC replicate, duplicate, or blank",TADA.ActivityType.Flag,QC_other,0,1
"Metadata indicates result(s) are aggregate, high-frequency continuous sensor results, not discrete samples",TADA.AggregatedContinuousData.Flag,Discrete,0,0
"Metadata indicates result(s) are aggregate, high-frequency continuous sensor results, not discrete samples",TADA.AggregatedContinuousData.Flag,Continuous,0,1
Uncommon analytical methods,TADA.AnalyticalMethod.Flag,Not Reviewed,0,0
Uncommon analytical methods,TADA.AnalyticalMethod.Flag,Invalid,0,1
Uncommon analytical methods,TADA.AnalyticalMethod.Flag,Valid,0,0
"Result value(s) outside of the national lower range for a given characteristic, possibly indicating non-sensical value(s)",TADA.ResultValueBelowLowerThreshold.Flag,Y,0,1
"Result value(s) outside of the national lower range for a given characteristic, possibly indicating non-sensical value(s)",TADA.ResultValueBelowLowerThreshold.Flag,Not Reviewed,0,0
Conflict between detection condition text and detection limit type or detection limit type is not in WQX domain tables (likely USGS/NWIS-specific) ,TADA.CensoredData.Flag,Detection condition or detection limit is not documented in TADA reference tables.,0,1
Conflict between detection condition text and detection limit type or detection limit type is not in WQX domain tables (likely USGS/NWIS-specific) ,TADA.CensoredData.Flag,Detection condition is missing and required for censored data ID.,0,1
Conflict between detection condition text and detection limit type or detection limit type is not in WQX domain tables (likely USGS/NWIS-specific) ,TADA.CensoredData.Flag,Conflict between Condition and Limit,0,1
Conflict between detection condition text and detection limit type or detection limit type is not in WQX domain tables (likely USGS/NWIS-specific) ,TADA.CensoredData.Flag,Non-Detect,0,0
Conflict between detection condition text and detection limit type or detection limit type is not in WQX domain tables (likely USGS/NWIS-specific) ,TADA.CensoredData.Flag,Over-Detect,0,0
Conflict between detection condition text and detection limit type or detection limit type is not in WQX domain tables (likely USGS/NWIS-specific) ,TADA.CensoredData.Flag,Other Condition/Limit Populated,0,0
"Metadata indicates result(s) are aggregate, high-frequency continuous sensor results, not discrete samples",TADA.AggregatedContinuousData.Flag,Discrete,0,0
"Metadata indicates result(s) are aggregate, high-frequency continuous sensor results, not discrete samples",TADA.AggregatedContinuousData.Flag,Continuous,0,1
Sample media is not water,TADA.ActivityMediaName,WATER,1,1
Spatial Flags,TADA.InvalidCoordinates.Flag,LAT_OutsideUSA,0,1
Spatial Flags,TADA.InvalidCoordinates.Flag,LONG_OutsideUSA,0,1
Spatial Flags,TADA.InvalidCoordinates.Flag,Imprecise_lessthan3decimaldigits,0,0
Spatial Flags,TADA.InvalidCoordinates.Flag,OK,0,0
MeasureQualifierCode indicates result value is suspect,TADA.MeasureQualifierCode.Flag,NA - Not Available,0,0
MeasureQualifierCode indicates result value is suspect,TADA.MeasureQualifierCode.Flag,Non-Detect,0,0
MeasureQualifierCode indicates result value is suspect,TADA.MeasureQualifierCode.Flag,Pass,0,0
MeasureQualifierCode indicates result value is suspect,TADA.MeasureQualifierCode.Flag,Suspect,0,1
MeasureQualifierCode indicates result value is suspect,TADA.MeasureQualifierCode.Flag,Over-Detect,0,0
MeasureQualifierCode indicates result value is suspect,TADA.MeasureQualifierCode.Flag,Not Reviewed,0,0
Invalid speciation for associated characteristic,TADA.MethodSpeciation.Flag,Valid,0,0
Invalid speciation for associated characteristic,TADA.MethodSpeciation.Flag,Invalid,0,1
Invalid speciation for associated characteristic,TADA.MethodSpeciation.Flag,Not Reviewed,0,0
"QAPP approval column is ""N"" (indicating not approved) or NA",QAPPApprovedIndicator,NA,0,1
"QAPP approval column is ""N"" (indicating not approved) or NA",QAPPApprovedIndicator,Y,0,0
"QAPP approval column is ""N"" (indicating not approved) or NA",QAPPApprovedIndicator,N,0,1
Quality Assurance Project Plan URL is missing,TADA.QAPPDocAvailable,Y_ProjectFileUrlProvided,0,0
Quality Assurance Project Plan URL is missing,TADA.QAPPDocAvailable,N,0,1
Invalid characteristic and fraction combination,TADA.SampleFraction.Flag,Valid,0,0
Invalid characteristic and fraction combination,TADA.SampleFraction.Flag,Invalid,0,1
Invalid characteristic and fraction combination,TADA.SampleFraction.Flag,Not Reviewed,0,0
Spatial Flags,TADA.InvalidCoordinates.Flag,LAT_OutsideUSA,0,1
Spatial Flags,TADA.InvalidCoordinates.Flag,LONG_OutsideUSA,0,1
Spatial Flags,TADA.InvalidCoordinates.Flag,Imprecise_lessthan3decimaldigits,0,0
Spatial Flags,TADA.InvalidCoordinates.Flag,OK,0,0
Result value is not numeric or result value is NA and no detection limit value is provided,TADA.ResultMeasureValueDataTypes.Flag,Non-ASCII Character(s),0,1
Result value is not numeric or result value is NA and no detection limit value is provided,TADA.ResultMeasureValueDataTypes.Flag,Blank,0,1
Result value is not numeric or result value is NA and no detection limit value is provided,TADA.ResultMeasureValueDataTypes.Flag,Coerced to NA,0,1
Result value is not numeric or result value is NA and no detection limit value is provided,TADA.ResultMeasureValueDataTypes.Flag,Numeric,0,0
Result value is not numeric or result value is NA and no detection limit value is provided,TADA.ResultMeasureValueDataTypes.Flag,Less Than,0,0
Expand All @@ -41,22 +45,22 @@ Result value is not numeric or result value is NA and no detection limit value i
Result value is not numeric or result value is NA and no detection limit value is provided,TADA.ResultMeasureValueDataTypes.Flag,Comma-Separated Numeric,0,0
Result value is not numeric or result value is NA and no detection limit value is provided,TADA.ResultMeasureValueDataTypes.Flag,Result Value/Unit Copied from Detection Limit,0,0
Result value is not numeric or result value is NA and no detection limit value is provided,TADA.ResultMeasureValueDataTypes.Flag,Result Value/Unit Estimated from Detection Limit,0,0
Result value is not numeric or result value is NA and no detection limit value is provided,TADA.ResultMeasureValueDataTypes.Flag,NA - Not Available,0,1
Result value is not numeric or result value is NA and no detection limit value is provided,TADA.ResultMeasureValueDataTypes.Flag,Text,0,1
Metadata indicates duplicative uploads of the same results by multiple organizations,TADA.ResultSelectedMultipleOrgs,N,0,1
Invalid units for a given characteristic and media combination,TADA.ResultUnit.Flag,Valid,0,0
Invalid units for a given characteristic and media combination,TADA.ResultUnit.Flag,NonStandardized,0,0
Invalid units for a given characteristic and media combination,TADA.ResultUnit.Flag,Invalid,0,1
Invalid units for a given characteristic and media combination,TADA.ResultUnit.Flag,Not Reviewed,0,0
"Measurement activity type code indicates it is a QC replicate, duplicate, or blank",TADA.ActivityType.Flag,Non_QC,0,0
"Measurement activity type code indicates it is a QC replicate, duplicate, or blank",TADA.ActivityType.Flag,QC_replicate,0,1
"Measurement activity type code indicates it is a QC replicate, duplicate, or blank",TADA.ActivityType.Flag,QC_duplicate,0,1
"Measurement activity type code indicates it is a QC replicate, duplicate, or blank",TADA.ActivityType.Flag,QC_calibration,0,1
"Measurement activity type code indicates it is a QC replicate, duplicate, or blank",TADA.ActivityType.Flag,QC_blank,0,1
"Measurement activity type code indicates it is a QC replicate, duplicate, or blank",TADA.ActivityType.Flag,QC_other,0,1
"Result value(s) outside of the national upper range for a given characteristic, possibly indicating non-sensical value(s)",TADA.ResultValueAboveUpperThreshold.Flag,Y,0,1
"Result value(s) outside of the national upper range for a given characteristic, possibly indicating non-sensical value(s)",TADA.ResultValueAboveUpperThreshold.Flag,N,0,0
"Result value(s) outside of the national upper range for a given characteristic, possibly indicating non-sensical value(s)",TADA.ResultValueAboveUpperThreshold.Flag,Not Reviewed,0,0
"Result value(s) outside of the national lower range for a given characteristic, possibly indicating non-sensical value(s)",TADA.ResultValueBelowLowerThreshold.Flag,Y,0,1
"Result value(s) outside of the national lower range for a given characteristic, possibly indicating non-sensical value(s)",TADA.ResultValueBelowLowerThreshold.Flag,N,0,0
"Result value(s) outside of the national lower range for a given characteristic, possibly indicating non-sensical value(s)",TADA.ResultValueBelowLowerThreshold.Flag,Not Reviewed,0,0
Invalid characteristic and fraction combination,TADA.SampleFraction.Flag,Valid,0,0
Invalid characteristic and fraction combination,TADA.SampleFraction.Flag,NonStandardized,0,0
Invalid characteristic and fraction combination,TADA.SampleFraction.Flag,Invalid,0,1
Invalid characteristic and fraction combination,TADA.SampleFraction.Flag,Not Reviewed,0,0
Metadata indicates duplicative uploads of the same results within a single organization,TADA.SingleOrgDup.Flag,Duplicate,0,1
Metadata indicates duplicative uploads of the same results by multiple organizations,TADA.ResultSelectedMultipleOrgs,N,0,1
MeasureQualifierCode indicates result value is suspect,TADA.MeasureQualifierCode.Flag,Non-Detect,0,0
MeasureQualifierCode indicates result value is suspect,TADA.MeasureQualifierCode.Flag,Pass,0,0
MeasureQualifierCode indicates result value is suspect,TADA.MeasureQualifierCode.Flag,Suspect,0,1
MeasureQualifierCode indicates result value is suspect,TADA.MeasureQualifierCode.Flag,Over-Detect,0,0
MeasureQualifierCode indicates result value is suspect,TADA.MeasureQualifierCode.Flag,Not Reviewed,0,0
Metadata indicates duplicative uploads of the same results within a single organization,TADA.SingleOrgDup.Flag,Unique,0,0
Loading

0 comments on commit 6edb886

Please sign in to comment.