Remove discovery documents that no longer have a submission attached

[sblack-usu]

When running the dsp report in September, an error occurred blocking the creation of the report. Some investigation found that there was a discoverable record that did not have a Submission attached to it. This is only an issue in creating the report when the provider is Zenodo as there is some metadata on the Submission document that is necessary for the report. This does not affect HydroShare, Earthchem, or external records.

A patch was introduced to the reporting endpoint to check if a submission exists for the discovery document. This fixed the reporting issue but there is still an issue with a discovery document without a Submission document. It look like the Submission document was deleted but the discovery document was not deleted with it. It is worth noting that there are legacy discoverable records that were imported and do not have a Submission attached to them. This is not the case for this record since there are no zenodo records in the set of legacy records.

The deletion should have occurred in the triggers file. It is possible that the deletion could have correlated with a deployment that interrupted the trigger. There is a pretty narrow window where this could occur. There are ways to protect against this by tracking change streams tokens.

We do have a daily job that updates discoverable datasets. The job is not catching this case because it looks at the submissions and then updates the discoverable documents. We could update it to check for non-legacy records that do not have a Submission.

[horsburgh]

@pkdash and @sblack-usu - I moved this issue from the dsp-back repository. I'm not sure whether it still exists, but if it does I've added it to the next milestone for DSP. If it has already been fixed, you can close it. I assigned this work to @pkdash

[pkdash]

@horsburgh I will be looking into this issue this week and if needed will make a fix to resolve it.

[pkdash]

@sblack-usu and @horsburgh It seems the daily scheduler already has the code to cleanup discovery records that are missing submission records.