Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider adding invalid dimensions to ingest metadata #24

Open
charparr opened this issue Apr 14, 2022 · 1 comment
Open

Consider adding invalid dimensions to ingest metadata #24

charparr opened this issue Apr 14, 2022 · 1 comment

Comments

@charparr
Copy link
Member

As it stands now invalid dimensions where the data is 100% null (e.g., "rcp85 scenario + "CRU-TS") are handled on an ad hoc basis in the API. If we continue the pattern of combining historical and future data in a single coverage, we'll continue to have the need to handle invalid dimensions. We should perhaps add the invalid dimensional combinations to the metadata in the ingest.json files so that the API can either drop them from the packaged data dictionary without much hassle (traversing a nested dictionary, checking for key combinations, etc.). They could probably even be dropped immediately after the response.

Something in like

invalid_dims = {
[0, 1, 1]...
}

But maybe encode with the actual names to make it clear.

@Joshdpaul
Copy link

Joshdpaul commented Jan 13, 2025

This would be a very useful feature for the CMIP6 (and someday, CMIP7) data where we have a dozen or more models, with varying availability of scenarios.

Currently in the API, our CMIP6 coverage in Rasdaman returns arrays of all zeros (as opposed to NaN or -9999) when a scenario is missing from a model. Without a hard-coded lookup table of model/scenario availability in the API, this zero return is difficult address. Including this lookup in the coverage metadata (rather than in the API config) would be a great solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants