Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to tas_2km_projected Rasdaman ingest #78

Open
BobTorgerson opened this issue Apr 25, 2024 · 0 comments
Open

Update to tas_2km_projected Rasdaman ingest #78

BobTorgerson opened this issue Apr 25, 2024 · 0 comments

Comments

@BobTorgerson
Copy link
Contributor

We want to mirror the table design in the current version of the ArcticEDS for precipitation that shows the minimum, mean, and maximum across all 5 models and 3 scenarios in the temperature section:

Precipitation
image

Temperature
image

Currently, we don't have all 5 models or all 3 scenarios available in the tas_2km_projected Rasdaman coverage, and only have 2 models: GFDL-3M3 & NCAR-CCSM4 (along with the 5ModelAvg) and 2 scenarios: RCP 4.5 & RCP 8.5. To allow for us to get the minimum, mean, and maximum across all of the models and scenarios, we will need to re-ingest the files for all of that data.

As I recall, we ended up going against this because of how long it takes to generate the temperature data for the ArcticEDS when we had the full dataset. However, with the newest version of Rasdaman on Zeus and the prospect of getting Rasdaman Enterprise, there may be some serious performance gains to look into for this dataset.

If we were to go forward with ingesting this dataset with the full coverage, we would need to download all of the GeoTIFFs in this TAS 2km dataset for the monthly data. We would then need to run:

https://github.com/ua-snap/rasdaman-ingest/blob/tas_projected_2km/iem/tas_projected_2km/merge.py

After merging the TIFs into NC files for tas, tasmax, and tasmin for each model, we would again merge those various variables as bands within an overarching NC file that would contain all of the variables for a given model and month combination using this script:

https://github.com/ua-snap/rasdaman-ingest/blob/tas_projected_2km/iem/tas_projected_2km/combine.py

After that was finished, we would need to modify the ingest.json file in the following areas to accomadate the new models and scenarios:
https://github.com/ua-snap/rasdaman-ingest/blob/9d6febc5670c799a299868d018bee7a43202fe57/iem/tas_projected_2km/ingest.json#L35C1-L38C1

"scenario": {
"0": "rcp45",
"1": "rcp85"
},

"statements": "import imp, os; luts = imp.load_source('luts', os.getenv('LUTS_PATH')); regex_str = 'combined_(5ModelAvg|GFDL-CM3|NCAR-CCSM4)_(01|02|03|04|05|06|07|08|09|10|11|12)_temperature.nc'",

And modifying the luts.py file to include the correct number of scenarios.

This would get us halfway there for getting the temperature table the way we want, and the rest would require changes to the data-api /eds/temperature endpoint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant