Check if CDS downloader of GloFAS river flood module is compatible with new CDS Engine API

[peanutfun]

See https://confluence.ecmwf.int/display/CKB/Please+read%3A+CDS+and+ADS+migrating+to+new+infrastructure%3A+Common+Data+Store+%28CDS%29+Engine

• Upgrade to cdsapi v0.7 compatible?
• Do we need to change download requests?

[elianekobler]

• Upgrade to cdsapi v0.7.3 works and is needed for downloading from the new API
• Downloading GloFAS v4 forecasts as well as computing flood hazards from it works
• Running setup_gumbel_fit() -> download_glofas_discharge ("historical",..) fails so far

[luseverin]

Hey, not exactly sure if this problem belongs to this specific issue, but I'll post it here:

Problem
Downloading reanalysis data fails, probably due to the last CDS update. Running the following code:

rf = RiverFloodInundation()
rf.download_reanalysis(
    countries=cntry,
    year=2024,
    system_version="version_3_1"#"version_3_1", "operational"  # Version mislabeled
)
ds_flood = rf.compute()

leads to the error:

HTTPError: 400 Client Error: Bad Request for url: https://ewds.climate.copernicus.eu/api/retrieve/v1/processes/cems-glofas-historical/execution
invalid request
Invalid integer for hmonth: 'january'

Additionally, running the same code with system_version="operational" leads to the following error:

HTTPError: 400 Client Error: Bad Request for url: https://ewds.climate.copernicus.eu/api/retrieve/v1/processes/cems-glofas-historical/execution
invalid request
Invalid date string: "operational". Should be yyyymmdd or yyyy-mm-dd

So apparently, they changed the way the reanalysis data is accessed through the API. It seems now that it requires more than the year but at least a month. I remember looking into it and not finding a fix straightforwadly, but I could have a look again when I have more time. @peanutfun I guess this is part of the ongoing issues resulting from the CDS update. I don't know if you are currently taking care of this specific issue, but let me know if there is something I can do about this.

[peanutfun]

@luseverin Thank you, I am aware. I adapted my local version already with a new default request and updated the functions adjusting it. No online yet, because I am still working on rebuilding the extreme value distributions on historical data.

The new default requests look like this:

DEFAULT_REQUESTS = {
    "historical": {
        "variable": ["river_discharge_in_the_last_24_hours"],
        "product_type": ["consolidated"],
        "system_version": ["version_4_0"],
        "hydrological_model": ["lisflood"],
        "data_format": "grib2",
        "download_format": "unarchived",
        "hyear": ["1979"],
        "hmonth": [f"{month:02}" for month in range(1, 13)],
        "hday": [f"{day:02}" for day in range(1, 32)],
    },
    "forecast": {
        "variable": ["river_discharge_in_the_last_24_hours"],
        "product_type": ["ensemble_perturbed_forecasts"],
        "system_version": ["operational"],
        "hydrological_model": ["lisflood"],
        "data_format": "grib2",
        "download_format": "unarchived",
        "year": ["2022"],
        "month": ["08"],
        "day": ["01"],
        "leadtime_hour": (np.arange(1, 31) * 24).astype(str).tolist(),
    },
}

Not sure where your second parsing error comes from, but I don't see it in my version. I think it's a follow-up error from the earlier parsing issue.