tutorial/Fetch_forecast_for_multiple_indicators.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fetch Forecast for multiple indicators\n",
    "\n",
    "Welcome to this tutorial! In this notebook, we will explore various methods to retrieve weather data from the ARPEGE and AROME models using a list of indicators. This tutorial aims to provide a comprehensive guide on how to efficiently access data for different scenarios.\n",
    "\n",
    "We will cover the following cases:\n",
    "- Retrieving data based on a list of indicators\n",
    "- For a single run date\n",
    "- Specifying the \"interval\" parameter for each indicator\n",
    "- Specifying the \"forecast_horizon\" parameter\n",
    "- Specifying the \"heights\" or \"pressures\" parameters for each indicator\n",
    "- For multiple runs\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "\n",
    "from meteole import ArpegeForecast  # or AromeForecast, AromePIForecast"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "\n",
    "logging.basicConfig(level=logging.INFO)\n",
    "logger = logging.getLogger(\"meteole\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Requirements notice** : TODO Link to the documentation to have application_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "APP_ID = \"<TO-BE-SET>\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Init Client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "client = ArpegeForecast(application_id=APP_ID)  # or AromeForecast, AromePIForecast"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Select random indicators to get\n",
    "random_indicators = random.sample(client.INDICATORS, 5)\n",
    "print(f\"Selected Indicators: {random_indicators}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### For One Run\n",
    "\n",
    "To retrieve data from a list of indicators, it is possible to do so from one or multiple runs common to all indicators. Let's start with the simple case of a single run\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if not hasattr(client, \"capabilities\"):\n",
    "    client.get_capabilities()\n",
    "\n",
    "capabilities = client.capabilities[client.capabilities[\"indicator\"] == random_indicators[0]]\n",
    "run = capabilities.run.max()\n",
    "\n",
    "print(run)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_combined = client.get_combined_coverage(\n",
    "    indicator_names=random_indicators,\n",
    "    runs=[run],\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_combined.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Specify intervals\n",
    "\n",
    "For an indicator, it is sometimes necessary to specify an \"interval\" parameter. To use the `get_combined_coverage` method, you must provide a list of intervals (`intervals`) associated with the list of indicators (`indicator_names`). If this is not done and an \"interval\" is required, a default interval will be used, and the default value will be logged as information. If you want to have multiple intervals for the same indicator, you need to duplicate the indicator in `indicator_names`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Depending on the indicators, the parameters intervals must be specified, if needed, we pick the first value\n",
    "intervals_list = []\n",
    "for indicator in random_indicators:\n",
    "    capabilities = client.capabilities[client.capabilities[\"indicator\"] == indicator]\n",
    "    intervals = capabilities.iloc[0][\"interval\"]\n",
    "    intervals_list.append(intervals)\n",
    "\n",
    "print(f\"Intervals: {intervals_list}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "capabilities.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_combined = client.get_combined_coverage(\n",
    "    indicator_names=random_indicators,\n",
    "    runs=[run],\n",
    "    intervals=intervals_list,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_combined.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Specify list of forecast_horizons\n",
    "\n",
    "The list of forecast horizons must be the same and valid for all indicators because the indicators are concatenated into columns for similar forecast horizons. If you want different forecast horizons for different indicators, you need to make multiple calls to `get_combined_coverage`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# if you don't know common forecast_horizons between list of indicators, we can pick 2 common forecast_horizons like that\n",
    "if intervals_list:\n",
    "    intervals_list = [None if interval == \"\" else interval for interval in intervals_list]\n",
    "list_coverage_id = [\n",
    "    client._get_coverage_id(indicator, run, interval) for indicator, interval in zip(random_indicators, intervals_list)\n",
    "]\n",
    "forecast_horizons = client.find_common_forecast_horizons(list_coverage_id)[:2]\n",
    "print(forecast_horizons)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_combined = client.get_combined_coverage(\n",
    "    indicator_names=random_indicators,\n",
    "    runs=[run],\n",
    "    intervals=intervals_list,\n",
    "    forecast_horizons=forecast_horizons,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_combined.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Specify list of 'heights' or 'pressures'\n",
    "\n",
    "Depending on the indicators, you need to select `heights` and `pressures` from a list of values. If you do not specify any values, a default value will be used. The list of `heights` or `pressures` must be the same length as `indicator_names`. If you want multiple heights or pressures for the same indicator, you need to duplicate the indicator in `indicator_names`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# To get heights, we can use `self.get_coverage_description` and pick a random height if height exist for the indicator\n",
    "heights = []\n",
    "for indicator, interval in zip(random_indicators, intervals_list):\n",
    "    coverage_id = client._get_coverage_id(indicator, run, interval)\n",
    "    description = client.get_coverage_description(coverage_id)\n",
    "\n",
    "    # Get a random height if heights exist for the indicator\n",
    "    possible_heights = description.get(\"heights\", [])\n",
    "    if possible_heights:\n",
    "        random_height = random.choice(possible_heights)\n",
    "        heights.append(random_height)\n",
    "    else:\n",
    "        heights.append(None)\n",
    "\n",
    "print(heights)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_combined = client.get_combined_coverage(\n",
    "    indicator_names=random_indicators,\n",
    "    runs=[run],\n",
    "    intervals=intervals_list,\n",
    "    forecast_horizons=forecast_horizons,\n",
    "    heights=heights,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_combined.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_combined.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### For multiple runs\n",
    "\n",
    "If you need to retrieve data for multiple runs, it is possible. Different runs will be concatenated into rows, unlike indicators which are concatenated into columns. Therefore, you need to specify different runs as each run applies to the entire set of indicators."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "capabilities = client.capabilities[client.capabilities[\"indicator\"] == random_indicators[0]]\n",
    "runs = capabilities[\"run\"].unique()[:2].tolist()\n",
    "\n",
    "print(runs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_combined = client.get_combined_coverage(\n",
    "    indicator_names=random_indicators,\n",
    "    runs=runs,\n",
    "    intervals=intervals_list,\n",
    "    forecast_horizons=forecast_horizons,\n",
    "    heights=heights,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_combined.head(2)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "meteole_env",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "undefined.undefined.undefined"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}