notebooks/3-advanced-usage-part-I.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Advanced Usage I\n",
    "\n",
    "**Concepts:**\n",
    "- Complex types, typed lists and dictionaries\n",
    "- Complex types, Enums and Unions\n",
    "- Datetime types\n",
    "- Custom types / hierarchical structures\n",
    "- Private attributes and class variables\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Complex Types\n",
    "\n",
    "We have already seen how to define classes / models using simple types, such as `int`, `float`, `str`, `bool`, etc. However in practice data structures are often more complex. They include for examples dictionaries, list of lists, or mutiple allowed types. Of course Pydantic also supports these more complex types, such as lists, dictionaries, enums, and unions. In the following we will see an overview of those types and how to use them:"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Typed Lists and Dictionaries\n",
    "\n",
    "Lists and dictionaries are very common data structures in Python. Pydantic supports typed lists and dictionaries, which means that we can also define the type of the elements in the list or the type of the values in the dictionary.\n",
    "Typed lists and dictionaries are defined using the `list` and `dict` generic types. For example, we can define a model with a list of floats as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel\n",
    "\n",
    "\n",
    "class LineV1(BaseModel):\n",
    "    \"\"\"A Line object represented by two lists of coordinates\"\"\"\n",
    "\n",
    "    x: list[float]\n",
    "    y: list[float]\n",
    "\n",
    "    def length(self):\n",
    "        \"\"\"Line length computed by summing over the distance between points\"\"\"\n",
    "        length = 0\n",
    "\n",
    "        for idx in range(len(self.x) - 1):\n",
    "            length += (\n",
    "                (self.x[idx] - self.x[idx + 1]) ** 2\n",
    "                + (self.y[idx] - self.y[idx + 1]) ** 2\n",
    "            ) ** 0.5\n",
    "\n",
    "        return length"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Instantiation of the model works as expected:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "line_v1 = LineV1(x=[0, 1, 3], y=[0, 1, 2])\n",
    "display(line_v1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(line_v1.length())"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The behavior is exactly the same as for simple types. So values are converted to the specified type if possible:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "line_v1 = LineV1(x=[0, 1, \"3\"], y=[0, True, 2])\n",
    "display(line_v1)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the type cannot be converted a `ValidationError` is raised:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "raises-exception"
    ]
   },
   "outputs": [],
   "source": [
    "line_v1 = LineV1(x=[0, 1, \"10^-2\"], y=[0, 1, 2])\n",
    "display(line_v1)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that pydantic indicates the index of the invalid value in the error message using `x -> 2`."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Typed dictionaries are defined in a similar way using the `dict` generic type:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel\n",
    "\n",
    "\n",
    "class LineV2(BaseModel):\n",
    "    \"\"\"A Line object represented by two lists of coordinates\"\"\"\n",
    "\n",
    "    coordinates: dict[str, list[float]]\n",
    "\n",
    "    def length(self):\n",
    "        \"\"\"Line length computed by summing over the distance between points\"\"\"\n",
    "        length = 0\n",
    "        x = self.coordinates[\"x\"]\n",
    "        y = self.coordinates[\"y\"]\n",
    "\n",
    "        for idx in range(len(x) - 1):\n",
    "            length += ((x[idx] - x[idx + 1]) ** 2 + (y[idx] - y[idx + 1]) ** 2) ** 0.5\n",
    "\n",
    "        return length"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we define some data an create the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "coordinates = {\n",
    "    \"x\": [0, 1, 3],\n",
    "    \"y\": [0, 1, 2],\n",
    "}\n",
    "\n",
    "line_v2 = LineV2(coordinates=coordinates)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For illustration purposes we also create a model with invalid data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "raises-exception"
    ]
   },
   "outputs": [],
   "source": [
    "coordinates = {\n",
    "    \"x\": [0, \"one\", 3],\n",
    "    \"y\": [0, 1, \"three\"],\n",
    "}\n",
    "\n",
    "line_v2 = LineV2(coordinates=coordinates)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again pydantic raises a meaningful error message and also indicates the keys and indices of the invalid values."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Enums and Union Types\n",
    "\n",
    "In many cases it is useful to provide users with a selection of valid values, such as a choice of strings. The data structure to handle this is called `Enum`. Enums are defined using the `Enum` generic type in Python. For example, we can define a selection for the addtional property of a line color:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel\n",
    "from enum import Enum\n",
    "\n",
    "\n",
    "class LineColor(str, Enum):\n",
    "    \"\"\"Line color enum\"\"\"\n",
    "\n",
    "    red = \"red\"\n",
    "    green = \"green\"\n",
    "    blue = \"blue\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And add this to the definiton of the line class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ColoredLine(BaseModel):\n",
    "    \"\"\"A Line object that can be used to represent a line.\"\"\"\n",
    "\n",
    "    x: list[float]\n",
    "    y: list[float]\n",
    "    color: LineColor = LineColor.red\n",
    "\n",
    "    def length(self):\n",
    "        \"\"\"Length of the line\"\"\"\n",
    "        length = 0\n",
    "\n",
    "        for idx in range(len(self.x) - 1):\n",
    "            length += (\n",
    "                (self.x[idx] - self.x[idx + 1]) ** 2\n",
    "                + (self.y[idx] - self.y[idx + 1]) ** 2\n",
    "            ) ** 0.5\n",
    "\n",
    "        return length"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On initialisation we can now pass a color:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "colored_line = ColoredLine(x=[0, 1, 2], y=[0, 1, 2], color=\"red\")\n",
    "display(colored_line)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's try an invalid color:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "raises-exception"
    ]
   },
   "outputs": [],
   "source": [
    "colored_line = ColoredLine(x=[0, 1, 2], y=[0, 1, 2], color=\"purple\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pydantic now gives a useful error message, saying that the value is not a valid enumeration member; and it also lists the valid choices.\n",
    "\n",
    "Now we try to modify the color of an exising instance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "colored_line.color = \"violet\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default pydantic only validates / parses the values on initialisation. If we want to validate the values on modification we can use the `validate_assignment` configuaration option:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ColoredLine(BaseModel):\n",
    "    \"\"\"A Line object that can be used to represent a line.\"\"\"\n",
    "\n",
    "    x: list[float]\n",
    "    y: list[float]\n",
    "    color: LineColor = LineColor.red\n",
    "\n",
    "    class Config:\n",
    "        \"\"\"Pydantic Config object\"\"\"\n",
    "\n",
    "        validate_assignment = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "raises-exception"
    ]
   },
   "outputs": [],
   "source": [
    "colored_line = ColoredLine(x=[0, 1, 2], y=[0, 1, 2], color=\"red\")\n",
    "colored_line.color = \"purple\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note:** that there is https://github.com/pydantic/pydantic-extra-types, which provides support for validation of CSS colors. `from pydantic.color import Color`, so there is actually no reason to implement this ourselves."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Union\n",
    "\n",
    "from pydantic import BaseModel\n",
    "from enum import Enum\n",
    "\n",
    "\n",
    "class LineColor(str, Enum):\n",
    "    \"\"\"Line color enum\"\"\"\n",
    "\n",
    "    red = \"red\"\n",
    "    green = \"green\"\n",
    "    blue = \"blue\"\n",
    "\n",
    "\n",
    "class LineV2(BaseModel):\n",
    "    \"\"\"A Line object that can be used to represent a line.\"\"\"\n",
    "\n",
    "    x: list[float]\n",
    "    y: list[float]\n",
    "    color: Union[LineColor, None] = LineColor.red"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Datetime Types\n",
    "\n",
    "Another very useful advanced type is the `datetime`, which lets users handle dates and times. Pydantic natively supports parsing and validation of datetime types. This relies on the Python standard library (see https://docs.python.org/3/library/datetime.html). Let's take a look at an example:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from datetime import datetime\n",
    "\n",
    "\n",
    "class LonLatTimeVector(BaseModel):\n",
    "    \"\"\"Represents a position on earth with time.\"\"\"\n",
    "\n",
    "    lon: float = 0.0\n",
    "    lat: float = 0.0\n",
    "    time: datetime = None\n",
    "\n",
    "\n",
    "position = LonLatTimeVector(lon=1.0, lat=2.0, time=datetime.now())\n",
    "display(position)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remember you can always check the type using `type()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(position.time)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So `datetime.now()` already creates a `datetime` object, but pydantic also supports other valid formats, such as:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Time in ISO format\n",
    "position = LonLatTimeVector(time=\"2021-01-01T00:00:00\")\n",
    "display(position)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Ints or floats interpred as unix time, i.e. seconds since 1970-01-01T00:00:00\n",
    "position = LonLatTimeVector(time=1609459200)\n",
    "display(position)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try to pass an invalid datetime:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "raises-exception"
    ]
   },
   "outputs": [],
   "source": [
    "position = LonLatTimeVector(time=\"2021-01-01\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is variety of other date and time related quantities, which might be useful:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from datetime import date, datetime, time, timedelta\n",
    "from pydantic import BaseModel\n",
    "\n",
    "\n",
    "class Model(BaseModel):\n",
    "    d: date = None\n",
    "    dt: datetime = None\n",
    "    t: time = None\n",
    "    td: timedelta = None\n",
    "\n",
    "\n",
    "m = Model(\n",
    "    d=1966280412345.6789,  # Unix time in seconds\n",
    "    dt=\"2032-04-23T10:20:30.400+02:30\",  # Time in ISO format\n",
    "    t=time(4, 8, 16),  # Time object hours, minutes, seconds, [milliseconds]\n",
    "    td=\"P3DT12H30M5S\",  # ISO 8601 duration format\n",
    ")\n",
    "\n",
    "display(m)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Other Types\n",
    "\n",
    "Pydantic has a few useful factory functions for building types that have additional validations built in. For example, `pydantic.conint` is a function with parameters that allow you to build a constrained integer. This could be used to create a `MovieRating` type that only allows integers from 0 to 100:\n",
    "```python\n",
    "from pydantic import BaseModel, conint\n",
    "\n",
    "MovieRating = conint(ge=0, le=100)\n",
    "\n",
    "class Movie(BaseModel):\n",
    "    name: str\n",
    "    year: int\n",
    "    rating: MovieRating\n",
    "```\n",
    "\n",
    "On top of the normal integer validations, `Movie` will throw a validation error if the value provided is an integer less than 0 or greater than 100.\n",
    "\n",
    "Other useful type factories exist, such as `conlist`, `confloat`, `constr`, `condate`, and more. Pydantic has also exposed specific instantiations of some of these type factories that are commonly used. A few examples include, but are not limited to, `NegativeInt`, `NonNegativeInt`, `PositiveInt`, `NonPositiveInt`, and the `float` counterparts. You can guess how each of these are created from `conint` and `confloat`.\n",
    "\n",
    "There are a variety of other useful types built into pydantic. We have already mentioned several above. Most of the types not mentioned above are mostly relevant for web development but it is nonetheless good to know that they exist. For an overview of those types see https://docs.pydantic.dev/1.10/usage/types/#pydantic-types and also https://github.com/pydantic/pydantic-extra-types (unreleased)."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Custom Data Types / Hierarchical Structures\n",
    "\n",
    "One of the most powerful features of Pydantic is the ability to combine models to create hierarchical structures. This is done simply by using an existing pydantic model as a new type. For example we easily define a triangle using three points:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel\n",
    "\n",
    "\n",
    "class Point(BaseModel):\n",
    "    \"\"\"Representation of a two-dimensional point coordinate.\"\"\"\n",
    "\n",
    "    x: float\n",
    "    y: float\n",
    "\n",
    "    def distance_to(self, other: \"Point\") -> float:\n",
    "        \"\"\"Computes the distance to another `PointV3`.\"\"\"\n",
    "        dx = self.x - other.x\n",
    "        dy = self.y - other.y\n",
    "        return (dx**2 + dy**2) ** 0.5\n",
    "\n",
    "\n",
    "class Triangle(BaseModel):\n",
    "    \"\"\"Representation of a triangle\"\"\"\n",
    "\n",
    "    point_a: Point = Point(x=0, y=0)\n",
    "    point_b: Point = Point(x=1, y=0)\n",
    "    point_c: Point = Point(x=0, y=1)\n",
    "\n",
    "    def circumference(self):\n",
    "        \"\"\"Circumference of the triangle\"\"\"\n",
    "        return (\n",
    "            self.point_a.distance_to(self.point_b)\n",
    "            + self.point_b.distance_to(self.point_c)\n",
    "            + self.point_c.distance_to(self.point_a)\n",
    "        )\n",
    "\n",
    "\n",
    "triangle = Triangle(\n",
    "    point_a=Point(x=0, y=0),\n",
    "    point_b=Point(x=0.5, y=0),\n",
    "    point_c=Point(x=0, y=0.5),\n",
    ")\n",
    "\n",
    "display(triangle)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(triangle.circumference())"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively in this case we can also pass a dictionary to the model without creating the `Point` instances first:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "triangle = Triangle(\n",
    "    point_a={\"x\": 0, \"y\": 0.5},\n",
    "    point_b={\"x\": 0.5, \"y\": 0},\n",
    "    point_c={\"x\": 0, \"y\": 0},\n",
    ")\n",
    "\n",
    "print(triangle.circumference())"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pydantic will automatically convert the dictionaries to `Point` instances by passing the arguments to `Point(**kwargs)`. This is already a preview of the serialization and deserialization of models into dictionaries and hierarchical languages such as JSON, YAML and TOML. Let's quickly verify the type of the point:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(triangle.point_a)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Of course this also works for list of `Point` objects. For example we can rewrite the definition of the `Line` class we introduced above as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel\n",
    "\n",
    "\n",
    "class Point(BaseModel):\n",
    "    \"\"\"Representation of a two-dimensional point coordinate.\"\"\"\n",
    "\n",
    "    x: float\n",
    "    y: float\n",
    "\n",
    "    def distance_to(self, other: \"Point\") -> float:\n",
    "        \"\"\"Computes the distance to another `PointV3`.\"\"\"\n",
    "        dx = self.x - other.x\n",
    "        dy = self.y - other.y\n",
    "        return (dx**2 + dy**2) ** 0.5\n",
    "\n",
    "\n",
    "class LineV3(BaseModel):\n",
    "    \"\"\"Line object represented by a list of points\"\"\"\n",
    "\n",
    "    points: list[Point]\n",
    "\n",
    "    def length(self):\n",
    "        \"\"\"Line length computed by summing over the distance between points\"\"\"\n",
    "        length = 0\n",
    "\n",
    "        for point, next_point in zip(self.points[:-1], self.points[1:]):\n",
    "            length += point.distance_to(next_point)\n",
    "\n",
    "        return length"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Which can now be used as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "line_v3 = LineV3(points=[Point(x=0, y=0), Point(x=1, y=1)])\n",
    "display(line_v3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(line_v3.length())"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you compare to our first implementation at the beginning, `LineV3` is more compact, readable and elegant. \n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Private Attributes and Class Variables\n",
    "\n",
    "Because of the way model attributes are defined in Pydantic, we cannot simply define a normal class attribute or \"normal\" Python attribute for a class. However in some cases we might want to introduce e.g. an internal attribute that is not part of the Pydantic model. The usual Python convention for non-public attributes is to prefix them with an underscore `_`. By convention attributes starting with an underscore are excluded from the Pydantic model.\n",
    "\n",
    "Let's try the following first:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel\n",
    "\n",
    "\n",
    "class LineV4(BaseModel):\n",
    "    \"\"\"Line object represented by a list of points\"\"\"\n",
    "\n",
    "    points: list[Point]\n",
    "    _color: LineColor = LineColor.red\n",
    "\n",
    "    def is_red(self):\n",
    "        return self._color == LineColor.red"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's instantiate the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "line_with_hidden_color = LineV4(\n",
    "    points=[Point(x=0, y=0), Point(x=1, y=1)],\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see that the `_color` attribute is not part of the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "line_with_hidden_color.__fields__"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But you can access it internally as usual, e.g. in the `is_red()` method we defined above:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "line_with_hidden_color.is_red()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However it cannot be modified from the outside:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "raises-exception"
    ]
   },
   "outputs": [],
   "source": [
    "line_with_hidden_color._color = \"blue\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This behaviour is obviously is different from a \"normal\" Python class attribute. \n",
    "\n",
    "Howver if you need to vary or manipulate internal attributes on instances of the model, you can declare them using `PrivateAttr` instead:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel, PrivateAttr\n",
    "\n",
    "\n",
    "class LineV5(BaseModel):\n",
    "    \"\"\"Line object represented by a list of points\"\"\"\n",
    "\n",
    "    points: list[Point]\n",
    "    _color: LineColor = PrivateAttr(LineColor.red)\n",
    "\n",
    "    # as this now behaves as a \"normal\" Python attribute, we need to set it on init\n",
    "    def __init__(self, **data):\n",
    "        super().__init__(**data)\n",
    "        self._color = data.get(\"_color\", LineColor.red)\n",
    "\n",
    "    def is_red(self):\n",
    "        return self._color == LineColor.red"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try the behaviour now:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "line_with_hidden_color = LineV5(\n",
    "    points=[Point(x=0, y=0), Point(x=1, y=1)], _color=\"violet\"\n",
    ")\n",
    "\n",
    "line_with_hidden_color.is_red()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "line_with_hidden_color._color = \"red\"\n",
    "line_with_hidden_color.is_red()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now this behaves like a normal Python class attribute. But note that it is also not being validated, just as a normal Python attribute. That's why we could set it to violet, even though it is not a valid color.\n",
    "\n",
    "In case you have many private attributes you can also use the config setting to achieve the equivalent behaviour:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel\n",
    "\n",
    "\n",
    "class LineV6(BaseModel):\n",
    "    \"\"\"Line object represented by a list of points\"\"\"\n",
    "\n",
    "    points: list[Point]\n",
    "    _color: LineColor = LineColor.red\n",
    "\n",
    "    class Config:\n",
    "        underscore_attrs_are_private = True\n",
    "\n",
    "    # as this now behaves as a \"normal\" Python attribute, we need to set it on init\n",
    "    def __init__(self, **data):\n",
    "        super().__init__(**data)\n",
    "        self._color = data.get(\"_color\", LineColor.red)\n",
    "\n",
    "    def is_red(self):\n",
    "        return self._color == LineColor.red"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "line_with_hidden_color = LineV5(points=[Point(x=0, y=0), Point(x=1, y=1)], _color=\"red\")\n",
    "line_with_hidden_color.is_red()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Theres is one more rare case to be covered: in some cases you might want to define a class variable, which is shared by all instances of the model. The \"normal\" syntax is occupied by the Pydantic model declaration. However we can still define class variables using the `ClassVar` type:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import ClassVar\n",
    "from pydantic import BaseModel\n",
    "\n",
    "\n",
    "class LineV6(BaseModel):\n",
    "    \"\"\"Line object represented by a list of points\"\"\"\n",
    "\n",
    "    points: list[Point]\n",
    "    color: ClassVar[LineColor] = LineColor.red\n",
    "\n",
    "    def is_red(self):\n",
    "        return self._color == LineColor.red"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again the variable is not part of the pydantic model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "line_global_color = LineV6(\n",
    "    points=[Point(x=0, y=0), Point(x=1, y=1)],\n",
    ")\n",
    "line_global_color.__fields__"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However it can be accessed and modified as a normal Python class variable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "other_line_global_color = LineV6(\n",
    "    points=[Point(x=0, y=0), Point(x=1, y=1)],\n",
    ")\n",
    "\n",
    "# We modify the class variable\n",
    "LineV6.color = LineColor.green\n",
    "\n",
    "# The change is reflected in both instances\n",
    "print(line_global_color.color)\n",
    "print(other_line_global_color.color)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3\n",
    "\n",
    "\n",
    "We will be using the [open-meteo](https://open-meteo.com/en/docs) free weather API to make a request to the <https://api.open-meteo.com/v1/forecast> endpoint. We will fetch temperature data for Austin, TX on an hourly cadence for all of 2023.\n",
    "\n",
    "To do this we will use the [requests](https://requests.readthedocs.io/en/latest/) third-party Python library. For anyone new to `requests`, here is a quick primer. Imagine you want to receive the response from the example URL: <https://example.com/api/v2/foo?bar=1&bap=scipy2023&baz=false>. Then the response can be obtained with:\n",
    "```python\n",
    "import requests\n",
    "\n",
    "api_endpoint = \"https://example.com/api/v2/foo\"\n",
    "params = {\"bar\": 1, \"bap\": \"scipy2023\", \"baz\": False}\n",
    "response = requests.get(url=api_endpoint, params=params)\n",
    "print(response.content)\n",
    "```\n",
    "The requests library has many additional features that you can find in the documentation linked above, but for this exercise, this should be all you need.\n",
    "\n",
    "Task: Write a script that makes a request to the open-meteo API enpoint with the following constraints:\n",
    "\n",
    "* Use Austin, TX as the location. Hint: you may need to look up the latitude and longitude.\n",
    "* You should request the hourly temperature 2m above ground.\n",
    "* Use GMT as the timezone.\n",
    "* The range of dates is from the beginning of 2023 to today.\n",
    "\n",
    "You should create a Pydantic model that parses and validates the response from the request. Your Pydantic model should be precise (i.e. include as many validations and specific type hints as possible) but also flexible (examples may include (a) allowing for accepting either celsius or fahrenheit or (b) allowing for either ISO8601 or unix time). Your model should also be hierarchical to account for the nested structure of the API response. In order to understand the expected response needed to build your model, you may want to spend a few minutes reading the open-meteo documentation (linked above) and/or playing around with the open-meteo UI (also at the linked documentation).\n",
    "\n",
    "After receiving the response, compute the daily average temperature and plot it as a function of time. You can use any plotting library you like, but we recommend [matplotlib](https://matplotlib.org/stable/index.html) and for the averaging you could use e.g. [pandas](https://pandas.pydata.org/).\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "scipy-2023-pydantic-tutorial",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}