Eac/refactor project (#9)

* WIP * major changes * moved configurable and dyname_class * linting and coverage * yamllint * pretty up cli inspect * Added comments to yaml files * improved printouts * added rail_project_library.ipynb * fix printout functions * added rail_plotting_control.ipynb * testing and fixes * fix pytest fixutre * added setup_project_area to test_project * updated example notebooks * updated rail_project_example.ipynb * linting * moved setup script to library to share it between notebooks and library * added execution * added execution * linting * coverage
LSSTDESC · Jan 31, 2025 · 672f0af · 672f0af
1 parent 78c8f35
commit 672f0af
Show file tree

Hide file tree

Showing 63 changed files with 6,069 additions and 1,745 deletions.
diff --git a/.yamllint.yaml b/.yamllint.yaml
@@ -0,0 +1,13 @@
+extends: default
+rules:
+  document-start: {present: false}
+  line-length:
+    max: 500
+    allow-non-breakable-words: true
+    allow-non-breakable-inline-mappings: true
+  truthy:
+    # "on" as a key in workflows confuses things
+    ignore: |
+      /.github/workflows/
+  indentation:
+    indent-sequences: consistent
diff --git a/examples/rail_plotting_control.ipynb b/examples/rail_plotting_control.ipynb
@@ -0,0 +1,274 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "13048f2e-11cf-4ed3-9764-47dbb5999776",
+   "metadata": {},
+   "source": [
+    "### Setup and teardown scripts to setup a test area"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8955dcb3-a5df-451b-805e-4099ebb0eb60",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "from rail.projects import library\n",
+    "\n",
+    "check_dir = os.path.basename(os.path.abspath(os.curdir))\n",
+    "if check_dir == 'examples':\n",
+    "    os.chdir('..')\n",
+    "\n",
+    "setup = library.setup_project_area()\n",
+    "assert setup == 0\n",
+    "\n",
+    "# use this to cleanup\n",
+    "# library.teardown_project_area()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "64dfec6f-136f-4aec-93ad-673392a777aa",
+   "metadata": {},
+   "source": [
+    "### import the plotting control and load the objects from the yaml file"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f50e7a45-1750-4808-93a4-0a88db64fb38",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from rail.plotting import control"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6eb09df5-c09b-45f9-b993-5dfe30e1f642",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "plot_groups = control.load_plot_group_yaml('tests/ci_plot_groups.yaml')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8b7cf610-fbf0-4927-a417-bc6fa5504832",
+   "metadata": {},
+   "source": [
+    "### Print the stuff that we have loaded"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "bba5bf79-10d3-4dc5-84c2-e5c03a9a88c5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "control.print_contents()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6121b538-dfe2-4aa8-bb84-3a42dd9ee440",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "control.print_classes()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cf8ae0c5-4f32-499f-a3b2-c3104bb1c210",
+   "metadata": {},
+   "source": [
+    "### let's have a look at the \"PlotGroup\" that we have loaded.\n",
+    "\n",
+    "A \"PlotGroup\" points at one or more plots and one or more datasets and will iterate over both and make all the possible plots"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f543e1d9-db9d-4eaf-a093-4194862b3be8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "for k, v in plot_groups.items():\n",
+    "    print(f\"{k}: {v}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c99c96ce-815c-4f41-9a15-c97fbb1f62a5",
+   "metadata": {},
+   "source": [
+    "Lets grab a particular plot_group"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4d34d6fe-fb77-4fe4-b9ab-99a7b9ea5bf8",
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "plot_group = plot_groups['zestimate_v_ztrue_test_plots']"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "54e5eb3f-a70c-41cb-b6c5-1714cda42a63",
+   "metadata": {},
+   "source": [
+    "And let's grab the associated datasets"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "739472c2-b190-4031-85d5-e92e98a3d24d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from rail.plotting.dataset_factory import RailDatasetFactory\n",
+    "dataset_dict = RailDatasetFactory.get_dataset_dict(plot_group.dataset_dict_name)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8f1c84b0-c451-4ba2-aae6-c6f193225620",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "for k, v in dataset_dict.items():\n",
+    "    print(f\"{k}: {v}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5b63d30b-51a1-474f-8b03-95315644c10c",
+   "metadata": {},
+   "source": [
+    "And lets get a particular dataset"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2f6571df-d4cd-4b47-b0dd-3fc605d244e8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "dataset_holder = dataset_dict['blend_baseline_trainz']\n",
+    "print(dataset_holder)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e125c7b4-7d16-4195-b4d5-3b14dd252e81",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "dataset_holder()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c8f87c3f-c9e5-49c3-96bf-0e7bc10efe63",
+   "metadata": {},
+   "source": [
+    "Now grab the associated plotters"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "10153c8d-49e1-4e1e-801c-a213fd3395fd",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from rail.plotting.plotter_factory import RailPlotterFactory\n",
+    "plotter_list = RailPlotterFactory.get_plotter_list(plot_group.plotter_list_name)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8519ec0c-ab5e-4c35-8e9d-4f10349b9afa",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "plotter_list"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3059d445-e894-4de2-b8e6-8e2f8168fc3c",
+   "metadata": {},
+   "source": [
+    "Here we make all the define plots, this makes 6 plots ( 3 datasets X 2 plotters )"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0718d7a7-5788-4a48-84f5-7b29b051395f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from rail.plotting.plotter import RailPlotter\n",
+    "RailPlotter.iterate(plotter_list, dataset_dict)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "bc640875-9dc0-4d11-9b2c-2c1c41e65e8e",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8dc792c9-f687-4a85-9810-7daf3de4db07",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.0"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}