Release v0.2.0

* make model hashes nicer; add model hash to model method * :class:`ModelHash` * support user-supplied models with no specified subspace * Skip virtual models in `models_to_yaml_list` (#66) * Action updates via dependabot (#92) * GHA: Add PyPI deployment workflow (#91) Related to #89 * Fix minimum Python requirement (#97) petab requires python>=3.10 * Doc: Fix heading levels in example (#96) Fixes the sphinx TOC and a typo. * Get rid of petab DeprecationWarnings (#93) Closes #90 * README.md: PyPI badge * Add some docs; ensure similar PEtab problems in distance calculation (#67) * File formats to RTD (#102) * Move test suite description to RTD (#101) Related to #95. * User-calibrated models (#87) * compute safe petab hash digest size; support custom digest size; refactor `CandidateSpace` excluded hashes; add `end_iteration`; rename `candidates`->`start_iteration`; change workflow as described in changelog * `ModelHash.__eq__`: support comparisons with `str` and partial hashes * repro: narrow bounds in test cases 1-8 * Add logo draft (#103) * add to RTD, closes #88 * Set up `setuptools_scm` (#100) Set up `setuptools_scm` and move most package configuration to `pyproject.toml`. Move git-based package URLs out of `pyproject.toml` (otherwise, PyPI upload will fail). Co-authored-by: Dilan Pathirana <[email protected]> * Doc: Update RTD landing page (#105) * Doc: Update RTD landing page Add a bit of content. #95. * Apply suggestions from code review Co-authored-by: Dilan Pathirana <[email protected]> * Formatting, spelling, links --------- Co-authored-by: Dilan Pathirana <[email protected]> * Switch from black/isort/... to ruff (#99) Replace black/isort/... by ruff. Run `pre-commit run --all-files`. Some additional rules should be enabled, but that will require additional effort. --------- Co-authored-by: Dilan Pathirana <[email protected]> * Update file format spec (#106) * add example to model space, to clarify compressed format Co-authored-by: Daniel Weindl <[email protected]> * Doc: fix list formatting (#107) * test case 0009: update predecessor model * Visualization methods (#36) * closes #104 --------- Co-authored-by: Daniel Weindl <[email protected]> Co-authored-by: Daniel Weindl <[email protected]> * Support SSR as criterion (#108) * closes #44 * changelog --------- Co-authored-by: Dilan Pathirana <[email protected]> Co-authored-by: Daniel Weindl <[email protected]> Co-authored-by: Daniel Weindl <[email protected]>
PEtab-dev · Nov 18, 2024 · 29aff05 · 29aff05
2 parents 8d97c58 + 819b4da
commit 29aff05
Show file tree

Hide file tree

Showing 78 changed files with 5,674 additions and 2,372 deletions.
diff --git a/.flake8 b/.flake8
diff --git a/.github/dependabot.yml b/.github/dependabot.yml
@@ -0,0 +1,11 @@
+version: 2
+updates:
+
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+    ignore:
+      - dependency-name: "*"
+        update-types: ["version-update:semver-patch", "version-update:semver-minor"]
+    target-branch: "develop"
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -18,7 +18,7 @@ jobs:
     strategy:
       matrix:
         # test on latest and minimum python version
-        python-version: ['3.11', '3.9']
+        python-version: ['3.13', '3.11']
 
     steps:
     - name: Check out repository

diff --git a/.github/workflows/deploy_release.yml b/.github/workflows/deploy_release.yml
@@ -0,0 +1,34 @@
+name: Deploy
+on:
+  release:
+    types:
+      - published
+
+jobs:
+  pypi:
+    name: Deploy PyPI
+    runs-on: ubuntu-latest
+    environment:
+      name: pypi
+      url: https://pypi.org/p/petab-select
+    permissions:
+      id-token: write
+
+    steps:
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: "3.12"
+
+    - uses: actions/checkout@v4
+      with:
+        fetch-depth: 20
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install setuptools wheel build
+        python -m build -s
+
+    - name: Publish a Python distribution to PyPI
+      uses: pypa/gh-action-pypi-publish@release/v1
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -4,31 +4,8 @@
 # `pre-commit run --all-files` as by default only changed files are checked
 
 repos:
-- repo: https://github.com/psf/black
-  rev: 23.10.0
-  hooks:
-  - id: black
-    description: The uncompromising code formatter
-- repo: https://github.com/pycqa/isort
-  rev: 5.12.0
-  hooks:
-  - id: isort
-    name: isort (python)
-  - id: isort
-    name: isort (cython)
-    types: [cython]
-  - id: isort
-    name: isort (pyi)
-    types: [pyi]
-- repo: https://github.com/nbQA-dev/nbQA
-  rev: 1.7.0
-  hooks:
-    - id: nbqa-black
-    - id: nbqa-pyupgrade
-      args: [--py36-plus]
-    - id: nbqa-isort
 - repo: https://github.com/pre-commit/pre-commit-hooks
-  rev: v4.5.0
+  rev: v5.0.0
   hooks:
   - id: check-yaml
     description: Check yaml files for parseable syntax
@@ -44,3 +21,19 @@ repos:
     description: Fix empty lines at ends of files
   - id: detect-private-key
     description: Detects the presence of private keys
+- repo: https://github.com/astral-sh/ruff-pre-commit
+  # Ruff version.
+  rev: v0.6.9
+  hooks:
+    # Run the linter.
+    - id: ruff
+      args:
+        - --fix
+        - --config
+        - pyproject.toml
+
+    # Run the formatter.
+    - id: ruff-format
+      args:
+        - --config
+        - pyproject.toml
diff --git a/.readthedocs.yaml b/.readthedocs.yaml
@@ -9,7 +9,7 @@ version: 2
 sphinx:
   builder: html
   configuration: doc/conf.py
-  fail_on_warning: True
+  fail_on_warning: False
 
 python:
   install:

diff --git a/README.md b/README.md
@@ -1,4 +1,6 @@
-# PEtab Select
+<img src="https://raw.githubusercontent.com/PEtab-dev/petab_select/logo/doc/logo/logo-wide.svg" height="200" alt="PEtab Select logo">
+
+[![PyPI - Version](https://img.shields.io/pypi/v/petab-select)](https://pypi.org/project/petab-select/)
 
 The repository for the development of the extension to PEtab for model
 selection, including the additional file formats and Python 3 package.
@@ -44,154 +46,3 @@ given the forward method and a predecessor model with 2 estimated parameters,
 if there are no models with 3 estimated parameters, but some models with 4
 estimated parameters, then the search may return candidate models with 4
 estimated parameters.
-
-## File formats
-
-Column or key names that are surrounding by square brackets
-(e.g. `[constraint_files]`) are optional.
-
-### Selection problem
-
-A YAML file with a description of the model selection problem.
-
-```yaml
-format_version: [string]
-criterion: [string]
-method: [string]
-model_space_files: [List of filenames]
-[constraint_files]: [List of filenames]
-[predecessor_model_files]: [List of filenames]
-```
-
-- `format_version`: The version of the model selection extension format (
-  e.g. `'beta_1'`)
-- `criterion`: The criterion by which models should be compared (e.g. `'AIC'`)
-- `method`: The method by which model candidates should be generated
-  (e.g. `'forward'`)
-- `model_space_files`: The filenames of model space files.
-- `constraint_files`: The filenames of constraint files.
-- `predecessor_model_files`: The filenames of predecessor (initial) model
-  files.
-
-### Model space
-
-A TSV with candidate models, in compressed or uncompressed format.
-
-| `model_subspace_id` | `petab_yaml` | [`sbml`] | `parameter_id_1`                                     | ... | `parameter_id_n`                                     |
-|---------------------|--------------|----------|------------------------------------------------------|-----|------------------------------------------------------|
-| (Unique) [string]   | [string]     | [string] | [string/float] OR [; delimited list of string/float] | ... | [string/float] OR [; delimited list of string/float] |
-
-- `model_subspace_id`: An ID for the model subspace.
-- `petab_yaml`: The PEtab YAML filename that serves as the base for a model.
-- `sbml`: An SBML filename. If the PEtab YAML file specifies multiple SBML
-  models, this can select a specific model by model filename.
-- `parameter_id_1`...`parameter_id_n` : Parameter IDs that are specified to
-  take specific values or be estimated. Example valid values are:
-    - uncompressed format:
-        - `0.0`
-        - `1.0`
-        - `estimate`
-    - compressed format
-        - `0.0;1.1;estimate` (the parameter can take the values `0.0` or `1.1`,
-          or be estimated according to the PEtab problem)
-
-### Constraints
-
-A TSV file with constraints.
-
-| `petab_yaml` | [`if`]                       | `constraint`                 |
-|--------------|------------------------------|------------------------------|
-| [string]     | [SBML L3 Formula expression] | [SBML L3 Formula expression] |
-
-- `petab_yaml`: The filename of the PEtab YAML file that this constraint
-  applies to.
-- `if`: As a single YAML can relate to multiple models in the model space file,
-  this ensures the constraint is only applied to the models that match
-  this `if` statement
-- `constraint`: If a model violates this constraint, it is skipped during the
-  model selection process and not optimized.
-
-### Model(s) (Predecessor models / model interchange / report)
-
-- *Predecessor models* are used to initialize an appropriate model selection
-  method. Model IDs should be unique here and compared to model IDs in any
-  model space files.
-- *Model interchange* refers to the format used to transfer model information
-  between PEtab Select and a PEtab-compatible calibration tool, during the
-  model selection process.
-- *Report* refers to the final results of the model selection process, which may
-  include calibration results from any calibrated models, or just the select
-  model.
-
-Here, the format for a single model is shown. Multiple models can be specified
-as a YAML list of the same format.
-
-The only required key is the PEtab YAML, as a model requires a PEtab problem.
-All other keys are maybe required, for the different uses of the format (e.g.,
-the report format should include `estimated_parameters`), or at different
-stages of the model selection process (the PEtab-compatible calibration tool
-should provide `criteria` for model comparison).
-
-```yaml
-[criteria]: [Dictionary of criterion names and values]
-[estimated_parameters]: [Dictionary of parameter IDs and values]
-[model_hash]: [string]
-[model_id]: [string]
-[parameters]: [Dictionary of parameter IDs and values]
-petab_yaml: [string]
-[predecessor_model_hash]: [string]
-[sbml]: [string]
-```
-
-- `criteria`: The value of the criterion by which model selection was
-  performed, at least. Optionally, other criterion values too.
-- `estimated_parameters`: Parameter estimates, not only of parameters specified
-  to be estimated in a model space file, but also parameters specified to be
-  estimated in the original PEtab problem of the model.
-- `model_hash`: The model hash, generated by the PEtab Select library.
-- `model_id`: The model ID.
-- `model_subspace_id`: Same as in the model space files.
-- `model_subspace_indices`: The indices that locate this model in its model
-  subspace.
-- `parameters`: The parameters from the problem (either values
-  or `'estimate'`) (a specific combination from a model space file, but
-  uncalibrated).
-- `petab_yaml`: Same as in model space files.
-- `predecessor_model_hash`: The hash of the model that preceded this model
-  during the model selection process.
-- `sbml`: Same as in model space files.
-
-## Test cases
-
-Several test cases are provided, to test the compatibility of a
-PEtab-compatible calibration tool with different PEtab Select features.
-
-The test cases are available in the `test_cases` directory, and are provided in
-the model format.
-
-| Test ID                                      | Criterion | Method           | Model space files | Compressed format | Constraints files | Predecessor (initial) models files |
-|----------------------------------------------|-----------|------------------|-------------------|-------------------|-------------------|------------------------------------|
-| 0001                                         | (all)     | (only one model) | 1                 |                   |                   |                                    |
-| 0002<sup>[1](#test_case_0002)</sup>          | AIC       | forward          | 1                 |                   |                   |                                    |
-| 0003                                         | BIC       | all              | 1                 | Yes               |                   |                                    |
-| 0004                                         | AICc      | backward         | 1                 |                   | 1                 |                                    |
-| 0005                                         | AIC       | forward          | 1                 |                   |                   | 1                                  |
-| 0006                                         | AIC       | forward          | 1                 |                   |                   |                                    |
-| 0007<sup>[2](#test_case_0007_and_0008)</sup> | AIC       | forward          | 1                 |                   |                   |                                    |
-| 0008<sup>[2](#test_case_0007_and_0008)</sup> | AICc      | backward         | 1                 |                   |                   |                                    |
-| 0009<sup>[3](#test_case_0009)</sup>          | AICc      | FAMoS            | 1                 | Yes               |                   | Yes                                |
-
-<a name="test_case_0002">1</a>. Model `M1_0` differs from `M1_1` in three
-parameters, but only 1 additional estimated parameter. The effect of this on
-model selection criteria needs to be clarified. Test case 0006 is a duplicate
-of 0002 that doesn't have this issue.
-
-<a name="test_case_0007_and_0008">2</a>. Noise parameter is removed, noise is
-fixed to `1`.
-
-<a name="test_case_0009">3</a>. This is a computationally expensive problem to
-solve. Developers can try a model selection initialized with the provided
-predecessor model, which is a model start that reproducibly finds the expected
-model. To solve the problem reproducibly <i>ab initio</i>, on the order of 100
-random model starts are required. This test case reproduces the model selection
-problem presented in https://doi.org/10.1016/j.cels.2016.01.002 .
diff --git a/changes.md b/changes.md
@@ -1,5 +1,57 @@
 # Changes
 
+## 0.2.0
+There are some **major breaking changes**, to support users providing previous calibration results, e.g. from previous model selection runs. The following changes are reflected in the notebook examples.
+- **breaking change** previously, calibration tools would call `candidates` at each iteration of model selection. `candidates` has now been renamed to `start_iteration`, and tools are now expected to run `end_iteration` after calibrating the iteration's models. This structure also simplifies the codebase for other features of PEtab Select.
+- **breaking change** previously, calibration tools would determine whether to continue model selection based on whether the candidate space contains any models. Now, calibration tools should rely on the `TERMINATE` signal provided by `end_iteration` to determine whether to continue model selection.
+- **breaking change** PEtab Select hides user-calibrated models from the calibration tool, until `end_iteration` is called. Hence, if a calibration tool does some analysis on the calibrated models of the current iteration, the tool should use the `MODELS` provided by `end_iteration`, and not the `MODELS` provided by `start_iteration`.
+In summary, here's some pseudocode showing the old way.
+```python
+from petab_select.ui import candidates
+while True:
+    # Get iteration models
+    models = candidates(...).models
+    # Terminate if no models
+    if not models:
+        break
+    # Calibrate iteration models
+    for model in models:
+        calibrate(model)
+    # Print a summary/analysis of current iteration models (dummy code)
+    print_summary_of_iteration_models(models)
+```
+And here's the new way. Full working examples are given in the updated notebooks, including how to handle the candidate space.
+```python
+from petab_select.ui import start_iteration, end_iteration
+from petab_select.constants import MODELS, TERMINATE
+while True:
+    # Initialize iteration, get uncalibrated iteration models
+    iteration = start_iteration(...)
+    # Calibrate iteration models
+    for model in iteration[MODELS]:
+        calibrate(model)
+    # Finalize iteration, get all iteration models and results
+    iteration_results = end_iteration(...)
+    # Print a summary/analysis of all iteration models (dummy code)
+    print_summary_of_iteration_models(iteration_results[MODELS])
+    # Terminate if indicated
+    if iteration_results[TERMINATE]:
+        break
+```
+- Other **major changes**
+    - Many thanks to @dweindl for:
+        - documentation! https://petab-select.readthedocs.io/
+        - GitHub CI fixes and GHA deployments to PyPI and Zenodo
+    - fixed a bug introduced in 0.1.8, where FAMoS "jump to most distant" moves were not handled correctly
+    - the renamed `candidates`->`start_iteration`:
+        - no longer accepts `calibrated_models`, as they are automatically stored in the `CandidateSpace` now with each `end_iteration`
+        - `calibrated_models` and `newly_calibrated_models` no longer need to be tracked between iterations. They are now tracked by the candidate space.
+        - exclusions via `exclude_models` is no longer supported. exclusions can be supplied with `set_excluded_hashes`
+    - model hashes are more readable and composed of two parts:
+        1. the model subspace ID
+        2. the location of the model in its subspace (the model subspace indices)
+    - users can now provide model calibrations from previous model selection runs. This enables them to skip re-calibration of the same models.
+
 ## 0.1.13
 - fixed bug when no predecessor model is provided, introduced in 0.1.11 (#83)
 

diff --git a/doc/api.rst b/doc/api.rst
@@ -16,5 +16,6 @@ petab-select Python API
    petab_select.model_space
    petab_select.model_subspace
    petab_select.petab
+   petab_select.plot
    petab_select.problem
    petab_select.ui