Merge branch 'release-v2.0.0' into 1225-20-rename-aindx-models

.github/workflows/run_dev_tests.yml

@@ -4,6 +4,9 @@ on:
   pull_request:
     branches:
       - dev
+  merge_group:
+    branches:
+      - dev
 
 jobs:
   linters:

.github/workflows/run_main_tests.yml

@@ -5,6 +5,10 @@ on:
     branches:
       - '*release*'
       - main
+  merge_group:
+    branches:
+      - '*release*'
+      - main
 
 jobs:
   linters:

CONTRIBUTING.md

@@ -121,4 +121,15 @@ where scope (optional) describes the packages affected by the code changes and t
 - **test**: Adding missing tests or correcting existing tests
 to set up your environment.
 
-When you are ready to open a pull request, please link any relevant issues and request a review. Thanks for contributing!
+When you are ready to open a pull request, please link any relevant issues and request a review. Thanks for contributing!
+
+## Release
+
+- From dev, create a branch called release-vX.Y.Z
+- Manually increment the version number in the aind_data_schema/__init__.py file to match
+- Manually increment the major/minor/patch versions of the core files as needed
+- Push the branch and open a PR into main
+- After this push, any last minute changes to the release-vX.Y.Z will have to done to via a PR
+- After review, use a merge commit to merge into main
+- Open a PR from main back into dev so they're synced again
+- Create a Github release with the corresponding tag, modify the auto-generated release notes to focus on the major changes that occurred

docs/source/quality_control.md

@@ -6,9 +6,9 @@ Quality control is a collection of **evaluations** based on sets of **metrics**
 
 `QCEvaluation`s should be generated during pipelines: before raw data upload, during processing, and during analysis by researchers.
 
-Each `QualityControl`, `QCEvaluation`, and `QCMetric` includes a `aind_data_schema.quality_control.State` which is a timestamped object indicating that the Overall QC/Evaluation/Metric passes, fails, or is in a pending state waiting for manual annotation.
+The overall `QualityControl`, each `QCEvaluation`, and each `QCMetric` can be evaluated to get a `aind_data_schema.quality_control.State` which indicates whether the Overall QC/Evaluation/Metric passes, fails, or is in a pending state waiting for manual annotation.
 
-The state of an evaluation is set automatically to the lowest of its metric's states. A single failed metric sets an entire evaluation to fail. While a single pending metric (with all other metrics passing) sets an entire evaluation to pending. An optional setting `QCEvaluation.allow_failed_metrics` allows you to ignore failures, which can be useful in situations where an evaluation is not critical for quality control.
+The state of an evaluation is set automatically to the lowest of its metric's states. A single failed metric sets an entire evaluation to fail. A single pending metric (with all other metrics passing) sets an entire evaluation to pending. An optional setting `QCEvaluation.allow_failed_metrics` allows you to ignore failures, which can be useful in situations where an evaluation is not critical for quality control.
 
 ## Details
 
@@ -30,77 +30,21 @@ Each `QCMetric` is a single value or set of values that can be computed, or obse
 
 `QCMetric`s have a `Status`. The `Status` should depend directly on the `QCMetric.value`, either by a simple function: "value>5", or by a qualitative rule: "Field of view includes visual areas". The `QCMetric.description` field should be used to describe the rule used to set the status. Metrics can be evaluated multiple times, in which case the new status should be appended the `QCMetric.status_history`.
 
+**Q: What is a metric reference?**
+
 Metrics should include a `QCMetric.reference`. References are intended to be publicly accessible images, figures, combined figures with multiple panels, or videos that support the metric or provide information necessary for manual annotation of a metric's status.
 
+See the AIND section for specifics about how references are rendered in the QC Portal.
+
 **Q: What are the status options for metrics?**
 
-In our quality control a metric's status is always `PASS`, `PENDING` (waiting for manual annotation), or `FAIL`. `PENDING` should be used when a user must manually annotated the metric's state.
+In our quality control a metric's status is always `PASS`, `PENDING` (waiting for manual annotation), or `FAIL`.
 
 We enforce this minimal set of states to prevent ambiguity and make it easier to build tools that can interpret the status of a data asset.
 
 ## Details for AIND users
 
-### Uploading QC
-
-#### Preferred workflow 
-
-**Metadata**
-
-If you are building `QCEvaluation` and `QCMetric` objects prior to raw data upload or in a capsule alongside your processing or analysis, your workflow is: 
-
-```
-from aind_data_schema.core.quality_control import QualityControl
-
-# Build your QCEvaluations and metrics
-evaluations = [QCEvaluation(), ...]
-
-# Build your QualityControl object
-qc = QualityControl(evaluations=evaluations)
-
-qc.write_standard_file()
-```
-
-The indexer will pick up this file alongside the other metadata files and handle it appropriately.
-
-**References**
-
-Each `QCMetric` you build should have an attached reference. Our preference is that you post these images to [FigURL](https://github.com/flatironinstitute/figurl/blob/main/doc/intro.md) and put the generated URL into the reference.
-
-We recommend that you write PNG files for images and static multi-panel figures, MP4 files for videos, and Altair charts for interactive figures.
-
-#### Alternate workflows
-
-**Metadata**
-
-We'll post documentation on how to append `QCEvaluations` to pre-existing quality_control.json files, via DocDB using the `aind-data-access-api`, in the future.
-
-**References**
-
-You can also place the references in the data asset itself, to do this include the relative path "qc_images/your_image.png" to that asset inside of the results folder.
-
-### QC Portal
-
-The QC Portal is a browser application that allows users to view and interact with the AIND QC metadata and to annotate ``PENDING`` metrics with qualitative evaluations. The portal is maintained by scientific computing, reach out to us with any questions or concerns.
-
-The portal works by pulling the metadata object from the Document Database (DocDB). When you make changes to metrics or add notes the **submit** button will be enabled, submitting pushes your updates to the DocDB along with a timestamp and your name.
-
-**Q: When does the state get set for the QCEvaluation and QualityControl objects?**
-
-The QC portal automatically calls ``QualityControl.evaluate_status()`` whenever you submit changes to the metrics. This first evaluates the individual `QCEvaluation` objects, and then evaluates the overall status.
-
-**Q: How do reference URLs get pulled into the QC Portal?**
-
-Each metric is associated with a reference figure, image, or video. The QC portal can interpret four ways of setting the reference field:
-
-- Provide a relative path to a file in this data asset's S3 bucket
-- Provide a url to a FigURL figure
-- Provide the name of one of the interactive plots, e.g. "ecephys-drift-map"
-
-<!-- There are many situations where it's helpful to be able to "swipe" between two images. If you have two identical images separated by a ';' the portal will allow users to swipe between them. For example, you might show snippets of the raw electrophysiology raster with detected spikes overlaid on the swipe. -->
-
-**Q: I saw fancy things like dropdowns in the QC Portal, how do I do that?**
-
-By default the QC portal displays dictionaries as tables where the values can be edited. We also support a few special cases to allow a bit more flexibility or to constrain the actions that manual annotators can take. Install the `aind-qcportal-schema` package and set the `value` field to the corresponding pydantic object to use these. 
+Instructions for uploading QC for viewing in the QC portal can be found [here](https://github.com/AllenNeuralDynamics/aind-qc-portal).
 
 ### Multi-asset QC
 
@@ -112,4 +56,4 @@ You should follow the preferred/alternate workflows described above. If your mul
 
 **Q: I want to be able to store data about each of the evaluated assets in this metric**
 
-Take a look at the `MultiAssetMetric` class in `aind-qc-portal-schema`. It allows you to pass a list of values which will be matched up with the `evaluated_assets` names. You can also include options which will appear as dropdowns or checkboxes. 
+Take a look at the `MultiAssetMetric` class in `aind-qc-portal-schema`. It allows you to pass a list of values which will be matched up with the `evaluated_assets` names. You can also include options which will appear as dropdowns or checkboxes.

examples/aibs_smartspim_procedures.json

@@ -1,6 +1,6 @@
 {
    "describedBy": "https://raw.githubusercontent.com/AllenNeuralDynamics/aind-data-schema/main/src/aind_data_schema/core/procedures.py",
-   "schema_version": "1.1.4",
+   "schema_version": "1.2.3",
    "subject_id": "651286",
    "subject_procedures": [
       {

examples/bergamo_ophys_session.json

@@ -1,6 +1,6 @@
 {
    "describedBy": "https://raw.githubusercontent.com/AllenNeuralDynamics/aind-data-schema/main/src/aind_data_schema/core/session.py",
-   "schema_version": "1.0.4",
+   "schema_version": "1.1.2",
    "protocol_id": [],
    "experimenter_full_name": [
       "John Doe"
@@ -53,7 +53,12 @@
                "index": 0,
                "imaging_depth": 150,
                "imaging_depth_unit": "micrometer",
-               "targeted_structure": "M1",
+               "targeted_structure": {
+                  "atlas": "CCFv3",
+                  "name": "Primary motor area",
+                  "acronym": "MOp",
+                  "id": "985"
+               },
                "fov_coordinate_ml": "1.5",
                "fov_coordinate_ap": "1.5",
                "fov_coordinate_unit": "micrometer",
@@ -148,6 +153,7 @@
          "stimulus_device_names": [],
          "speaker_config": null,
          "light_source_config": [],
+         "objects_in_arena": null,
          "output_parameters": {},
          "reward_consumed_during_epoch": null,
          "reward_consumed_unit": "microliter",

examples/bergamo_ophys_session.py

@@ -15,6 +15,7 @@
     StimulusModality,
     Stream,
 )
+from aind_data_schema_models.brain_atlas import CCFStructure
 
 # If a timezone isn't specified, the timezone of the computer running this
 # script will be used as default
@@ -56,7 +57,7 @@
                 FieldOfView(
                     index=0,
                     imaging_depth=150,
-                    targeted_structure="M1",
+                    targeted_structure=CCFStructure.MOP,
                     fov_coordinate_ml=1.5,
                     fov_coordinate_ap=1.5,
                     fov_reference="Bregma",

examples/data_description.json

@@ -1,6 +1,6 @@
 {
    "describedBy": "https://raw.githubusercontent.com/AllenNeuralDynamics/aind-data-schema/main/src/aind_data_schema/core/data_description.py",
-   "schema_version": "1.0.3",
+   "schema_version": "1.0.4",
    "license": "CC-BY-4.0",
    "platform": {
       "name": "Electrophysiology platform",

examples/ephys_session.json

@@ -1,6 +1,6 @@
 {
    "describedBy": "https://raw.githubusercontent.com/AllenNeuralDynamics/aind-data-schema/main/src/aind_data_schema/core/session.py",
-   "schema_version": "1.0.4",
+   "schema_version": "1.1.2",
    "protocol_id": [],
    "experimenter_full_name": [
       "Max Quibble",
@@ -37,7 +37,12 @@
                "coordinate_transform": "behavior/calibration_info_np2_2023_04_24.npy",
                "calibration_date": "2023-04-25T00:00:00Z",
                "notes": "Moved Y to avoid blood vessel, X to avoid edge. Mouse made some noise during the recording with a sudden shift in signals. Lots of motion. Maybe some implant motion.",
-               "primary_targeted_structure": "LGd",
+               "primary_targeted_structure": {
+                  "atlas": "CCFv3",
+                  "name": "Dorsal part of the lateral geniculate complex",
+                  "acronym": "LGd",
+                  "id": "170"
+               },
                "other_targeted_structure": null,
                "targeted_ccf_coordinates": [
                   {
@@ -70,7 +75,12 @@
                "coordinate_transform": "behavior/calibration_info_np2_2023_04_24.py",
                "calibration_date": "2023-04-25T00:00:00Z",
                "notes": "Trouble penetrating. Lots of compression, needed to move probe. Small amount of surface bleeding/bruising. Initial Target: X;10070.3\tY:7476.6",
-               "primary_targeted_structure": "LC",
+               "primary_targeted_structure": {
+                  "atlas": "CCFv3",
+                  "name": "Locus ceruleus",
+                  "acronym": "LC",
+                  "id": "147"
+               },
                "other_targeted_structure": null,
                "targeted_ccf_coordinates": [
                   {
@@ -172,7 +182,12 @@
                "coordinate_transform": "behavior/calibration_info_np2_2023_04_24.npy",
                "calibration_date": "2023-04-25T00:00:00Z",
                "notes": "Moved Y to avoid blood vessel, X to avoid edge. Mouse made some noise during the recording with a sudden shift in signals. Lots of motion. Maybe some implant motion.",
-               "primary_targeted_structure": "LGd",
+               "primary_targeted_structure": {
+                  "atlas": "CCFv3",
+                  "name": "Dorsal part of the lateral geniculate complex",
+                  "acronym": "LGd",
+                  "id": "170"
+               },
                "other_targeted_structure": null,
                "targeted_ccf_coordinates": [
                   {
@@ -205,7 +220,12 @@
                "coordinate_transform": "behavior/calibration_info_np2_2023_04_24.py",
                "calibration_date": "2023-04-25T00:00:00Z",
                "notes": "Trouble penetrating. Lots of compression, needed to move probe. Small amount of surface bleeding/bruising. Initial Target: X;10070.3\tY:7476.6",
-               "primary_targeted_structure": "LC",
+               "primary_targeted_structure": {
+                  "atlas": "CCFv3",
+                  "name": "Locus ceruleus",
+                  "acronym": "LC",
+                  "id": "147"
+               },
                "other_targeted_structure": null,
                "targeted_ccf_coordinates": [
                   {
@@ -306,6 +326,7 @@
          "stimulus_device_names": [],
          "speaker_config": null,
          "light_source_config": [],
+         "objects_in_arena": null,
          "output_parameters": {},
          "reward_consumed_during_epoch": null,
          "reward_consumed_unit": "microliter",
@@ -348,6 +369,7 @@
          "stimulus_device_names": [],
          "speaker_config": null,
          "light_source_config": [],
+         "objects_in_arena": null,
          "output_parameters": {},
          "reward_consumed_during_epoch": null,
          "reward_consumed_unit": "microliter",

examples/ephys_session.py

@@ -16,6 +16,7 @@
     Stream,
     VisualStimulation,
 )
+from aind_data_schema_models.brain_atlas import CCFStructure
 
 session = Session(
     experimenter_full_name=["Max Quibble", "Finn Tickle"],
@@ -122,7 +123,7 @@
                     arc_angle=5.2,
                     module_angle=8,
                     coordinate_transform="behavior/calibration_info_np2_2023_04_24.npy",
-                    primary_targeted_structure="LGd",
+                    primary_targeted_structure=CCFStructure.LGD,
                     manipulator_coordinates=Coordinates3d(x=8422, y=4205, z=11087.5),
                     calibration_date=datetime(year=2023, month=4, day=25, tzinfo=timezone.utc),
                     notes=(
@@ -137,7 +138,7 @@
                     targeted_ccf_coordinates=[CcfCoords(ml=6637.28, ap=4265.02, dv=10707.35)],
                     assembly_name="Ephys_assemblyB",
                     coordinate_transform="behavior/calibration_info_np2_2023_04_24.py",
-                    primary_targeted_structure="LC",
+                    primary_targeted_structure=CCFStructure.LC,
                     manipulator_coordinates=Coordinates3d(x=9015, y=7144, z=13262),
                     calibration_date=datetime(year=2023, month=4, day=25, tzinfo=timezone.utc),
                     notes=(
@@ -170,7 +171,7 @@
                     targeted_ccf_coordinates=[CcfCoords(ml=8150, ap=3250, dv=7800)],
                     assembly_name="Ephys_assemblyA",
                     coordinate_transform="behavior/calibration_info_np2_2023_04_24.npy",
-                    primary_targeted_structure="LGd",
+                    primary_targeted_structure=CCFStructure.LGD,
                     manipulator_coordinates=Coordinates3d(x=8422, y=4205, z=11087.5),
                     calibration_date=datetime(year=2023, month=4, day=25, tzinfo=timezone.utc),
                     notes=(
@@ -185,7 +186,7 @@
                     targeted_ccf_coordinates=[CcfCoords(ml=6637.28, ap=4265.02, dv=10707.35)],
                     assembly_name="Ephys_assemblyB",
                     coordinate_transform="behavior/calibration_info_np2_2023_04_24.py",
-                    primary_targeted_structure="LC",
+                    primary_targeted_structure=CCFStructure.LC,
                     manipulator_coordinates=Coordinates3d(x=9015, y=7144, z=13262),
                     calibration_date=datetime(year=2023, month=4, day=25, tzinfo=timezone.utc),
                     notes=(

examples/mri_session.json

@@ -1,6 +1,6 @@
 {
    "describedBy": "https://raw.githubusercontent.com/AllenNeuralDynamics/aind-data-schema/main/src/aind_data_schema/core/session.py",
-   "schema_version": "1.0.4",
+   "schema_version": "1.1.2",
    "protocol_id": [
       "dx.doi.org/10.57824/protocols.io.bh7kl4n6"
    ],