Pre-release revisions and alignment with CDS v5.0.4 #169

Bankso · 2025-02-08T01:59:39Z

Fixes #151 and adds DOI columns noted in #159

This is an extension of the data model refactor and alignment with CDS model v5.0.4.

This update reflects the current Synapse table schemas used in grant-specific projects and UNION tables for GrantView, PublicationView, DatasetView, ToolView, and EducationalResource components.

Also included in this update are updated models for participant, specimen, file, and assay metadata, aligned with CDS v5.04 and mapped via this document.

Changelog

Added valid values for CDS data model, v5.0.4
Added study- and specimen-focused attributes, to better address some data elements requested by CDS model
Added DOI columns for Dataset View, Tool View, and Educational Resource models
Removed GeoMx attributes covered by Sequencing models
Added missing "Key" attributes, to support data model linkages
Specific model types mapped to CDS 5.0.4 and updated:
- Study
- Biospecimen
- Individual
- Model
- File View
- Sequencing Level 1 and 2
- Imaging Level 2, 3, and Channel
Models are also available for:
- Data Sharing Plan
- Sequencing Level 3
- Imaging Level 1 and 4
- GeoMx Level 1, 2, 3, Image, ROI/Annotations (updated), and Auxiliary files (added)
- Visium Level 1, 2, 3, 4, and Auxiliary files (updated)
- Sequencing RNA Level 1 (added)

Sequencing Level 1 model contains everything needed, so GeoMx Level 1 can be limited to just GeoMx-specific attributes

Sequencing Level 2 model contains everything needed, so GeoMx Level 2 can be limited to just GeoMx-specific attributes

Sequencing Level 3 model contains everything needed, so GeoMx Level 3 can be limited to just GeoMx-specific attributes

Already contained in Sequencing Level 1

Include attributes from shared that capture workflow details

- added DSP Dataset Metadata attribute and added to schema def - update descriptions to provide better guidance

Add "Metadata" as a file level, to help indicate that the information must be mapped onto a metadata template

CV updates from script

Using schematic 24.12.1

Add attribute GeoMx Imaging AOI Coordinates and update schema definition. This attribute allows a submitter to identify specific coordinate/mask files to be used with a GeoMx image

Using schematic 24.12.1

Retain Publication Dataset Alias to simplify curation and database updates

Move json to json_schemas and csvs to templates

Add output file handling

Replace Component Key attributes in Collection with Component Table Id attributes, which will be used to log table subsets generated for sharing alongside file Datasets

Using schematic v24.12.1

Add data type qualifier to description

Added DOI attributes for Dataset View, Tool View, and Educational Resource models Also added Keys to Educational Resource, but retained tool and dataset link attributes, in case the ed res uses resources from outside of Synapse

For mapping and conversion

Align valid values from CDS v5.0.4

These are stored in visium-specific folders

Remove unneeded attributes, covered by File View template

These values live in sequencing-specific folders, CSVs in shared were redundant

Equivalent to tissueOrganOriginCDS.csv, so I deleted this file and replaced mapping reference with reference to tissueOrganOriginCDS.csv

Align mappings with current set of valid values

Added Biospecimen Treatment Response, Incidence Type Added valid values for Acquisition method Removed Biospecimen Pathology

Updated by update valid values script

Updated by update valid values script; Moved attributes NGS Raw Reads and NGS Unique Bases from sequencing level 2 to sequencing level 1 and added to level 1 model definition. These are requested by CDS model

Updated by update valid values script; removed attributes now in sequencing level 1

Updated by update valid values script

via make

generated via schematic 24.12.1

This will be maintained locally until a clear need for storage in the repo is identified

Bankso · 2025-02-08T02:08:11Z

@aditigopalan please let me know if anything looks off! I was able to generate a google sheet template using the JSON-LD, so it at least works 😄

Bankso added 30 commits December 13, 2024 14:03

Update Collection schema definition

2a605ec

Remove redundant GeoMx Level 1 attributes

074c563

Sequencing Level 1 model contains everything needed, so GeoMx Level 1 can be limited to just GeoMx-specific attributes

Remove redundant GeoMx Level 2 attributes

12aaf5d

Sequencing Level 2 model contains everything needed, so GeoMx Level 2 can be limited to just GeoMx-specific attributes

Remove redundant GeoMx Level 3 attributes

020770a

Sequencing Level 3 model contains everything needed, so GeoMx Level 3 can be limited to just GeoMx-specific attributes

Add coverage to Sequencing Level 1 schema def

ad006d5

Remove redundant coverage attribute from Sequencing Level 2

c018e12

Already contained in Sequencing Level 1

Add Sequencing Level 3 attributes to capture workflow

c07f61d

Include attributes from shared that capture workflow details

Define Collection model

6b94045

Add GrantView Key to Dataset View

dd0e9c4

Add GrantView Key to Educational Resource

fa49c74

Add GrantView Key to Publication View

0cb5184

Add GrantView Key to DataDSP

7de7915

Add GrantView Key to Tool View

3183e7f

Update annotationProperty.csv

dfdbd3e

- added DSP Dataset Metadata attribute and added to schema def - update descriptions to provide better guidance

Update processLevel.csv

9c827b7

Add "Metadata" as a file level, to help indicate that the information must be mapped onto a metadata template

Update annotationProperty.csv

1294689

CV updates from script

make and schema convert

8946550

Using schematic 24.12.1

Update all_valid_values.csv

c17e23f

Update release_workflow.sh

e477f5b

Update annotationProperty.csv

4e8d934

Add attribute GeoMx Imaging AOI Coordinates and update schema definition. This attribute allows a submitter to identify specific coordinate/mask files to be used with a GeoMx image

Collate CSV and convert to JSON-LD

4b6152d

Using schematic 24.12.1

Update annotationProperty.csv

1633490

Retain Publication Dataset Alias to simplify curation and database updates

Organize schema convert outputs

2a7643b

Move json to json_schemas and csvs to templates

Update release_workflow.sh

9e78642

Add output file handling

Update annotationProperty.csv

8dcc1b0

Replace Component Key attributes in Collection with Component Table Id attributes, which will be used to log table subsets generated for sharing alongside file Datasets

Make and schema convert

df4b61a

Using schematic v24.12.1

Update annotationProperty.csv

9de1831

Add data type qualifier to description

Add DOI attributes for components

f273ecf

Added DOI attributes for Dataset View, Tool View, and Educational Resource models Also added Keys to Educational Resource, but retained tool and dataset link attributes, in case the ed res uses resources from outside of Synapse

Mark DOI attribute as required for resources

39e1b71

Import CDS valid values

7bdc434

For mapping and conversion

Bankso added 26 commits February 7, 2025 17:07

Update tissueOrganOriginCDS.csv

e40e406

Align valid values from CDS v5.0.4

Update treatmentOutcome.csv

b2e2701

Align valid values from CDS v5.0.4

Update treatmentType.csv

16b4267

Align valid values from CDS v5.0.4

Update tumorGrade.csv

d5d7baf

Align valid values from CDS v5.0.4

Update deIdMethod.csv

28d93aa

Align valid values from CDS v5.0.4

Update indexDate.csv

35d2708

Align valid values from CDS v5.0.4

Update studyLicense.csv

ffdf76e

Align valid values from CDS v5.0.4

Remove unused terms

a334c19

These are stored in visium-specific folders

Update annotationProperty.csv

e0c2dd4

Remove unneeded attributes, covered by File View template

Remove unneeded files

1b65157

These values live in sequencing-specific folders, CSVs in shared were redundant

Delete resectionBiopsySiteCDS.csv

d4615dc

Equivalent to tissueOrganOriginCDS.csv, so I deleted this file and replaced mapping reference with reference to tissueOrganOriginCDS.csv

Update mapping.yaml

d916219

Align mappings with current set of valid values

Biospecimen updates

720548a

Added Biospecimen Treatment Response, Incidence Type Added valid values for Acquisition method Removed Biospecimen Pathology

Update annotationProperty.csv

8cafc51

Updated by update valid values script

Update annotationProperty.csv

c968185

Updated by update valid values script

Update annotationProperty.csv

2eb167e

Updated by update valid values script

Update annotationProperty.csv

1b3a7c8

Updated by update valid values script

Update annotationProperty.csv

fabf22c

Updated by update valid values script

Update annotationProperty.csv

0539af2

Updated by update valid values script; Moved attributes NGS Raw Reads and NGS Unique Bases from sequencing level 2 to sequencing level 1 and added to level 1 model definition. These are requested by CDS model

Update annotationProperty.csv

32f9698

Updated by update valid values script; removed attributes now in sequencing level 1

Update annotationProperty.csv

083ce94

Updated by update valid values script

Update all_valid_values.csv

f4df02b

Updated by update valid values script

Update mc2.model.csv

26da7cf

via make

Update mc2.model.jsonld

b6a911d

generated via schematic 24.12.1

Update .gitignore

be73e58

Delete cds_vocab directory

9b5ea40

This will be maintained locally until a clear need for storage in the repo is identified

Bankso requested a review from aditigopalan as a code owner February 8, 2025 01:59

Bankso added the major PR label for a major update label Feb 8, 2025

Bankso mentioned this pull request Feb 8, 2025

End of 25-1 data model release #167

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pre-release revisions and alignment with CDS v5.0.4 #169

Pre-release revisions and alignment with CDS v5.0.4 #169

Bankso commented Feb 8, 2025

Bankso commented Feb 8, 2025

Pre-release revisions and alignment with CDS v5.0.4 #169

Are you sure you want to change the base?

Pre-release revisions and alignment with CDS v5.0.4 #169

Conversation

Bankso commented Feb 8, 2025

Changelog

Bankso commented Feb 8, 2025