Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decoding pipeline #731

Merged
merged 117 commits into from
Jan 10, 2024
Merged

Decoding pipeline #731

merged 117 commits into from
Jan 10, 2024

Conversation

edeno
Copy link
Collaborator

@edeno edeno commented Dec 21, 2023

Description

This PR will:

  • update the existing pipeline to use merge tables
  • have a table to store the decoding results
  • use the new non_local_detector package for decoding

Checklist:

  • This PR should be accompanied by a release: (yes/no/unsure)
  • (If release) I have updated the CITATION.cff
  • I have updated the CHANGELOG.md
  • I have added/edited docs/notebooks to reflect the changes

To-do:

  • Position Group for multiple position entries
  • Waveform Group for grouping clusterless marks
  • Sorted Spikes Group for grouping sorted units
  • Convert model classes into dictionaries and back again
  • Write file cleanup for .nc files
  • Write notebooks for each step of the pipeline
  • Document 1D decoding requires setting track_graph, track_order, track_spacing in environment variable
  • Document the difference between estimate parameters and not estimate parameters
  • Save initial conditions and discrete transition matrix (possibly just in the xarray)
  • Update notebook on using GPUs with decoding
  • Load spatial environments easily
  • Add section on decoding visualization with figurl in notebook
  • Decoding parameters override insert
  • Test adding track_graph to Environment and storing in decoding table
  • Test on production data to make sure there is a reasonable decode

@edeno edeno requested a review from CBroz1 December 21, 2023 21:34
@edeno
Copy link
Collaborator Author

edeno commented Dec 21, 2023

@CBroz1 would you mind taking a look at src/spyglass/decoding/v1/waveform_features.py? The saving of the waveform features is still not working but the rest should be good to review.

@samuelbray32
Copy link
Collaborator

  • Added functions to the SpikesortingOutput table to make it easier to fetch information from upstream tables.
  • Used these and other changes to allow UnitWaveformFeatures to accept from the spikesorting v0 pipeline

@edeno
Copy link
Collaborator Author

edeno commented Jan 9, 2024

@samuelbray32 was able to get existing data through to decoding.
Clusterless:
image
Sorted Spikes:
image

@edeno edeno requested a review from CBroz1 January 9, 2024 19:09
Copy link
Member

@CBroz1 CBroz1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I think the last step is to make a decision about #749 - to add, add/tweak and place elsewhere, or to close

src/spyglass/decoding/v1/clusterless.py Outdated Show resolved Hide resolved
src/spyglass/decoding/v1/clusterless.py Outdated Show resolved Hide resolved
src/spyglass/decoding/v1/sorted_spikes.py Outdated Show resolved Hide resolved
src/spyglass/decoding/v1/waveform_features.py Outdated Show resolved Hide resolved
src/spyglass/spikesorting/imported.py Outdated Show resolved Hide resolved
src/spyglass/spikesorting/imported.py Outdated Show resolved Hide resolved
@edeno
Copy link
Collaborator Author

edeno commented Jan 10, 2024

Looks good. I think the last step is to make a decision about #749 - to add, add/tweak and place elsewhere, or to close

I would like to merge this in master as is and then in a separate PR think about how to add that since I haven't tested it out.

@edeno edeno removed the request for review from khl02007 January 10, 2024 17:30
@edeno edeno merged commit 582e993 into master Jan 10, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add clusterless / clustered decoding parameter schema and decoded data schema
3 participants