-
Notifications
You must be signed in to change notification settings - Fork 45
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tidy position #870
Merged
Merged
Tidy position #870
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
edeno
approved these changes
Jun 25, 2024
edeno
approved these changes
Jun 26, 2024
7 tasks
CBroz1
added a commit
to CBroz1/spyglass
that referenced
this pull request
Oct 22, 2024
edeno
pushed a commit
that referenced
this pull request
Oct 29, 2024
* Use provided epoch * Save video to temp dir * Remove open-cv support * WIP: Multithread, RAM hungry * Limit number of workers * Save file images in batches * Reduce RAM cost, remove cv2 dep * Update changelog * Get debug arg from params * Revert merge error #870, #975 * Adjust for final frame. Resume from existing * Resume from fail * except IndexError for final frame * Delay delete temp files until complete * Explicit error messages * Return video object for debugging
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR primarily seeks to reduce complexity in the position pipeline to prepare for future expansion. To that end ...
pyproject.toml
specifies the headless version ofopencv
for testingmake_video
position/position_merge.py
- removed unused imports as holdover from Remove classes for usused tables #1003position/v1/
...dlc_decorators.py
served limited utility to enforce thesource
enum present inDLCModelSource
dlc_utils.py
OutputLogger
context was replaced by afile_log
decorator that serves the same purpose without requiring the additional indent_convert_mp4
had some redundancy eliminated by extracting components into separate functionsdlc_utils_makevid.py
was added to collapse similar functions across DLC and Trodes implementationsdlc_reader.py
- Complex class that should be refactoreddlc_utils.py
- directory lookup functions that are holdovers fromelement_deeplabcut
common_usage.py
Changes elsewhere include...
tests
directoryAssertionError
->FileNotFoundError
)populate_all_common
- while debugging theVideoFile
ingestion process for the test files, Iget_config
.get_config
now caches that a config does not exist, so spyglass doesn't attempt to look up the config and warn on fail to find many times in ingestget_config
also accepts an optional arg to determine which table called for the config filePositionIntervalMap
, related topop_all
redundancy, discussed in Populate should leave evidence of success state #849convert_epoch_interval...
changed to check for null entries, and delete/repopulate onpopulate_missing
This PR depends on two features fetched from #1002
dj_helper_fn.py:L264
-fetch_nwb
limits thequery_table
restriction to only the pk to avoid a restriction with a python dict being run in SQLtests/data_downloader.py
- refactor of how towait
for downloads in progressChecklist:
CITATION.cff
CHANGELOG.md