Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Hanning filtering #3604

Open
wants to merge 85 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
6b9350c
Skipping peaks if enough have been collected
yger Nov 4, 2024
da9bd9d
fix
yger Nov 4, 2024
7f89e87
Merge branch 'SpikeInterface:main' into sc2_recording_slices
yger Nov 5, 2024
7ff0d12
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 5, 2024
611170d
Docs and cleaning n_jobs within sc2
yger Nov 13, 2024
9f86617
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 13, 2024
019b399
Merge branch 'clustering_components_api' into sc2_recording_slices
yger Nov 13, 2024
dfd9bff
Merge branch 'clustering_components_api' into sc2_recording_slices
yger Nov 13, 2024
bd525b2
Exclude 32 chan probe by default for motion correction
yger Nov 13, 2024
fd9ee0b
One pass to get the prototype
yger Nov 15, 2024
8e37d54
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 15, 2024
1c8443f
Verbose flag
yger Nov 15, 2024
80cca25
WIP
yger Nov 15, 2024
801a362
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 15, 2024
48026e3
WIP
yger Nov 15, 2024
472022d
Merge branch 'sc2_recording_slices' of github.com:yger/spikeinterface…
yger Nov 15, 2024
d3bee9a
Fixes for the clustering
yger Nov 15, 2024
0865dcd
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 15, 2024
f4ff9ab
WIP
yger Nov 19, 2024
28229d8
WIP
yger Nov 19, 2024
70673ce
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 19, 2024
6002da4
WIP
yger Nov 21, 2024
2310811
WIP
yger Nov 21, 2024
0054974
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 21, 2024
9c61e71
Merge branch 'main' into sc2_recording_slices
yger Nov 21, 2024
2885aeb
WIP
yger Nov 22, 2024
94cae34
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 22, 2024
3b39a05
job_kwargs propagated everywhere
yger Nov 22, 2024
c5fecf4
Merge branch 'sc2_recording_slices' of github.com:yger/spikeinterface…
yger Nov 22, 2024
28431f4
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 22, 2024
24b22da
Removing noise templates
yger Nov 22, 2024
0b7674c
Merge branch 'sc2_recording_slices' of github.com:yger/spikeinterface…
yger Nov 22, 2024
ce8432a
WIP
yger Nov 22, 2024
a396e1b
Skipping waveforms re-detection if matched filtering
yger Nov 27, 2024
cb560e5
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 27, 2024
7c5bce7
In case of both peak detection, waveforms should be aligned
yger Nov 27, 2024
047cac4
Merge branch 'sc2_recording_slices' of github.com:yger/spikeinterface…
yger Nov 27, 2024
94c63f3
WIP
yger Nov 27, 2024
d39874f
Merge branch 'SpikeInterface:main' into sc2_recording_slices
yger Nov 28, 2024
0d1907f
Merge branch 'sc2_recording_slices' of github.com:yger/spikeinterface…
yger Nov 28, 2024
57a9cc9
Whitening
yger Nov 29, 2024
02cad49
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 29, 2024
29fda81
Merge branch 'main' of github.com:spikeinterface/spikeinterface into …
yger Dec 4, 2024
b9af739
Merge branch 'main' of github.com:spikeinterface/spikeinterface into …
yger Dec 5, 2024
06711d1
Merge branch 'SpikeInterface:main' into sc2_recording_slices
yger Dec 11, 2024
d18c2ef
Merge branch 'sc2_recording_slices' of github.com:yger/spikeinterface…
yger Dec 19, 2024
a6e3c26
WIP
yger Dec 19, 2024
1de6503
WIP
yger Jan 8, 2025
c51fd1a
Merge branch 'main' of github.com:spikeinterface/spikeinterface into …
yger Jan 8, 2025
e4d7e72
Harmonization of get_prototype_and_waveforms
yger Jan 8, 2025
158347c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jan 8, 2025
437d8b4
Docstrings and cosmetics
yger Jan 8, 2025
2cad313
Docstrings
yger Jan 8, 2025
c2cacd9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jan 8, 2025
1272008
Merge branch 'main' into sc2_recording_slices
yger Jan 8, 2025
0e8603d
Merge branch 'main' of github.com:spikeinterface/spikeinterface into …
yger Jan 8, 2025
6119609
Merge branch 'sc2_recording_slices' of github.com:yger/spikeinterface…
yger Jan 8, 2025
fb9747d
Refactoring get_prototype
yger Jan 9, 2025
6b3aba1
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jan 9, 2025
e839894
Docstrings
yger Jan 9, 2025
731c713
Merge branch 'sc2_recording_slices' of github.com:yger/spikeinterface…
yger Jan 9, 2025
7c32a65
Removing artefactual templates due to matched filtering
yger Jan 9, 2025
c1c8b06
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jan 9, 2025
f994cd8
Fix
yger Jan 9, 2025
472b99c
Merge branch 'sc2_recording_slices' of github.com:yger/spikeinterface…
yger Jan 9, 2025
7cdb239
Cleaning and adapting code for clustering
yger Jan 9, 2025
efbb9a7
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jan 9, 2025
05507ab
Adding Hanning filtering
yger Jan 9, 2025
4a22c9b
Merge branch 'SpikeInterface:main' into hanning_filter
yger Jan 13, 2025
6d5d180
Merge branch 'SpikeInterface:main' into sc2_recording_slices
yger Jan 13, 2025
b901241
Fixing engine
yger Jan 13, 2025
70be7a4
Typos
yger Jan 13, 2025
50123d1
Hanning filter not centered
yger Jan 13, 2025
db6b8d2
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jan 13, 2025
46eba7b
Revert split related stuffs
yger Jan 13, 2025
13fa06c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jan 13, 2025
5c1983c
Fix
yger Jan 13, 2025
64e6e67
WIp
yger Jan 13, 2025
1805ab6
Merge branch 'sc2_recording_slices' into hanning_filter
yger Jan 13, 2025
db801a9
WIP
yger Jan 13, 2025
6b12f41
Merge branch 'hanning_filter' of github.com:yger/spikeinterface into …
yger Jan 13, 2025
23dfa93
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jan 13, 2025
1668fcb
Merge branch 'main' into hanning_filter
yger Jan 14, 2025
98610fe
Sync
yger Jan 15, 2025
3bf97e9
Params conflict
yger Jan 16, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions src/spikeinterface/sortingcomponents/clustering/circus.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
from spikeinterface.core.recording_tools import get_noise_levels, get_channel_distances
from spikeinterface.sortingcomponents.peak_selection import select_peaks
from spikeinterface.sortingcomponents.waveforms.temporal_pca import TemporalPCAProjection
from spikeinterface.sortingcomponents.waveforms.hanning_filter import HanningFilter
from spikeinterface.core.template import Templates
from spikeinterface.core.sparsity import compute_sparsity
from spikeinterface.sortingcomponents.tools import remove_empty_templates
Expand Down Expand Up @@ -125,11 +126,13 @@ def main_function(cls, recording, peaks, params, job_kwargs=dict()):
radius_um=radius_um,
)

node2 = TemporalPCAProjection(
recording, parents=[node0, node1], return_output=True, model_folder_path=model_folder
node2 = HanningFilter(recording, parents=[node0, node1], return_output=False)

node3 = TemporalPCAProjection(
recording, parents=[node0, node2], return_output=True, model_folder_path=model_folder
)

pipeline_nodes = [node0, node1, node2]
pipeline_nodes = [node0, node1, node2, node3]

if len(params["recursive_kwargs"]) == 0:
from sklearn.decomposition import PCA
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
import pytest


from spikeinterface.sortingcomponents.waveforms.hanning_filter import HanningFilter

from spikeinterface.core.node_pipeline import (
PeakRetriever,
ExtractDenseWaveforms,
run_node_pipeline,
)


def test_hanning_filter(generated_recording, detected_peaks, chunk_executor_kwargs):
recording = generated_recording
peaks = detected_peaks

# Parameters
ms_before = 1.0
ms_after = 1.0

# Node initialization
peak_retriever = PeakRetriever(recording, peaks)

extract_waveforms = ExtractDenseWaveforms(
recording=recording, parents=[peak_retriever], ms_before=ms_before, ms_after=ms_after, return_output=True
)

hanning_filter = HanningFilter(recording=recording, parents=[peak_retriever, extract_waveforms])
pipeline_nodes = [peak_retriever, extract_waveforms, hanning_filter]

# Extract projected waveforms and compare
waveforms, denoised_waveforms = run_node_pipeline(recording, nodes=pipeline_nodes, job_kwargs=chunk_executor_kwargs)
assert waveforms.shape == denoised_waveforms.shape
47 changes: 47 additions & 0 deletions src/spikeinterface/sortingcomponents/waveforms/hanning_filter.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
from __future__ import annotations


from typing import List, Optional
import numpy as np
from spikeinterface.core import BaseRecording
from spikeinterface.core.node_pipeline import PipelineNode, WaveformsNode, find_parent_of_type


class HanningFilter(WaveformsNode):
"""
Hanning Filtering to remove border effects while extracting waveforms

Parameters
----------
recording: BaseRecording
The recording extractor object
return_output: bool, default: True
Whether to return output from this node
parents: list of PipelineNodes, default: None
The parent nodes of this node
"""

def __init__(
self,
recording: BaseRecording,
return_output: bool = True,
parents: Optional[List[PipelineNode]] = None,
):
waveform_extractor = find_parent_of_type(parents, WaveformsNode)
if waveform_extractor is None:
raise TypeError(f"HanningFilter should have a single {WaveformsNode.__name__} in its parents")

super().__init__(
recording,
waveform_extractor.ms_before,
waveform_extractor.ms_after,
return_output=return_output,
parents=parents,
)

self.hanning = np.hanning(self.nbefore + self.nafter)[:, None]
yger marked this conversation as resolved.
Show resolved Hide resolved
self._kwargs.update(dict())

def compute(self, traces, peaks, waveforms):
denoised_waveforms = waveforms * self.hanning
return denoised_waveforms
Loading