Kilosort takes extremely long to sort 4-shank Neuropixel recording

[guidomeijer]

Hi there, I recently switched to KS4 and got good results sorting a NP1 recording. However, when trying on a NP2 4-shank recording it takes an extreme amount of time. The sorting runs in 8 hours but the step that comes after (recalculating spike templates) would take an estimated 160 hours. And this is for only one of the four shanks. The only thing I noticed is that the allocated GPU memory was a bit low (700 MB), this was much higher when sorting the NP1 recording. I have a local installation of Kilosort which is launched by SpikeInterface and run using the run_sorter_by_property function to run the sorting per shank. I use all the default settings of Kilosort.

Setup:
NVIDIA RTX 4080
64 GB RAM
Ubuntu 20.04
kilosort 4.0.2
spikeinterface 0.100.2

Any ideas would be appreciated!

[jacobpennington]

Hello,

For multi-shank probes, we've been recommending that people artificially stack contacts vertically in the probe layout, treating them as a single shank with a ~100 micron gap between shanks. This will remove the need to sort shanks one at a time. Others have encountered long sorting times before doing this (as well as some related errors), and this temporary workaround fixed the issue. We'll be updating the code in the near future to explicitly handle separate shanks so that this won't be necessary.

As for why it would be running for so long when sorting a single shank, I don't know. It may have something to do with how SpikeInterface is running the sorter, but that would be a question for their team. I would recommend you try running Kilosort directly (with the change suggested above), instead of through SpikeInterface, if you continue to have issues.

[guidomeijer]

Hi Jacob, thanks for your reply. Your suggestion works great, it runs in a couple of hours now!