Re-enable n_jobs in multi-comparison

[alejoe91]

@paul-aparicio I realized that the n_jobs parameter was actually not used.

This PR adds it back and updates the parallelization using the ProcessPoolExecutor. This will run each comparison in parallel.
A simple test shows that the results are >10x faster (on my machine - 8 CPUs).

Can you test this PR? you just need to add the n_jobs=-1 to the compare_multiple_sorters

[paul-aparicio]

Absolutely! Thank you so much

[alejoe91]

Refactor this

[paul-aparicio]

Hi, just to make sure I am doing the right thing (sorry for lack of experience), I should use gh pr checkout 3747 and then rerun the analysis with n_jobs=-1 and agreement_method='distance', is that correct?

[alejoe91]

As long as you have spikeinterface installed from sources, yes! You can also try the other agreement method (counts), since this will parallelize over comparisons across sorters

[paul-aparicio]

Hi, I completed my tests. I did not find much difference, maybe I downloaded the incorrect patch? I tested agreement_method='counts' (took ~7.5 hours to complete both before and after) and agreement_method='count' (took ~7.75 hours to compete before and after the change). I am supposed to patch with #3747? Thanks!

[alejoe91]

How did you install from this PR? My suggestion is:

pip uninstall -y spikeinterface
git clone https://github.com/alejoe91/spikeinterface.git
git checkout multicomp-parallel
cd spikeinterface
pip install -e .
cd ..