Add levels to plot benchmarks

[alejoe91]

Replacement for #3301

This adds the option to aggregate levels when plotting benchmark results.

[yger]

Looks good to me, but I'll need to play a bit with it to understand exactly the levels. Each time a function in plot_tools will be added, it will need to handle case keys for grouping then, am I right?

[yger]

Why a special focus on gt_unit_locations? I mean such a function could be great, to compute quantities on the main analyzer, but maybe it could be more generic and allow any extension to be computed? Or otherwise, let's discuss why only gt_unit_locations

[alejoe91]

This is because the the plot_performance_vs_depth_and_snr uses it

[alejoe91]

Changed in last commit to have a more general compute_analyzer_extension and get_gt_unit_locations (which uses the gt_unit_location property if available)

[alejoe91]

@yger let me share some examples. You can check the complex study in the test, which has 8 datasets and 16 cases (tdc2 and sc2).

Now, if you run the plotting functions without levels, you would get 16 cases, which is very messy... instead, let's say I'm only interested in the effect of ["sorter_name"] and ["sorter_name", "processing"] (the last level could be probe, or in my case a different stream/hybrid case).

Here's what you get:

plot_run_times

plot_performances_ordered

performance VS SNR

performance VS SNR+Depth

performance comparison

[yger]

Nice, thanks for sharing the example, this is great. And can you provide a custom palette for your cases? Let say for example I want SC2 to only be orange and TDC2 to only be blue (with different alpha depending on cases). Is it doable ? I'll read it, but this seems very practical to ease the lecture of the graphs

[alejoe91]

Nice, thanks for sharing the example, this is great. And can you provide a custom palette for your cases? Let say for example I want SC2 to only be orange and TDC2 to only be blue (with different alpha depending on cases). Is it doable ? I'll read it, but this seems very practical to ease the lecture of the graphs

Let me add some customization there

[alejoe91]

@yger with the last commit you can specify colors:

Examples:

plot_run_times(
    study,
    levels_to_keep=["sorter_name", "processing"], 
    colors={"no-preprocess": "b", "preprocess": "r"}
)

or:

plot_performances_ordered(
    study, levels_to_keep=["sorter_name", "processing"], 
    colors={
    ("tdc2", "no-preprocess"): "b", ("tdc2", "preprocess"): "c", ("sc2", "no-preprocess"): "r", ("sc2", "preprocess"): "y"}, 
    orientation="horizontal"
)

If you specify wrong colors based on the levels, you'll get this:

plot_run_times(study, levels_to_keep=["sorter_name", "processing"], colors={"no-preprocess": "b", "tdc2": "r"})

Error:

AssertionError: colors must have a color for each palette key: ['no-preprocess' 'preprocess']

[yger]

This looks good to me. It might be out of the scope of this PR, but all these plotting functions in benchmarks are working for Study object. Would it be worth making them also work for single Comparisons? Not sure this is crucial, just a thought

[alejoe91]

You mean for a single Benchmark?

[yger]

Yes, I am not using these syntax anymore and maybe this will be deprecated, but for a while we had functions such as plot_performances(comp), where comp was a single Benchmark (GTComparison for example ?). But i should read the code again, maybe this is now a Study and all these plots in widgets will still work

[alejoe91]

Ok, extended this also to plot_unit_counts: