Select too many columns and it just breaks

[mccalluc]

Construct a CSV with 20 columns and just keep adding them. Shiny can't keep up and the UI becomes unresponsive.

I think (but haven't tested this hypothesis) that plot generation is expensive.

• Display only one preview plot at a time?
• Hard limit on the number of columns that can be selected?
• Less recalculation? Make things lazier / less reactive?
• More caching of results?

[mccalluc]

Started a PR

• Add a warning about a large number of columns #289

but I wonder if part of the solution is non-blocking operations with Shiny. Need to do more research.

[mccalluc]

There are at least two problems using non blocking operations. 😦

They are intended as a wrapper for calculations: What was reactive is now called as a regular function. Since the slow part for us is the plot, I'm not sure how to change this to a function invocation:

ui.output_plot("histogram_preview_plot", height="300px"),

But even if that worked, while non blocking operations don't block other code, they still block themselves:

An extended task can run concurrently to reactive code and to other extended tasks–that’s its whole purpose. However, a single extended task object cannot run itself multiple times concurrently. If you call sum_values() while it is already running, it will enqueue the new invocation and run it after the first one completes.

They acknowledge that this is often not desired: The fix is a widget that blocks the user from enqueing more operations. I don't know that a UI that blocked user operations would be any better than one where they get backlogged.

👉 New idea: There are many plotting libraries out there: speed might not usually be the deciding factor, but worth looking at in this case.

[mccalluc]

No obvious switch that just makes it faster. possible next steps

• Time the rendering code: if we'll be making experiments to speed this up, we need numbers. Playing with the UI and seeing how it feels is not enough.
• Switch out matplotlib backends. Does it make a difference?
• Profile the whole thing: I've assumed plot_histogram is the bottleneck, but not super confident
• Is it incrementally updating the plot? That could be expensive. Make sure we're only rendering once.