Discussion on structure and auto-building of documentation

[JoeZiminski]

Further to the previous meeting, I thought we could discuss here what to do about stucture and automatic building for the documentation.

I think we agreed that:

• it would not be possible to change the examples such that they all build quickly, so some kind of method of building the docs with / without the long-to-build docs would be necessary
• We should build custom index.rst pages that can link to both sphinx-built and non-sphinx-built documentation for How To's and Tutorials (i.e. we will not use the auto-genereated sphinx gallery)

In terms of automation options, two possibilities are:

• Replace the current manual-building of how-to with a similar system, but where these are generated automatically. This is proposed in Try "Handle motion/drift" documentation as a sphinx gallery #2879 (requiring some rejigging so the sphinx outputs are not pushed to git).
• A better option that we touched on would be to use the sphinx-build tags to edit the keyword sphinx uses to build the .py files. At the moment there are two modes - .py files prefixed with plot_ are run in full when converted to .rst. Otherwise, the python code is not actually run and the .py file is converted to .rst (so you get no plots or code output). It would be cool to use the tag like sphinx-build -b html doc doc/build -t long-time-builds that changes the keywords used to fully build the long-docs. For example it could extend the plot_ prefix to also include long_build. Then, when the tag is not used, these long-build docs will just be converted to .rst only (i.e. without running the python code) and it will be fast. But when properly building the docs, we can use the tag to ensure that they are build with running the python code and images etc. are generated.

Handling docs with heavy data / environment requirements

• There are some cases where GPU's are required (e.g. comparing across sorters) or when difficult-to-obtain data is used. These are very difficult / impossible to auto-build. For this, there was some discussion about having a separate docs section, 'blog' or 'showcase' or something. These are dated / versioned and it is made clear the API used in code examples is from a specific SI version, and that these are not mean to be run-along. These can also link to corresponding tutorials if people want to try actually running similar examples. That's nice as we will never need to maintain these difficult-to-run docs going forward.

Blockers

• It would be nice as a first step to move the existing large how-to's (e.g. drift, runing neuropixels) to the auto-build method we decide on. This requires re-writing the docs into .py file for sphinx-gallery format. That's okay, but where the documentation requires a complex environment / other data it is not possible to the CI to build build so it's also necessary to rewrite parts of these documentation to use synthetic data at the same time as introducing the new build machinery. But, in some cases (drift docs) this changes the interpretation of the data and so a lot of the docs article changes. In this case, the PR like Try "Handle motion/drift" documentation as a sphinx gallery #2879 becomes bloated with sphinx-machinery changes and article-content changes. But I'm not sure of the best way around this.

[zm711]

I plan to read and comment soon, but I have a presentation I'm working on at the moment. Feel free to ping me on this @JoeZiminski next week :)

[chrishalcrow]

I don't think we should worry about replacing real data with synthetic data. I think it's only 3 tutorials we'd need to make alterations to (motion, neuropixles and quickstart) and there are ways we can make up for the real data (link to seminars/papers which include real data? Make and link to a blog?). Replacing the mearc downloads in the other tutorials with generated data is also simple.

Your solution in #2879 is great, though the build tags solution does sound simpler. Seems like Sphinx suggests something like that here https://sphinx-gallery.github.io/stable/configuration.html#parsing-and-executing-examples-via-matching-patterns ?