Desired structure for examples in the code base

[zm711]

Inspired by #2316,

There are two main places to put actual examples in the documentation:

The modules_gallery or the how_to. It is currently unclear what the separate goals for these two sections are/ought to be. Either way both are stored in examples, but modules_gallery are run by sphinx-gallery whereas how_to are run locally (by Sam or Alessio) and added to the documentation later.

Topics of Discussion

What are the purposes for these two separate example streams?
Are they currently appropriately discoverable (ie do they need better names)?
Do they need to be improved (ie even we are keeping the modules_gallery what improvements need to be made to make it as useful as possible)?
What rules (if any) do developers need to know to provide examples for the documentation code base?

Outputs

If there is a clear consensus/set of rules we can add this to the developer docs so we can keep track of this and try to ensure our examples documentation follows a rough set of guidelines (even if they are not hard rules)

[samuelgarcia]

Thanks for this clear summary.

Clearly the organisation is not good and the main purpose was technical because the sphinx-gallery is making a separate folder.

[JoeZiminski]

Thanks! Overall I think the docs are very comprehensive and contain a lot of important information. Mostly I would suggest structural changes so that users can quickly find relevant information related to their use case. As a relatively naive user of the documentation I would suggest the following changes, mostly based on the [https://diataxis.fr/] approach

1. Keep the 'How to' section and a 'Tutorials' section and fold some existing pages into these. I think for the most part the module gallery examples can be renamed and form additional 'How To's. Conceptually, tutorials and 'how to's' are distinct. 'How To' are short, recipes to tell you how to do specific things (e.g. the current 'append and or / concatenate segments' becomes 'How to analyse multi-segment recordings'). Tutorials are longer, covering a complete workflow from start to finish (I believe SI already has some of these in a repo). For example there may be 2-3 main tutorials that go into a lot of detail. Then the 'How to' sections can be very targetted but not go into a lot of detail / background.

2. I would suggest re-pinning sphinx version to a more recent one e.g. 7.2. There have been many cool new features (e.g. dropdowns, as far as I can tell because they didn't work when I tried in 5.1.1. Nor some newer sphinx-gallery features. That been said, I am not 100% sure the problems I encountered were due to the version but I am pretty sure. This may be a nightmare, but I guess it will have to be done at some point anyway.

3. Remove or move some less important sections from the front page. I think a problem with the docs is that there is (necessarily) a lot of information, but it is not clear as a new user where to go. From a user's perspective, it makes sense for things to be organised in terms of finding the information for specific use cases, rather than how the package is structured in terms of the SI implementation.

Thing I would suggest are removed from the front page are:

• Importing Spikeinterface (move to installation?)
• Installing Spike Sorters (moved to installation?)
• Viewers (not sure where moved)
• Then maybe (if feasable) modules documentation and modules example gallery folded into 'tutorials' and 'how to'

Would be happy to help with any of these!

[zm711]

I think 1 is a great idea and 2 I think is more related to sphinx's own recommendation to pin once you get things working, but it's easy enough to do a test PR with it pinned to a recent version just to see how things look.

For 3 maybe we create a section called Getting Started which contains

• importing spikeinterface
• installing spikeinterface
• installing spikesorters

Then we have a Tutorials and Examples section (as you described), but since the module_gallery is created by sphinx-gallery combining that with How to/tutorials will just require some toctree troubleshooting.

I personally still like having a narrative section. So for me I would like to have the modules documentation in its narrative form as a separate section. I think some duplication between the narrative docs and the example docs would be nice. For me I don't need to see examples, but I always go to the Widgets docs to remember the names of functions.

So in my dream world the index would be:

Overview
Getting Started
Modules Documentation
Examples and Tutorials
API
Developer
Release Notes
Contact us

I also don't know where Viewers would go.

[h-mayorquin]

I think that the core principle is that documentation (and decisions about documentation) should be made in terms of how they will be used and not in terms of how we are producing it. The latter is a programming problem that we can solve with scripts or CI and coupling these two concepts has if they were one has lead to the current confusion.

@JoeZiminski said:

Keep the 'How to' section and a 'Tutorials' section and fold some existing pages into these. I think for the most part the module gallery examples can be renamed and form additional 'How To's. Conceptually, tutorials and 'how to's' are distinct. 'How To' are short, recipes to tell you how to do specific things (e.g. the current 'append and or / concatenate segments' becomes 'How to analyse multi-segment recordings'). Tutorials are longer, covering a complete workflow from start to finish (I believe SI already has some of these in a repo). For example there may be 2-3 main tutorials that go into a lot of detail. Then the 'How to' sections can be very targetted but not go into a lot of detail / background.

I agree with this and I will add that having a framework like the one proposed by Daniele Procida and already used in projects like Django will avoid a lot of discussion among us of where things should be:

https://www.writethedocs.org/videos/eu/2017/the-four-kinds-of-documentation-and-why-you-need-to-understand-what-they-are-daniele-procida/

[zm711]

@h-mayorquin,

I actually support this (in case that wasn't clear). For me I meant more from an indexing perspective I would like it to be split at a first level as:

Tutorials and How tos
Explanation and Reference

Or if you prefer:

Practical
Theoretical

Right now I think how_to (more structured like traditional full tutorials) vs gallery (which are examples, but also easy quick how_to)
makes it hard for users to discover where to go. Our modules doc is set up mostly as explanation and the API page is reference. So I think we have all the types of docs, but I don't think we have them indexed in the best way. Does that make sense?

[zm711]

@h-mayorquin, just wanted to bring this back on your radar. With the big api change maybe we can also get @chrishalcrow's opinion on re-organizing some of the documentation to best facilitate end-user learning. :)

[chrishalcrow]

I've also been learning to use spikeinterface in the past couple of months, so just had the "new user" experience!

I think there's a consensus that we should have a Tutorials page with longer form learning guides. Then "how to"s which are shorter. To me, the current modules documentation seems closer to the "Explanation" part of in @h-mayorquin 's graph. Then the API is a full reference guide. So my proposal would be

1. Tutorials
   • Installation
   • Getting Started
   • Specific, longer form stuff.
2. How to's / Gallery
   • Most of the stuff in modules example gallery
   • Cool stuff
3. Module documentation
4. API

And these four cover the four quadrants of the graph.

Or my reading of @zm711 's proposal is something like:

1. Practical
   • Tutorials
   • How to's
2. Theoretical
   • Module documentation
   • API

Which works too!

Just to explain the sphinx-build complications. In the tutorials (and maybe other places), you often want to use some real dataset to show off some features. And sometimes do some long calculations. We don't want these to become part of the CI, so at the moment we write a .py file, convert it to a jupyter notebook, run it, save the output, convert it to .rst all locally. Then put that (+ the output'd figures etc) in the doc/how_to folder. At the moment we separate this type of output from the simpler output, but there's no technical reason to do this (I believe).

We should definitely update sphinx! I've been playing with some fun sphinx-design stuff too! (TABS!
)

I also think we should improve the "how to contribute" documentation for people with little git experience (me!). Neo has a nice page: https://neo.readthedocs.io/en/latest/contributing.html which I've been using.

If we agree on an overall structure, I have time to work on this stuff.

[zm711]

I'm fine with either layout. Probably the first is a bit closer to what I would expect the toctree to be

Tutorials
Installation
Getting Started
Specific, longer form stuff.
How to's / Gallery
Most of the stuff in modules example gallery
Cool stuff
Module documentation
API

Maybe @samuelgarcia and @alejoe91 can weigh in if they feel strongly against it?

I also think we should improve the "how to contribute" documentation for people with little git experience (me!). Neo has a nice page: https://neo.readthedocs.io/en/latest/contributing.html which I've been using.

I think @h-mayorquin did a big push for that and I did the most recent update, but if you have more suggestions I'd be happy to hear them. I think eventually it'd be nice to be able to say make a dev environment run pytest and you're good to go. But that doesn't work for windows yet, so I also want to be careful how much we promise there. I don't even know who wrote neo's... but kudos to them!

Finally,

Just to explain the sphinx-build complications.

Maybe @h-mayorquin has a better idea but couldn't we do like a cron job of running this. So once a month (a quarter ou trimestre) it refreshes the examples and then opens a PR that we could merge or close. Right now the big tutorials get some edits but don't necessarily get rerun, so we've had stale tutorials for a while (before your big push @chrishalcrow!). I think a cron opening a PR will at least force us to not let those get stale.

[zm711]

Also closing this. Let's move future discussions of this over to #2800 since we've completed the broad reorganization and now everything else will be more granular changes.

[JoeZiminski]

Hey @zm711 would you mind reopening this for some additional discussion I'd like to have on How To vs. Tutorials?

At present (on main branch) these are split based on whether they are built with sphinx-gallery or not. However thematically some are how-to and some are full tutorials (I have my take on their split here, although how I think handling-drift is a tutorial).

This is a little bit tricky because sphinx-gallery by default uses a 'gallery examples' frontpage that displays cards that link to each individual sphinx-gallery page. In theory this makes it difficult to mix-and-match sphinx-gallery pages and non-sphinx-gallery pages linked to from the same page.

For the 'Tutorials', I think it makes sense that these are all sphinx-gallery pages. For 'How To' however, some necessarily contain a reasonable amount of code while others have no code. I think it is best to use sphinx-gallery for anything with code, but it is a pain to use them for non-code pages because the formatting is kind of annoying and you lose some intrinsic sphinx features.

For the 'How To' page I might suggest we make our own index.rst with cards that we can link to either sphinx-gallery pages or non-sphinx gallery pages. For this we lose the auto-generated frontpage sphinx gallery but it is quite simple to set your own links to the pages sphinx creates, as at the end sphinx-gallery just creates an .rst.

There are a few long-form tutorials with a lot of code in the 'How To' section (like the handling drift) that are manually built. I would suggest converting these to sphinx-gallery pages.

Therefore a plan could be:

1. Convert all manually-built code pages into sphinx-gallery pages
2. Ensure anything that is thematically a 'tutorial' is a sphinx-gallery page and have the tutorials section be a normal sphinx gallery page with the autogenerated sphinx-gallery examples frontpage.
3. Anything that is a 'How To' is made either a normal sphinx page (if it has no / very little code) or a sphinx-gallery page (if it has code and code-generated images). The index for this page will be a custom index page with cards, that link to the various how-to articles.

I'm very happy to get started on this ASAP if there is agreement.

[zm711]

Sure @JoeZiminski I'll reopen for discussion!

[h-mayorquin]

Convert all manually-built code pages into sphinx-gallery pages

Agree strongly with this. I think is only one thing, right? It is also making the diff worse because we need the images there.

This is a little bit tricky because sphinx-gallery by default uses a 'gallery examples' frontpage that displays cards that link to each individual sphinx-gallery page. In theory this makes it difficult to mix-and-match sphinx-gallery pages and non-sphinx-gallery pages linked to from the same page.

Can we change this? I would not like a sphinx-gallery default to determine the way that we structure our documentation. That is, maybe we like the format of sphinx-gallery presentation and want everything to look like it but we shouldn't have to if we don't want or is not the best.

[JoeZiminski]

I agree with this, I played around with building a custom index 'gallery-like' page that can link to both normal pages and sphinx-gallery pages and it worked well. I'll open a PR to start on (1) as I don't think there will be much disagreement on it, and do an example for (2) and (3).

[chrishalcrow]

I agree with the plan @JoeZiminski ! Sounds great.

A (much worse!) version of the idea, which tries to keep the auto-generated sphinx frontpage, is currently seen on the Tutorials page: https://spikeinterface.readthedocs.io/en/latest/tutorials/index.html
which contain a normal page From WaveformExtractor to SortingAnalyzer and the galleries.

I'm not a huge fan of the sphinx-gallery cards, so I like that we'll have some styling control of them on the HowTo page!

[JoeZiminski]

That's nice I think that's very useful too, if there are many tutorials or how-tos all on one page it might be useful to split them up and link between them as text links! I'll push a PR soon on converting the article on working with drift correction and see what people say.

[zm711]

Are we done with this issue for now? ie should we open a fresher issue for newer discussions?

[JoeZiminski]

I think not quite, once #2881 is solved it will unlock everything related to this as it will allow us to place pages on How To / Tutorials however we like. I believe the approach outlined in #2879 is not optimal and from our discussions the best approach is to have everything as sphinx-gallery using simulated data and then build / don't build certain files using the sphinx machinery based on the tags (i.e. for fast local building). I'm really keen on this, it's definitely on the TODO list but new things keep pushing it further down 😅