Cut the preamble at the first top-level heading

[Julow]

Closes #235 and #631

If the preamble contains a level 1 heading, this heading and the rest of the preamble is moved out of the 'header' to the 'content' of the page.
That part of the preamble is rendered after the TOC and appears in it.

Heading of level 2 or higher are kept in the preamble.

cc @Drup, @dbuenzli

[dbuenzli]

Heading of level 2 or higher are kept in the preamble.

It should be no heading. The idea is that the preamble never has any heading, it's what comes before the implicit level 0 heading (module or .mld page title) and the first heading. Basically the preamble is like an abstract, only a sequence of block level items.

If you do it the way it is in this PR then these headings won't appear in the toc (and you'll soon get a duplicate of #235 with a lower level which people do not apply religiously) or if you make them appear in the toc it will be absurd because typically when you downgrade to a mobile experience the toc comes after the preamble like this:

abstract
toc
contents

[Drup]

Code looks ok to me.

It should be any heading. The idea is that the preamble never has any heading, it's what comes before the implicit level 0 heading (module or .mld page title) and the first header. Basically the preamble is like an abstract, only a sequence of block level items.

I don't know, I think it's rather nice to be able to have some structure in your preamble. In any case, it's only in the first comment, and the specification is clear. And indeed, they do not appear in the TOC, which is fair.

[Drup]

Could you use labels for everything here ?

Also, comments are already lists, no need to have a list of lists.

There starts to be a lot of those helpers floating around, I think we'll need to put them in a dedicated module at some point.

[Julow]

I don't think labels would be good here, arguments have all different types and some are expected to be literals, for example:

              make_page modname `Mod url [ t.doc; expansion_doc ] items

Also, comments are already lists, no need to have a list of lists.

I wanted to abstract the code that knows what comments are and that concatenate them in a specific way, instead of repeating that everywhere.

I've pushed some refactoring making my intent clearer.

[dbuenzli]

And indeed, they do not appear in the TOC, which is fair.

Well what we are going to have is duplicates of #235 for lower levels... People do not apply these things religiously. Keep things simple and usable, in particular you don't want for people to have to scroll for pages to access the toc when the viewport becomes narrow. As mentioned in #235 take wikipedia pages as an example here.

[dbuenzli]

In any case, it's only in the first comment,

And again comment should not be the unit here. If I have two unattached doc strings paragraphs before the first heading these two paragraphs should go in the preamble.

[dbuenzli]

Just to make things clear. That's the model I'd like to give to odoc users with respect to preambles and tocs:

1. The preamble is an abstract for the module or the .mld page. It is any text that appears before the first heading or ocaml definition in the page. It should be kept short ("the wikipedia page model").
2. The toc is the list of all headings in the module, in order and up to level 3.
3. On narrow viewports the preamble comes before the toc ("the wikipedia page model", but concretely that's up to the stylesheet discretion which can change visual order).

This model is simple, easily discovered and remembered. In particular it doesn't require conditional thinking and enforces good usability. I enjoin you to write down in words the documentation for the model that this PR proposes and compare it with the above and ponder on the fact that it will have to be (likely) discovered and remembered by users.

Regarding page usability, if we allow people to section their preamble they may not realize that on narrow viewports this may push toc access much further down and hamper page usability. It also means that for example on .mld pages if for some reasons someone writes with only level 2 headers all the content will be in the .odoc-preamble and there will be no toc. This means that in general what constitutes the toc and the markup structure for content is dependent on how one uses headings which I personally find to be a terrible idea, especially from a styling perspective (content can now be both in .odoc-preamble and .odoc-content, that was not the idea). As a result this will likely only bring in user complaints that things do not work properly because no-one is going to read the inexisting documentation about how all this works and that the model of this PR is not as evident as it might seem to you.

Now @Drup "finds it nice" to be able to section the abstract, I don't think there's anything wrong with writing these sections in the content, especially on broad viewports, there will be no difference; so I'd like a concrete example that would not be well served by the model I give above which, again, is simple to discover, understand and remember, enforces better usability and leads to a more consistent markup structure for content.

[lpw25]

I think we were picturing a model where the preamble was more like the introduction section of an article, rather than the wikipedia-style lead section. The lead section model is appealing though, and would indeed require cutting the comment at the first heading of any sort.

I disagree about having multiple comments though. I strongly prefer using only the first comment. It allows leaving the preamble without needing a section or OCaml value. It also keeps the mli and the docs in a similar state. Most mli files have no sections and using all comments before an item will lead to a lot of arbitrary stuff being pulled into preambles unintentionally.

[dbuenzli]

I think we were picturing a model where the preamble was more like the introduction section of an article, rather than the wikipedia-style lead section.

I think that in practice if you peruse a bit .mlis you'll mostly find:

• Nothing…
• A synopsis line
• A synopsis line and a few paragraphs

But again in the "wikipedia-style", for those .mlis which then follow with non-1 headings and text it won't make any difference on broad viewports. Only in the narrow ones: the toc will come before these (and it will consistently include all the headings in the toc, which is definitively a good thing).

I don't think you really loose anything w.r.t. vision you had for preambles -- which I have to say I'm not too fond of anyways, I don't want to have to scroll through walls of text to access the api defs (even nowadays that we have a toc). Longer blurbs of text can be deferred to sections at the end of the module and/or .mld files.

I disagree about having multiple comments though. I strongly prefer using only the first comment. It allows leaving the preamble without needing a section or OCaml value.

I could almost be convinced by that but it has two problems in my opinion.

1. There is no such concept in .mld files. So you won't be able to control that in .mld files anyways. In general I prefer if .mld files work the same way as .mli files (now the astute arguer will tell me that an .mld file is just one big comment… so that it's consistent, point taken).
2. If I'm in the middle of a .mli file and write:

   (** Para1 *)
   
   (** Para2 *)
   

   This will yield two paragraphs and markup wise there's no difference with:

   (** Para1 
   
       Para2 *)
   

   So when we detect the preamble I find it bizzare that suddenly this makes a difference, namely that in the first case only Para1 will end up in the preamble and in the second case both Para1 and Para2 will. It breaks my mental model of the ocamldoc language.

If there's really a strong feeling that you want to be able to break an initial sequence of paragraphs I think we should rather introduce an explicit construct like {---}, bonus point that would work in .mld files aswell.

Most mli files have no sections and using all comments before an item will lead to a lot of arbitrary stuff being pulled into preambles unintentionally.

Most poorly documented .mlis (which are plenty) indeed do not have sections. But in any case I suspect that if an .mli starts with a sequence of documentation paras these are likely all in the same comment anyways, so they will end up in the preamble even if we consider only the first comment. In general I don't think it will be a big deal (and easily fixable).

[lpw25]

I think you've convinced me about no section headers in the pre-amble.

If I'm in the middle of a .mli file and write:
(** Para1 *)

(** Para2 )
This will yield two paragraphs and markup wise there's no difference with:
(* Para1

Para2 *)

That isn't true when (* Para1 *) is the comment attached to a definition, and for me the first comment in a signature is a form of attached comment. I really think that using only the first comment will be the least surprising option for users.

[dbuenzli]

That isn't true when (* Para1 *) is the comment attached to a definition, and for me the first comment in a signature is a form of attached comment. I really think that using only the first comment will be the least surprising option for users.

Ok I won't fight about this. It is true that sometimes the notion of comment "counts" in the ocamldoc language.

[Julow]

Following the discussion, I changed the rule to cut the preamble at the first heading, regardless of its level.