Hello fellow peers. It is I, Josh, looking beyond our deployment and release strategy and process into the great beyond. I see..... documentation pains. The Readme owl, it takes our money, yet leaves us thirsty for more. I'm experimenting with alternative technological solutions for docs-generation and hosting. Please, find your opinions and spin them in 🧵 What MUST we have? What are nice-to-haves? What can we not live without, etc...

This is a really epic and important project!

And for the record, MkDocs is currently in my cross-hairs for generation. It seems like we can get most (all?) our beloved features with very minimal messaging of our Markdown docs I hope to have a PoC next week maybe?

Must have: Docs versions Nice to have: Suggested edits that become pull requests

I use Sphinx extensively at work and generally think it's compelling, especially its support for cross-references. The biggest reason to use it is API generation from docstring imo - it's better than alternatives like Handsdown I hate RST so would strongly recommend Myst to use Markdown The Furo theme is very nice. I extensively researched several Sphinx themes and it was the clear winner. It's used by pip docs (and now qiskit.org!) My main complaints are: • somewhat cryptic error messages • slow build times, but the project I work on has 5000 HTML pages hah • its caching sometimes gets corrupted, which results in confusing dev experience

Must have: Flexible docs structure (none of ReadMe's two level only constraints)

Must have: cross-referencing (as with the current doc: urls)

How precise? Only the page? Or also parts of the page like the

#

anchor? Should the tool error when cross-references are now invalid?

Anchors, and, ideally, yes, cross-references should be validated

this is the document that Eric referred to re: Sphinx research (among other solutions) they have done https://docs.google.com/document/d/1bZE8PlF9oRzcPQz4-JUFr5vfD0LFHH4V3Nj2k221CFM/edit

I think one of the benefits of MkDocs is that RST (and therefore Sphinx) didn't quite make it out of the Python ecosystem. Markdown on the other hand....

You can use Markdown with Sphinx via Myst! I'd never recommend RST to a project not already using it

Well the argument was the format was as ubiquitous, so the tools in the space are likely less ubiquitous

Custom HTML/Javascript! I’d like to plug in Algolia for our doc search

So the good news is I'm finding a bunch of dead references 😅

Good feature to have: ability to maintain multiple slugs for the same page

In MkDocs, without a plugin (there are several) it uses the file path. There's also a plugin to validate links

I learned python by writing plugins for sphinx. I still like sphinx and rst - Markdown always feels so feature-poor to me, even if most others don’t like ReStructured Text. I would appreciate MyST which allows sphinx and RST’s features to shine through the Markdown facade when needed.

OK, my running demo will be at https://thejcannon.github.io/pants/docs The idea being that URLs could be replaced directly (

<http://pantsbuild.org|pantsbuild.org>

->

<http://thejcannon.github.io/pants|thejcannon.github.io/pants>

)

Another nice to have: Ability to bundle the entire docsite into Pants, for easy offline access. A la

rustup docs --book

That's a different project. Likely much easier, but related

Oh, another nice feature - ability to only have a few most recent versions show up in the version selector (while the underlying docs are still available for direct links)

Makes sense. My stretch goal is to support all of 2.x. Maybe not out of the gate, but if we can export the markdown and stash it somewhere, then we can migrate it (but that does require fixing links, creating the nav, etc...)

I think anything that can be done automatically we should do all the way back to 2.0, but anything that requires human tending, we should only do to 2.15.x

Fixing broken links during migration likely won't kill me

Also, it might be that we don't do versioning in MkDocs, instead letting readthedocs do that (if we choose that provider)

Yeah, this is somewhat tricky to pull off. The way we do it for qiskit.org is similar to readthedocs but we host ourselves. Each version is a distinct standalone docs build. We deploy it to a URL with the version number included. Then, other docs builds have a "Previous Releases" dropdown that points to those builds. Readthedocs does the same thing. The trickiest part is how to have prior docs versions point to new docs versions, like 2.3 makes clear 2.16 exists in its dropdown. Readthedocs works around this by dynamically adding the version selector after-the-matter to the site, rather than it being included in the original docs build

That's makes sense. I suspect the readthedocs support will likely be simplest. I also noticed they have "PR build preview" support which would be absolutely amazing

That's fine with me, as long as we can get the functionality we want

I have reference docs! https://thejcannon.github.io/pants/docs/reference/global-options/

And nesting of folders!!

Yeah. Well I made it loosely resemble the Nav from the existing site. It doesn't have that "header" looking thing, so each directory is a dropdown