I started adding Sphinx support 🙂 Thoughts on this target modeling?

I like less boiler plate

me too. And it seems less likely you're gonna be setting metadata on an individual file. Unlike Python, you're not really saying "this particular file has a test timeout of 20s" Sphinx seems pretty fast from playing with it, so not a huge deal if dependencies aren't precise. That'd be good to confirm from Sphinx users tho

Sounds like a start! Any support is better than none. And it can always be improved later. And by using the feedback from initial implementation

I added some stuff to our sphinx building to monkeypatch the generated rst code to change up the depth of table of contents it shows and such. Sphinx really doesn't like namespace packages (we had to add dummy

__init__.py

modules throughout if a directory didn't contain any python modules). Other than that, we never needed to do much. It was all handled through

conf.py

and associated extensions.

Okay yeah I like this

sphinx_project

target type a lot! For example, we can add a field for what mode(s?) you want to generate in, e.g. HTML. So you change on a per-project basis

i think we should take care of autodocs and python-source dependencies from the docs, but practically this type of caching/incremental build also could be offloaded to Sphinx itself.

Can't the sphinx project generate file level targets. That keeps the boilerplate down while still being open to file level metadata via overrides.

Can't the sphinx project generate file level targets.

Possibly. I think I'm gonna start w/ just

sphinx_project

for now, then we could probably add if there's demand

We built some scripts to work alongside Pants in the build chain for when it came to the documentation build step, so this would be a great feature to have to ultimately get rid of those. Is the

package

goal on a

sphinx_project

expected to be similar to invoking the Makefile that comes with every Sphinx project? Would the output type be specified in the goal's args or the target's params (e.g., "make html", "make pdf")? Not sure if this would affect the design choice, but we also make heavy use of the IPython Sphinx extensions module that allows your

.rst

to drop into an IPython kernel, do some Python, and then save the outputs/plots in a way that a user would experience if they had a console or notebook open. In our build chain we had to make sure that any given project on the repo had a correctly setup virtual env for their docs sub-project to be built correctly and make this type of documentation possible. That turned out to be a bit of a headache, esp. for first-party code (maybe because we have become so accustomed to Pants doing this so elegantly in other situations!); hopeful whatever becomes of this new target, these sorts of things are easily handled :)

Hi! The output type would be in the target's fields. And yes,

package

is like invoking

make

to build it I ended up reverting Sphinx in 2.13 because CI was flaky and the design needs more work. But, it's a foundation and I hope to pick it back up (I want to use Sphinx for Pants itself)

I hope it's successful when you do! When the time comes and you need some early adopters for testing, let me know.

Will do! And lmk if you have any interest in implementing, we're definitely always happy to help contributors

We are planning at some point to commit time to exploring Pants’ internals with the aim of being able to contribute changes we need in the future, but I am not sure how far we will get (to an extent it depends on how institutionalized Pants becomes within our org—I'm trying to make a good case for it over Bazel and our own horribly inadequate dynamic language build tools). You or someone else will likely beat us to the documentation goal, but we'll see ;) One of the first things we’d want to do is add support for the Q programming language (very common for projects to use both or spin up a Q service that the Python interacts with) since it is so widely used here (and at banks/hedge funds in general).