I want to run a `sphinx application Sphinx` in a script I kn

Question

I want to run a `sphinx application Sphinx` in a script I know a bit different use than normal I d like to do this with Pants but it can t find `sphinxcontrib serializinghtml` This is a transitive dependency of `sphinx` Are there any options other than adding that transitive dependency to my `requirements txt`

fast-nail-55400 · Answer

Not really While it may be a transitive dependency for one requirement your proposed use is as a direct dependency of your code so you should be declaring it as such

fast-nail-55400 · Answer

That is add it as a requirement

bitter-ability-32190 · Answer

There s some thoughts around transitive dependencies as targets But nothing concrete

fast-nail-55400 · Answer

And although I imagine that `sphinx` would not remove this dependency from their own dependency graph they are free to do so Your code should not rely on assumptions built into sphinx when your code could just declare the dependency it needs

high-yak-85899 · Answer

Yeah that all makes sense to me The only part where I slightly deviate in opinion is that calling `sphinx application Sphinx` has a valid input with `html` as the builder name which makes it seem like I shouldn t need to separately say oh by the way I know internally you need this library to do the thing

high-yak-85899 · Answer

And for the same reasoning they re free to change it

high-yak-85899 · Answer

So who am I to say that s the right library to use

fast-nail-55400 · Answer

yeah sounds like they are exposing what could be an implementation detail then

fast-nail-55400 · Answer

software is too hard

enough-analyst-54434 · Answer

< high yak 85899> what version of Sphinx The `sphinxcontrib serializinghtml` project is direct dependency of Sphinx main and so you d only need a proper module mapping for that i if even that

high-yak-85899 · Answer

Yeah it s already in the environment We ere on 4 5 at the moment I ll try out a module mapping I guess I didn t think to try that first since we don t have that package in our direct dependencies so I thought module mappings would complain

enough-analyst-54434 · Answer

```$ pex sphinx c import sphinxcontrib serializinghtml print sphinxcontrib serializinghtml file tmp tmpt 1zhtm0 bootstrap pex environment py 654 PEXWarning The `pkg resources` package was loaded from a pex vendored version when declaring namespace packages defined by 1 sphinxcontrib devhelp==1 0 2 namespace packages sphinxcontrib 2 sphinxcontrib jsmath==1 0 1 namespace packages sphinxcontrib 3 sphinxcontrib serializinghtml==1 1 5 namespace packages sphinxcontrib 4 sphinxcontrib qthelp==1 0 3 namespace packages sphinxcontrib These distributions should fix their `install requires` to include `setuptools` pex warnings warn home jsirois pex installed wheels 352a9a00ae864471d3a7ead8d7d79f5fc0b57e8b3f95e9867eb9eb28999b92fd sphinxcontrib serializinghtml 1 1 5 py2 py3 none any whl sphinxcontrib serializinghtml init py```

enough-analyst-54434 · Answer

So it totally works with latest Sphinx

enough-analyst-54434 · Answer

< high yak 85899> you only need a module mapping if you import the sphinxcontrib code like I did in this example proof If you don t you don t

high-yak-85899 · Answer

Right so then 4 5 must not declare it as a proper dependency

enough-analyst-54434 · Answer

Note the NS package warnings though You ll want an execution mode= venv pex binary to work around issues there

enough-analyst-54434 · Answer

Nope works fine ```$ pex sphinx==4 5 c import sphinxcontrib serializinghtml print sphinxcontrib serializinghtml file tmp tmp361dpp9r bootstrap pex environment py 654 PEXWarning The `pkg resources` package was loaded from a pex vendored version when declaring namespace packages defined by 1 sphinxcontrib devhelp==1 0 2 namespace packages sphinxcontrib 2 sphinxcontrib jsmath==1 0 1 namespace packages sphinxcontrib 3 sphinxcontrib serializinghtml==1 1 5 namespace packages sphinxcontrib 4 sphinxcontrib qthelp==1 0 3 namespace packages sphinxcontrib These distributions should fix their `install requires` to include `setuptools` pex warnings warn home jsirois pex installed wheels 352a9a00ae864471d3a7ead8d7d79f5fc0b57e8b3f95e9867eb9eb28999b92fd sphinxcontrib serializinghtml 1 1 5 py2 py3 none any whl sphinxcontrib serializinghtml init py```

high-yak-85899 · Answer

So then what am I doing wrong with my `python source` When I do `dependencies transitive` it s only showing that `sphinx` is a dependency

high-yak-85899 · Answer

``` reqs click reqs jinja2 reqs pyyaml reqs sortedcontainers reqs sphinx reqs sphinx rtd theme reqs types PyYAML requirements txt reqs```

enough-analyst-54434 · Answer

Pants `dependencies` goal only does 1st party and direct 3rdparty

enough-analyst-54434 · Answer

It doesn t do resolves or incorporate their results

enough-analyst-54434 · Answer

Basically it only lists text in your repo

high-yak-85899 · Answer

So perhaps I need to make a dedicate `pex binary` target I was hoping just the `python source` would work now that running that is separated from a pex in newer pants versions

enough-analyst-54434 · Answer

It might help if you include the actual real problem A command line with full output

enough-analyst-54434 · Answer

This should work with the latest machinations of the runnable python source but I m not sure what versions of Pants have the latest fixes there it recently uses venvs

high-yak-85899 · Answer

A minimal repro is just instantiating `sphinx application Sphinx` with a builder set to ` html ` and you get ```Running Sphinx v4 5 0 Traceback most recent call last File home nathanael cache pants named caches pex root venvs 874a56f50430c8505af5832bda4ffa33c6fe9d8c c4fe7bf08b2a1259d39d005551c5bf25de7c32ef lib python3 8 site packages sphinx registry py line 425 in load extension mod = import module extname File usr lib python3 8 importlib init py line 127 in import module return bootstrap gcd import name level package level File lt frozen importlib bootstrap gt line 1014 in gcd import File lt frozen importlib bootstrap gt line 991 in find and load File lt frozen importlib bootstrap gt line 975 in find and load unlocked File lt frozen importlib bootstrap gt line 671 in load unlocked File lt frozen importlib bootstrap external gt line 848 in exec module File lt frozen importlib bootstrap gt line 219 in call with frames removed File home nathanael cache pants named caches pex root venvs 874a56f50430c8505af5832bda4ffa33c6fe9d8c c4fe7bf08b2a1259d39d005551c5bf25de7c32ef lib python3 8 site packages sphinx builders epub3 py line 12 in lt module gt from sphinx builders import epub base File home nathanael cache pants named caches pex root venvs 874a56f50430c8505af5832bda4ffa33c6fe9d8c c4fe7bf08b2a1259d39d005551c5bf25de7c32ef lib python3 8 site packages sphinx builders epub base py line 15 in lt module gt from sphinx builders html import BuildInfo StandaloneHTMLBuilder File home nathanael cache pants named caches pex root venvs 874a56f50430c8505af5832bda4ffa33c6fe9d8c c4fe7bf08b2a1259d39d005551c5bf25de7c32ef lib python3 8 site packages sphinx builders html init py line 1295 in lt module gt import sphinxcontrib serializinghtml NOQA ModuleNotFoundError No module named sphinxcontrib serializinghtml The above exception was the direct cause of the following exception Traceback most recent call last File home nathanael cache pants named caches pex root venvs 874a56f50430c8505af5832bda4ffa33c6fe9d8c c4fe7bf08b2a1259d39d005551c5bf25de7c32ef pex line 236 in lt module gt runpy run module module name run name= main alter sys=True File usr lib python3 8 runpy py line 207 in run module return run module code code init globals run name mod spec File usr lib python3 8 runpy py line 97 in run module code run code code mod globals init globals File usr lib python3 8 runpy py line 87 in run code exec code run globals File home nathanael devel monorepo astranis python astranis service action triggers documenter config documenter py line 77 in lt module gt build File home nathanael cache pants named caches pex root venvs 874a56f50430c8505af5832bda4ffa33c6fe9d8c c4fe7bf08b2a1259d39d005551c5bf25de7c32ef lib python3 8 site packages click core py line 829 in call return self main args kwargs File home nathanael cache pants named caches pex root venvs 874a56f50430c8505af5832bda4ffa33c6fe9d8c c4fe7bf08b2a1259d39d005551c5bf25de7c32ef lib python3 8 site packages click core py line 782 in main rv = self invoke ctx File home nathanael cache pants named caches pex root venvs 874a56f50430c8505af5832bda4ffa33c6fe9d8c c4fe7bf08b2a1259d39d005551c5bf25de7c32ef lib python3 8 site packages click core py line 1066 in invoke return ctx invoke self callback ctx params File home nathanael cache pants named caches pex root venvs 874a56f50430c8505af5832bda4ffa33c6fe9d8c c4fe7bf08b2a1259d39d005551c5bf25de7c32ef lib python3 8 site packages click core py line 610 in invoke return callback args kwargs File home nathanael devel monorepo astranis python astranis service action triggers documenter config documenter py line 70 in build sphinx = application Sphinx File home nathanael cache pants named caches pex root venvs 874a56f50430c8505af5832bda4ffa33c6fe9d8c c4fe7bf08b2a1259d39d005551c5bf25de7c32ef lib python3 8 site packages sphinx application py line 219 in init self setup extension extension File home nathanael cache pants named caches pex root venvs 874a56f50430c8505af5832bda4ffa33c6fe9d8c c4fe7bf08b2a1259d39d005551c5bf25de7c32ef lib python3 8 site packages sphinx application py line 380 in setup extension self registry load extension self extname File home nathanael cache pants named caches pex root venvs 874a56f50430c8505af5832bda4ffa33c6fe9d8c c4fe7bf08b2a1259d39d005551c5bf25de7c32ef lib python3 8 site packages sphinx registry py line 428 in load extension raise ExtensionError Could not import extension s extname sphinx errors ExtensionError Could not import extension sphinx builders epub3 exception No module named sphinxcontrib serializinghtml ```

enough-analyst-54434 · Answer

Great it is running in a venv

enough-analyst-54434 · Answer

Can you do `ls 1 home nathanael cache pants named caches pex root venvs 874a56f50430c8505af5832bda4ffa33c6fe9d8c c4fe7bf08b2a1259d39d005551c5bf25de7c32ef lib python3 8 site packages ` and provide the output

high-yak-85899 · Answer

Untitled

high-yak-85899 · Answer

So only one of the sphinxcontrib dependencies

enough-analyst-54434 · Answer

Ok ```PEX EXTRA SYS PATH pth``` That s the problem Some tools don t handle this which supports building venvs from symlink farms You can turn that off with `pex binary`

enough-analyst-54434 · Answer

Just a sec for the details

enough-analyst-54434 · Answer

You need Pants gt = 2 15 x and ```pex binary execution mode= venv venv site packages copies=True ``` <https www pantsbuild org v2 15 docs reference pex binary codevenv site packages copiescode>

high-yak-85899 · Answer

Sounds good I ll try updating sometime soon