So my adhoc_tool targets always rebuild. Is there ...
# general
p
So my adhoc_tool targets always rebuild. Is there a way I can figure out why? Or is this is suppose to happen.
w
https://www.pantsbuild.org/2.18/docs/ad-hoc-tools/integrating-new-tools-without-plugins#runnable-targets >> Output values will be cached by Pants, and future invocations with identical inputs will be retrieved from the cache instead of being re-executed. If your process has behavior that is not fully defined by its inputs, Pants' behavior may be unexpected or inconsistent. https://www.pantsbuild.org/2.18/docs/ad-hoc-tools/integrating-new-tools-without-plugins#chaining-processes-together A lot/most of it should be cached - any chance the runnable are not pure (and I guess idempotent)?
p
Hmm how can I debug that? I'm essentially building some sphinx api docs..
Copy code
python_sources(name="conf")

files(name="index", sources=["index.md"])

"""
Unfortunately, we _cannot_ glob over dependencies, as such we need to explicitly define the dependencies which are required to
build documentation. As such, if you add a new python package it will not be included in the documentation unless you add it to the
list below.
"""
DOC_DEPS = [
    ":conf",
    ":index",
    "//blah/blah",
    .....
]

SPHINX_DEPS = [
    ".build/requirements:reqs-dev#myst-parser",
    "//.build/requirements:reqs-dev#sphinx",
]

# NOTE(G3): In the future everything below could be a macro

# Create a pex binary that executes the sphinx-apidoc console script
pex_binary(
    name="sphinx-apidoc",
    script="sphinx-apidoc",
    dependencies=SPHINX_DEPS,
)

adhoc_tool(
    name="build-apidoc",
    runnable=":sphinx-apidoc",
    args=[
        "--force",
        "--module-first",
        "--output-dir",
        # NOTE(G3): In later steps, when the output is unfurled.. this places the documentation in the worktree
        # appropriately from the root of the workspace
        "blah/docs/source",
        # NOTE(G3): Relative to this BUILD file
        "./../blah",
    ],
    execution_dependencies=DOC_DEPS,
    output_directories=["blah/docs"],
    #root_output_directory="./",
    log_output=True,
)

# Create a pex binary that executes the sphinx-build console script with all
# transitive dependencies from the DOC_DEPS
pex_binary(
    name="sphinx-build",
    script="sphinx-build",
    dependencies=SPHINX_DEPS + DOC_DEPS,
)

adhoc_tool(
    name="build-html",
    runnable=":sphinx-build",
    args=["-M", "html", ".", "_build"],
    execution_dependencies=[":build-apidoc"] + DOC_DEPS,
    output_directories=["_build/html"],
    #root_output_directory="./_build/html",
    log_output=True,
)
archive(name="docs", files=[":build-html"], format="zip")
If I build the docs archive subsequent runs don't rebuild. When I pass the output of build-html to a pex_binary.. it always runs.
Copy code
experimental_wrap_as_resources(
    name="docs_resources",
    inputs=["//blah/docs:build-html"],
)

pex_binary(
    name="blah_server",
    entry_point="blah.main",
    restartable=True,
    dependencies=[
        ":docs_resources",
        ":generated_css_resource",
        ":templates",
    ],
)
Oh hmm I think DOC_DEPS
Copy code
# Create a pex binary that executes the sphinx-build console script with all
# transitive dependencies from the DOC_DEPS
pex_binary(
    name="sphinx-build",
    script="sphinx-build",
    dependencies=SPHINX_DEPS + DOC_DEPS,
)
Shouldn't be here... the build-html should happen each time the docs change, but not also trigger the sphinx-build.