## Context We have an internal CI process which gates merges to trunk on a variety of custom validation: audit, unit tests, etc. That process supports a number of processes external to Pants for a given repo managed by Pants: - Publishing Python packages to internal Artifactory - Publishing multiple Lambda Layers && Functions of different languages from a single repository - Interpolating the $latest version of a given Lambda Layer while building the deployable artifact for a Function - etc A number of these use Pants (or its outputs) as part of their logic. To use the "interpolation" example: - Pants builds a zip file for a Function - We then crack open that zip file, find and read its internal config file (used for UpdateFunctionConfiguration) - If it consumes Layers, we find $latest for each (at time of publishing), and rewrite them into the file : - AWS only supports

"Layers": ["arn:aws:lambda:us-east-1:123456789012:layer:MyLayer:<INT>"]

- We thus can support

"Layers": ["arn:aws:lambda:us-east-1:123456789012:layer:MyLayer:*"]

, to allow Functions to always publish FunctionVersions which consume the $latest version of any internal Layers - We then publish the mutated zip file to S3, where our CD tooling consumes it for publishing Function Versions. ## How we call Pants - We have a module gh.{path}.pants.exec, containing a

_exec_pants_in_repo()

function intended be the sole executor of Pants shell cmds within our CI process -- consumers handle the resulting

subprocess.CompletedProcess

as appropriate (inspecting exit code, parsing stdout/stderr, etc) for each use case. - Within the same module are commands such as `exec_pants_list_changed`:

py
PANTS_LIST_CHANGED = "./pants filter --changed-since=origin/{refspec} --changed-dependees=transitive --target-type={target_type}"
...

def exec_pants_list_changed(..., target_type: Optional[Union[str, Iterable[str]]] = DEFAULT_TARGET_TYPE) -> Tuple[str]:
    """
    Run PANTS_LIST_CHANGED and return stdout as a tuple of strings-per-line

    Example: THIS stdout
        src/python/gh-foo:gh-foo
        src/python/gh-bar:gh-bar

    BECOMES THIS tuple

        ("src/python/gh-foo:gh-foo", "src/python/gh-bar:gh-bar")
    """

- CI consumers may call

exec_pants_list_changed

and use the results to post internal audit entries, determine which Targets have schemas to publish, etc. ## Why not a Plugin? These are the docs I wrote into the above module

"""
-- Why not call the Pants Target API directly? --

As of 2022-04-26, the Pants Plugin API is not stable (<https://www.pantsbuild.org/docs/plugins-overview>).

It can (and does) make breaking changes between minor version releases, and offers neither documentation nor guidance
for Pants execution from non-shell processes.

In an IDEAL world, Pants' API would have the same maturity as, say, the Pulumi Automation API. In THAT world, yes, we'd
adopt "Pants execution from within Python". UNTIL they reach that level of stability, we've chosen to continue with our
existing (stable, if not ideal) subprocess design.


-- Why not use Pants plugins? --

See <https://www.pantsbuild.org/docs/target-api-new-targets> for a description of the functionality the question implies.

While we COULD conceivably do "./pants gh-merge-to-trunk" via a custom plugin, the immaturity of the API's has made the value
proposition a tenuous one. We have an existing process with excellent documentation and test coverage, and no plans to
migrate at this time.
"""