I’ve started looking at plugin development and have managed to add a new linter by copy-pasting the

docformatter

one and tweaking it to use

vulture

. All works fine — there is a lot to learn, but I think the approach taken when structuring plugin code is fairly intuitive for a newcomer. I have a specific question about the

flake8

linter available out-of-the-box with Pants. The nice thing about

flake8

is that there are tons of plugins (https://github.com/DmytroLitvinov/awesome-flake8-extensions#clean-code) which means that after installing any of those into a Python environment that has

flake8

installed, those plugins will be picked up by the

flake8

and their rules will run as well. I wonder what would be a smart thing to do to bring

flake8

plugins support in Pants into my monorepo? The

flake8

requirement is specified at https://github.com/pantsbuild/pants/blob/release_2.5.1rc3/src/python/pants/backend/python/lint/flake8/subsystem.py#L18 so I wonder how could I extend it to use multiple

flake8

plugins (each being a PyPI package) when

./pants lint

goal is run?

set the channel topic: https://www.pantsbuild.org/docs/plugins-overview

set the channel description: Help with writing Pants Build System plugins (NB: this is an unstable API!).

set the channel description: Help with writing Pants Build System plugins (NB: this is an unstable API!). https://www.pantsbuild.org/docs/plugins-overview

cleared channel topic

cleared channel description

set the channel description: Help with writing Pants Build System plugins (NB: this is an unstable API!). https://www.pantsbuild.org/docs/plugins-overview

set the channel topic: https://www.pantsbuild.org/docs/plugins-overview

oh my. sorry for the spam.

just heads up, I have a draft version of YAPF Python formatter plugin added to Pants. https://github.com/pantsbuild/pants/pull/12317. Feel free to checkout and experiment.

I’m a bit interested in starting to look at a docker plugin (https://github.com/pantsbuild/pants/issues/2648) as a POC. Probably more work than I would finish in a sensible amount of time, but think it would be fun, none the less.

I am still having trouble with relocated file targets - specifically getting a snapshot with the moved files: I would expect this to work (I have removed all my plugin code from this function so it's just running the one

Get

and then exiting)

@rule(level=LogLevel.DEBUG)
async def package_into_image(
    field_set: DockerPackageFieldSet,
    union_membership: UnionMembership,
) -> BuiltPackage:
    target_name = field_set.address.target_name
    transitive_targets = await Get(
        TransitiveTargets, TransitiveTargetsRequest([field_set.address])
    )
    for t in transitive_targets.dependencies:
        if type(t) is RelocatedFiles:
            s = await Get(
                SourceFiles,
                SourceFilesRequest,
                SourceFilesRequest(
                    sources_fields=[t.get(Sources)],
                    for_sources_types=[RelocatedFilesSources],
                )
            )
            <http://logger.info|logger.info>('relocated files: %s', s.snapshot.files)
    return BuiltPackage(
        digest=s.snapshot.digest,
        artifacts=([BuiltPackageArtifact(f, ()) for f in s.snapshot.files]),
    )

has output:

[nate@ragin-cajun pants-docker]$ ./pants package test_docker:dockerized_flask_app
12:04:18.11 [INFO] relocated files: ()

I thought I’d try out the docker support on the

example-python

repo, and manage to run a local built version of

pants

. However, when I try to load in the experimental docker package, it fails…

hey team! I’ve been thinking about extending Pants with a new target

debian_package

that would let users create a Debian package from a PEX file. This has been my preferred way to distribute PEX files so far. I package PEX files and then run a few lines of Bash that produce Debian packages as part of the CI pipeline. However, I think it could have been a useful feature to have available OOTB as another binary artifact target so that people don’t need to rely on custom shell scripts. I’ve described the idea in here: https://github.com/pantsbuild/pants/issues/12421. Keen to know what you think.

anyone know of an OpenAPI/swagger codegen plugin? if not, would the protobuf plugin be a good place to start looking if i were to try to create one?

Given a

GoalSubsystem

, how can I iterate over the options (in my

@goal_rule

)?

subsystem.options

I want to get all of the options registered for my subsystem that have modified values.

@fast-nail-55400: anything to know about the experimental

terraform

backend? is it usable?

How do I run pants in dev mode? I could have sworn I was using interactive debugging while writing a plugin on Friday, but can't seem to get it to work now? resolved:

--no-dynamic-ui

So, Trying to build a simplistic docker plugin to build a docker image using a

Dockerfile

, files from any other unbuilt dependencies, a and the outputs of other

package

targets. (minimal requirement is a

.pex

file, and a

.sh

script built into an image) Have gotten the latter working and running

find

inside a chroot reveals the

.pex

file (great!)

.
./examples.docker_image
./examples.docker_image/hello-world.pex'

Struggling to get the former bits working, and I've tried enough things from the examples in the docs I'm a bit lost. WIP code here: https://github.com/xlevus/pants2-plugins/pull/1/files

Is there a way to inspect the rule graph? Haven't yet seen how to work out what rules provide what dependencies, (and in turn, what they require) i.e. The only way I know to get a

ProcesResult

by giving it a

Process

input is from reading the docs.

@fast-nail-55400: re target usage for Go

messing around with the

find_needle

plugin in the testing docs and pushed an example here: https://github.com/njgrisafi/pants-example/tree/find-needle getting a test error:

request = HydrateSourcesRequest(field=<class 'pants.engine.target.Sources'>(alias='sources', address=//:t1, value={repr(self.value)}, default={repr(self.default)}), for_sources_types=(<class 'pants.engine.target.Sources'>,), enable_codegen=False)

    def mock_hydrate_sources(request: HydrateSourcesRequest) -> HydratedSources:
        # Our rule only looks at `HydratedSources.snapshot.files`, so we mock all other fields. We
        # include the file `needle.txt` for the target `:t2`, but no other targets.
        files = (
            ("needle.txt", "foo.txt")
            if request.field.address.target_name == "t2"
            else ("foo.txt", "bar.txt")
        )
>       mock_snapshot = Snapshot(EMPTY_DIGEST, files=files, dirs=())
E       TypeError: PySnapshot.__new__()() takes at most 0 arguments (3 given)

Any ideas why this is?

does anyone know if there any plugins for building webpack bundles?

Hello! I’ve been working on this yesterday and today and now am able to build bundles using webpack through pants: https://github.com/compyman/pants-node One problem i’ve been having is trying to integrate this with my docker plugin: https://github.com/compyman/pants-docker I have added the package as a requirement & included it in the python_library dependencies but it isn’t able to import the

sendwave

module? this seems like i’ve just done something very dumb again but i’ve been brushing up against it all morning… anyway if anyone has any ideas I could use a fresh set of eyes

(here’s the pr that adds the failing import: https://github.com/compyman/pants-node/pull/1)

Trying my hand at writing a plugin to publish things places. I think I 'get' how the rules work. https://github.com/xlevus/pants2-plugins/blob/publish/src/xlvs/pants/publish/publish_subsystem.py#L79 But stuck here. my

requests

looks good

[TwinePublishRequest(package_field_set=PythonDistributionFieldSet(address=Address(examples/pypi_repository:example_package), provides=<class 'pants.backend.python.target_types.PythonProvidesField'>(alias='provides', value=<pants.backend.python.macros.python_artifact.PythonArtifact object at 0x7f9ff81a2e80>, default=None)), field_set=TwineFieldSet(address=Address(examples/pypi_repository:example_package)))]

but when I get to

oof

I get:

12:32:01.88 [ERROR] Encountered 5 rule graph errors:
  No installed rules return the type PrePublishRequest, and it was not provided by potential callers of @rule(xlvs.pants.publish.publish_subsystem:51:to_publish_request(PrePublishRequest) -> PublishRequest, gets=[Get(BuiltPackage, ArchiveFieldSet), Get(BuiltPackage, PexBinaryFieldSet), Get(BuiltPackage, PythonDistributionFieldSet), Get(BuiltPackage, DockerImageFieldSet)]).
    If that type should be computed by a rule, ensure that that rule is installed.
    If it should be provided by a caller, ensure that it is included in any relevant Query or Get.
  No source of dependency Get(BuiltPackage, ArchiveFieldSet) for @rule(xlvs.pants.publish.publish_subsystem:51:to_publish_request(PrePublishRequest) -> PublishRequest, gets=[Get(BuiltPackage, ArchiveFieldSet), Get(BuiltPackage, PexBinaryFieldSet), Get(BuiltPackage, PythonDistributionFieldSet), Get(BuiltPackage, DockerImageFieldSet)]). All potential sources were eliminated: []
  No source of dependency Get(BuiltPackage, DockerImageFieldSet) for @rule(xlvs.pants.publish.publish_subsystem:51:to_publish_request(PrePublishRequest) -> PublishRequest, gets=[Get(BuiltPackage, ArchiveFieldSet), Get(BuiltPackage, PexBinaryFieldSet), Get(BuiltPackage, PythonDistributionFieldSet), Get(BuiltPackage, DockerImageFieldSet)]). All potential sources were eliminated: []
  No source of dependency Get(BuiltPackage, PexBinaryFieldSet) for @rule(xlvs.pants.publish.publish_subsystem:51:to_publish_request(PrePublishRequest) -> PublishRequest, gets=[Get(BuiltPackage, ArchiveFieldSet), Get(BuiltPackage, PexBinaryFieldSet), Get(BuiltPackage, PythonDistributionFieldSet), Get(BuiltPackage, DockerImageFieldSet)]). All potential sources were eliminated: []
  No source of dependency Get(BuiltPackage, PythonDistributionFieldSet) for @rule(xlvs.pants.publish.publish_subsystem:51:to_publish_request(PrePublishRequest) -> PublishRequest, gets=[Get(BuiltPackage, ArchiveFieldSet), Get(BuiltPackage, PexBinaryFieldSet), Get(BuiltPackage, PythonDistributionFieldSet), Get(BuiltPackage, DockerImageFieldSet)]). All potential sources were eliminated: []

I'm not requesting a

PrePublishRequest

?? I'm doing

Get(PublishRequest, PrePublishRequest, request)

[edit] Solved

I suppose this is a plugin Api question/feedback.. when creating files in a digest with

FileContent

I happened to pass a

str

rather than

bytes

. What happens was…. deadlock (or something, the tests time out). No warning or nothing. I’m sure mypy would catch this, but it took me a while to figure out why it wouldn’t work.. (as an empty string worked just fine, but that’s most likely an edge case, then…)

I found the tip to use

pants_requirement

for plugins to be a bit too narrow for our use case. Locking down each version of the plugin to a single pants version is a bit hard.. I’d like a little more flexibility, so I opted for:

#pants_requirement(name="pants", dist="pantsbuild.pants")
python_requirement_library(
    name="pants",
    requirements=["pantsbuild.pants >=2.6.0, <3.0"],
    module_mapping={
        "pantsbuild.pants": ["pants"],
    },
)

Instead.. sure enough, if there are breaking changes before 3.0, we’ll have to address them, but this will at least not require us to release a new version of the plugin for every pants release. Am I missing something, like, do I shoot my self horrible in the foot by doing this? (it is not a in-repo plugin, but stand alone, pushed to a cheese chop registry)

I've started writing a plugin to provide support for

pants publish

I've got it to the point that it 'works' and can push

python_distribution

targets to

pypi_repository

targets. would be great if someone could spare a few minutes to sanity-check it, https://github.com/xlevus/pants2-plugins/pull/2/files

Ok fully stumped on Union Rules now. https://github.com/xlevus/pants2-plugins/pull/2/files#diff-173d7b1e4de36693f6c6b3b1aaf75f4252b8ddf52115ccea743d30af8d97b89bR130 When this runs w/ the commented MultiGet uncommented, I get errors:

No installed rules return the type DockerPublishRequest, and it was not provided by potential callers of @rule(xlvs.pants.publish.docker.docker_push:8:publish_docker_image(DockerPublishRequest) -> PublishedPackageSet).
    If that type should be computed by a rule, ensure that that rule is installed.
    If it should be provided by a caller, ensure that it is included in any relevant Query or Get.
  No installed rules return the type PypiPublishRequest, and it was not provided by potential callers of @rule(xlvs.pants.publish.aws_s3.s3_upload:7:upload_s3(PypiPublishRequest) -> PublishedPackageSet).
    If that type should be computed by a rule, ensure that that rule is installed.
    If it should be provided by a caller, ensure that it is included in any relevant Query or Get.

Yet, with the

MultiGet

call uncommented, my prints are:

>> DockerPublishRequest(built_package=BuiltPackage(digest=Digest('e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 0), artifacts=()), fieldset=DockerPublishFieldSet(address=Address(examples/container_registry:image)))
Publishing 'examples/pypi_repository:example_package' to 'examples/pypi_repository:privatepypi'.
>> PypiPublishRequest(built_package=BuiltPackage(digest=Digest('66de45797a5ab5514223ade6a42dfd721b0099cf27860cc8a66c15eba606b3b3', 105), artifacts=(BuiltPackageArtifact(relpath='xlvs-pants2-pypi-example-0.0.1', extra_log_lines=()),)), fieldset=PublishTargetFieldSet(address=Address(examples/pypi_repository:example_package)))
Publishing 'examples/pypi_repository:example_package' to 'examples/pypi_repository:s3_bucket'.
>> S3PublishRequest(built_package=BuiltPackage(digest=Digest('66de45797a5ab5514223ade6a42dfd721b0099cf27860cc8a66c15eba606b3b3', 105), artifacts=(BuiltPackageArtifact(relpath='xlvs-pants2-pypi-example-0.0.1', extra_log_lines=()),)), fieldset=PublishTargetFieldSet(address=Address(examples/pypi_repository:example_package)))
FrozenOrderedSet([<class 'xlvs.pants.publish.python.targets.PypiPublishRequest'>, <class 'xlvs.pants.publish.docker.targets.DockerPublishRequest'>, <class 'xlvs.pants.publish.aws_s3.targets.S3PublishRequest'>])

Which looks like: • My

publish_request

type are the right subclasses. • my

PublishRequest

subclasses are a part of the

UnionMembership