Hi everyone, I am just getting started with pants and was wondering if there are any plugins to package as docker image?

Hey all: First draft of a plugin to build node package is here: https://github.com/compyman/pants-node! Thanks for everyones help on it. We need it for a very specific use case (replacing the heroku node buildpack to generate a webpack bundle) so that is what it is optimized for. It also integrates with my pants-docker plugin so you can include your generated node stuff into a docker image. As always an feature requests or ideas/opportunities for improvement are very welcome! Thanks

thinking about creating a new plugin that requires to know source files git information. For example, we would need to know if a source file has been added, removed, modified. Is there a way to get this info in pants now? I guess another option is to create a custom rule to run gitpython. Maybe something like this?

source_files_git_info = await Get(
    SourceFilesGitInfo,
    SourceFilesGitInfoRequest(source_files),
)

as long as we have access to the git repo I assume this would be possible

hi all! Cross posting from #general -> https://pantsbuild.slack.com/archives/C046T6T9U/p1631695955493700 I want to develop a plugin to replace some of our custom tooling in the monorepo, so any “get started” advice (or examples) would be super useful

Going through the docs, it seems it should be enough to add to

plugins

to register a published plugin, but my experience is that I also need to add it to

backend_packages

. Bug or feature?

Looking at the terraform module, and that it uses infer dependencies, compared to the docker plugin, where I’ve used the inject dependencies.. I was still a bit confused about the difference between the two, but after reading up on the doc string of the request types of each, I get the feeling that I ought to use the infer dependency variant also in the docker plugin, as it is analysing the Dockerfile source .. should I change this?

Hey! I'm working on a pants plugin for pulumi, and since pulumi is distributed as a pre-compiled go binary I want to use ExternalTool to install it. Only problem is that pulumi has multiple executables in it's archive that it expects to be on the PATH in runtime, but AFAICT you can only specify one executable with

generate_exe

. Is there a mechanism I am missing to get multiple executables with the same archive? Or maybe I can make an ExternalTool for each executable I need and rely on caching to not download the archive n times?

I would like to do “extra” steps, post publish, that depends on details from what was published. A kind of “post rule” hook if you will. Is there a mechanism for this already, or do you have other suggestions for how I may get what I want? More specifically, so is it to notify other parts of our system that a new version of X has been built and published. I’m quite certain we’ll want a custom plugin for this case, but that plugin would still need to be able to fish these details out in some way.. and that some way could perhaps be some form of generic hook-ability into the rule graph during execution, or some such ?

Is it possible to get an output digest from an InteractiveProcess run? In my plugin I would like to be able to run an interactive process in it's chroot and see how a subset of the files have changed, but it seems like that might only be supported for a non-interactive process

I’m trying to create a special

package

rule that works with

python_library

as target. What I did was create a target

class DagsDependencies(Dependencies):
    """Simple inheritance for dependencies."""


class Dags(Target):
    """DAG tasks generated by a library."""

    alias = "dags"
    core_fields = (*COMMON_TARGET_FIELDS, DagsDependencies)

and then

@dataclass(frozen=True)
class DagsFieldSet(PackageFieldSet):
    required_fields = (DagsDependencies,)

    output_path: OutputPathField
    dependencies: DagsDependencies


@rule(level=<http://LogLevel.INFO|LogLevel.INFO>)
async def package_dags(field_set: DagsFieldSet) -> BuiltPackage:
    output_filename = field_set.output_path.value_or_default(
        field_set.address, file_ending="yaml"
    )
    ???

however, what I want no is to access the sources in the

dependencies

, ie, loop over the dependencies and get the source files of those that are

python_library

. I have been trying and reading source code, and I couldn’t manage. Any hints?

So I'm finally getting down to business and looking at a way to integrate pants with an RPM-based system. I see several challenges to getting this to work, and I want to start laying them out here so I can understand the problem space before I start hacking away: 1. How can I tell pants how python modules can be provided by these system packages, as opposed to pip-installable packages? 2. How will I be able to run build steps with these (and only) system packages installed? Containers? Chroot? 3. If I use one of these solutions for build steps, I imagine I'll need to run multiple commands to set up and clear the build environment. Is this compatible with the way Pants executes rules now?

I'm looking to add an option to

pants

to have a "strict constraints" mode and will likely need some friendly nudges by the Pants devs to navigate the codebase. Ideally in "strict_constraints" mode, all of a Python source's dependencies must exist in the

requirements_constraints

file. This ensures nobody adds a dependency to a script without also adding it to the constraints file. (Additionally, this will help in migrating to

pants

as it's hard for me to see which modules aren't getting found automagically and require a

module_mapping

entry).

When I upgrade my

pants

main

branch, I'm seeing the following:

Bootstrapping pants_deps with requirements:
  /home/joshuacannon/work/pants/3rdparty/python/requirements.txt
/home/joshuacannon/work/pants/build-support/pants_venv: line 33: /home/joshuacannon/.cache/pants/pants_dev_deps/Linux.x86_64.x86_64.py38.venv/bin/pip: No such file or directory

This isn't the first time it has happened too 🤔

Is there a preferred way of profiling

pants

? It being a Python app, I could try to see if I could use

py-spy

, but wanted to ping y'all first

Would it makes sense to extend

per_file_caching

option to each linter itself? I'm thinking some linters are faster than others (

pylint

traverses imports and does crazy inference, whereas

black

does very little work). The tipping point here would be something like

pylint

is faster to read from the cache than run, but for

black

the difference might be in the noise

I tried to be a good boy scout and refactor some duplicated code in

pants.backend.python.target_types_rules

into a helper method, but then Pants gets upset

Exception: pants.backend.python.target_types_rules:134:generate_targets_from_python_tests() returned a result value that did not satisfy its constraints: <coroutine object _generate_targets_helper at 0x7f84820224c0>

(moving to development)

I'm looking for publicly-viewable examples of the Pants plugin API in use, with an eye toward making it easier in the future for folks to discover the API's potential and connect with potential collaborators. I'm aware of these plugins by @chilly-magazine-21545 @average-australia-85137 @magnificent-hamburger-46133 @rapid-bird-79300 @ambitious-actor-36781 etc. Additions to the list would be very much much appreciated! https://docs.google.com/spreadsheets/d/1F3k4T8OkQpZ1pGA6YK-kh5AkbP3ma-C-QDljFi33ZTk/edit?usp=sharing

@limited-insurance-37393 I think we will want to clean this up and open source it for the community at some point, but it getting active development in our monorepo makes the ergonomics difficult until it stabilizes. For now here's our current code packaged in a zip, which we are using in production today 🙂

pdl-pulumi-plugin.zip

FYI the Plugin Upgrade Guide has now been updated to Pants 2.8 and the plugin docs shouldn't be stale anymore. Apologies that that got so out of date! https://www.pantsbuild.org/docs/plugin-upgrade-guide I'm also really hoping to rewrite the example-plugin repo in 2022 to be easier to follow and have more examples

Plugin API feedback wrgt using type hints.. I had to apply this diff, in order to avoid issues in pants: 🧵

FYI I added a new debugging guide for Rule Graph issues, feedback welcomed. It includes a new script to help isolate where issues come from: https://www.pantsbuild.org/docs/rules-api-tips#debugging-rule-graph-issues

I'm playing around with trying to profile a Pants runs by using a WorkUnitCallback, and noticed some gaps in the workunit events that the callback sees. Namely a huge gap where

resolving requirements.pex

could be 🤔

Are there any decent tutorials on creating a plugin in 2.8? I understand the plugin API is not stable.

Let's say I have a list of possible resource paths (relative to some file). How would one request the address for the valid ones (tossing the invalid ones?) E.g. If I have

data/foo.json

I know is relative to some dir

./bar

(e.g.

./bar/data.foo.json

), if it's a valid resource I want it's address, otherwise no-op)

What should the constraints look like that

pants_requirements

produce when running with a

PANTS_SHA

version.. ? Currently I get: 🧵

I'm getting a message: "Unknown entity: prettier" when trying to get the most bare bones plugin working w/

./pants help prettier

🧵

Hello guys, I'm trying to run a simple hand written codegen plugin for pants, but

execute_codegen

function is not get called, although

prepare

is called and

isgentarget

returns

True

. How can I debug this thing?

./pants --version 1.30.4