Hey folks, I've got questions about how I should be using Sources: 1. Ansible has several different artifact types (roles, collections, playbooks). I should be creating a different type of source for each of them, right?

2. some of the Ansible artifact types have a directory structure where each folder contains a different type of resource, and the subcomponents don't really make sense independently. For example, a Role has a folder for "tasks", "meta", "templates", and so on. Should I be creating a different type of Source for each of those? Should I need to put a BUILD file in each of those folders?

Looking for some general guidance on new language plugin development. Right now, I'm adding linters/formatters to a couple of the languages I develop in, so I can monorepo and basically run the same sets of commands on pre-commit and CI for most/all of my repos, and not really think more about it. Giant win with some of my projects, neutral on others. However, after that, I'll be testing out compilation checks ("does this PR compile?") - which is where my question comes in. While it's possible to aggregate all files independently, and compile them, if there is a language with a primary mechanism for handling files/dependencies (think

Package.swift

in swift or

package.json

with Node stuff), should those be pulled in and used as a compilation target in lieu of bundling all sources? I'm not sure if a Slack thread is even the best mechanism for this, as it almost feels like a Pants "evolution" type of question. I'm kinda caught between the two cases, since I don't like duplicating work, but I see the value in going all native Pants too, but, then I think about overreach - and it all spirals in my brain. To clear up something/anything, what I'm trying to describe would be the difference between the following (noting that in both cases, the entire folder and subfolders would be pulled into a hermetic environ): a)

swift build

in a folder with a

Package.swift

which then references the sources/tests b)

swiftc Sources/File1.swift Sources/File2.swift

in the "manual" version which would have a

swift_sources

target

Hey! The plugin docs mention something about streaming output, but there's not a lot there. Is there a way to do that in our own rules?

I have the following code (pants 2.11 rc0) to migrate to the new options system

Getting an error while trying to test a plugin I'm writing, haven't seen this one before:

File "/home/jreed/.cache/pants/named_caches/pex_root/venvs/s/ec6445a5/venv/lib64/python3.9/site-packages/_pytest/config/__init__.py", line 1579, in parse_warning_filter
    category: Type[Warning] = warnings._getcategory(category_)  # type: ignore[attr-defined]
  File "/usr/lib64/python3.9/warnings.py", line 262, in _getcategory
    raise _OptionError("invalid module name: %r" % (module,)) from None
warnings._OptionError: invalid module name: 'pymysql'

This is the first time i'm trying pytest with pants on this container, wonder if it's possible that something is interfering

❗ Format plugin authors: If you see the following ❗

09:51:09.16 [ERROR] 1 Exception encountered:

  AttributeError: 'builtins.Digest' object has no attribute 'digest'

FATAL: exception not rethrown
Aborted (core dumped)

You've upgraded to a version past https://github.com/pantsbuild/pants/pull/14865 Upgrading is easy,

input

and

output

are now

Snapshot

in

FmtResult

. The PR includes multiple examples of how to fix your formatter plugin.

Is

ConfigFilesRequest

the canonical mechanism to grab a config file from a source root? Like a

.clang-format

or

.eslint.rc

or similar? If so, is there any formal documentation around it outside of source code?

Hey folks, I'm (trying to) write tests for my plugin. I'm using the RuleRunner approach just to get some tests for whether anything works at all. I'm hitting a problem with a task to build a PEX, seems it can't find a Python interpreter because the search path is empty.

E           pants.engine.process.BinaryNotFoundError: Was not able to locate a Python interpreter to execute rule code.
E           Please ensure that Python is available in one of the locations identified by `[python-bootstrap] search_path`, which currently expands to:
E             []

I feel like I've overlooked something obvious, can someone help me out?

So I'm going to start writing a (set of?) plugins that might seem a little out of order, and I'm not sure where to start. Basically, I've got a set of things that needs to happen. 1. Do some 'codegen' to inline python code 2. Do some more codegen to generate YAMLs file based on some targets/fields in the BUILD files & environment vars 3. Do some more codegen to create another YAML file referencing the other yaml files 4.

deploy

this to 'the cloud' using a binary blob 5. Run a subset of tests against this code in 'the cloud' 6. also have the ability to

deploy

this manually. of cause, hope to leverage the whole Pants 'don't repeat work' feature to skip steps #1 to #4 if the code doesn't change. I think I've got a grasp on doing the codegen. But can't picture how to make the deploy the code as a dependency for tests work And it's been nearly 6 months since I last used Pants.

How would one go about • having

pants package

also package the dependencies for a given field-set • creating a 'manifest' for all targets who did something as a result of

pants package

? I've got

pants package

generating the files I need to deploy "a widget" using some union rules (those are neat). But those things also depend on other "widgets". And then I need to create a manifest file of all the widgets packaged. Or am I going to have to create a custom goal command to do this?

Stylistic question: If I'm pulling in all files (regardless of extension) in a target, should I create a new target for that (i.e.

ansible_sources

), or just use a

files

or

resources

field? While ansible execution files are largely yaml, there can be any extension of files which are used as resources or references. There is a semantic difference to be had, but not a practical or functional one I feel. For example, I can do something like the following:

files(
    name="myansiblesources",
    sources=["**/*"],
)

or

ansible_sources(
    name="myansiblesources",
    sources=["**/*"],
)

ansible_deployment(
    name="helloansible",
    dependencies=[":myansiblesources"],
    playbook="playbook.yml",
    #inventory="", # TODO: Could this be setup like the Docker registries? in pants.toml
    #tags="", # TODO: Might need a different name to not overlap
    #timeout="",
)

Or, I could get even more specific by calling out the YAML files and then the supplementary files to help with formatting/linting I guess... In summary, I'm not sure about the API 🙂 I'm going forward to get something functional, but in terms of something maximally usable/pragmatic + pants-like

I think

ansible_sources

makes sense, it's where you might put any future extra fields you need that apply ansible-specific metadata to those sources (if such a thing becomes relevant)

Hello! Curious if anyone has made a plugin for

./pants fmt ::

to run

black

over Jupyter Notebooks. Haven't dipped my toes into plugins just yet but feel this might be a decent starting point.

Are

mypy

types included with

pantsbuild.pants

? In a custom plugin, I'm running into a bunch of these... Maybe it's a Monday morning thing, but I thought I had fixed these before, but can't recall anymore:

pants-plugins/experimental/fooify/rules.py:4: error: Cannot find implementation or library stub for module named "experimental.fooify.subsystem"
pants-plugins/experimental/fooify/rules.py:5: error: Cannot find implementation or library stub for module named "experimental.fooify.target_types"
pants-plugins/experimental/fooify/rules.py:6: error: Cannot find implementation or library stub for module named "pants.core.goals.package"
pants-plugins/experimental/fooify/rules.py:7: error: Cannot find implementation or library stub for module named "pants.engine.fs"
pants-plugins/experimental/fooify/rules.py:8: error: Cannot find implementation or library stub for module named "pants.engine.rules"
pants-plugins/experimental/fooify/rules.py:9: error: Cannot find implementation or library stub for module named "pants.engine.target"
pants-plugins/experimental/fooify/rules.py:15: error: Cannot find implementation or library stub for module named "pants.engine.unions"
pants-plugins/experimental/fooify/rules.py:16: error: Cannot find implementation or library stub for module named "pants.util.logging"

How does pants terminate a Process if it gets cancelled? I have a bash script I am running with a Process that needs to do some cleanup on exit that doesn't seem to be doing so reliably. Wondering what I might be doing wrong

"Way more red than green" AKA Pants 2.12.x formatter simpliffication: https://gist.github.com/thejcannon/b6a84c7bfb8b43c4b5af9153379bd62c

Hmm… there’s an issue with using

pants_requirements()

target on the 2.11.0rc1 release.. 🧵

New Pants plugin template using cookiecutter: https://github.com/kaos/cookiecutter-pants-plugin

I'm wondering if I missed something in the docs - is there an easy way to have multiple rules share the same option flags?

I'm just getting

ValueError: empty pattern

When doing a

Get(TransitiveTargets, TransitiveTargetsRequest(...)

Cross-post: https://pantsbuild.slack.com/archives/C0D7TNJHL/p1649424055196659

Whaaaaaaaaaaaaa? amazing

I needed this yesterday!

That's so cool!

is it documented anywhere on how to get all the dependencies for targets?

How do you rename a file in a digest?

I need to extract a zipfile coming from a

DigestContents

. Python's

zipflle

seemingly only works on files on disk. Anyone know how to either materialize the

DigestContents

(into a tempdir or something) or parse the zipfile in-memory?

🧪 ⚡ (It's an absolutely gross misuse of a lot of Pants' features, but IT WORKS!) ⚡ 🧪 I just made a collection of in-house plugins which altogether allows me to relatively easily define Py Extension Modules (including using

pybind11

) and they get inferred, compiled, and included in the dependees' sandbox. Hop in! 🧵

I'm starting to work on some tests for my plugin. I think I need option 4 https://www.pantsbuild.org/docs/rules-api-testing#approach-4-run_pants-integration-tests-for-pants but the example test is failing for me:

pants.base.build_root.BuildRoot.NotFoundError: No build root detected. Pants detects the build root by looking for at least one file from ['pants', 'BUILDROOT', 'BUILD_ROOT'] in the cwd and its ancestors. If you have none of these files, you can create an empty file in your build root.

From the code I would expect

setup_tmpdir

to do this for me. I tried adding an empty file called BUILDROOT to the sources in the example but that didn't fix it either. Is this a known issue? Or am I missing something? I am running 2.11.0.rc2