Hi all, have a question. Was looking at the PR merged in for black support assuming it would be available under the lint goal. After digging through some threads, it seems like this is a part of a larger effort of a v2 feature set. Attempted to run

./pants lint-v2 <target>

on

1.23.0rc0

and ran into an error:

ERROR: Not a registered union type: <class 'pants.engine.rules.TargetWithSources'>

Which led me here: https://github.com/pantsbuild/pants/pull/8490. And I figured I was heading deeper down the rabbit hole. So question is, are v2 goals usable outside of pants development itself? and if so what's the uplift of bringing them into our project

(1) v2 goals are usable outside of pants development itself, but development on that front is currently fast and furious -- see https://github.com/pantsbuild/pants/pull/8760 which i did over thanksgiving, and which contains a ton of hacks that required a lot of implementation knowledge. (2) the biggest single blocker to using v2 rules for your own repo right now is probably https://github.com/pantsbuild/pants/issues/4535 -- designing the v2 target API. in short, v1 targets expect to have an entire build graph available, and that's part of why v2 goals are faster (

./pants list

is only in v2 --

./pants fast-filedeps

also works). the approach i took in #8760 to hack around that is just as slow as v1, as a result. (3)

@union

and

UnionRule

are intended to make `@rule`s extensible by allowing acting on an abstract description of an object, and are some of the more complex parts of this quickly-evolving API. see https://github.com/pantsbuild/pants/pull/8542 for an example of an attempt to try to make using them simpler -- but the takeaway is that error messages are bad, and we're sorry, and we're definitely working to fix that as we speak as non-Twitter folks begin to start picking up the v2 engine.

Hey @aloof-angle-91616 thanks for the quick reply! I'm in transit right now but I will take a look at all the PRs you provided in the morning. Will definitely take a stab at the implementation because bringing this into our workflow would be pretty helpful. Will circle back tomorrow and let you know what I find. Also thanks for the help! Will join the #engine channel and move discussion there if it comes up

Hey @damp-quill-59187, for this particular case, I discovered the same thing today when trying to use Black on a project outside of the Pants repo itself. It’s now a known issue, but I haven’t had a chance to figure out the fix yet. Hoping to spend some time on it this week or early next week The above PRs and engine channel are good overall context about V2, although not particularly relevant to trying to use Black

(if you productionize the use of black, i and twitter would be extremely interested!)

(if you productionize the use of black, i and twitter would be extremely interested!)

It, and V2 isort, are good to go generally. We are about to start using V2 isort in Pants internally now. The issue is in how we distribute the plugins via

backend_packages

. You’re supposed to activate them by adding

pants.backend.python.lint.black

or

pants.backend.python.lint.isort

to your `pants.ini`’s

backend_packages

and that works for Pants but not for external users due to the stack trace Tennyson posted at the start of this thread.

yes! i am referring to support in the monorepo which despite its name does not consume pants from source

Hey @hundreds-father-404, thanks for the heads up. I'll take a look at it and see if there's anything that quickly jumps out at me, but obviously not super familiar with the Pants internals presently. Thanks for the resources and if you happen to make any progress in your efforts, feel free to ping me. Happy to help out where I can.

if the full stack trace was provided i might be able to help, i don’t have the context eric does

@aloof-angle-91616 will update here with it in the am

thank you!! no rush!!! just making it clear i am open to help

adding @wonderful-iron-54019 for context here

@damp-quill-59187 @wonderful-iron-54019 this is now my highest priority issue I’m working on. Hope to have a fix in either this week’s dev release or next week’s

Hey Eric, should I pull down a new version? When I tried that previously, it couldn't find the module

We got it working with

1.24.0.dev1

will try to update to that

Can you try

pants.backend.python.lint.isort

please?

When I went down this path the last time, I tried to see if I could figure out where pants was sourcing the modules from/where it was installed

Yes, add

backend_packages: +[
    "pants.backend.python.lint.black",
  ]

And don’t include

black

as a part of that option for now

lint-v2 seems to run now, although I'm unsure, i'll try to edit things so the python isn't parseable and see if i fails

To see if isort is working correctly, try reording one of your imports to not be alphabetical. For example,

from animals import dog, cat

instead of

from animals import cat, dog

Yep, looks like that worked actually

Note when testing that you can only have one or the other formatters installed currently. You cannot have both

isort

and

black

in your

backend_packages

. (We’re working on fixing that)

is there anything additional I'd need to do to enable black?

You should be able to simply add to

pants.ini

backend_packages: +[
    "pants.backend.python.lint.black",
  ]

cool, so I'm going to try the following: Copy black directory into pants by hand Remove isort and add black to backend run lint-v2

Awesome, glad you got it working! Still an open question why we were both able to get isort working how it’s supposed to be setup, but you had to go through all this extra trouble for Black. That should not be the case. I’ll ping this thread once we get things working as intended so that you can remove this workaround from your codebase Thank you for raising this issue and iterating on a workaround! Also let us know any feedback on

fmt-v2

and

lint-v2

! They’re seeing a lot of active development. One nice benefit is that once you run the command on a target, so long as no changes are made to that target, the result will be cached. So, you can run

./pants fmt-v2 ::

and Black will only run over any new changes!