Ive noticed that pretty much all the linting/codegen/whatever stuff has been built around calling executables that have been installed... Currently we've built all our tooling around just doing it in straight python. but it seems unidiomatic, and is even causing problems (ie. im building a custom linter and cant work out how to get pants to show its output) Is it better to convert stuff to pex_binary targets, and then somehow call those? Would be neat if there was like a

PantsTargetProcess

that took an

Address

instead of a .exe

Is it better to convert stuff to pex_binary targets, and then somehow call those?

Are you referring to the targets you're calling tooling upon? Or the new linters themselves?

around calling executables that have been installed...

The pattern for Python lint/fm tooling (and other languages, if those support pip installs) is for your newly defined Subsystem to be used to create a pex which is then called inside your rules. I was also able to get away with this with

clang-format

Is that what you're referring to as well, or do you mean system installed tools? Or, do you mean creating custom lint/fmt code in the plugin itself?

I think he means the latter. It would be nice if Pants allowed you to place tool code in-tree and then just point at that tool-code's target address and say run me a Process that uses that address as the tool binary / argv0. That would cover the in-repo plugin case. It would further be nice to make it easy to turn that same arrangement into a plugin distribution that could be installed by others with minimal fuss. As things stand today, there is a bit more work to get in-repo tool binary sources massaged into the Digest of a binary you can use to run a Process.

Currently we’ve built all our tooling around just doing it in straight python. but it seems unidiomatic, and is even causing problems (ie. im building a custom linter and cant work out how to get pants to show its output)

if you are running

@rule

code which is intended to have side-effects, you’ll need to watch out for how things are marked as cacheable. for example:

LintResults

extends

EngineAwareReturnType

and sets `cacheable=False`: https://github.com/pantsbuild/pants/blob/eeb3fd97710c9f37fd7344ee584f263065ac33e7/src/python/pants/core/goals/lint.py#L160-L162 … to indicate that the the

@rule

that produces it should always re-run to render its message

@wide-midnight-78598 Yeah, we've written custom in-repo, python-based linters and formatters. And instead of calling them with

Process

we're just importing and calling the python directly. Which in the case of a linter, means faking an exception/return into something that looks like a

ProcessResult

Yeah, @ambitious-actor-36781 you really want to use a Process for a host of reasons. For one, most Python things care about the Python version and Pants runs in one, say 3.7 and your code may need another, say 3.10. This can be handled out-of-process but not in-process.

yeah. But by the sounds of it, there isn't a mechanism to easily do it entirely in-repo right?

Its not easy, but it's not super hard either. I can point to examples, just a sec.

Any reason I couldn't just get the packaged artefact (.pex) for a known-targets address?

Not sure, give it a whirl.

hmm. it can be done! but will need more work than I can do at work. Might work on it tonight