I played a bit with possibilities around Allow using a shell command/adhoc tool as part of any goal (check, lint, fmt, fix, deploy, ...) in this comment: https://github.com/pantsbuild/pants/issues/17729#issuecomment-1741198969 . There is a crude demo of having a user put in a small amount of configuration and get some of the tools they have installed participate in

pants lint

That's interesting. I was thinking of it as enhancing

BUILD

files to allow more functionality to be specified there, but it seems you're approaching it from the perspective of "make plugins simpler/more stable" which seems like a reasonable one too. 🤔 Do you have an idea about how it might work to have an in-repo executable that runs as the linter/formatter? (For example's sake, let's say a

pex_binary

that is checking

.md

files)

Yes have some thoughts about In repo python code though it would be separated into the plugins dir with no possibility of dep on non-plugin code in the same repo. Using a PEX on a third party lib (like flake8 or mdformat) should also be smooth in terms of user API. We would take the golden path that already exists around PythonToolBase and make the "simple" case very easy to express.

My thinking was target-based linting/formatting etc... How does this compare?

Not sure yet how it would compare. Will push it a bit further to feel what fmt/fix would look like and then make it into some proposal with an accompanying unpolished/poc impl.

Second question would be does this work completely in a plugin or are there core pants changes?

So far it looks possible with no core changes. I did have to add a little escape hatch somewhere in the subsystem mechanics, but I've been doing it quickly and maybe it is avoidable.

You'd be surprised. I think making ad-hoc linting/testing (less so formatting, but still...) is extremely desirable

I definitely see value in something like this, for those languages or tools that come out where we want to offer support that’s a bit more built into the system. Something that would pique my interest is whether we’d have a setup robust enough to pull our linters/formatters into it - reducing our code base

That is an interesting target... To explore in the direction of something that could plausibly replace some of the built-in linters. That has a bit different profile than the "I have a program I use to lint stuff, let me use it in pants lint without having to program Get" user persona I had in mind. I'm getting enough encouragement to keep pushing. I'll try to enumerate the features of some of the built-in linters and consider which are easy and hard to replicate with the approach in the poc.

I think @careful-address-89803 Had been messing around with similar things in the past too

I still think a target-based approach will likely be the most flexible solution, FWIW

This is really cool! The thing I was working on was on the python side, it would essentially take the python package you wanted to install and a cli invocation and synth all the rules to make a linter (including lockfiles and export). https://github.com/pantsbuild/pants/pull/16412 This has a few killer usecases for me. One of them is pulling unsupported tools/languages into the pants world. For example, running yamllint on yaml files. Another is on pulling ad-hoc checks into the pants world. For example, we have a few scripts to check that a MR is ready to merge, such as incrementing the version number, adding to the changelog, and so on. Great work!

@careful-address-89803 I just looked through your PR description in detail and the src code at a glance. I am so glad to know I'm not the only one finding myself barking up this same tree. Your approach looks way more feature-filled for the case that it solves. I need to study it. Next week I'll try to put some doc together with our two approaches and measure them against some of the use-cases. Then we can figure out which one to take as a base for further development.

my MR is a bit old. The generation of export rules got spun off into its own MR (now merged), and lockfile rules got a similar treatment. I hope to look deeply into your MR later. One difference I saw is that metalint allows many of its "internal" components to be overridden, either as passed-in args or in the subclass of MetalintTool. That gives it a lot of flexibility, although I'm not sure its necessary; ByoTool might just be powerful enough. I think every other feature metalint has would be pretty easy to port over/into ByoTool. I think one area that ByoTool might struggle in is leveraging dependency inference. I think dependency inference would require moving from pathglobs to fieldsets, and I think the logic there might get a little involved. Although, we handle dependencies by delegating to each langauge's implementation, so it might turn out to be simple. It also might be fine for ByoTool to just not support that. If you only have to lint a few files (or the file type doesn't even have dependencies!) that's probably fine, and if it becomes a problem then that's a sign to write a full plugin.

Interesting, so I had ByoTool using fieldsets at first, but switched it since I was thinking of users who had, say

.md

files or

.yaml

or anything else that doesn't already have a natural target aside from

file

. I was targeting a user who wouldn't need to know about fieldsets (and how they are not the same as targets and all that!). I didn't realize that dependency inference comes up during these kinds of tasks. I was studying the implementation of shellcheck, flake8, yamlllint and black (for fmt). My impression was that dep inference and target dependencies in general are needed mostly for

check

. But that lint, fmt, fix generally are expected to operate on each source file in isolation (they may need lint configuration files).

I think I'm this instance, path gonna are likely closer to what will be most useful for the user. Since they're already outside of plugins, fields sets likely won't be something they understand, and defining targets just for lint seems silly I think what @careful-address-89803 meant wasn't dep inference, but dependencies. Without a dependencies field you don't get dependencies in the sandbox. I think that's fine since most of what the user wants to do right involve dependencies, and if they need them they can just write a plugin

Just wanted to show some kind of progress here, ended up re-documenting how flake8 works just to establish a kind of baseline, and haven't gotten started with making the feature list covering existing linters and then our various proposals. I haven't figured out how to share google drive files but, for now this is something on my personal drive which should be readable by anyone with the link. Will make it editable as soon as we can figure out some pants-friendly home for it. https://docs.google.com/presentation/d/15P0YdoX1T4fX0DpNzwwXxP1bMVZqUZ_GB9Jkn3rQwRQ/edit?usp=sharing

At some point we really ought to chat, in richer detail, about all the work you've done, and the direction I have stuffed in my head. FWIW I see a lot of overlap, and thats a good thing. Unfortunately I never got the stomach to further the efforts I put in a while back, so if you wanna take the helm, please do 🙂

oh, don't worry! this work is just pre-work, the stuff in the slides. Like just establishing a baseline of what there is now in pants main, not any of our pilots/proposals. I have not been pouring any further effort into the particular implementation I was hacking on, but rather into how we can have a frame for the discussion.

Always 🙂

out for the day, only allowing myself to work on pants until lunch. After that need to do responsible grown-up stuff like look for my next $-provider. 😛

I'd prefer if we chunked off some time on a calendar to go through it, otherwise knowing myself, it'll be hard to focus on it 🙂

+1 let's do it. I will dm on Monday to figure out a time in the rest of that week.

• The slides are great! Thank you for putting them together • As a minor but concrete use case, something I would eventually like to do is move all of our

pre-commit

checks into pants. Many of them are hardly more than oneliners like

end-of-file-fixer

and

check-executables-have-shebangs

and not so much a self contained tool.

You can techincally do that today with https://www.pantsbuild.org/docs/reference-experimental_test_shell_command

hmm, now that I actually look most of these are python scripts. With the shell command I'm at the mercy of "whatever python people happen to have installed", and don't get pants to find (or provide!) the right python?

Depends on how its implemented

@curved-manchester-66006 could you describe a bit more how you have those set up? Could you add it in the issue. I'm looking for more examples and particles to throw at the various emerging proposals.

Sure, I'll elaborate in a comment there.

@bitter-ability-32190 https://github.com/pantsbuild/pants/issues/17729#issuecomment-1766816206 is a proposal for a user api that leans on targets for building the runnables but leaves the rest of the configuration in the conf side. It looks promising to me ...

Looks absolutely amazing. I'm thrilled by this