Why plugin API for

check

goal is different from

lint

? Aren't they supposed to support all the same features like partitioning? Why

check

doesn't have that?

fix

,

fmt

, and

lint

share a lot of rules, so they work very similarly.

check

is separate, so unless someone has copied some of the fix/fmt/lint features to

check

, then the featureset might naturally differ. I don't know specifically about whether

check

has support for partitioning, or what support it has. At least, I believe this is the case. I'm going off my memory 😉, so there's always a chance my humanity is affecting my memory.

Yeah, but the question is why?

fmt

and

lint

used to use separate rules as well. I think around when

fix

was added, they were refactored to use a shared set of rules.

I havent’ looked into this, but I thought mypy uses partitions and the check rule? However, maybe those are hand rolled partitions?

I don't see this Batch magic in the Check request, probably hand rolled

Oh yeah, I recall that now - not there. It would be nice if there was a parallelization/partitioning option by default on the goals, and then specific backends could opt-in/opt-out. While I wasn’t around during any of the design discussions, just off the top of my head, I can imagine that the spirit of the

check

being typechecking/compilation - it’s expected you would put all of the files into a single batch to be operated on (but then, if you type check on multiple Python versions or something, that’s it’s own thing)

Yeah, but the check goal assumes you have some kind of checker that needs not only the sources of the target, but also the dependencies. This is the main difference between lint and check, right? So I want to use check goal for a checker for helm deployments that will build the image and then check that it can import the correct python module inside the docker image. Now every helm deployment is in it's own partition, not cool

Let's see: This is the discussion about adding a fix goal: https://github.com/pantsbuild/pants/issues/13504 This is where batching (partitioning) was added to fmt / lint: https://github.com/pantsbuild/pants/pull/14186 This adds partitioning for MyPy and Pylint (a little bit of discussion about generalizing partitioning): https://github.com/pantsbuild/pants/pull/14321 This is where lint and fmt were combined (it links to a design doc) https://github.com/pantsbuild/pants/pull/14903 This refactors the MyPy and Pylint partitioning: https://github.com/pantsbuild/pants/pull/15141 This refactors lint to prepare to add fix (this describes why some of the design decisions were made): https://github.com/pantsbuild/pants/pull/16735 This refactors fmt following the changes to lint: https://github.com/pantsbuild/pants/pull/16980 This generalizes partitioning logic: https://github.com/pantsbuild/pants/pull/17153 The fix goal was eventually added here as a copy of fmt: https://github.com/pantsbuild/pants/pull/17202 This defines the difference between partition and batch: https://github.com/pantsbuild/pants/pull/17263 This is where fmt and fix were combined: https://github.com/pantsbuild/pants/pull/17223 This refactored fix / fmt / lint partitioning: https://github.com/pantsbuild/pants/pull/17288 This refactors partitioning a bit more to let test partitioning use some of the same logic: https://github.com/pantsbuild/pants/pull/17255 This makes test use the shared partitioning logic: https://github.com/pantsbuild/pants/pull/17134 This adds pytest batching: https://github.com/pantsbuild/pants/pull/17385 This makes fix and lint use more similar partitioning: https://github.com/pantsbuild/pants/pull/19796 So, I think check does not share as much logic because no one has taken the time to figure out what partitioning logic might be shared by check and other goals, and then refactor to make that happen.

Cool! Thank you Jacob for the detailed answer