It seems general consensus for semantic fixers has landed on a new

fix

goal. However this brings up a follow-up issue, which goal should run these to validate code?

lint

or

check

. Which then, IMO, turns into a question of, have we really nailed down the difference between

lint

and

check

w.r.t. which belongs where? I know at least for Python the delineation is fuzzy and even has a few maintainers unsure. I'd like to open a discussion for really nailing down exactly how someone can determine which tool belongs where, and why (with a slight lens on languages where it isn't obvious, like Python or JS). (Note this isnt a discussion about where fixers go, we can have that discussion after this one)

I agree with a) It seems we have reached consensus around

fix

and

fmt

existing as separate goals. b) I too don't love the difference between

check

and

lint

, but specifically for non-compiled languages. For compiled languages, I think it's great having a goal only to run compilation w/o running linters For compiled languages, we have not yet formalized whether

check

should exclusively run compilation, or also be able to run linters? If so, how do we determine whether the linter belongs to

lint

vs

check

?

For context, the only definitions we define (that I can tell) are: `lint`: Run all linters and/or formatters in check mode. `check`: Run type checking or the lightest variant of compilation available for a language.

fixers generally fix lints/warning/deprecations, rather than typechecking errors, so seems like

lint

.

I don't disagree, but that's not this discussion 😛

i think it is… as the definition you pasted indicates: “is it a type checking error or is it a lint”

I should've been more specific by: "I'd like to open a discussion for really nailing down exactly how someone can determine which tool belongs where, and why (with a slight lens on languages where it isn't obvious, like Python or JS)." I meant any tool not just fixers

what are the ambiguities that you see with the existing definition:

is it a type checking error or is it a lint

?

I think a major example of the ambiguity is Pylint - what should it be? It's not really doing type checking, but several have argued it should be in

check

. In the past, it's been proposed that we use the dividing line as

does this tool need to consider transitive deps?

Which leads to my question about compiled languages. If a JVM linting tool needed transitive deps, what goal should it belong to?

Well specifically

mypy

claims to be both a

static type checker

as well as a

lint-like tool

. Languages like Python which use type hints can still be completely valid runnable code with incorrect or missing type hints, just as they can with one or more lint errors.

If we stick with

check

only being type-checking or compilation, then I think there's little ambiguity. The issue would be if we moved Pylint to

check

, or had compiled-language linters using

check

Pylint, somewhat like

mypy

only more extreme is both a linter and "typechecker". It complains about names as well as type issues.

the transitive deps razor is the most convincing for me, although i haven’t thought all the way through what the implications would be.

I like the razor "is this compilation, or if a non-compiled language, is it typechecking" For the razor being transitive deps, my main question is how do we apply that for compiled languages?

(are there other implementations that would get pulled in other than

pylint

?)

Also, to muddy the waters I just filed "Add a flag to `pylint` to disable including dependencies in the sandbox #16303" because I want to do that in my repo 🙂

the reason why it might be useful is folks not realizing that

--changed-dependees=transitive check

is necessary, while dependees/dependencies are not necessary for

lint

. e.g. https://github.com/pantsbuild/pants/issues/16240

Personally, I'd prefer we don't make users consider it at all A la https://github.com/pantsbuild/pants/pull/14182 which is why I don't see the razor as well. Again, for tools like

mypy

and

pylint

it is "valid" to run without dependencies. And we could allow configurability for such. For compiled languages, where you can't make forward progress if your code doesn't compile, the razor seems more natural and users are more dependency-aware (due to tools like GNU make)

Yeah Python really muddies the water here. I'd argue the best solution is to stop supporting

pylint

👿

I'll abstain, I absolutely don't care about the answer to this sort of question in tools I use. I only care we have a clearly communicable stable principle. You're taking that on Josh, so thank you.

I'm also wondering if there are truly lint tools which require dependencies that arent type checkers or compilers 🤔

To add more flame to the discussion heh, one option is for

check

to be called

compile

, and it only works with compiled languages. Linting tools are always

lint

, and compilation is always

compile

. No ambiguity what "check" means (Ack renaming Yet Again would be obnoxious.)

@hundreds-father-404: reminder that

check

for rust doesn’t actually compile

In seriousness though... I don't think there is a stable principle to be found here. Pylint bills itself as a lint tool, has "lint" in it's name, and has been in the

lint

goal for a long time. I don't think there's a compelling reason to move it

Additionally, what about `fix`ers that require dependencies, whould they be run in

check

?

So, imo we have two options to clear up the ambiguity: 1)

check

== compile or type check. For compiled languages, this means all linting tools are run via

lint

, regardless of transitive deps 2) get rid of

check

, and add

compile

which is solely for compiled languages.

I don't think "needs transitive deps" is a particularly good razor in a cross-language context

Agreed. "needs transitive deps" makes sense from a strictly Python perspective, but seems to lose value for compiled languages, where I suspect you usually do need those deps present

isn’t that exactly why compilation goes in that goal…?

To be clear, compilation going in

check

makes sense. Rather, for compiled languages, this is my concern

Which leads to my question about compiled languages. If a JVM linting tool needed transitive deps, what goal should it belong to?

We have not thought about that question, afaict

…anyway, i feel like each time we increase/change the scope of one of these threads we start going in circles. if the goal is to decide whether to move

pylint

, discussing that more concretely would be helpful. the rest of the discussion is too abstract.

I believe the goal is to figure out what the razor line is between

check

vs

lint

. We've said in the past that it's "does this need transitive deps", but haven't formally committed to it because: a) Pylint would be miscategorized w/ that definition b) we haven't confirmed we're okay with linting tools for compiled languages being in

check

So, do we: 1. stick to that razor line? 2. Change razor to something like "either compilation or type checking, but nothing else"? 3. Get rid of

check

and add

compile

? Motivation is

fmt

vs

fix

relates to this conversation, specifically confirming whether

fix

tools should always run with

lint

, rather than

check

@bitter-ability-32190 does that sound right?

Loosely. I'm just concerned that we don't have a well-defined delineation, and our current unspoken one (of requiring deps) isn't valid. And all this in the context of "where does tool X belong?" (and why?)

@bitter-ability-32190 so out of the three options, which are you in favor of? 1) keep razor as "needs transitive deps" 2) change razor to "is compilation or type checking" 3) merge

check

into

lint

, but add

compile

for compiled languages

4) A new razor/definition which likely is closer to "is compilation" (I don't think the goal should be renamed, due to tools like

cargo check

- "This will essentially compile the packages without performing the final step of code generation [...] Some diagnostics and errors are only emitted during code generation, so they inherently won't be reported with

cargo check

." In a C++ backend,

check

v

compile

, I could see a user being amenable to

check

running the compiler but with certain settings to make the operation fast (skips optimization for instance). Where

compile

seems like it does the user-required work, and produces an artifact (either the

obj

or the built lib). Honestly I wish one of us had more TypeScript experience, as that is a "compiled" scripting language and might also help us decide.

TypeScript can vary quite a bit, some people use on-the-fly dynamic compilation, sometimes people require an explicit build step; this depends on a number of things related to your project an whether you're using esx modules or commonjs modules or something like that

Thanks for that context. I think we support that workflow already; note that w/ compiled languages, you don't first have to run

check

to then run things like

package

. Pants will do whatever is necessary beforehand, i.e. on-the-fly dynamic compilation

check

solely exists to allow you explicitly check that compilation works

Yeah I don't think there would be a meaningful distinction between "check" and "compile" in typescript, but i'm not entirely sure. People tend to define lots of commands in their

package.json

file, which then gets run with

npm run my-command

like make files...if you wanted to get a picture of how it's actually used, it might be interesting to sample those in major JS projects

I'm slightly leaning towards "check is compile (and maybe should even be named compile), lint is everything else, including the read-only mode of formatters and of fixers"

I walk away from my computer for an hour, and this is what happens... My "vocal" thoughts : • Not a fan of "transitive dependencies" in the discussion, strictly because

check

is a user-facing API and as a user, I don't want dependency graphs in my brain while figuring out which 3-5 letters to type (regardless of how trivial or not) • Need to be careful with

compile

as a goal because I think it is conflated with "build" or "compile and link" for a lot/most developers, which this goal wouldn't do... Not opposed to a

compile

goal, but that's something to think about ◦ In my C/C++ backend,

check

performs compilation, but there is a separate Linking step that happens on

package

- but I know that's how it should work, not suggesting it's intuitive to someone who is new to Pants • Typescript (the entire JS ecosystem actually) is a strange one, because of all the pseudo-isolated concepts of bundling, tree-shaking (optimization), transpilation, etc - I've been trying to add TS to the JS backend for a while, but I'm stuck in a semantics loop - so excited to hear the result of all this • I'd be fine keeping

check

with the Rust definition mentioned above ("This will essentially compile the packages without performing the final step of code generation") ◦ I'd only want to see

check

changed to

compile

if we're entirely on board, but changing the documentation and slowly moving plugins around while keeping the same name would allow an eventual deprecation and re-naming, without it being too crazy ◦ Could also add to

check

- for non- ahead-of-time compiled languages,

check

will perform typechecking (if possible) • Mentioned above, but I wonder if a

check

vs

lint

rule could be "your code won't run"? ◦ This does exclude interpreted languages from

check

but they can run linters, and this would also be where something like

mypyc

could go for Python, because

check

failures would become make/break

@witty-crayon-22786 How do you feel the definition of

check

shedding itself of "typechecking" and being much closer to "light compilation"

Semi-related aside: Apparently Typescript refers to

tsc

as a compiler. https://www.typescriptlang.org/docs/handbook/compiler-options.html So, go figure 🤷

https://pantsbuild.slack.com/archives/C046T6T9U/p1658868458322549?thread_ts=1658857985.109339&cid=C046T6T9U that seems like it would mean removing

mypy

from

check

which doesn’t make sense to me for practical purposes.

doesn’t make sense to me for practical purposes

Can you elaborate?

i use

check

to decide whether it’s going to be worthwhile to run a bunch of tests… i don’t care about the rest of

lint

for that purpose, and so i don’t

lint

until i’m ~done with code

Would you say

check

then is something a la "check for code correctness?"

it would be for

python

, but not for the other languages. would be better not to have to type out the specific

check

implementation i want (

scalac

,

rustc

, etc)

So would we agree that the "dependency" razor between the two is no longer our unspoken delineator? If so, I think we're making forward progress 🙂

agreed that that is an attempt to solve another problem that might potentially be better solved by https://github.com/pantsbuild/pants/pull/14182 but it continues to not help you answer the question of where to put

pylint

, so whether that is progress depends on the goal of the thread

Ultimately the goal of the thread is to nail down definitions, so that fixers aren't a hot potato exactly like

pylint

is 🙂

So, to you,

check

is more about ensuring it’s even worth testing (or executing the next goal)?

yea. “typechecking” is, to be clear.

How do you feel about the fact that

mypy

can fail, but tests can pass?

no compiler/typechecker can solve that problem statically, but they’re still worth running… not sure i understand the question

Well for a compiled language I'd expect if `check`fails, tests can't be run at all let alone pass

Yeah, that is an alternative wording for what I mean with option 2:

2) change razor to "is compilation or type checking"

And it actually matches our current

./pants help

definition. I'm generally very open to that outcome, as long as we follow it through to its logical conclusion: 1) For compiled languages, linting tools should not be run with

check

. Only the compilers themselves are

check

, e.g.

cargo check

vs

cargo clippy

(linter) 2) Still don't know what Pylint is. I think it's less of a type checker, it's not MyPy

I would expect pylint to stay. It is a linter which, in a time before typecheckers, included some light "type checking". In today's world, no one would consider it a typechecker

Makes sense to me 🙂 A nice benefit of option 2 is that it will result in zero changes to users. We haven't yet added any compiled language linters under

check

. This would only tighten up our semantics

It's probably indicative of a broader problem in that you can't ad-hoc pick out a member of a union when running any goal today. That'd be nice: "I know I want to run X that happens to be in goal Y, but I have to run everything else in that goal too :/".

It's probably indicative of a broader problem in that you can't ad-hoc pick out a member of a union when running any goal today.

Sort of.

./pants fmt --only=black --only=isort

exists, along with

./pants --black-skip fmt

The --only stuff is super-ad-hoc and boilerplately though.

Oh, this thread was way longer than my brain can absorb right now. 😛 (in the middle of trying to secure a place to stay in Toronto) I’ll just throw this wild idea out here and do a 🎤 💧 I feel a big issue is trying to find goal vocabulary that will make sense for any tool for any language. That feels like it could be an impossible task. So how about instead look for a way to more dynamically tie tools to goals in a more configurable way. That is,

lint

is just a configuration for running tools

a

b

and

c

. for instance. If a users feels strongly that it doesn’t make sense can move tool

b

to

check

instead, in their project.

Yeah, that's what we had in v1.

The only minor detail there is how to run them in "check my code, don't change it" mode.

I feel a big issue is trying to find goal vocabulary that will make sense for any tool for any language. That feels like it could be an impossible task. So how about instead look for a way to more dynamically tie tools to goals in a more configurable way. That is,

lint

is just a configuration for running tools

a

b

and

c

. for instance. If a users feels strongly that it doesn’t make sense can move tool

b

to

check

instead, in their project.

While I like a good mic drop as much as the next person, this feels like it's moving in the wrong direction - basically over-configurability. It's the whole "anything can be anything" which is usually fine in the moment, but I've had to maintain projects with this type of dev tool configuration (and sometimes even production configuration), and it's a hard pass. It ends up moving this entire Slack thread to multiple companies over multiple years of bike-shedding I'm an advocate of getting tools that make sane decisions, with some configuration as an aside: Black, Prettier, SvelteKit, etc

+1. And if you don't like how Pants configured something, you're very welcome to disable the backend and re-implement with your own plugin - that is a key design goal of the engine to enable that

No more makefiles! I'm turning to pants to escape that customized this is fine fire

So, the next steps in this discussion: 1) Is anyone advocating for MyPy moving to

lint

, and

check

solely being compilation, aka "option 3"? If not, we can probably stop considering this 2) Do folks agree that for compiled languages, linting tools should never run under

check

, as they are not compilation?

0) Is there agreement that

check

means compilation without linking/codegen and for non-compiled languages, it's a best-effort typecheck (mypy, pyright, tsc, Flow, etc)?

Personally, and with the upcoming cache feature I need to test things, but Im still interested in batched mypy runs. Other than that, I don't think which goal it is in important once we fix all the other known issues (--changed=implicit, and parallel goals)