https://pantsbuild.org/ logo
#general
Title
# general
b

bitter-ability-32190

07/26/2022, 5:53 PM
It seems general consensus for semantic fixers has landed on a new
fix
goal. However this brings up a follow-up issue, which goal should run these to validate code?
lint
or
check
. Which then, IMO, turns into a question of, have we really nailed down the difference between
lint
and
check
w.r.t. which belongs where? I know at least for Python the delineation is fuzzy and even has a few maintainers unsure. I'd like to open a discussion for really nailing down exactly how someone can determine which tool belongs where, and why (with a slight lens on languages where it isn't obvious, like Python or JS). (Note this isnt a discussion about where fixers go, we can have that discussion after this one)
👀 1
1
h

hundreds-father-404

07/26/2022, 5:56 PM
I agree with a) It seems we have reached consensus around
fix
and
fmt
existing as separate goals. b) I too don't love the difference between
check
and
lint
, but specifically for non-compiled languages. For compiled languages, I think it's great having a goal only to run compilation w/o running linters For compiled languages, we have not yet formalized whether
check
should exclusively run compilation, or also be able to run linters? If so, how do we determine whether the linter belongs to
lint
vs
check
?
b

bitter-ability-32190

07/26/2022, 5:56 PM
w

witty-crayon-22786

07/26/2022, 6:19 PM
fixers generally fix lints/warning/deprecations, rather than typechecking errors, so seems like
lint
.
👍 1
b

bitter-ability-32190

07/26/2022, 6:20 PM
I don't disagree, but that's not this discussion 😛
w

witty-crayon-22786

07/26/2022, 6:21 PM
i think it is… as the definition you pasted indicates: “is it a type checking error or is it a lint”
b

bitter-ability-32190

07/26/2022, 6:22 PM
I should've been more specific by: "I'd like to open a discussion for really nailing down exactly how someone can determine which tool belongs where, and why (with a slight lens on languages where it isn't obvious, like Python or JS)." I meant any tool not just fixers
1
w

witty-crayon-22786

07/26/2022, 6:23 PM
what are the ambiguities that you see with the existing definition:
is it a type checking error or is it a lint
?
one i see is that “lint” is not very well defined… type checking is though.
JS has a few different type checkers though, so i don’t think there is likely to be ambiguity there: “run
flow
, the typescript compiler, etc”
h

hundreds-father-404

07/26/2022, 6:26 PM
I think a major example of the ambiguity is Pylint - what should it be? It's not really doing type checking, but several have argued it should be in
check
. In the past, it's been proposed that we use the dividing line as
does this tool need to consider transitive deps?
Which leads to my question about compiled languages. If a JVM linting tool needed transitive deps, what goal should it belong to?
1
b

bitter-ability-32190

07/26/2022, 6:26 PM
Well specifically
mypy
claims to be both a
static type checker
as well as a
lint-like tool
. Languages like Python which use type hints can still be completely valid runnable code with incorrect or missing type hints, just as they can with one or more lint errors.
h

hundreds-father-404

07/26/2022, 6:28 PM
If we stick with
check
only being type-checking or compilation, then I think there's little ambiguity. The issue would be if we moved Pylint to
check
, or had compiled-language linters using
check
b

bitter-ability-32190

07/26/2022, 6:30 PM
Pylint, somewhat like
mypy
only more extreme is both a linter and "typechecker". It complains about names as well as type issues.
👍 1
I think the question I have is "should code be presumably runnable if
check
fails?". Or put another way, when we say "type checking" do we mean "strict" or "loose" checking?
w

witty-crayon-22786

07/26/2022, 6:36 PM
the transitive deps razor is the most convincing for me, although i haven’t thought all the way through what the implications would be.
h

hundreds-father-404

07/26/2022, 6:37 PM
I like the razor "is this compilation, or if a non-compiled language, is it typechecking" For the razor being transitive deps, my main question is how do we apply that for compiled languages?
w

witty-crayon-22786

07/26/2022, 6:37 PM
(are there other implementations that would get pulled in other than
pylint
?)
b

bitter-ability-32190

07/26/2022, 6:38 PM
Also, to muddy the waters I just filed "Add a flag to `pylint` to disable including dependencies in the sandbox #16303" because I want to do that in my repo 🙂
Personally, I'm not a fan of the transitive deps razor, as it isn't user-friendly IMO in these kinds of cases. Users likely don't think about
pylint
requiring transitive deps because its only ever been run in an env containing all possible deps. Combine that with the obvious name association and it can get really confusing.
2
w

witty-crayon-22786

07/26/2022, 6:50 PM
the reason why it might be useful is folks not realizing that
--changed-dependees=transitive check
is necessary, while dependees/dependencies are not necessary for
lint
. e.g. https://github.com/pantsbuild/pants/issues/16240
(haven’t responded there yet, but: i need to point this out)
b

bitter-ability-32190

07/26/2022, 6:52 PM
Personally, I'd prefer we don't make users consider it at all A la https://github.com/pantsbuild/pants/pull/14182 which is why I don't see the razor as well. Again, for tools like
mypy
and
pylint
it is "valid" to run without dependencies. And we could allow configurability for such. For compiled languages, where you can't make forward progress if your code doesn't compile, the razor seems more natural and users are more dependency-aware (due to tools like GNU make)
I'm gonna do the annoying thing and pull in a few vocal community voices 🙂 CC @wide-midnight-78598 @flat-zoo-31952 @happy-kitchen-89482 @enough-analyst-54434 @curved-television-6568
❤️ 1
f

flat-zoo-31952

07/26/2022, 7:01 PM
Yeah Python really muddies the water here. I'd argue the best solution is to stop supporting
pylint
👿
🎉 1
1
e

enough-analyst-54434

07/26/2022, 7:01 PM
I'll abstain, I absolutely don't care about the answer to this sort of question in tools I use. I only care we have a clearly communicable stable principle. You're taking that on Josh, so thank you.
👍 2
b

bitter-ability-32190

07/26/2022, 7:02 PM
I'm also wondering if there are truly lint tools which require dependencies that arent type checkers or compilers 🤔
👍 1
h

hundreds-father-404

07/26/2022, 7:03 PM
To add more flame to the discussion heh, one option is for
check
to be called
compile
, and it only works with compiled languages. Linting tools are always
lint
, and compilation is always
compile
. No ambiguity what "check" means (Ack renaming Yet Again would be obnoxious.)
w

witty-crayon-22786

07/26/2022, 7:03 PM
@hundreds-father-404: reminder that
check
for rust doesn’t actually compile
👍 1
f

flat-zoo-31952

07/26/2022, 7:03 PM
In seriousness though... I don't think there is a stable principle to be found here. Pylint bills itself as a lint tool, has "lint" in it's name, and has been in the
lint
goal for a long time. I don't think there's a compelling reason to move it
b

bitter-ability-32190

07/26/2022, 7:04 PM
Additionally, what about `fix`ers that require dependencies, whould they be run in
check
?
h

hundreds-father-404

07/26/2022, 7:04 PM
So, imo we have two options to clear up the ambiguity: 1)
check
== compile or type check. For compiled languages, this means all linting tools are run via
lint
, regardless of transitive deps 2) get rid of
check
, and add
compile
which is solely for compiled languages.
f

flat-zoo-31952

07/26/2022, 7:05 PM
I don't think "needs transitive deps" is a particularly good razor in a cross-language context
1
👍 2
h

hundreds-father-404

07/26/2022, 7:06 PM
Agreed. "needs transitive deps" makes sense from a strictly Python perspective, but seems to lose value for compiled languages, where I suspect you usually do need those deps present
w

witty-crayon-22786

07/26/2022, 7:06 PM
isn’t that exactly why compilation goes in that goal…?
h

hundreds-father-404

07/26/2022, 7:08 PM
To be clear, compilation going in
check
makes sense. Rather, for compiled languages, this is my concern
Which leads to my question about compiled languages. If a JVM linting tool needed transitive deps, what goal should it belong to?
We have not thought about that question, afaict
w

witty-crayon-22786

07/26/2022, 7:09 PM
…anyway, i feel like each time we increase/change the scope of one of these threads we start going in circles. if the goal is to decide whether to move
pylint
, discussing that more concretely would be helpful. the rest of the discussion is too abstract.
h

hundreds-father-404

07/26/2022, 7:14 PM
I believe the goal is to figure out what the razor line is between
check
vs
lint
. We've said in the past that it's "does this need transitive deps", but haven't formally committed to it because: a) Pylint would be miscategorized w/ that definition b) we haven't confirmed we're okay with linting tools for compiled languages being in
check
So, do we: 1. stick to that razor line? 2. Change razor to something like "either compilation or type checking, but nothing else"? 3. Get rid of
check
and add
compile
? Motivation is
fmt
vs
fix
relates to this conversation, specifically confirming whether
fix
tools should always run with
lint
, rather than
check
@bitter-ability-32190 does that sound right?
b

bitter-ability-32190

07/26/2022, 7:22 PM
Loosely. I'm just concerned that we don't have a well-defined delineation, and our current unspoken one (of requiring deps) isn't valid. And all this in the context of "where does tool X belong?" (and why?)
👍 1
mypy
and
pylint
come up often because they are deeply related in a typechecking sense, are in different goals, and have varying degrees of requiring dependencies / "checking" the code for "compilability".
👍 1
Personally, I don't agree with the razor at all. • Those tools don't require dependencies (they simply work best when provided dependencies) • I can still run all my code and have high confidence in it without using them at all (vs. say a C++ typechecker, which you'd fail to produce an obj if it failed) • We run MyPy in a similar fashion to
pylint
as well as
flake8
, but without the batching and parallelization. (Part of this of course, is that open ticket about possibly treating it as a compiler, but to me this is completely a Pants-ism. Ask anyone in the Python community if you should run
mypy
recursively like a compiler and they'll go "what?!") •
lint
runs linters and formatters. should
check
encompass fixers that require deps, if thats the razor?
👍 1
h

hundreds-father-404

07/26/2022, 7:43 PM
@bitter-ability-32190 so out of the three options, which are you in favor of? 1) keep razor as "needs transitive deps" 2) change razor to "is compilation or type checking" 3) merge
check
into
lint
, but add
compile
for compiled languages
b

bitter-ability-32190

07/26/2022, 7:48 PM
4) A new razor/definition which likely is closer to "is compilation" (I don't think the goal should be renamed, due to tools like
cargo check
- "This will essentially compile the packages without performing the final step of code generation [...] Some diagnostics and errors are only emitted during code generation, so they inherently won't be reported with
cargo check
." In a C++ backend,
check
v
compile
, I could see a user being amenable to
check
running the compiler but with certain settings to make the operation fast (skips optimization for instance). Where
compile
seems like it does the user-required work, and produces an artifact (either the
obj
or the built lib). Honestly I wish one of us had more TypeScript experience, as that is a "compiled" scripting language and might also help us decide.
1
f

flat-zoo-31952

07/26/2022, 8:10 PM
TypeScript can vary quite a bit, some people use on-the-fly dynamic compilation, sometimes people require an explicit build step; this depends on a number of things related to your project an whether you're using esx modules or commonjs modules or something like that
👍 1
I'm not hugely experienced in typescript though. Nobody uses on-the-fly dynamic compliation for prod code, only for doing things quickly in development and in test, but it does exist as an option.
h

hundreds-father-404

07/26/2022, 8:15 PM
Thanks for that context. I think we support that workflow already; note that w/ compiled languages, you don't first have to run
check
to then run things like
package
. Pants will do whatever is necessary beforehand, i.e. on-the-fly dynamic compilation
check
solely exists to allow you explicitly check that compilation works
f

flat-zoo-31952

07/26/2022, 8:18 PM
Yeah I don't think there would be a meaningful distinction between "check" and "compile" in typescript, but i'm not entirely sure. People tend to define lots of commands in their
package.json
file, which then gets run with
npm run my-command
like make files...if you wanted to get a picture of how it's actually used, it might be interesting to sample those in major JS projects
👍 1
1
h

happy-kitchen-89482

07/26/2022, 8:30 PM
I'm slightly leaning towards "check is compile (and maybe should even be named compile), lint is everything else, including the read-only mode of formatters and of fixers"
1
But could be convinced otherwise
w

wide-midnight-78598

07/26/2022, 8:36 PM
I walk away from my computer for an hour, and this is what happens... My "vocal" thoughts : • Not a fan of "transitive dependencies" in the discussion, strictly because
check
is a user-facing API and as a user, I don't want dependency graphs in my brain while figuring out which 3-5 letters to type (regardless of how trivial or not) • Need to be careful with
compile
as a goal because I think it is conflated with "build" or "compile and link" for a lot/most developers, which this goal wouldn't do... Not opposed to a
compile
goal, but that's something to think about ◦ In my C/C++ backend,
check
performs compilation, but there is a separate Linking step that happens on
package
- but I know that's how it should work, not suggesting it's intuitive to someone who is new to Pants • Typescript (the entire JS ecosystem actually) is a strange one, because of all the pseudo-isolated concepts of bundling, tree-shaking (optimization), transpilation, etc - I've been trying to add TS to the JS backend for a while, but I'm stuck in a semantics loop - so excited to hear the result of all this • I'd be fine keeping
check
with the Rust definition mentioned above ("This will essentially compile the packages without performing the final step of code generation") ◦ I'd only want to see
check
changed to
compile
if we're entirely on board, but changing the documentation and slowly moving plugins around while keeping the same name would allow an eventual deprecation and re-naming, without it being too crazy ◦ Could also add to
check
- for non- ahead-of-time compiled languages,
check
will perform typechecking (if possible) • Mentioned above, but I wonder if a
check
vs
lint
rule could be "your code won't run"? ◦ This does exclude interpreted languages from
check
but they can run linters, and this would also be where something like
mypyc
could go for Python, because
check
failures would become make/break
2
b

bitter-ability-32190

07/26/2022, 8:47 PM
@witty-crayon-22786 How do you feel the definition of
check
shedding itself of "typechecking" and being much closer to "light compilation"
w

wide-midnight-78598

07/26/2022, 8:54 PM
Semi-related aside: Apparently Typescript refers to
tsc
as a compiler. https://www.typescriptlang.org/docs/handbook/compiler-options.html So, go figure 🤷
w

witty-crayon-22786

07/26/2022, 8:55 PM
https://pantsbuild.slack.com/archives/C046T6T9U/p1658868458322549?thread_ts=1658857985.109339&cid=C046T6T9U that seems like it would mean removing
mypy
from
check
which doesn’t make sense to me for practical purposes.
b

bitter-ability-32190

07/26/2022, 8:57 PM
doesn’t make sense to me for practical purposes
Can you elaborate?
w

witty-crayon-22786

07/26/2022, 8:58 PM
i use
check
to decide whether it’s going to be worthwhile to run a bunch of tests… i don’t care about the rest of
lint
for that purpose, and so i don’t
lint
until i’m ~done with code
failing typechecks are not lints to me… they’re “highly likely to cause a crash”
b

bitter-ability-32190

07/26/2022, 9:00 PM
Would you say
check
then is something a la "check for code correctness?"
Also, and I know this a little in-the-weeds, but
./pants lint --only=mypy ...
would be your spiritual equivalent
w

witty-crayon-22786

07/26/2022, 9:02 PM
it would be for
python
, but not for the other languages. would be better not to have to type out the specific
check
implementation i want (
scalac
,
rustc
, etc)
at which point we’re back to
./pants lint --typecheckers
or something.
imo, typechecking is well defined, so i feel like it remains a nearly sufficient razor. it’s only “lint” that is nebulous.
b

bitter-ability-32190

07/26/2022, 9:05 PM
So would we agree that the "dependency" razor between the two is no longer our unspoken delineator? If so, I think we're making forward progress 🙂
👍 1
w

witty-crayon-22786

07/26/2022, 9:09 PM
agreed that that is an attempt to solve another problem that might potentially be better solved by https://github.com/pantsbuild/pants/pull/14182 but it continues to not help you answer the question of where to put
pylint
, so whether that is progress depends on the goal of the thread
b

bitter-ability-32190

07/26/2022, 9:12 PM
Ultimately the goal of the thread is to nail down definitions, so that fixers aren't a hot potato exactly like
pylint
is 🙂
❤️ 1
Also, gotta nit here:
it would be for
python
, but not for the other languages. would be better not to have to type out the specific
check
implementation i want (
scalac
,
rustc
, etc)
You'd have
./pants lint --only=mypy check ...
😬 1
So, to you,
check
is more about ensuring it's even worth testing (or executing the next goal)? How do you feel about the fact that
mypy
can fail, but tests can pass?
w

witty-crayon-22786

07/26/2022, 9:39 PM
So, to you,
check
is more about ensuring it’s even worth testing (or executing the next goal)?
yea. “typechecking” is, to be clear.
How do you feel about the fact that
mypy
can fail, but tests can pass?
no compiler/typechecker can solve that problem statically, but they’re still worth running… not sure i understand the question
b

bitter-ability-32190

07/26/2022, 9:40 PM
Well for a compiled language I'd expect if `check`fails, tests can't be run at all let alone pass
I think I can see the delineation clearer with your lens. Lint is about finding code issues that aren't type or syntax. Check is for checking code syntax and types How do others who have chimed in feel? @wide-midnight-78598 @happy-kitchen-89482 @hundreds-father-404 @flat-zoo-31952 (For the underlying discussion, all fixers go into lint)
h

hundreds-father-404

07/26/2022, 9:48 PM
Yeah, that is an alternative wording for what I mean with option 2:
2) change razor to "is compilation or type checking"
And it actually matches our current
./pants help
definition. I'm generally very open to that outcome, as long as we follow it through to its logical conclusion: 1) For compiled languages, linting tools should not be run with
check
. Only the compilers themselves are
check
, e.g.
cargo check
vs
cargo clippy
(linter) 2) Still don't know what Pylint is. I think it's less of a type checker, it's not MyPy
b

bitter-ability-32190

07/26/2022, 9:50 PM
I would expect pylint to stay. It is a linter which, in a time before typecheckers, included some light "type checking". In today's world, no one would consider it a typechecker
1
👍 1
h

hundreds-father-404

07/26/2022, 9:53 PM
Makes sense to me 🙂 A nice benefit of option 2 is that it will result in zero changes to users. We haven't yet added any compiled language linters under
check
. This would only tighten up our semantics
I do generally agree w/ Stu that I find it convenient to run MyPy as a separate command from Flake8 and Black
e

enough-analyst-54434

07/26/2022, 9:56 PM
It's probably indicative of a broader problem in that you can't ad-hoc pick out a member of a union when running any goal today. That'd be nice: "I know I want to run X that happens to be in goal Y, but I have to run everything else in that goal too :/".
In Pants v1 this wasn't easy, but was possible. Goals were just names you could arrange or re-arrange any number of tasks into.
h

hundreds-father-404

07/26/2022, 9:57 PM
It's probably indicative of a broader problem in that you can't ad-hoc pick out a member of a union when running any goal today.
Sort of.
./pants fmt --only=black --only=isort
exists, along with
./pants --black-skip fmt
e

enough-analyst-54434

07/26/2022, 9:57 PM
The --only stuff is super-ad-hoc and boilerplately though.
👍 1
c

curved-television-6568

07/26/2022, 10:09 PM
Oh, this thread was way longer than my brain can absorb right now. 😛 (in the middle of trying to secure a place to stay in Toronto) I’ll just throw this wild idea out here and do a 🎤 💧 I feel a big issue is trying to find goal vocabulary that will make sense for any tool for any language. That feels like it could be an impossible task. So how about instead look for a way to more dynamically tie tools to goals in a more configurable way. That is,
lint
is just a configuration for running tools
a
b
and
c
. for instance. If a users feels strongly that it doesn’t make sense can move tool
b
to
check
instead, in their project.
🇨🇦 3
e

enough-analyst-54434

07/26/2022, 10:10 PM
Yeah, that's what we had in v1.
b

bitter-ability-32190

07/26/2022, 10:12 PM
The only minor detail there is how to run them in "check my code, don't change it" mode.
This is feeling more like a consensus. Thanks to all who provided opinions 🙂 They are and always will be much appreciated
❤️ 3
w

wide-midnight-78598

07/26/2022, 10:53 PM
I feel a big issue is trying to find goal vocabulary that will make sense for any tool for any language. That feels like it could be an impossible task. So how about instead look for a way to more dynamically tie tools to goals in a more configurable way. That is,
lint
is just a configuration for running tools
a
b
and
c
. for instance. If a users feels strongly that it doesn’t make sense can move tool
b
to
check
instead, in their project.
While I like a good mic drop as much as the next person, this feels like it's moving in the wrong direction - basically over-configurability. It's the whole "anything can be anything" which is usually fine in the moment, but I've had to maintain projects with this type of dev tool configuration (and sometimes even production configuration), and it's a hard pass. It ends up moving this entire Slack thread to multiple companies over multiple years of bike-shedding I'm an advocate of getting tools that make sane decisions, with some configuration as an aside: Black, Prettier, SvelteKit, etc
👍 1
1
h

hundreds-father-404

07/26/2022, 10:55 PM
+1. And if you don't like how Pants configured something, you're very welcome to disable the backend and re-implement with your own plugin - that is a key design goal of the engine to enable that
p

proud-dentist-22844

07/26/2022, 11:50 PM
No more makefiles! I'm turning to pants to escape that customized this is fine fire
👍 2
h

hundreds-father-404

07/26/2022, 11:55 PM
So, the next steps in this discussion: 1) Is anyone advocating for MyPy moving to
lint
, and
check
solely being compilation, aka "option 3"? If not, we can probably stop considering this 2) Do folks agree that for compiled languages, linting tools should never run under
check
, as they are not compilation?
w

wide-midnight-78598

07/27/2022, 12:05 AM
0) Is there agreement that
check
means compilation without linking/codegen and for non-compiled languages, it's a best-effort typecheck (mypy, pyright, tsc, Flow, etc)?
👍 2
b

bitter-ability-32190

07/27/2022, 12:29 AM
Personally, and with the upcoming cache feature I need to test things, but Im still interested in batched mypy runs. Other than that, I don't think which goal it is in important once we fix all the other known issues (--changed=implicit, and parallel goals)