https://pantsbuild.org/ logo
#welcome
Title
# welcome
f

fresh-xylophone-16214

07/27/2022, 12:02 AM
Hi Pants Team, I'm responsible for tooling for my company's Python monorepo and Pants looks like a good fit. The only issue is that we're heavily dependent on Pyright for type checking. Are you aware of anyone who's integrated Pyright into their build pipeline?
👋 4
h

hundreds-father-404

07/27/2022, 12:04 AM
Welcome! I don't think I've heard of someone using *Pants + Pyright, but we're definitely happy to help you implement a plugin, and I think that could be interesting to open source 🙂
How fancy is your Pyright usage? For example, are you using plugins?
f

fresh-xylophone-16214

07/27/2022, 12:07 AM
That would be great! I'm happy to contribute to an open source plugin. Pyright doesn't support plugins, which simplifies things.
👍 1
We're not doing anything fancy.
h

hundreds-father-404

07/27/2022, 12:15 AM
Great! So imo the main trick to writing a Pants plugin is figuring out first how would you run this outside of Pants, like what argv do you use, env vars you need to set, etc. Then, it becomes fairly mechanical to port to Pants Would you want to open a GitHub issue about Pyright, and/or start a thread in #plugins? I'm super happy to talk through next steps 🙂 (figuring out how to install will be a major question)
@wide-midnight-78598 wrote a good blog post about their experience developing a plugin: https://sureshjoshi.com/development/first-pants-plugin
1
f

fresh-xylophone-16214

07/27/2022, 12:19 AM
Sure thing! I'll take a stab at the plugin then post or create an issue with the details. Installing will be a pain since it's an npm package and requires NodeJS to run. Someone did create a package on pypy for it that wraps the NodeJS call, but I'm not sure how well it works.
h

hundreds-father-404

07/27/2022, 12:21 AM
the easiest way to get something working is hardcoding the path to the Pyright binary, like
/bin/path/to/pyright
as the first value in
argv
. It will allow you to get everything else working Then, we can generalize it. For example, in your org, it may be sufficient to assume everyone has the binary installed in the same place. Or, Pants has code to discover the pre-installed binary; or we have code to install it for you, e.g. via the pip package
f

fresh-xylophone-16214

07/27/2022, 12:22 AM
Good call
h

hundreds-father-404

07/27/2022, 12:23 AM
Our guide for adding a typechecker: https://www.pantsbuild.org/docs/plugins-typecheck-goal I recommend first reading the general docs on the Target API and Rules API. There can definitely be a learning curve w/ the Rules API, which we want to improve via better error messages and better docs. Definitely do feel free to ask for help 🙂
f

fresh-xylophone-16214

07/27/2022, 12:23 AM
Thanks. I missed that page in the docs.
❤️ 1
w

wide-midnight-78598

07/27/2022, 12:35 AM
We have a super basic (and incomplete) Node implementation via npx (see Prettier: https://github.com/pantsbuild/pants/tree/main/src/python/pants/backend/javascript/lint/prettier) But I'd never really thought about cases where you have one backend's system lint/check another's. I don't see why it wouldn't work 🤯
❤️ 1
h

hundreds-father-404

07/27/2022, 12:36 AM
But I'd never really thought about cases where you have one backend's system lint/check another's. I don't see why it wouldn't work
It should! The rules API is designed specifically to make this type of integration possible, where things are highly pluggable
💰 1
w

wide-midnight-78598

07/27/2022, 12:37 AM
Oh like via the field_sets?
Copy code
@dataclass(frozen=True)
class PrettierFmtFieldSet(FieldSet):
    required_fields = (JSSourceField,)

    sources: JSSourceField
If that changed to the pyfields?
h

hundreds-father-404

07/27/2022, 12:37 AM
@wide-midnight-78598 do you think it would be useful to land your generic NPX support in Pants to help out plugin authors?
w

wide-midnight-78598

07/27/2022, 12:38 AM
do you think it would be useful to land your generic NPX support in Pants to help out plugin authors
Yeah, but I don't think it's there yet and I'm hesitant for people to build off of an incomplete work. There is some caching work that is slightly over my head, and then I'd like API reviews to make sure it's sane (since we're using NPX calls directly)
f

fresh-xylophone-16214

07/27/2022, 12:38 AM
Nice! It's because Pyright was build by Microsoft as part of VS Code and used by the editor to enable IntelliSense and refactor features. AFAIK, all of the VS Code language parsers are written in Typescript.
h

hundreds-father-404

07/27/2022, 12:38 AM
Oh like via the field_sets?
The idea is we'd have generic rules like
NpxProcess
, very similar to how we have the `Pex`/`PexRequest` and
PexProcess
types. Any rule is free to use those how they want So then the Pyright plugin would indeed use a
CheckFieldSet
on Python targets. But inside its rules, it would use
NpxProcess
w

wide-midnight-78598

07/27/2022, 12:39 AM
Sweet: https://github.com/pantsbuild/pants/blob/5087766b3cbe401c0a71e16c45151ab7feefac22/src/python/pants/backend/javascript/lint/prettier/rules.py#L61-L73 So, in that case, I think it's just caching help and then someone not-me who knows JS/Node to sanity check what's in there
h

hundreds-father-404

07/27/2022, 12:39 AM
There is some caching work that is slightly over my head, and then I'd like API reviews to make sure it's sane (since we're using NPX calls directly)
I think we'd be able to help with this. Having
NpxProcess
landed as a generic "building block" would be great. Helps out plugin authors, and gives a good foundation for folks to build out a JavaScript/TS backend
b

busy-vase-39202

07/27/2022, 12:40 AM
Aside: welcome! 👋 I see you're already in really good hands with Eric (a core maintainer) and SJ (a core contributor and who's authored many plugins). Please don't hesitate to ask as many questions as you need to. We try very hard to maintain a kind and responsive community here. We also welcome and pay attention to feedback, so do let us know how we can improve the API, the docs, etc. Cheers!
🙏 2
h

hundreds-father-404

07/27/2022, 12:41 AM
wait lol, I forgot we (meaning SJ) already landed Prettier in Pants as an experimental backend 🙈 And indeed
NpxProcess
exists already
w

wide-midnight-78598

07/27/2022, 12:41 AM
🙂
b

busy-vase-39202

07/27/2022, 12:42 AM
Hahaha. Terrific!
w

wide-midnight-78598

07/27/2022, 12:42 AM
I wasn't sure if there was another "landing" process where it gets moved into a generic backend, instead of dedicated. https://github.com/pantsbuild/pants/issues/15489 That is the ticket that I think needs to be fixed, but the whole append caching vs named confused me
👍 1
f

fresh-xylophone-16214

07/27/2022, 12:42 AM
Prettier is handy for formatting JSON, YAML, and Markdown in Python projects.
👍 2
h

hundreds-father-404

07/27/2022, 12:45 AM
I wasn't sure if there was another "landing" process where it gets moved into a generic backend, instead of dedicated.
It's fine for a backend to use rules defined in another backend, so long as its
register.py
activates everything necessary. For example, you can activate the Black backend without activating the Python backend, and it works! (Only, no target definitions) So no need to move it somewhere else
🤯 1
w

wide-midnight-78598

07/27/2022, 12:48 AM
De-railing aside, @fresh-xylophone-16214 looks like there are mechanisms for you to give this a shot today without too much overhead. If you have any trouble, I can help out as needed
👍 2
b

bitter-ability-32190

07/27/2022, 12:00 PM
I've been wanting to add pyright support myself, since I've been really pleased with it's type checking (sometimes better than mypy) Glad to hear it might be picked up (the only reason I hadn't is the fact that it needs node to run, which seems to be resolved above). I look forward to this, and feel free to reach out with trouble as well 🙂
👍 2
w

wide-midnight-78598

07/27/2022, 12:27 PM
Last note on this, I think the NPX approach works well as of recent NPM versions - for the longest time, even cached NPX'd tools would have a few seconds of startup/resolution time. In my testing that I mentioned to Eric (lost to Slack's 10k limit), I think I mentioned that a recent test was in the order of 100ms or something equally awesome. ... some seconds later... Yep, just checked - under 250ms after first install, so pyright should be good. I think it was still faster outside of Pants for startup, but not appreciably
l

lemon-oxygen-72498

10/07/2022, 3:06 PM
Any progress on adding
pyright
support since this discussion? (I couldn't find any in this Slack or in GH issues). Similarly to @fresh-xylophone-16214 I'm evaluating
pants
to help scale our python monorepo, but we use
pyright
so far, not
mypy
.
b

bitter-ability-32190

10/07/2022, 3:07 PM
I don't think so. I don't even see an issue (query) wanna file one? 🙂
l

lemon-oxygen-72498

10/07/2022, 3:08 PM
Yep, doing it right away 👍
🙏 1
w

wide-midnight-78598

10/07/2022, 3:21 PM
This would be a great example of where a "meta linter" would work well. Should only need a subsystem + tests - but, that's not the case yet, unfortunately
b

bitter-ability-32190

10/07/2022, 3:21 PM
pyright
woulnd't be a linter 😤
Buuuut also, this one is an
npm
tool distributed by PyPI, so even weirder 😛
2
(Not to trash on
pyright
. I find it is better than
mypy
. Which is hilarious, because
mypy
should be the shining example, but when you have corporation $$$ you can do anything)
w

wide-midnight-78598

10/07/2022, 3:23 PM
meta checker
🤯 1
b

bitter-ability-32190

10/07/2022, 3:28 PM
lol this is why I love this community. We're fighting over who gets to hack at it 😈
w

wide-midnight-78598

10/07/2022, 3:28 PM
🙂 Someone else is mooooore than welcome - I just figured since I landed Prettier, it's probably the least effort for me to hack something quick.
h

happy-kitchen-89482

10/07/2022, 3:30 PM
@fresh-xylophone-16214 if you could chime in on https://github.com/pantsbuild/pants/issues/17141, that would be great!
l

lemon-oxygen-72498

10/07/2022, 3:41 PM
I must say, coming from a functional programming background, I was impressed by
pyright
inference capabilities. It infers pretty good types, even on datascience libraries that are notoriously hard to typecheck, with lots of union types. User experience has been really good so far with `pyright`: it's fast, silent when it should be, and when it outputs an error it's on point. In addition there is feedback as you type in
vscode
which I believe
mypy
doesn't support (although I'm no
mypy
expert, so may miss something).
b

bitter-ability-32190

10/07/2022, 3:43 PM
Yes
pyright
comes embedded with a LSP for Python. Also the turnaround time on issues for that tool is insane. Eric Traut is a beast
w

wide-midnight-78598

10/07/2022, 3:56 PM
Neat...
Copy code
11:55:34.61 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Run Pyright on 8 files.' failed with exit code 1.
stdout:
No configuration file found.
No pyproject.toml file found.
Assuming Python platform Darwin
Searching for source files
Found 8 source files
pyright 1.1.274
/private/var/folders/18/q1r7phps28nc9rx5j_0t3jmm0000gp/T/pants-sandbox-G31wI5/helloworld/greet/greeting.py
  /private/var/folders/18/q1r7phps28nc9rx5j_0t3jmm0000gp/T/pants-sandbox-G31wI5/helloworld/greet/greeting.py:9:8 - warning: Import "pkg_resources" could not be resolved from source (reportMissingModuleSource)
/private/var/folders/18/q1r7phps28nc9rx5j_0t3jmm0000gp/T/pants-sandbox-G31wI5/helloworld/main.py
  /private/var/folders/18/q1r7phps28nc9rx5j_0t3jmm0000gp/T/pants-sandbox-G31wI5/helloworld/main.py:4:6 - error: Import "colors" could not be resolved (reportMissingImports)
/private/var/folders/18/q1r7phps28nc9rx5j_0t3jmm0000gp/T/pants-sandbox-G31wI5/helloworld/translator/translator_test.py
  /private/var/folders/18/q1r7phps28nc9rx5j_0t3jmm0000gp/T/pants-sandbox-G31wI5/helloworld/translator/translator_test.py:4:8 - error: Import "pytest" could not be resolved (reportMissingImports)
2 errors, 1 warning, 0 informations
b

bitter-ability-32190

10/07/2022, 3:56 PM
🤯
w

wide-midnight-78598

10/07/2022, 3:57 PM
I cheated and pretended it was a linter, but 🤷
l

lemon-oxygen-72498

10/07/2022, 4:01 PM
I'll log off now because I'm in Europe, talk to you again on Monday. Thanks for the support of https://github.com/pantsbuild/pants/issues/17141, really appreciated ❤️ If
pyright
support makes it soon to
pants
, it'll make a net difference regarding possible acceptance of
pants
in my team 🙏
👋 4
w

wide-midnight-78598

10/08/2022, 1:37 AM
Soooo, out of curiousity - has anyone setup
pyright
to any degree to help me setting it up (or, even better, takeover my PR to add config) I have it running on input sources, naively - but I've never used
pyright
outside of VS Code before, so I'm not a great person to be handling the config aspects
l

lemon-oxygen-72498

10/10/2022, 8:50 AM
Hey guys, it's Monday morning in Europe so here I am again 👋 Let me know how I can help/test things out.
w

wide-midnight-78598

10/10/2022, 4:20 PM
Holiday Monday for me, but if you could take a look at this? https://github.com/pantsbuild/pants/pull/17163 That's a naive, no custom config implementation - I've never used
pyright
out of VSCode, so I'm clueless about args
l

lemon-oxygen-72498

10/11/2022, 8:59 AM
@wide-midnight-78598> yes I will. Lots of meetings for me today so I will maybe only try it out tomorrow. Thanks for the help 🙏
👍 1
w

wide-midnight-78598

10/11/2022, 12:33 PM
The PR got merged, so it's the most basic level of usability right now - but it's in! 🙂
b

bitter-ability-32190

10/11/2022, 1:04 PM
This makes me feel like we could very easily design a scenario where pyright is used for typechecking through pants, but also with pyrights multi-project-support, you could easily have editor support (with multiple resolves!!!!) After and
export
I smell a blog post
b

busy-vase-39202

10/11/2022, 3:10 PM
Do eeeeeeeet, mwahaha.
b

bitter-ability-32190

10/11/2022, 3:25 PM
Well if I could give my engs a one-button in-pants and in-editor experience I'd be elated
OMG I have the perfect name. Now I have to work on this