https://pantsbuild.org/ logo
Join Slack
Powered by
Hi Pants Team, I'm responsible for tooling for my ...
# welcome
f
Hi Pants Team, I'm responsible for tooling for my company's Python monorepo and Pants looks like a good fit. The only issue is that we're heavily dependent on Pyright for type checking. Are you aware of anyone who's integrated Pyright into their build pipeline?
👋 4
h
Welcome! I don't think I've heard of someone using *Pants + Pyright, but we're definitely happy to help you implement a plugin, and I think that could be interesting to open source 🙂
How fancy is your Pyright usage? For example, are you using plugins?
f
That would be great! I'm happy to contribute to an open source plugin. Pyright doesn't support plugins, which simplifies things.
👍 1
We're not doing anything fancy.
h
Great! So imo the main trick to writing a Pants plugin is figuring out first how would you run this outside of Pants, like what argv do you use, env vars you need to set, etc. Then, it becomes fairly mechanical to port to Pants Would you want to open a GitHub issue about Pyright, and/or start a thread in #plugins? I'm super happy to talk through next steps 🙂 (figuring out how to install will be a major question)
@wide-midnight-78598 wrote a good blog post about their experience developing a plugin: https://sureshjoshi.com/development/first-pants-plugin
➕ 1
f
Sure thing! I'll take a stab at the plugin then post or create an issue with the details. Installing will be a pain since it's an npm package and requires NodeJS to run. Someone did create a package on pypy for it that wraps the NodeJS call, but I'm not sure how well it works.
h
the easiest way to get something working is hardcoding the path to the Pyright binary, like 
/bin/path/to/pyright
as the first value in 
argv
. It will allow you to get everything else working Then, we can generalize it. For example, in your org, it may be sufficient to assume everyone has the binary installed in the same place. Or, Pants has code to discover the pre-installed binary; or we have code to install it for you, e.g. via the pip package
f
Good call
h
Our guide for adding a typechecker: https://www.pantsbuild.org/docs/plugins-typecheck-goal I recommend first reading the general docs on the Target API and Rules API. There can definitely be a learning curve w/ the Rules API, which we want to improve via better error messages and better docs. Definitely do feel free to ask for help 🙂
f
Thanks. I missed that page in the docs.
❤️ 1
w
We have a super basic (and incomplete) Node implementation via npx (see Prettier: https://github.com/pantsbuild/pants/tree/main/src/python/pants/backend/javascript/lint/prettier) But I'd never really thought about cases where you have one backend's system lint/check another's. I don't see why it wouldn't work 🤯
❤️ 1
h
But I'd never really thought about cases where you have one backend's system lint/check another's. I don't see why it wouldn't work
It should! The rules API is designed specifically to make this type of integration possible, where things are highly pluggable
💰 1
w
Oh like via the field_sets?
Copy code
@dataclass(frozen=True)
class PrettierFmtFieldSet(FieldSet):
    required_fields = (JSSourceField,)

    sources: JSSourceField
If that changed to the pyfields?
h
@wide-midnight-78598 do you think it would be useful to land your generic NPX support in Pants to help out plugin authors?
w
do you think it would be useful to land your generic NPX support in Pants to help out plugin authors
Yeah, but I don't think it's there yet and I'm hesitant for people to build off of an incomplete work. There is some caching work that is slightly over my head, and then I'd like API reviews to make sure it's sane (since we're using NPX calls directly)
f
Nice! It's because Pyright was build by Microsoft as part of VS Code and used by the editor to enable IntelliSense and refactor features. AFAIK, all of the VS Code language parsers are written in Typescript.
h
Oh like via the field_sets?
The idea is we'd have generic rules like 
NpxProcess
, very similar to how we have the `Pex`/`PexRequest` and 
PexProcess
types. Any rule is free to use those how they want So then the Pyright plugin would indeed use a 
CheckFieldSet
on Python targets. But inside its rules, it would use 
NpxProcess
w
Sweet: https://github.com/pantsbuild/pants/blob/5087766b3cbe401c0a71e16c45151ab7feefac22/src/python/pants/backend/javascript/lint/prettier/rules.py#L61-L73 So, in that case, I think it's just caching help and then someone not-me who knows JS/Node to sanity check what's in there
h
There is some caching work that is slightly over my head, and then I'd like API reviews to make sure it's sane (since we're using NPX calls directly)
I think we'd be able to help with this. Having 
NpxProcess
landed as a generic "building block" would be great. Helps out plugin authors, and gives a good foundation for folks to build out a JavaScript/TS backend
b
Aside: welcome! 👋 I see you're already in really good hands with Eric (a core maintainer) and SJ (a core contributor and who's authored many plugins). Please don't hesitate to ask as many questions as you need to. We try very hard to maintain a kind and responsive community here. We also welcome and pay attention to feedback, so do let us know how we can improve the API, the docs, etc. Cheers!
🙏 2
h
wait lol, I forgot we (meaning SJ) already landed Prettier in Pants as an experimental backend 🙈 And indeed 
NpxProcess
exists already
w
🙂
b
Hahaha. Terrific!
w
I wasn't sure if there was another "landing" process where it gets moved into a generic backend, instead of dedicated. https://github.com/pantsbuild/pants/issues/15489 That is the ticket that I think needs to be fixed, but the whole append caching vs named confused me
👍 1
f
Prettier is handy for formatting JSON, YAML, and Markdown in Python projects.
👍 2
h
I wasn't sure if there was another "landing" process where it gets moved into a generic backend, instead of dedicated.
It's fine for a backend to use rules defined in another backend, so long as its 
register.py
activates everything necessary. For example, you can activate the Black backend without activating the Python backend, and it works! (Only, no target definitions) So no need to move it somewhere else
🤯 1
w
De-railing aside, @fresh-xylophone-16214 looks like there are mechanisms for you to give this a shot today without too much overhead. If you have any trouble, I can help out as needed
👍 2
b
I've been wanting to add pyright support myself, since I've been really pleased with it's type checking (sometimes better than mypy) Glad to hear it might be picked up (the only reason I hadn't is the fact that it needs node to run, which seems to be resolved above). I look forward to this, and feel free to reach out with trouble as well 🙂
👍 2
w
Last note on this, I think the NPX approach works well as of recent NPM versions - for the longest time, even cached NPX'd tools would have a few seconds of startup/resolution time. In my testing that I mentioned to Eric (lost to Slack's 10k limit), I think I mentioned that a recent test was in the order of 100ms or something equally awesome. ... some seconds later... Yep, just checked - under 250ms after first install, so pyright should be good. I think it was still faster outside of Pants for startup, but not appreciably
l
Any progress on adding 
pyright
support since this discussion? (I couldn't find any in this Slack or in GH issues). Similarly to @fresh-xylophone-16214 I'm evaluating 
pants
to help scale our python monorepo, but we use 
pyright
so far, not 
mypy
.
b
I don't think so. I don't even see an issue (query) wanna file one? 🙂
l
Yep, doing it right away 👍
🙏 1
w
This would be a great example of where a "meta linter" would work well. Should only need a subsystem + tests - but, that's not the case yet, unfortunately
b
pyright
woulnd't be a linter 😤
Buuuut also, this one is an 
npm
tool distributed by PyPI, so even weirder 😛
➕ 2
(Not to trash on 
pyright
. I find it is better than 
mypy
. Which is hilarious, because 
mypy
should be the shining example, but when you have corporation $$$ you can do anything)
w
meta checker
🤯 1
b
lol this is why I love this community. We're fighting over who gets to hack at it 😈
w
🙂 Someone else is mooooore than welcome - I just figured since I landed Prettier, it's probably the least effort for me to hack something quick.
h
@fresh-xylophone-16214 if you could chime in on https://github.com/pantsbuild/pants/issues/17141, that would be great!
l
I must say, coming from a functional programming background, I was impressed by 
pyright
inference capabilities. It infers pretty good types, even on datascience libraries that are notoriously hard to typecheck, with lots of union types. User experience has been really good so far with `pyright`: it's fast, silent when it should be, and when it outputs an error it's on point. In addition there is feedback as you type in 
vscode
which I believe 
mypy
doesn't support (although I'm no 
mypy
expert, so may miss something).
b
Yes 
pyright
comes embedded with a LSP for Python. Also the turnaround time on issues for that tool is insane. Eric Traut is a beast
w
Neat...
Copy code
11:55:34.61 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Run Pyright on 8 files.' failed with exit code 1.
stdout:
No configuration file found.
No pyproject.toml file found.
Assuming Python platform Darwin
Searching for source files
Found 8 source files
pyright 1.1.274
/private/var/folders/18/q1r7phps28nc9rx5j_0t3jmm0000gp/T/pants-sandbox-G31wI5/helloworld/greet/greeting.py
  /private/var/folders/18/q1r7phps28nc9rx5j_0t3jmm0000gp/T/pants-sandbox-G31wI5/helloworld/greet/greeting.py:9:8 - warning: Import "pkg_resources" could not be resolved from source (reportMissingModuleSource)
/private/var/folders/18/q1r7phps28nc9rx5j_0t3jmm0000gp/T/pants-sandbox-G31wI5/helloworld/main.py
  /private/var/folders/18/q1r7phps28nc9rx5j_0t3jmm0000gp/T/pants-sandbox-G31wI5/helloworld/main.py:4:6 - error: Import "colors" could not be resolved (reportMissingImports)
/private/var/folders/18/q1r7phps28nc9rx5j_0t3jmm0000gp/T/pants-sandbox-G31wI5/helloworld/translator/translator_test.py
  /private/var/folders/18/q1r7phps28nc9rx5j_0t3jmm0000gp/T/pants-sandbox-G31wI5/helloworld/translator/translator_test.py:4:8 - error: Import "pytest" could not be resolved (reportMissingImports)
2 errors, 1 warning, 0 informations
b
🤯
w
I cheated and pretended it was a linter, but 🤷
l
I'll log off now because I'm in Europe, talk to you again on Monday. Thanks for the support of https://github.com/pantsbuild/pants/issues/17141, really appreciated ❤️ If 
pyright
support makes it soon to 
pants
, it'll make a net difference regarding possible acceptance of 
pants
in my team 🙏
👋 4
w
Soooo, out of curiousity - has anyone setup 
pyright
to any degree to help me setting it up (or, even better, takeover my PR to add config) I have it running on input sources, naively - but I've never used 
pyright
outside of VS Code before, so I'm not a great person to be handling the config aspects
l
Hey guys, it's Monday morning in Europe so here I am again 👋 Let me know how I can help/test things out.
w
Holiday Monday for me, but if you could take a look at this? https://github.com/pantsbuild/pants/pull/17163 That's a naive, no custom config implementation - I've never used 
pyright
out of VSCode, so I'm clueless about args
l
@wide-midnight-78598> yes I will. Lots of meetings for me today so I will maybe only try it out tomorrow. Thanks for the help 🙏
👍 1
w
The PR got merged, so it's the most basic level of usability right now - but it's in! 🙂
b
This makes me feel like we could very easily design a scenario where pyright is used for typechecking through pants, but also with pyrights multi-project-support, you could easily have editor support (with multiple resolves!!!!) After and 
export
I smell a blog post
b
Do eeeeeeeet, mwahaha.
b
Well if I could give my engs a one-button in-pants and in-editor experience I'd be elated
OMG I have the perfect name. Now I have to work on this
Open in Slack
PreviousNext
Powered by