https://pantsbuild.org/ logo
Join Slack
Powered by
Hey Everyone, I’m trying to add a linter to pants,...
# general
e
Hey Everyone, I’m trying to add a linter to pants, and I’m struggling to understand from where I should import the object PythonSource referred here https://www.pantsbuild.org/docs/plugins-lint-goal (section two)
b
The docs have drifted some from the state of the repo. What version of Pants are you using?
What worked for me is modeling my linter after one of the existing ones in https://github.com/pantsbuild/pants/tree/main/src/python/pants/backend/python/lint
e
2.9
b
flake8
is simple enough to start there
e
Thanks 🙂
b
If you're supporting a third-party tool, feel free to make a PR to add it to Pants itself. And don't hesitate to reach out with more questions!
e
Sure 🙂 Let’s make it work first 😄
🙌 1
b
Also, we're hoping to improve the docs/examples for adding new plugins, but the plugin API changes so often and there's never enough time to work on this stuff 😐
(I like linting, in general, what linter are you adding?)
e
I’m trying to add import-linter
👀 1
🙌 1
b
I think we actually have a use-case for this in my day-job's repo. Godspeed!
🤩 2
e
To be honest, I’m completly lost in this code. I got this error: 
Inative_engine.PantsdClientException: The pantsd process was killed during the run
🤯 1
I don’t know from where it comes from
b
If you're able to share the code (ideally a github link) feel free
Even a draft PR against the Pants repo, if you're able to 😉
w
@echoing-london-29138 I'll again plug something I wrote (https://sureshjoshi.com/development/first-pants-plugin) It won't help directly make a linter, but it might help with the "what do all these things do" problems that I ran into (and frankly STILL run into)
🙌 1
My recommendation would also be to follow the "in-repo" plugins mechanism to start, as I think it's a lot faster to iterate upon and it less... daunting... https://www.pantsbuild.org/docs/plugins-overview#in-repo-plugins
Also, while it's not exactly a linter, I made an 
ansible check
plugin, and it might get you some of what you need (while probably not being the best example of canonical Pants code, to be fair) https://github.com/sureshjoshi/pants-plugins/blob/226dd5b1cff1746feb3a55aa697600bb2248802f/pants-plugins/experimental/ansible/rules.py#L32-L80
e
Thanks for your advice 🙂 Maybe you know how to fix this one ? 
ModuleNotFoundError: No module named 'pants.testutil'
I’m trying to write integration test in order to be able to find my mistake
w
Did you pip install that?
e
I try to run the test with pants
I guess it should be able to find by itsefl
w
There are some deps to install
e
That’s weird because I already added the pants_requirement(name=“pants”) in my main plugin folder
w
Hmm, how are you running the tests?
e
./pants test pants-plugins/import_lint/rules_integration_test.py:rules_integration_test
w
Ah, okay okay. One sec
e
I’ve added the 
pants_requirement(name="pants")
In directly in my plugin, but now I have this error No module named ‘pants’
w
Do you have a sample repo?
b
Pants is slightly odd when it comes to dependencies for it's plugins. When you run pants the plugin code will be in the Pants env, but when you go to test or lint Pants won't be in the list of your requirements (depending on your scenario)
You should try setting your constraints lockfile to the output of this script: https://www.pantsbuild.org/docs/python-third-party-dependencies#tip-set-up-a-virtual-environment-optional
It'll make a constraints file that includes your normal dependencies AND pants itself
(This has gotten much improved in Pants 2.10 I'm told, which supports multiple lockfiles. So one for your normal repo code and one for your pants plguins 🎉 )
w
@bitter-ability-32190 I'm pretty sure it should just work - I'm running my tests, and I don't have pants mentioned in the lockfiles
b
@wide-midnight-78598 AFAIK it depends somewhat on how you configure pants to treat deps. In our repo we set a constraints file and tell Pants it can't pull deps from anywhere but there. I believe there are certain scenarios in which Pants will pull deps from PyPI if not told otherwise
w
@echoing-london-29138
I’ve added the 
pants_requirement(name="pants")
In directly in my plugin, but now I have this error Where exactly did you add that?
@bitter-ability-32190 Fair point, assuming it's a barebones repo though - I thought the python_requirements(name="pants") in a BUILD file in pants-plugins would be enough
b
I would assume in a barebones repo (no other requirement constraining) it is enough
e
@wide-midnight-78598 in the build file on the specific plugin folder
w
What happens if you put it in a top-level, like this one https://github.com/sureshjoshi/pants-plugins/blob/main/pants-plugins/BUILD
e
I did as well and I have the pants.testutil not found
w
Could you make a sample repo and put it here? It'd be a bit easier to debug that way
➕ 2
@bitter-ability-32190 Thoughts on a pantsbuild/plugin template repo? We have example plugins, but I'm thinking just a simple, mostly empty version that people can use. No need for a cookiecutter or anything, just a baseline repo (https://docs.github.com/en/repositories/creating-and-managing-repositories/creating-a-repository-from-a-template)
b
At the rate of plugin changes, I'd fear it'd get dusty without the requisite TLC. That's not to say it wouldn't be useful, but as others have pointed out it might be more useful to spend the energy on reducing the boilerplate/wiring first.
I'm hoping that I can dust off this PR in the nearish future: https://github.com/pantsbuild/pants/pull/14238 I have high hopes it'll help with boilerplate reduction
w
Fair point, but scaffolding the pants.toml, pythonpath, a couple build files - even if out of date, there are still 10 required files that won't go away. I'll build one in my repo, and can transfer it if there is interest
🙌 1
b
That's a great start 🙂
w
https://github.com/sureshjoshi/pants-plugin-template I'll fill this out tonight and see where we stand
e
Sorry, I fixed my issue by adding this
Copy code
python_sources()

python_requirement(
    name="pants.testutil",
    modules=["pants.testutil"],
    requirements=["pantsbuild.pants.testutil"],
)

python_tests(
    name="rules_integration_test",
    sources=["rules_integration_test.py"],
    timeout=240,
)
I will share with my code later, (End of the day for me 😄 )
👍 1
w
please do share when you get a chance, I don't think that should strictly be necessary
✅ 2
h
Sorry, yeah, we should make it much clearer that the test utils for your plugins are not in the main pantsbuild.pants artifact, but in pantsbuild.pants.testutil
e
Hey 🙂 I’ve opened this PR https://github.com/pantsbuild/pants/pull/14648 with the code that I have. I tried to respect the convention that you have 🙂 If someone has a chance to try and maybe to help me debug, it would be really great
🙌 1
w
Hey @echoing-london-29138 What are you trying to debug?
b
e
Hey SJ, there is a unit test in the rules test file
I would like to be able to make a unit test working at first 😄
And then to be able to use this linter in the lint goal
w
Ah okay @echoing-london-29138 - sorry I thought you were just submitting it for PR review. Understood!
e
I would if I was able to make it work 😅
w
I'm building right now, will take a quick look and see if there is anything obvious
Hey @echoing-london-29138 - so right now, I think your problem is imports, because trying to run the tests fall over entirely on import problems right now. Also, where is 
PythonLockfileRequest
? You import that, but I don't see it anywhere in the codebase (note: I'm working on your forked repo in the 
add_import _linter
branch)
e
I don’t use it. When I run the test I’m able to have import-linter working but I struggle to configure the folders and files
w
Its strange, because I can't even get imports working when I pull your repo down
e
I’m trying
w
I'm testing something on the side - I'm trying to isolate some code and settings and see what could be causing it
e
I forgot to change some import name
I try to rebase onto the 2.9 in order to have the same context as I have on my project
w
So, I'm building a standalone version in one of my repos - it might be easier to build off of, as it's a simplified version, if you're interested
Runs on 2.11.0
https://github.com/sureshjoshi/pants-plugins/tree/25-import-linter/pants-plugins/experimental/importlint I'm running into that namespace problem you had earlier - and that appears to be because the linter runs at the root directory, and my examples are nested
Okay, so it seems to work, sorta - weird configuration for that tool. 
./pants lint helloworld:
results in
Copy code
13:39:16.18 [INFO] Completed: experimental.importlint.rules.run_myplugin_lint - importlinter succeeded.
=============
Import Linter
=============

---------
Contracts
---------

Analyzed 4 files, 2 dependencies.
---------------------------------


Contracts: 0 kept, 0 broken.



✓ black succeeded.
✓ flake8 succeeded.
✓ importlinter succeeded.
e
Are you able to configure it ?
If you setup a .importlinter file ?
That’s super cool 🙂 Thanks a lot 🙂 for the help
w
The configuration isn't correctly handled. I just hardcoded a single field inside the rule. There are several ways to handle an "actual" configuration file, and I honestly don't know what the canonical way is. I've seen a ConfigRequest (I think isort and black do this), adding a file/resource, adding something into a target, etc. Maybe @bitter-ability-32190 would know the correct way to specify a configuration file for linters or formatters? You could alternatively put it in the pants.toml, or maybe pull in a pyproject?
b
I'd say just look at the other linters/formatters 🙂
w
But, @echoing-london-29138 - the non-configuration part of the plugin is roughly correct, depending on nuances from the import-lint tool (which I'd never seen before), and you could always MultiGet to merge some of the async stuff, but 🤷
e
I’ll process that tomorrow morning, it’s a huge help that you just provided to me 🙂 Thanks again 🙂
w
👍
@echoing-london-29138 Can you edit your PR's title to start with "[WIP]" ?
e
Done
👍 1
5 Views
Open in Slack
PreviousNext
Powered by