https://pantsbuild.org/ logo
#general
Title
# general
e

echoing-london-29138

02/25/2022, 2:22 PM
Hey Everyone, I’m trying to add a linter to pants, and I’m struggling to understand from where I should import the object PythonSource referred here https://www.pantsbuild.org/docs/plugins-lint-goal (section two)
b

bitter-ability-32190

02/25/2022, 2:26 PM
The docs have drifted some from the state of the repo. What version of Pants are you using?
What worked for me is modeling my linter after one of the existing ones in https://github.com/pantsbuild/pants/tree/main/src/python/pants/backend/python/lint
e

echoing-london-29138

02/25/2022, 2:26 PM
2.9
b

bitter-ability-32190

02/25/2022, 2:27 PM
flake8
is simple enough to start there
e

echoing-london-29138

02/25/2022, 2:27 PM
Thanks 🙂
b

bitter-ability-32190

02/25/2022, 2:28 PM
If you're supporting a third-party tool, feel free to make a PR to add it to Pants itself. And don't hesitate to reach out with more questions!
e

echoing-london-29138

02/25/2022, 2:29 PM
Sure 🙂 Let’s make it work first 😄
🙌 1
b

bitter-ability-32190

02/25/2022, 2:29 PM
Also, we're hoping to improve the docs/examples for adding new plugins, but the plugin API changes so often and there's never enough time to work on this stuff 😐
(I like linting, in general, what linter are you adding?)
e

echoing-london-29138

02/25/2022, 2:48 PM
I’m trying to add import-linter
👀 1
🙌 1
b

bitter-ability-32190

02/25/2022, 2:49 PM
I think we actually have a use-case for this in my day-job's repo. Godspeed!
🤩 2
e

echoing-london-29138

02/25/2022, 3:20 PM
To be honest, I’m completly lost in this code. I got this error:
Inative_engine.PantsdClientException: The pantsd process was killed during the run
🤯 1
I don’t know from where it comes from
b

bitter-ability-32190

02/25/2022, 3:21 PM
If you're able to share the code (ideally a github link) feel free
Even a draft PR against the Pants repo, if you're able to 😉
w

wide-midnight-78598

02/25/2022, 3:48 PM
@echoing-london-29138 I'll again plug something I wrote (https://sureshjoshi.com/development/first-pants-plugin) It won't help directly make a linter, but it might help with the "what do all these things do" problems that I ran into (and frankly STILL run into)
🙌 1
My recommendation would also be to follow the "in-repo" plugins mechanism to start, as I think it's a lot faster to iterate upon and it less... daunting... https://www.pantsbuild.org/docs/plugins-overview#in-repo-plugins
Also, while it's not exactly a linter, I made an
ansible check
plugin, and it might get you some of what you need (while probably not being the best example of canonical Pants code, to be fair) https://github.com/sureshjoshi/pants-plugins/blob/226dd5b1cff1746feb3a55aa697600bb2248802f/pants-plugins/experimental/ansible/rules.py#L32-L80
e

echoing-london-29138

02/25/2022, 4:15 PM
Thanks for your advice 🙂 Maybe you know how to fix this one ?
ModuleNotFoundError: No module named 'pants.testutil'
I’m trying to write integration test in order to be able to find my mistake
w

wide-midnight-78598

02/25/2022, 4:18 PM
Did you pip install that?
e

echoing-london-29138

02/25/2022, 4:19 PM
I try to run the test with pants
I guess it should be able to find by itsefl
w

wide-midnight-78598

02/25/2022, 4:20 PM
There are some deps to install
e

echoing-london-29138

02/25/2022, 4:23 PM
That’s weird because I already added the pants_requirement(name=“pants”) in my main plugin folder
w

wide-midnight-78598

02/25/2022, 4:23 PM
Hmm, how are you running the tests?
e

echoing-london-29138

02/25/2022, 4:24 PM
./pants test pants-plugins/import_lint/rules_integration_test.py:rules_integration_test
w

wide-midnight-78598

02/25/2022, 4:24 PM
Ah, okay okay. One sec
e

echoing-london-29138

02/25/2022, 4:25 PM
I’ve added the
pants_requirement(name="pants")
In directly in my plugin, but now I have this error No module named ‘pants’
w

wide-midnight-78598

02/25/2022, 4:25 PM
Do you have a sample repo?
b

bitter-ability-32190

02/25/2022, 4:26 PM
Pants is slightly odd when it comes to dependencies for it's plugins. When you run pants the plugin code will be in the Pants env, but when you go to test or lint Pants won't be in the list of your requirements (depending on your scenario)
You should try setting your constraints lockfile to the output of this script: https://www.pantsbuild.org/docs/python-third-party-dependencies#tip-set-up-a-virtual-environment-optional
It'll make a constraints file that includes your normal dependencies AND pants itself
(This has gotten much improved in Pants 2.10 I'm told, which supports multiple lockfiles. So one for your normal repo code and one for your pants plguins 🎉 )
w

wide-midnight-78598

02/25/2022, 4:29 PM
@bitter-ability-32190 I'm pretty sure it should just work - I'm running my tests, and I don't have pants mentioned in the lockfiles
b

bitter-ability-32190

02/25/2022, 4:29 PM
@wide-midnight-78598 AFAIK it depends somewhat on how you configure pants to treat deps. In our repo we set a constraints file and tell Pants it can't pull deps from anywhere but there. I believe there are certain scenarios in which Pants will pull deps from PyPI if not told otherwise
w

wide-midnight-78598

02/25/2022, 4:31 PM
@echoing-london-29138
I’ve added the 
pants_requirement(name="pants")
In directly in my plugin, but now I have this error Where exactly did you add that?
@bitter-ability-32190 Fair point, assuming it's a barebones repo though - I thought the python_requirements(name="pants") in a BUILD file in pants-plugins would be enough
b

bitter-ability-32190

02/25/2022, 4:32 PM
I would assume in a barebones repo (no other requirement constraining) it is enough
e

echoing-london-29138

02/25/2022, 4:34 PM
@wide-midnight-78598 in the build file on the specific plugin folder
w

wide-midnight-78598

02/25/2022, 4:35 PM
What happens if you put it in a top-level, like this one https://github.com/sureshjoshi/pants-plugins/blob/main/pants-plugins/BUILD
e

echoing-london-29138

02/25/2022, 4:36 PM
I did as well and I have the pants.testutil not found
w

wide-midnight-78598

02/25/2022, 4:38 PM
Could you make a sample repo and put it here? It'd be a bit easier to debug that way
2
@bitter-ability-32190 Thoughts on a pantsbuild/plugin template repo? We have example plugins, but I'm thinking just a simple, mostly empty version that people can use. No need for a cookiecutter or anything, just a baseline repo (https://docs.github.com/en/repositories/creating-and-managing-repositories/creating-a-repository-from-a-template)
b

bitter-ability-32190

02/25/2022, 4:51 PM
At the rate of plugin changes, I'd fear it'd get dusty without the requisite TLC. That's not to say it wouldn't be useful, but as others have pointed out it might be more useful to spend the energy on reducing the boilerplate/wiring first.
I'm hoping that I can dust off this PR in the nearish future: https://github.com/pantsbuild/pants/pull/14238 I have high hopes it'll help with boilerplate reduction
w

wide-midnight-78598

02/25/2022, 4:52 PM
Fair point, but scaffolding the pants.toml, pythonpath, a couple build files - even if out of date, there are still 10 required files that won't go away. I'll build one in my repo, and can transfer it if there is interest
🙌 1
b

bitter-ability-32190

02/25/2022, 4:52 PM
That's a great start 🙂
w

wide-midnight-78598

02/25/2022, 4:55 PM
https://github.com/sureshjoshi/pants-plugin-template I'll fill this out tonight and see where we stand
e

echoing-london-29138

02/25/2022, 5:08 PM
Sorry, I fixed my issue by adding this
Copy code
python_sources()

python_requirement(
    name="pants.testutil",
    modules=["pants.testutil"],
    requirements=["pantsbuild.pants.testutil"],
)

python_tests(
    name="rules_integration_test",
    sources=["rules_integration_test.py"],
    timeout=240,
)
I will share with my code later, (End of the day for me 😄 )
👍 1
w

wide-midnight-78598

02/25/2022, 5:09 PM
please do share when you get a chance, I don't think that should strictly be necessary
2
h

happy-kitchen-89482

02/25/2022, 7:24 PM
Sorry, yeah, we should make it much clearer that the test utils for your plugins are not in the main pantsbuild.pants artifact, but in pantsbuild.pants.testutil
e

echoing-london-29138

02/28/2022, 5:00 PM
Hey 🙂 I’ve opened this PR https://github.com/pantsbuild/pants/pull/14648 with the code that I have. I tried to respect the convention that you have 🙂 If someone has a chance to try and maybe to help me debug, it would be really great
🙌 1
w

wide-midnight-78598

02/28/2022, 5:26 PM
Hey @echoing-london-29138 What are you trying to debug?
b

bitter-ability-32190

02/28/2022, 5:27 PM
e

echoing-london-29138

02/28/2022, 5:27 PM
Hey SJ, there is a unit test in the rules test file
I would like to be able to make a unit test working at first 😄
And then to be able to use this linter in the lint goal
w

wide-midnight-78598

02/28/2022, 5:28 PM
Ah okay @echoing-london-29138 - sorry I thought you were just submitting it for PR review. Understood!
e

echoing-london-29138

02/28/2022, 5:31 PM
I would if I was able to make it work 😅
w

wide-midnight-78598

02/28/2022, 5:32 PM
I'm building right now, will take a quick look and see if there is anything obvious
Hey @echoing-london-29138 - so right now, I think your problem is imports, because trying to run the tests fall over entirely on import problems right now. Also, where is
PythonLockfileRequest
? You import that, but I don't see it anywhere in the codebase (note: I'm working on your forked repo in the
add_import _linter
branch)
e

echoing-london-29138

02/28/2022, 5:54 PM
I don’t use it. When I run the test I’m able to have import-linter working but I struggle to configure the folders and files
w

wide-midnight-78598

02/28/2022, 5:55 PM
Its strange, because I can't even get imports working when I pull your repo down
e

echoing-london-29138

02/28/2022, 6:04 PM
I’m trying
w

wide-midnight-78598

02/28/2022, 6:06 PM
I'm testing something on the side - I'm trying to isolate some code and settings and see what could be causing it
e

echoing-london-29138

02/28/2022, 6:24 PM
I forgot to change some import name
I try to rebase onto the 2.9 in order to have the same context as I have on my project
w

wide-midnight-78598

02/28/2022, 6:25 PM
So, I'm building a standalone version in one of my repos - it might be easier to build off of, as it's a simplified version, if you're interested
Runs on 2.11.0
https://github.com/sureshjoshi/pants-plugins/tree/25-import-linter/pants-plugins/experimental/importlint I'm running into that namespace problem you had earlier - and that appears to be because the linter runs at the root directory, and my examples are nested
Okay, so it seems to work, sorta - weird configuration for that tool.
./pants lint helloworld:
results in
Copy code
13:39:16.18 [INFO] Completed: experimental.importlint.rules.run_myplugin_lint - importlinter succeeded.
=============
Import Linter
=============

---------
Contracts
---------

Analyzed 4 files, 2 dependencies.
---------------------------------


Contracts: 0 kept, 0 broken.



✓ black succeeded.
✓ flake8 succeeded.
✓ importlinter succeeded.
e

echoing-london-29138

02/28/2022, 6:40 PM
Are you able to configure it ?
If you setup a .importlinter file ?
That’s super cool 🙂 Thanks a lot 🙂 for the help
w

wide-midnight-78598

02/28/2022, 6:44 PM
The configuration isn't correctly handled. I just hardcoded a single field inside the rule. There are several ways to handle an "actual" configuration file, and I honestly don't know what the canonical way is. I've seen a ConfigRequest (I think isort and black do this), adding a file/resource, adding something into a target, etc. Maybe @bitter-ability-32190 would know the correct way to specify a configuration file for linters or formatters? You could alternatively put it in the pants.toml, or maybe pull in a pyproject?
b

bitter-ability-32190

02/28/2022, 6:44 PM
I'd say just look at the other linters/formatters 🙂
w

wide-midnight-78598

02/28/2022, 6:45 PM
But, @echoing-london-29138 - the non-configuration part of the plugin is roughly correct, depending on nuances from the import-lint tool (which I'd never seen before), and you could always MultiGet to merge some of the async stuff, but 🤷
e

echoing-london-29138

02/28/2022, 6:47 PM
I’ll process that tomorrow morning, it’s a huge help that you just provided to me 🙂 Thanks again 🙂
w

wide-midnight-78598

02/28/2022, 6:48 PM
👍
@echoing-london-29138 Can you edit your PR's title to start with "[WIP]" ?
e

echoing-london-29138

02/28/2022, 7:11 PM
Done
👍 1
5 Views