#general
Title
e
echoing-london-29138
02/25/2022, 2:22 PMHey Everyone, I’m trying to add a linter to pants, and I’m struggling to understand from where I should import the object PythonSource referred here https://www.pantsbuild.org/docs/plugins-lint-goal (section two)
b
bitter-ability-32190
02/25/2022, 2:26 PMThe docs have drifted some from the state of the repo. What version of Pants are you using?
What worked for me is modeling my linter after one of the existing ones in https://github.com/pantsbuild/pants/tree/main/src/python/pants/backend/python/lint
e
echoing-london-29138
02/25/2022, 2:26 PM2.9
b
bitter-ability-32190
02/25/2022, 2:27 PMflake8Here's that same link, but on the
2.9.xe
echoing-london-29138
02/25/2022, 2:27 PMThanks 🙂
b
bitter-ability-32190
02/25/2022, 2:28 PMIf you're supporting a third-party tool, feel free to make a PR to add it to Pants itself.
And don't hesitate to reach out with more questions!
e
echoing-london-29138
02/25/2022, 2:29 PMSure 🙂 Let’s make it work first 😄
🙌 1
b
bitter-ability-32190
02/25/2022, 2:29 PMAlso, we're hoping to improve the docs/examples for adding new plugins, but the plugin API changes so often and there's never enough time to work on this stuff 😐
(I like linting, in general, what linter are you adding?)
e
echoing-london-29138
02/25/2022, 2:48 PMI’m trying to add import-linter
👀 1
🙌 1
b
bitter-ability-32190
02/25/2022, 2:49 PMI think we actually have a use-case for this in my day-job's repo.
Godspeed!
🤩 2
e
echoing-london-29138
02/25/2022, 3:20 PMTo be honest, I’m completly lost in this code. I got this error:
Inative_engine.PantsdClientException: The pantsd process was killed during the run🤯 1
I don’t know from where it comes from
b
bitter-ability-32190
02/25/2022, 3:21 PMIf you're able to share the code (ideally a github link) feel free
Even a draft PR against the Pants repo, if you're able to 😉
w
wide-midnight-78598
02/25/2022, 3:48 PM@echoing-london-29138 I'll again plug something I wrote (https://sureshjoshi.com/development/first-pants-plugin)
It won't help directly make a linter, but it might help with the "what do all these things do" problems that I ran into (and frankly STILL run into)
🙌 1
My recommendation would also be to follow the "in-repo" plugins mechanism to start, as I think it's a lot faster to iterate upon and it less... daunting...
https://www.pantsbuild.org/docs/plugins-overview#in-repo-plugins
Also, while it's not exactly a linter, I made an
ansible checke
echoing-london-29138
02/25/2022, 4:15 PMThanks for your advice 🙂
Maybe you know how to fix this one ?
ModuleNotFoundError: No module named 'pants.testutil'w
wide-midnight-78598
02/25/2022, 4:18 PMDid you pip install that?
e
echoing-london-29138
02/25/2022, 4:19 PMI try to run the test with pants
I guess it should be able to find by itsefl
w
wide-midnight-78598
02/25/2022, 4:20 PMThere are some deps to install
e
echoing-london-29138
02/25/2022, 4:23 PMThat’s weird because I already added the pants_requirement(name=“pants”) in my main plugin folder
w
wide-midnight-78598
02/25/2022, 4:23 PMHmm, how are you running the tests?
e
echoing-london-29138
02/25/2022, 4:24 PM./pants test pants-plugins/import_lint/rules_integration_test.py:rules_integration_test
w
wide-midnight-78598
02/25/2022, 4:24 PMAh, okay okay. One sec
e
echoing-london-29138
02/25/2022, 4:25 PMI’ve added the
pants_requirement(name="pants")w
wide-midnight-78598
02/25/2022, 4:25 PMDo you have a sample repo?
b
bitter-ability-32190
02/25/2022, 4:26 PMPants is slightly odd when it comes to dependencies for it's plugins. When you run pants the plugin code will be in the Pants env, but when you go to test or lint Pants won't be in the list of your requirements (depending on your scenario)
You should try setting your constraints lockfile to the output of this script:
https://www.pantsbuild.org/docs/python-third-party-dependencies#tip-set-up-a-virtual-environment-optional
It'll make a constraints file that includes your normal dependencies AND pants itself
(This has gotten much improved in Pants 2.10 I'm told, which supports multiple lockfiles. So one for your normal repo code and one for your pants plguins 🎉 )
w
wide-midnight-78598
02/25/2022, 4:29 PM@bitter-ability-32190 I'm pretty sure it should just work - I'm running my tests, and I don't have pants mentioned in the lockfiles
b
bitter-ability-32190
02/25/2022, 4:29 PM(For 2.10) https://www.pantsbuild.org/v2.10/docs/plugins-overview#building-in-repo-plugins-with-pants specifically the last
pants.toml@wide-midnight-78598 AFAIK it depends somewhat on how you configure pants to treat deps. In our repo we set a constraints file and tell Pants it can't pull deps from anywhere but there. I believe there are certain scenarios in which Pants will pull deps from PyPI if not told otherwise
w
wide-midnight-78598
02/25/2022, 4:31 PM@echoing-london-29138
I’ve added theIn directly in my plugin, but now I have this error Where exactly did you add that?pants_requirement(name="pants")
@bitter-ability-32190 Fair point, assuming it's a barebones repo though - I thought the python_requirements(name="pants") in a BUILD file in pants-plugins would be enough
b
bitter-ability-32190
02/25/2022, 4:32 PMI would assume in a barebones repo (no other requirement constraining) it is enough
e
echoing-london-29138
02/25/2022, 4:34 PM@wide-midnight-78598 in the build file on the specific plugin folder
w
wide-midnight-78598
02/25/2022, 4:35 PMWhat happens if you put it in a top-level, like this one
https://github.com/sureshjoshi/pants-plugins/blob/main/pants-plugins/BUILD
e
echoing-london-29138
02/25/2022, 4:36 PMI did as well and I have the pants.testutil not found
w
wide-midnight-78598
02/25/2022, 4:38 PMCould you make a sample repo and put it here? It'd be a bit easier to debug that way
➕ 2
@bitter-ability-32190 Thoughts on a pantsbuild/plugin template repo? We have example plugins, but I'm thinking just a simple, mostly empty version that people can use. No need for a cookiecutter or anything, just a baseline repo (https://docs.github.com/en/repositories/creating-and-managing-repositories/creating-a-repository-from-a-template)
b
bitter-ability-32190
02/25/2022, 4:51 PMAt the rate of plugin changes, I'd fear it'd get dusty without the requisite TLC. That's not to say it wouldn't be useful, but as others have pointed out it might be more useful to spend the energy on reducing the boilerplate/wiring first.
I'm hoping that I can dust off this PR in the nearish future: https://github.com/pantsbuild/pants/pull/14238
I have high hopes it'll help with boilerplate reduction
w
wide-midnight-78598
02/25/2022, 4:52 PMFair point, but scaffolding the pants.toml, pythonpath, a couple build files - even if out of date, there are still 10 required files that won't go away.
I'll build one in my repo, and can transfer it if there is interest
🙌 1
b
bitter-ability-32190
02/25/2022, 4:52 PMThat's a great start 🙂
w
wide-midnight-78598
02/25/2022, 4:55 PMhttps://github.com/sureshjoshi/pants-plugin-template
I'll fill this out tonight and see where we stand
e
echoing-london-29138
02/25/2022, 5:08 PMSorry, I fixed my issue by adding this
Copy code
python_sources()
python_requirement(
name="pants.testutil",
modules=["pants.testutil"],
requirements=["pantsbuild.pants.testutil"],
)
python_tests(
name="rules_integration_test",
sources=["rules_integration_test.py"],
timeout=240,
)I will share with my code later, (End of the day for me 😄 )
👍 1
w
wide-midnight-78598
02/25/2022, 5:09 PMplease do share when you get a chance, I don't think that should strictly be necessary
✅ 2
h
happy-kitchen-89482
02/25/2022, 7:24 PMSorry, yeah, we should make it much clearer that the test utils for your plugins are not in the main pantsbuild.pants artifact, but in pantsbuild.pants.testutil
e
echoing-london-29138
02/28/2022, 5:00 PMHey 🙂 I’ve opened this PR https://github.com/pantsbuild/pants/pull/14648 with the code that I have. I tried to respect the convention that you have 🙂 If someone has a chance to try and maybe to help me debug, it would be really great
🙌 1
w
wide-midnight-78598
02/28/2022, 5:26 PMHey @echoing-london-29138 What are you trying to debug?
b
bitter-ability-32190
02/28/2022, 5:27 PM@wide-midnight-78598 https://pantsbuild.slack.com/archives/C046T6T9U/p1646060178879819
👍 1
e
echoing-london-29138
02/28/2022, 5:27 PMHey SJ, there is a unit test in the rules test file
I would like to be able to make a unit test working at first 😄
And then to be able to use this linter in the lint goal
w
wide-midnight-78598
02/28/2022, 5:28 PMAh okay @echoing-london-29138 - sorry I thought you were just submitting it for PR review. Understood!
e
echoing-london-29138
02/28/2022, 5:31 PMI would if I was able to make it work 😅
w
wide-midnight-78598
02/28/2022, 5:32 PMI'm building right now, will take a quick look and see if there is anything obvious
Hey @echoing-london-29138 - so right now, I think your problem is imports, because trying to run the tests fall over entirely on import problems right now. Also, where is
PythonLockfileRequestadd_import _lintere
echoing-london-29138
02/28/2022, 5:54 PMI don’t use it. When I run the test I’m able to have import-linter working but I struggle to configure the folders and files
w
wide-midnight-78598
02/28/2022, 5:55 PMIts strange, because I can't even get imports working when I pull your repo down
e
echoing-london-29138
02/28/2022, 6:04 PMI’m trying
w
wide-midnight-78598
02/28/2022, 6:06 PMI'm testing something on the side - I'm trying to isolate some code and settings and see what could be causing it
e
echoing-london-29138
02/28/2022, 6:24 PMI forgot to change some import name
I try to rebase onto the 2.9 in order to have the same context as I have on my project
w
wide-midnight-78598
02/28/2022, 6:25 PMSo, I'm building a standalone version in one of my repos - it might be easier to build off of, as it's a simplified version, if you're interested
Runs on 2.11.0
https://github.com/sureshjoshi/pants-plugins/tree/25-import-linter/pants-plugins/experimental/importlint
I'm running into that namespace problem you had earlier - and that appears to be because the linter runs at the root directory, and my examples are nested
Okay, so it seems to work, sorta - weird configuration for that tool.
./pants lint helloworld: Copy code
13:39:16.18 [INFO] Completed: experimental.importlint.rules.run_myplugin_lint - importlinter succeeded.
=============
Import Linter
=============
---------
Contracts
---------
Analyzed 4 files, 2 dependencies.
---------------------------------
Contracts: 0 kept, 0 broken.
✓ black succeeded.
✓ flake8 succeeded.
✓ importlinter succeeded.e
echoing-london-29138
02/28/2022, 6:40 PMAre you able to configure it ?
If you setup a .importlinter file ?
That’s super cool 🙂 Thanks a lot 🙂 for the help
w
wide-midnight-78598
02/28/2022, 6:44 PMThe configuration isn't correctly handled. I just hardcoded a single field inside the rule.
There are several ways to handle an "actual" configuration file, and I honestly don't know what the canonical way is. I've seen a ConfigRequest (I think isort and black do this), adding a file/resource, adding something into a target, etc.
Maybe @bitter-ability-32190 would know the correct way to specify a configuration file for linters or formatters? You could alternatively put it in the pants.toml, or maybe pull in a pyproject?
b
bitter-ability-32190
02/28/2022, 6:44 PMI'd say just look at the other linters/formatters 🙂
w
wide-midnight-78598
02/28/2022, 6:45 PMBut, @echoing-london-29138 - the non-configuration part of the plugin is roughly correct, depending on nuances from the import-lint tool (which I'd never seen before), and you could always MultiGet to merge some of the async stuff, but 🤷
e
echoing-london-29138
02/28/2022, 6:47 PMI’ll process that tomorrow morning, it’s a huge help that you just provided to me 🙂 Thanks again 🙂
w
wide-midnight-78598
02/28/2022, 6:48 PM👍
@echoing-london-29138 Can you edit your PR's title to start with "[WIP]" ?
e
echoing-london-29138
02/28/2022, 7:11 PMDone
👍 1
5 Views