https://pantsbuild.org/ logo
Join Slack
Powered by
I’m trying to debug an issue with running `pylint`...
# general
a
I’m trying to debug an issue with running 
pylint
. We have a lot of apps and for some in particular, when I run 
./pants --changed-since=foo lint
(which for this PR runs it on all the repo, since I changed every BUILD file) I get a lot of nonsensical errors, like complaints about modules not having members they definitely should have. This only runs on the CI, since it’d take forever on my mac, but I thought I could debug it by trying to lint the file, however… When I do that, it doesn’t happen. Even if I try to lint the whole directory. This seems to be related to namespace packages. The simplest example is we have a folder called 
audit
in which we have several packages, one of them being 
lib
. So in 
audit/lib
we have a BUILD file and then the sources are in 
audit/lib/src/audit/lib/
(with another BUILD in 
audit/lib/src/
) and tests in 
audit/lib/tests/
. When we run it on the CI:
Copy code
audit/lib/tests/unit/test_validate_event.py:73:24: E1101: Module 'audit' has no 'lib' member (no-member)
And, yes, the tests depend on 
"audit/lib/src:lib"
. However, as I said, when I just run pylint locally on 
audit/
, I don’t get this problem. I’m really at a loss at how I can debug this. Also, this problem appeared (presumably) after updating from 2.6 to 2.12 and is still present in 2.13, but it could just be that this PR runs against a lot of targets and that’s related somehow. Any ideas?
e
On latest 2.13 on PyPI you have 
--keep-sandboxes=on_failure
. That will print out the path of a /tmp dir you can examine. It contains a 
__run.sh
script at the root that models what the Pants Rust engine actually executes. That should shed light on things.
h
This is the relevant change I can think of between 2.6 and 2.13: https://github.com/pantsbuild/pants/pull/13918
Otherwise, only major change is resolves, which you aren't yet using. The rest looks like refactoring https://github.com/pantsbuild/pants/commits/main/src/python/pants/backend/python/lint/pylint/rules.py
a
Ugh, last time I tried to run something on the CI I couldn’t get it to work. 🙂
h
What's the behavior of 
./pants lint --only=pylint ::
? The namespace packages clue is a good one, I believe users have had issues with Pylint before with this. iirc Pylint doesn't like when Pants runs on all namespaces together, and often without Pants people run Pylint on smaller packages-at-a-time
e
The usefullness in the sandboxes is not so much running the 
__run.sh
script - you know the results already - it's examining the layout of files.
a
Let me try to run that on my mac, our security team promised they pushed a change to crowdstrike that should make it less of a pain when dealing with pants. 🙂
👍 1
h
iirc Pylint doesn't like when Pants runs on all namespaces together, and often without Pants people run Pylint on smaller packages-at-a-time
I think in the past folks have debugged by running on a single Pylint file, vs only one namespace package, vs everything. 
./pants --changed-since=<foo> list
could be informative to see which of those 3 categories your command is falling into
e
The folks at Color took the tack of adding a CI step that tarred / uploaded 
/tmp/pants-sandbox-*
so they could grab the failed sandboxes and share.
I think they too had issues accessing CI machines easily.
a
We’re moving to CircleCI where this won’t be possible, but there we’ll be running every app/lib in its own env, so it won’t be an issue.
e
Ah, the CircleCI feature that allows you to run a job and drop to a web ssh session will be useful for this sort of thing going forward.
a
That’s explicitly disabled, since there will be credentials littered all over the place. 🙂
I’m not sure why we can do it on our current CI, I was surprised to find out it works (Though, I personally couldn’t get it to work)
Okay, so it’s definitely not related to 
--changed-since
, I got the same errors with just filtering by a tag and 
::
👍 1
But, I obviously forgot to use 
--keep-sandboxes=on_failure
😞
h
do you get the failure on individual files? If not, then this is indeed likely namespace packages Because Pylint considers transitive deps, I recommend trying running on some of your simplest files with no or few deps
a
I don’t get failure on individual files, but as I said, it’s weird I don’t get it on 
lint audit
okay, actually, I do get failure on individual files if I use 
./pylint_runner.pex_pex_shim.sh
from the sandbox
it’s quite weird, because the file is in there
OH!
Copy code
>>> import audit.post.health
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/private/var/folders/_j/1pch58t96g7d7sk4t4dp7qxm0000gp/T/pants-sandbox-LAoavj/audit/post/src/audit/post/health.py", line 3, in <module>
    from <http://aio.app|aio.app> import run_app
ModuleNotFoundError: No module named 'aio'
>>>
Does pylint try to actually import the module?
h
I believe pylint does actually follow imports, unlike Flake8. It's why we have to make sure all transitive deps are available in the sandbox
a
it could figure out that it’s not a problem with that module, but something that module imports
okay, then the next question is, why isn’t 
aio
in there, 
<http://audit.post|audit.post>
has 
aio:lib
in the dependencies
oh, okay. wait, maybe that’s not it.
ah, okay, I know what the problem is
so the sandbox is populated with exactly the directory structure from the repo. so, as I said, we have 
<http://audit.post|audit.post>
as a package that is actually in 
audit/post/src/audit/post/
so when you try to import 
<http://audit.post|audit.post>
in the root there, it will fail
I’m not sure I’m expalining this right. But, yeah, this is definitely a bigger issue than I thought initially, because with this folder structure, things will never work
👍 1
h
Got it. So, it sounds like more a Pylint issue that Pants is surfacing because of the way it invokes Pylint? That's exactly what it was for the previous user, iirc
a
It’s not a pylint issue, I think. It’s a pants issue. The path to that file in the repo shouldn’t matter, the path to the files should.
I think mypy is doing the right thing, btw, I had to fix an issue earlier related to this: two apps had BUILD in the same folder as the code, so they both got a main.py
But imagine this, you have folder 
foo
in which you have a few packages. The code that should be copied shouldn’t be in 
foo
, it should be whatever the packages have inside.
e
The knob controlling this is Pants source roots configuration. Have you been down that road yet?
It's talked about here in case not: https://www.pantsbuild.org/docs/source-roots
a
I haven’t, but it’s weird that this one in particular behaves differently from building packages and mypy.
Yeah, I was reading that.
Is this related to 
./pants roots
?
Because this reports the correct thing (ish)
Copy code
$ ./pants roots | grep ^audit
audit/lib
audit/lib/src
audit/post
audit/post/src
audit/store
audit/store/src
audit/view
audit/view/src
In this case, the non src versions are just used for tests
e
Yeah, if 
./pants roots
looks correct (the entries are what you need to 
sys.path
to import things), then that at least is sorted.
The fact the tests are housed a level above src/ is fishy though given the error in layout you described.
a
Why?
e
We'll, natively you could imagine a bug with nested source roots where the wrong one is picked.
a
But it’s not using the sources root, it’s just copying the files as they are in the repo
e
That's not a problem if sys.path is adjusted. What env vars are in __run.sh?
Presumably there is a 
PEX_EXTRA_SYS_PATH
accounting for this?
a
Sorry, got distracted by something. Yeah, that has the roots. Let me try to use that as 
PYTHONPATH
Okay, so that works.
e
What version of Pants is this?
a
2.13.0rc1
, but we had it with 
2.12.0
as well
e
There was recent activity around PEX_EXTRA_SYS_PATH plumbing to be more transparent to Python interpreters in certain situations.
a
Copy code
$ ./pylint_runner.pex_pex_shim.sh audit/post/src/audit/post/main.py
************* Module post.main
audit/post/src/audit/post/main.py:25:43: E1101: Module '<http://audit.post|audit.post>' has no 'health' member (no-member)
But then, after exporting 
PYTHONPATH
:
Copy code
$ pylint audit/post/src/audit/post/main.py
************* Module post.main
audit/post/src/audit/post/main.py:71:4: E1101: Module 'asyncio' has no 'run' member (no-member)
(we’ll deal with that later :D)
But, yeah, it seems that the behaviour is different between the two.
e
And, just to button up the details, you're using the same exact value for PYTHONPATH that is the value for PEX_EXTRA_SYS_PATH in __run.sh?
a
Yeah.
e
Ok. I have a team meeting the next hour, but this is potentially enough detail to run with investigating things. Thanks for digging.
Wait a sec.
The two bits above are not symmetric. Do you export PEX_EXTRA_SYS_PATH when you run 
$ ./pylint_runner.pex_pex_shim.sh audit/post/src/audit/post/main.py
? The __run.sh script does this when it runs that pex, but you look like you're running the PEX directly in your snip above.
a
Oh, I’m stupid, you’re right.
e
Ok, does that make the errors align?
a
No. 🙂
Now I get no errors from the pylint runner
And yet, if I run the whole thing, it reports errors
e
Ok, losing track a bit. If you unexport PYTHONPATH & PEX_EXTRA_SYS_PATH and then run 
__run.sh
what happens - good things or bad things?
a
Bad things.
Copy code
(crpython-3.7) ip-10-42-1-179:pants-sandbox-LAoavj user$ ./pylint_runner.pex_pex_shim.sh audit/post/src/audit/post/main.py

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

(crpython-3.7) ip-10-42-1-179:pants-sandbox-LAoavj user$ ./pylint_runner.pex_pex_shim.sh $'--rcfile=.pylintrc' $'--jobs=6' architect/themis/themis/config/config_definer.py architect/themis/themis/exceptions.py architect/themis/themis/extractors.py architect/themis/themis/main.py architect/themis/themis/models.py architect/themis/themis/parameters.py architect/themis/themis/template_filters.py architect/themis/themis/utils.py armiger/armiger/__init__.py armiger/armiger/config/__init__.py armiger/armiger/config/config_definer.py armiger/armiger/db.py armiger/armiger/lib/__init__.py armiger/armiger/lib/payload.py armiger/armiger/main.py armiger/armiger/metrics.py armiger/tests/integration/test_process_architect_breaches_prevented.py armiger/tests/integration/test_process_payload.py armiger/tests/integration/test_processor_integration.py armiger/tests/unit/lib/test_payload.py armiger/tests/unit/test_processor.py audit/lib/tests/unit/test_meta_schemas.py audit/lib/tests/unit/test_models.py audit/lib/tests/unit/test_schema.py audit/lib/tests/unit/test_validate_event.py audit/lib/tests/unit/test_validate_user_schema_validator.py audit/lib/tests/unit/test_validate_user_schemas.py audit/post/src/audit/post/__init__.py audit/post/src/audit/post/api.py audit/post/src/audit/post/config.py audit/post/src/audit/post/health.py audit/post/src/audit/post/main.py audit/post/src/audit/post/metrics.py audit/post/src/audit/post/middlewares.py audit/post/src/audit/post/validation.py audit/post/tests/integration/test_api.py audit/post/tests/integration/test_health.py audit/post/tests/unit/test_api.py audit/store/src/audit/store/__init__.py audit/store/src/audit/store/audit_db.py audit/store/src/audit/store/config.py audit/store/src/audit/store/health.py audit/store/src/audit/store/main.py audit/store/src/audit/store/prometheus.py audit/store/src/audit/store/sqs.py audit/store/src/audit/store/utils.py audit/store/tests/integration/test_main.py audit/store/tests/unit/test_utils.py audit/view/src/audit/view/__init__.py audit/view/src/audit/view/audit_db.py audit/view/src/audit/view/config.py audit/view/src/audit/view/main.py audit/view/tests/integration/test_events.py auth/auth/__init__.py auth/auth/authorisation.py auth/auth/config/__init__.py auth/auth/config/config_definer.py auth/auth/entity.py auth/auth/error.py auth/auth/external_api/__init__.py auth/auth/external_api/db.py auth/auth/external_api/error.py auth/auth/external_api/middleware.py auth/auth/external_api/tokens.py auth/auth/external_api/utils.py auth/auth/main.py auth/auth/middleware.py auth/auth/test_utils/__init__.py auth/auth/test_utils/fixtures.py auth/auth/utils.py auth/auth/validate.py auth/tests/integration/external_api/test_external_api_middleware.py auth/tests/integration/external_api/test_external_token_validation.py auth/tests/integration/test_authorisation.py auth/tests/integration/test_get_ip_whitelist.py auth/tests/integration/test_middleware.py auth/tests/unit/external_api/test_tokens.py auth/tests/unit/test_authorization.py auth/tests/unit/test_middleware.py auth/tests/unit/test_utils.py auth/tests/unit/test_validate.py beluga/beluga/__init__.py beluga/beluga/config/config_definer.py beluga/beluga/db.py beluga/beluga/endpoints/__init__.py beluga/beluga/endpoints/roles.py beluga/beluga/endpoints/tenants.py beluga/beluga/endpoints/users.py beluga/beluga/endpoints/utils.py beluga/beluga/mailer.py beluga/beluga/main.py beluga/client/beluga_client/client.py beluga/client/tests/unit/test_client.py beluga/tests/integration/test_roles.py beluga/tests/integration/test_tenants.py beluga/tests/integration/test_users.py beluga/tests/unit/test_mailer.py common/audit/src/common/audit/__init__.py common/audit/src/common/audit/helpers.py common/audit/src/common/audit/pytest_plugin.py common/audit/src/common/audit/utils_for_tests.py common/audit/tests/unit/test_helpers.py common/celery/src/common/celery/__init__.py common/celery/src/common/celery/celery.py common/celery/src/common/celery/config.py common/config/src/common/config/__init__.py common/config/tests/unit/test_config.py common/demo/src/common/demo/__init__.py common/demo/src/common/demo/utils.py common/demo/tests/integration/__init__.py
************* Module test_validate_event
audit/lib/tests/unit/test_validate_event.py:73:24: E1101: Module 'audit' has no 'lib' member (no-member)
audit/lib/tests/unit/test_validate_event.py:74:24: E1101: Module 'audit' has no 'lib' member (no-member)
audit/lib/tests/unit/test_validate_event.py:79:24: E1101: Module 'audit' has no 'lib' member (no-member)
************* Module test_validate_user_schema_validator
audit/lib/tests/unit/test_validate_user_schema_validator.py:142:24: E1101: Module 'audit' has no 'lib' member (no-member)
audit/lib/tests/unit/test_validate_user_schema_validator.py:144:8: E1101: Module 'audit' has no 'lib' member (no-member)
audit/lib/tests/unit/test_validate_user_schema_validator.py:147:8: E1101: Module 'audit' has no 'lib' member (no-member)
audit/lib/tests/unit/test_validate_user_schema_validator.py:150:8: E1101: Module 'audit' has no 'lib' member (no-member)
audit/lib/tests/unit/test_validate_user_schema_validator.py:155:8: E1101: Module 'audit' has no 'lib' member (no-member)
audit/lib/tests/unit/test_validate_user_schema_validator.py:160:8: E1101: Module 'audit' has no 'lib' member (no-member)
************* Module post.main
audit/post/src/audit/post/main.py:25:43: E1101: Module 'audit' has no 'post' member (no-member)

------------------------------------------------------------------
Your code has been rated at 9.90/10 (previous run: 9.90/10, +0.00)

(crpython-3.7) ip-10-42-1-179:pants-sandbox-LAoavj user$
(same thing happens if I deactivate the venv, so it’s not that)
e
But yet you're saying __run.sh has a correct PEX__EXTRA_SYS_PATH and when you use that when running the pex.shim itself on the sdame set of files as the __run.sg, things work?
a
No. 
__run.sh
seems to have the correct stuff in there, but when I run the check on all the files it fails.
e
Does an audit/ dir have an 
__init__.py
?
a
Nope.
If I run pylint directly, I get errors like 
beluga/tests/integration/test_users.py:90:4: E1701: Async context manager 'async_generator' doesn't implement __aenter__ and __aexit__. (not-async-context-manager)
, which is false:
Copy code
@contextlib.asynccontextmanager
async def setup_kevlar_environment(
    name: str, allowed_client_tenants: List[Tuple[str, str, str]]
) -> AsyncGenerator[None, None]:
I think the solution here will be to just run the linting per package
Or, oh.
I wonder if using one shard? was that the name, will fix it
e
a
Sorry, I don’t understand the question
e
I edited the question after swearing at Slack.
a
yeah, reading now 🙂
e
Pants allows you to configure the pylint version, so you might try 
[pylint] version = "pylint==2.14.5"
.
a
I’m trying, but apparently, 
Could not find a version that satisfies the requirement pylint==2.14.5
e
Maybe just try "pylint<3"
The only explicit mention of a ns package fix is in the 2.12 changelog: https://pylint.pycqa.org/en/latest/whatsnew/2/2.12/full.html
a
And tha’ts what I hate about pants. When it works, it works, when it doesn’t, it’s several extra steps to figure it out 🙂
e
Yeah, we definetly get more in your way currently than a tool like tox. There are really just a handful of things to learn to debug things generally to be fair, but it is a steep learning curve I think.
Hmm, 2.14.5 should have worked - unless you're running Python 3.7.0 or 3.7.1: https://pypi.org/project/pylint/2.14.5/#files
pylint 2.14,5 has a Requires-Python of >= 3.7.2
a
My requirements are > 3.7, I’m on 3.7.4 I think
e
Ok, makes no sense then. Nonetheless, using 
<3
and seeing what pip picks is the way to go for the moment.
a
<3
ended up with 2.13.9
e
Ok.
a
I pushed it to the CI to see, but gonna try to understand why 2.14 isn’t working
e
Well, 2.13.9 says Requires-Python >= 3.6.2 so that implies Pants is maybe picking a Python 3.6 interpreter to run Pylint with.
I'm not familiar with that rule set, looking....
Do you have any custom interpreter constraint configuration in pants.toml that would veer away from 
>=3.7
?
a
Nah
e
Hrm, no clue why 2.14.5 wasn't picked unless Python 3.7.0 or 3.7.1 then.
a
Copy code
export CPPFLAGS= LC_CTYPE=en_GB.UTF-8 LDFLAGS= PATH=$'/Users/user/.poetry/bin:/usr/local/opt/openjdk@8/bin:/Users/user/.pyenv/shims:/Users/user/.pyenv/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Wireshark.app/Contents/MacOS:/Users/user/golang/bin:/usr/local/opt/go/libexec/bin:/Users/user/PycharmProjects/tools/aws/bin:/Users/user/.local/bin' PEX_IGNORE_RCFILES=true PEX_PYTHON_PATH=$'/Users/user/.pyenv/versions/crpython-3.7/bin:/Users/user/.pyenv/versions/3.10.0/bin:/Users/user/.pyenv/versions/3.10.3/bin:/Users/user/.pyenv/versions/3.6.6/bin:/Users/user/.pyenv/versions/3.6.8/bin:/Users/user/.pyenv/versions/3.7.4/bin:/Users/user/.pyenv/versions/3.8.12/bin:/Users/user/.pyenv/versions/3.9.5/bin:/Users/user/.pyenv/versions/aws-venv/bin:/Users/user/.pyenv/versions/boto_new/bin:/Users/user/.pyenv/versions/boto_old/bin:/Users/user/.pyenv/versions/catapult/bin:/Users/user/.pyenv/versions/compare-requirements/bin:/Users/user/.pyenv/versions/crpython-3.7/bin:/Users/user/.pyenv/versions/ipython/bin:/Users/user/.pyenv/versions/jupyter-test/bin:/Users/user/.pyenv/versions/kite-venv/bin:/Users/user/.pyenv/versions/pants-test/bin:/Users/user/.pyenv/versions/py-3.6.6/bin:/Users/user/.pyenv/versions/py-3.6.6-test-alembik/bin:/Users/user/.pyenv/versions/py-3.7/bin:/Users/user/.pyenv/versions/py-3.7-vtdownload/bin:/Users/user/.pyenv/versions/python-for-pants/bin:/Users/user/.pyenv/versions/sashimi_test/bin:/Users/user/.pyenv/versions/simon/bin:/Users/user/.pyenv/versions/test_mypy/bin:/Users/user/.pyenv/versions/test_mypy2/bin:/Users/user/.pyenv/versions/test_pip/bin:/Users/user/.pyenv/versions/tle-test/bin:/Users/user/.pyenv/versions/tle-test2/bin:/Users/user/.pyenv/versions/tle-test3/bin:/Users/user/.pyenv/versions/toolsvenv/bin' PEX_ROOT=.cache/pex_root PEX_SCRIPT=pex3
cd /private/var/folders/_j/1pch58t96g7d7sk4t4dp7qxm0000gp/T/pants-sandbox-9pCkWl
$'/Users/user/.pyenv/versions/crpython-3.7/bin/python3.7' ./pex lock create --tmpdir .tmp --python-path $'/Users/user/.pyenv/versions/crpython-3.7/bin:/Users/user/.pyenv/versions/3.10.0/bin:/Users/user/.pyenv/versions/3.10.3/bin:/Users/user/.pyenv/versions/3.6.6/bin:/Users/user/.pyenv/versions/3.6.8/bin:/Users/user/.pyenv/versions/3.7.4/bin:/Users/user/.pyenv/versions/3.8.12/bin:/Users/user/.pyenv/versions/3.9.5/bin:/Users/user/.pyenv/versions/aws-venv/bin:/Users/user/.pyenv/versions/boto_new/bin:/Users/user/.pyenv/versions/boto_old/bin:/Users/user/.pyenv/versions/catapult/bin:/Users/user/.pyenv/versions/compare-requirements/bin:/Users/user/.pyenv/versions/crpython-3.7/bin:/Users/user/.pyenv/versions/ipython/bin:/Users/user/.pyenv/versions/jupyter-test/bin:/Users/user/.pyenv/versions/kite-venv/bin:/Users/user/.pyenv/versions/pants-test/bin:/Users/user/.pyenv/versions/py-3.6.6/bin:/Users/user/.pyenv/versions/py-3.6.6-test-alembik/bin:/Users/user/.pyenv/versions/py-3.7/bin:/Users/user/.pyenv/versions/py-3.7-vtdownload/bin:/Users/user/.pyenv/versions/python-for-pants/bin:/Users/user/.pyenv/versions/sashimi_test/bin:/Users/user/.pyenv/versions/simon/bin:/Users/user/.pyenv/versions/test_mypy/bin:/Users/user/.pyenv/versions/test_mypy2/bin:/Users/user/.pyenv/versions/test_pip/bin:/Users/user/.pyenv/versions/tle-test/bin:/Users/user/.pyenv/versions/tle-test2/bin:/Users/user/.pyenv/versions/tle-test3/bin:/Users/user/.pyenv/versions/toolsvenv/bin' $'--output=lock.json' --no-emit-warnings $'--style=universal' --resolver-version pip-2020-resolver --target-system linux --target-system mac $'--indent=2' -r __pip_args.txt --no-pypi $'--index=<https://pypi.org/simple/>' --manylinux manylinux2014 --interpreter-constraint $'CPython==3.7.*' $'pylint==2.14.5'
That 
--python-path
is weird.
e
That is not PYTHONPATH, it's PATH for Python executables.
a
I know, but why would it care?
e
How would you honor 
--interpreter-constraint $'CPython==3.7.*'
? You need to know where to look for Pythons
Keeping in mind the constraint can be more general / wider and encompass multiple major/minor versions in general.
a
I thought that part was taken care of before
e
I'm not sure what you mean. At any rate - this explains 2.13.9. I didn't understand locks were involved here. If you lock for Python 3.7.* then of course 2.14.5 is no good since both 3.7.0 and 3.7.1 are members of 3.7.*
The lock might get checked in and later used on a machine with 3.7.0
So locking 2.14.5 would be an error.
a
Okay.
e
Ok - so that curiosity is explained. Now to see if this fixed your CI.
a
I put 
interpreter_constraints = ["CPython>=3.7.4"]
and it’s still not working 🙂
e
Its being selecting 2.14.5 or the original pylint issue with ns packages?
a
And I meant that I thought the interpreter is selected before making a sandbox to try to install things.
e
Locking doesn't install things
a
It’s refusing to install 
2.14.5
e
There are many little processes, one to lock, one to then build a tool from the lock, one to then run the tool, etc.
If its still picming 2.13.9, that's fine for now. This thread is really chaotic to grasp as a remote debugger; so how about seeing if the 2.13.9 lock works to solve the original NS packages issue 1st, then we can unwind the stack to the 2.14.5 issues.
a
It doesn’t.
e
Ok. Your remote setup is to opaque for me and 20 questions is not working out well. Let me try to gin up a repro with a few files in 2 seprate ns package roots and go forward from there to isolate Pants issue or Pex issue or Pylint issue.
Do you have any special pylint config I should emulate? A 
.pylintrc
or any other means of configuring Pylint outside of Pants?
I'm not sure if that will be relevant, but in the interests of covering bases.
Ok, I get no repro using the simplest things. Can you confirm your PEX_EXTRA_SYS_PATH does not contain any of the test sys.path entry points and just the sys,path entries for the source code, the ones nested deeper?
From the rule code it looks like the PEX_EXTRA_SYS_PATH should be properly trimmed to just those roots owning linted sources, but to be safe in my repro attempt assumptions.
The only other intersting quirk I can think of I'm not simulating is if you have internal 
python_distribution
targets that share in the namespace. If those are dependencies of linted source files, they'll be turned into wheels and installed on the sys.path and might be the angle the issue is coming in on.
Oh, you do have tests in the lists of files up there; so that means you do have nested PEX_EXTRA_SYS_PATH entries.
I'm a little slow. I just realized this was not an import error, but a proxy maybe of that style issue coming from pylint linting: 
E1101: Module 'audit' has no 'lib' member (no-member)
a
Sorry, let me catch up.
So. We do have a 
.pylintrc
, will paste it in the next message. Then, I’m not sure I understand the question about PEX_EXTRA_SYS_PATH. We do not have any python_distribution in our code. And, yes, I can import that.
.pylintrc.cpp
.pylintrc.cpp
e
Ok, thank you.
I was basically asking if PEX_EXTRA_SYS_PATH had all the 
./pants roots
you listed earlier. Things like 
PEX_EXTRA_SYS_PATH=audit/lib:audit/lib/src:...
where there are nested dirs in the list.
a
Yeah, it did.
e
Yeah, so 
audit/lib
is not actually a valid sys.path entry IIUC, 
audit/lib/tests
~is. I don't think that's at play, but, again, covering bases.
a
I’m going to go to bed now, it’s 3AM here… But, yeah, I have a way better understanding of the issue now and will look into it, over the weekend or on Monday.
😴 1
e
IOW, I think the 
./pants roots
are bad based on the file lists you show above being passed to pylint.
a
Thank you for the help!
❤️ 1
e
Ok. Great. It may be best to collect things in an issue at that point. This is wild to grok in Slack.
➕ 1
10 Views
Open in Slack
PreviousNext
Powered by