Hi folks! I’m having an issue where a package I’m ...
# general
a
Hi folks! I’m having an issue where a package I’m consuming from PyPI (via the
python_requirements
macro) has started including a module whose name clashes with a local module in my workspace, leading to dependency inference warnings like this (and other issues when both modules are used by a target):
Copy code
22:22:22.84 [WARN] The target src/python/foo/health_score/ios/file_line_count_insight.py:ios_health_score imports `foo.health_score.health_score_check.HealthScoreCheck`, but Pants cannot safely infer a dependency because more than one target owns this module, so it is ambiguous which to use: ['3rdparty/python:foo-sdk', 'src/python/foo/health_score/health_score_check.py:lib'].
(
foo-sdk
includes both a
foo_sdk
and a
foo
module.) Is there any way exclude a specific module from a package? Some magic with a handwritten
python_requirement_library
target perhaps? Or am I better off renaming my local module?
w
unfortunately it doesn’t look like there is a way to do this with pip…
but, i’m surprised that the macro is declaring a clashing requirements target: it should only declare targets for things literally listed in
requirements.txt
and it sounds like the thing you want to avoid is a transitive dependency of something in your
requirements.txt
…?
h
@witty-crayon-22786 Iiuc, there is a
python_requirement_library
(created by the macro) that has the module
foo
. There is also first party code
foo.bar.baz
. Sounds like that's resulting in ambiguity
👍 1
@alert-airplane-2123 does the 3rd party requirement only use more precise submodules, like
foo.a.b
and
foo.a.b
for example? And your first party code is
foo.x
for example If that's true, you can use
module_mapping
to override the default module of the 3rd-party requirement:
Copy code
python_requirements(
  module_mapping={"foo": ["foo.a", "foo.b"]}
)
Whereas right now, the requirement
foo
defaults to exporting the modules
["foo"]
, which captures all submodules
👀 1
a
Appreciate the help! To be more concrete: the 3rd party package is
slack-sdk
(https://github.com/slackapi/python-slack-sdk), which contains a
slack_sdk
module as well as a
slack
module, and all of my first-party code lives under
src/python/slack
(and
tests/python/slack_tests
). I have test targets in my project that import both
slack.some_first_party_module
and
slack_sdk
, and it appears that when a test imports
slack.some_first_party_module
, it’s getting
slack
from
slack-sdk
.
Copy code
==================================== ERRORS ====================================
_ ERROR collecting tests/python/slack_tests/health_score/ios/test_health_score_processor.py _
ImportError while importing test module '/private/var/folders/m2/7q9pbw453p19bpl150ntdfr40000gp/T/process-executionGT5ET7/tests/python/slack_tests/health_score/ios/test_health_score_processor.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/Users/jschear/.pyenv/versions/3.8.9/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/python/slack_tests/health_score/ios/test_health_score_processor.py:5: in <module>
    import slack.health_score.ios.ios_health_score_processor as ios_health_score_processor
/Users/jschear/.cache/pants/named_caches/pex_root/venvs/short/8a731efb/lib/python3.8/site-packages/slack/__init__.py:7: in <module>
    from slack_sdk.rtm import RTMClient  # noqa
/Users/jschear/.cache/pants/named_caches/pex_root/venvs/short/8a731efb/lib/python3.8/site-packages/slack_sdk/rtm/__init__.py:16: in <module>
    import aiohttp
E   ModuleNotFoundError: No module named 'aiohttp'
=============================== warnings summary ===============================
(My sense is that the
ModuleNotFoundError
is a bit of a red herring here — ideally
site-packages/slack/__init__.py
wouldn’t be executed at all when
slack.health_score.ios.ios_health_score_processor
is imported.) I only get the ambiguous dependency inference warnings if I specify a module_mapping of
"slack-sdk": ["slack_sdk", "slack"]
, but I hit those `ModuleNotFoundError`s regardless of the module_mapping. (I’ve tried
"slack-sdk": ["slack_sdk", "slack"]
and
"slack-sdk": ["slack_sdk"]
.)
h
Hey @alert-airplane-2123, I'm sorry I dropped the ball on replying here! Are you still having issues with this?
a
Hey, no worries! I haven’t been able to resolve this yet — I’m probably just going to rename the module in my project to avoid the collision.