I'm getting a `No module named 'pkg_resources'` er...
# general
s
I'm getting a
No module named 'pkg_resources'
error when I include
mongomock
in my
thirdparty/python_test/requirements.txt
and depending on it from another BUILD file like so:
Copy code
python_tests(
    name="tests",
    sources = ["test/test_*.py", "test/conftest.py"],
    dependencies=[":fixtures", "thirdparty/python:pymongo", "thirdparty/python_test:mongomock"]
)
The usual fix for this error from what I can see is updating setuptools... But I'm not having this error outside of Pants/Pex so not sure how to fix this in Pants. Any ideas?
e
You will find alot of libraries do this sort of thing. Here mongomock uses pkg_resources: https://github.com/mongomock/mongomock/blob/develop/mongomock/__version__.py#L1 But they do not declare this as an
install_requires
. You get away with this outside of Pants beacuse either by default, or by sheer dint of its popularity / use by other distributions you installed in your venv, you happen to have setuptools already.
What version of Pants are you experimenting with?
The workaround is different depending on 2.7.x or 2.8.x IIRC.
👍 1
s
2.7.1 but I'm happy to upgrade if it's better
e
Its not significantly better, its just easier to get right perhaps, the same thing is really going on under the covers.
Ok, do you already have a version of setuptools defined that other code uses in your repo in a Pants target? Likely in a requirements file you've hooked up?
s
Not currently I'm just bootstrapping Pants into our monorepo ... so it's all very early stages at the moment
e
OK. Last question: do you use a requirements file that's hooked up to Pants via a python_requirements BUILD target?
s
yeah
e
Ok, then like so: 1. remove mongomock from that requirements file. 2. Add in the existing BUILD file:
Copy code
python_requirement_library(
    name="mongomock",
    requirements=["mongomock", "setuptools"],
)
If you slready had a version of setuptools defined in the requirements file, you'd instead do the following for 2.:
Copy code
python_requirement_library(
    name="mongomock",
    requirements=["mongomock"],
    dependencies=[":setuptools"]
)
h
Or if you can upgrade to 2.8, it would look like: 1. Add
setuptools
to your
requirements.txt
2. Update
python_requirements
to this
Copy code
python_requirements(
   overrides={"mongomock": {"dependencies": ":setuptools"}},
)
I personally like this approach because you get to still have all your deps in
requirements.txt
like before
e
Similar to default module mappings we should probably have a mechanism to auto-patch libraries like this that are known deficient. Its a bit titchier than default module mappings though because of the extra requirements vs dependencies approaches and because its probably much more version sensitive than module names. The module name is a brand after all, so maintainers are really unlikely to change that version to version.
Right now, even with 2.8.x's improved syntax, the ugly fact remains that every user of mongomock and Pants right now has to stumble through all this. Not awesome.
👍 1
s
Thanks folks 👍
❤️ 1
h
Hey @stale-nightfall-29801, can you please take a look at these new docs? https://www.pantsbuild.org/v2.8/docs/python-third-party-dependencies#requirements-with-undeclared-dependencies I'm curious if this would have helped you this morning and if you have feedback to improve it
I also linked to those new docs in the troubleshooting guide for missing imports: https://www.pantsbuild.org/v2.8/docs/troubleshooting#import-errors-and-missing-dependencies
s
Hi Eric, sorry I'm in the UK so this was in the afternoon and I'd gone home by the time this came through. I'll look today, thank you.
Yeah these documents are exactly what I was after 👏 I've not tried with 2.8 yet though as there's breaking changes and deprecation warnings around
python_distribution
I've not got time to read up on right now.