https://pantsbuild.org/ logo
Join Slack
Powered by
Hello all, I am setting up a proof of concept Pant...
# general
w
Hello all, I am setting up a proof of concept Pants build for an existing small Python monorepo at work. First of all, thank you for this tool and for providing this forum for getting assistance! I have encountered a problem with (I think) a pytest dependency clashing with a project dependency, and I would appreciate any pointers you can give me on resolving it. In a nutshell, the project depends on version 21.4.0 of the attrs library, but an older version, 21.2.0, is pulled in as a pytest dependency (I think), and tests fail because they run with an incompatible mix of code from both versions. I am not sure if this indicates a bug in Pants or perhaps an error on my part. I'll dump details in the thread.
To set expectations about what kinds of mistakes I might be making, this is my first time using Pants, and though I've been using Python casually since just before the 2.0 release, this is my first time doing application programming in Python.
Running 
./pants test ::
I get the following error:
Copy code
/Users/david/.cache/pants/named_caches/pex_root/venvs/s/305a3081/venv/lib/python3.9/site-packages/attrs/_init_.py:31: in <module>
    from attr._next_gen import asdict, astuple
E   ImportError: cannot import name 'asdict' from 'attr._next_gen' (/Users/david/.cache/pants/named_caches/pex_root/venvs/s/305a3081/venv/lib/python3.9/site-packages/attr/_next_gen.py)
This is suspicious because the attr and attrs packages are both provided by the attrs library, so they should not be out of sync with each other. Looking inside the site-packages directory referenced in the error message, I discovered that attrs/__init__.py is from attrs 21.4.0, but attr/_next_gen.py is from an older version. I am using constraints.txt, generated by poetry as suggested in the documentation, and it specifies attrs 21.4.0:
Copy code
attrs==21.4.0; (python_version >= "2.7" and python_full_version < "3.0.0") or (python_full_version >= "3.5.0")
I deleted the pants caches and re-ran ./pants test ::, and found attrs wheels with versions 21.2.0 and 21.4.0 in the cache. Trying to figure out where the 21.2.0 wheel came from, I ran pants with --no-process-cleanup and -ldebug, looked inside the directories mentioned with "Preserving local process execution dir", and found that the 21.2.0 wheels were present only in the process execution directories related to running tests. I looked inside the process execution directory for "Building pytest.pex from pytest_default_lockfile.txt" and found this in pytest_default_lockfile.txt:
Copy code
attrs==21.2.0; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.6" \
    --hash=sha256:149e90d6d8ac20db7a955ad60cf0e6881a3f20d37096140088356da6c716b0b1 \
    --hash=sha256:ef6aaac3ca6cd92904cdd0d83f629a15f18053ec84e6432106f7a4d04ae4f5fb
This is where version 21.2.0 of attrs is coming from, but I don't know where to go from here in terms of debugging further and resolving the problem. Any tips or assistance would be much appreciated!
p
e
I think that's not quite right. This seems to be the 1st instance of a need for tool shading in Python. We've supported this for JVM tools in v1, but never for Python. @wooden-library-2965 the issue here is two separate resolves - one for the pytest test tool and one for the code under test - are combined. The 1st on the 
sys.path
wins for any overlaps. Since we don't support shading the pytest tool depencies yet, you'll have to force the pytest tool to use the same version of attrs. See here for the recipe: https://www.pantsbuild.org/docs/python-third-party-dependencies#tool-lockfiles You'll want to add an extra requirement for pytest on attrs directly with the version your code needs. This assumes that version is compatible with pytest (transitive) requirements. I haven't checked.
And @wooden-library-2965 thanks for that level of debugging. You provided some crucial information there.
w
Thanks, using a pytest lockfile with
Copy code
extra_requirements.add = ["attrs==21.4.0"]
worked!
🙌 1
e
Excellent. Note there will come a day when this won't work and shading will be the only answer, but we have ideas there: https://github.com/pantsbuild/pants/issues/9206
👍 1
Open in Slack
PreviousNext
Powered by