https://pantsbuild.org/ logo
#general
Title
# general
r

rhythmic-battery-45198

04/11/2022, 8:01 PM
Hey guys, I am getting a
pex.environment.ResolveError
in
2.11.0rc2
Seems to be related to extra dependencies. Adding details to๐Ÿงต
โœ… 1
Copy code
pex.environment.ResolveError: Failed to resolve requirements from PEX environment @ /home/dave/.cache/pants/named_caches/pex_root/unzipped_pexes/780e9986364e8711fb1c8e29713d51ade52ba5f4.
Needed cp38-cp38-manylinux_2_31_x86_64 compatible dependencies for:
 1: matplotlib>=3.0; extra == "plotting"
    Required by:
      FingerprintedDistribution(distribution=GPy 1.9.9 (/home/dave/.cache/pants/named_caches/pex_root/installed_wheels/8b63348de1556ed17a912bcab858509d4eb977b29413a06be18bb5b0844755be/GPy-1.9.9-py3-none-any.whl), fingerprint='b577ba75477ade4f390be4e7b738268b0944f129e21a78f2902894d695771347')
    But this pex had no 'matplotlib' distributions
My
requirements.txt
does not include the
[plotting]
extra dependencies - it is just
GPy==1.9.9
. (
pex: Building pex :: Resolving GPy==1.9.9
)
pex seems to be skipping
matplotlib
earlier in the logs.
Copy code
pex: Skipping activation of `matplotlib>=3.0; extra == "plotting"` due to environment marker de-selection
If I change my requirement to
GPy[plotting]==1.9.9
and regenerate the lockfile, the resolve error goes away, as expected.
e

enough-analyst-54434

04/11/2022, 9:42 PM
Copy code
Needed cp38-cp38-manylinux_2_31_x86_64 compatible dependencies for:
 1: matplotlib>=3.0; extra == "plotting"
    Required by:
      FingerprintedDistribution(distribution=GPy 1.9.9 (/home/dave/.cache/pants/named_caches/pex_root/installed_wheels/8b63348de1556ed17a912bcab858509d4eb977b29413a06be18bb5b0844755be/GPy-1.9.9-py3-none-any.whl), fingerprint='b577ba75477ade4f390be4e7b738268b0944f129e21a78f2902894d695771347')
    But this pex had no 'matplotlib' distributions
That can mean there is a
matplotlib
embedded in the Pex, but not one loadable for Python 3.8. Can you run Pants with
--no-process-cleanup
and then go in to the sandbox and run
PEX_TOOLS=1 ./repository.pex info -i4
and report what the repository PEX has in its manifest?
@#$! slack - did not mean for that to go to channel.
h

hundreds-father-404

04/11/2022, 9:44 PM
Iirc there is an option to undo it if you click on the message's settings. Otherwise in the past I have deleted the message and resent ๐Ÿ˜•
e

enough-analyst-54434

04/11/2022, 9:57 PM
Ah, yup. From the channel it shows up as an option, but not from here in the thread. Thanks Eric.
โค๏ธ 1
h

hundreds-father-404

04/11/2022, 9:59 PM
Ah, I did not know that is why it sometimes works and does not work! ๐Ÿคท
r

rhythmic-battery-45198

04/11/2022, 9:59 PM
Thanks John - will try this in the morning
Looks like the sandbox doesn't have a
repository.pex
,
Copy code
process-execution1lKr3q ls
local_dists.pex  pex  pytest.pex  requirements.pex  __run.sh  source_files
I dug into it and found the root cause though. While I have a direct dependency on
GPy
, it turns out that I also have a transitive dependency on
GPy[plotting]
. This results in the
pex.environment.ResolveError
. If I change my direct dependency to
GPy[plotting]
, the error goes away. Not sure if this might be a bug or a gap in my understanding.
e

enough-analyst-54434

04/12/2022, 3:29 PM
Ah, yes
s/repository.pex/requirements.pex/
. That rings a bell, let me check the archives here before asking you for more data.
This was what was ringing: https://github.com/pantsbuild/pex/issues/616 That instance though was fixed by modern Pex. Can you provide me either all the data I layed out before (with
s/repository.pex/requirements.pex/
) or else the name / version of the transitive dependency that depends on
GPy[plotting]
? Hopefully this is easy to repro with just Pex and those 2 requirements.
r

rhythmic-battery-45198

04/12/2022, 3:38 PM
Sure thing,
emukit==0.4.8
depends on
GPy[plotting]>=1.9.9
Thanks for the help - and this is not a blocker on my end since I can bubble the
[plotting]
extra up to my requirements.txt as an easy workaround.
e

enough-analyst-54434

04/12/2022, 3:44 PM
Great, thanks. For apples, what are the interpreter constraints you're using?
r

rhythmic-battery-45198

04/12/2022, 3:45 PM
Copy code
[python]
interpreter_constraints = [
  "==3.8.*"
]
lockfile_generator = 'pex'
enable_resolves = true
e

enough-analyst-54434

04/12/2022, 3:45 PM
Thanks
Ok, Yeah - easy to repro with:
Copy code
$ pex3 lock create --style universal --resolver-version pip-2020-resolver --interpreter-constraint "==3.8.*" GPy emukit==0.4.8 -o lock.json
$ pex --python python3.8 --lock lock.json -o all_from_lock.pex
$ pex-tools all_from_lock.pex info -i4
{
    "bootstrap_hash": "194811fc66631583971925a2a1c59e3bfb6c6d49",
    "build_properties": {
        "pex_version": "2.1.78"
    },
    "code_hash": "7506e475892b062c4c5f74f795b09a08a9feb157",
    "distributions": {
        "Cython-0.29.28-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl": "300800a1c9828650ffeba8325b1afc978c026b1fe81eb923852afc363909ecb3",
        "GPy-1.10.0-py3-none-any.whl": "7618d519f3b108be265f2f963987cf559a560d0c0b836a8f1013c9c87d573aae",
        "decorator-5.1.1-py3-none-any.whl": "7ed6dc469274d0b89fff2c3b5a28d1ddc0937210510a7b13016a8327fc28bc3b",
        "emcee-3.1.1-py2.py3-none-any.whl": "e404ed544fe2c8678b16388b8c35abea3a404a626068d10d0bbb617da540a044",
        "emukit-0.4.8-py3-none-any.whl": "706dfd068037f73b383d3cf8c65c3c926dc67a499e6ba336adebde2585f6c204",
        "numpy-1.22.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "fb7879661e502744c416238e5779680ce4e6d2cb6c3c70f30bd364d1475befe9",
        "paramz-0.9.5-py3-none-any.whl": "809087cc272c458d865f491fe8a112e6f200a4776fda9d2271bb4f22acc5db9c",
        "scipy-1.8.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "bcf91dc99a0fbf9915203b26a1371b5798eab488bf7cbc696b4f83b6ce53b060",
        "setuptools-62.1.0-py3-none-any.whl": "8f28465858fe3db86603d42b80b8e3d7f306d805bbae398986878e1d593e2d50",
        "six-1.16.0-py2.py3-none-any.whl": "3e1c439c88d2e7681372427bab751b3fc99969891e95a714fed9604bf7710213"
    },
    "emit_warnings": true,
    "ignore_errors": false,
    "includes_tools": false,
    "inherit_path": "false",
    "interpreter_constraints": [],
    "pex_hash": "e4b596cc533c996c1b31f32ba75804be1dd13b79",
    "pex_path": null,
    "requirements": [
        "GPy",
        "emukit==0.4.8"
    ],
    "strip_pex_env": true,
    "venv": false,
    "venv_bin_path": "false",
    "venv_copies": false,
    "venv_site_packages_copies": false,
    "pex_root": "/home/jsirois/.pex"
}
No matplotlib via the transitive extra. I'll file a Pex bug here shortly.
๐Ÿ‘ 1
r

rhythmic-battery-45198

04/13/2022, 4:04 PM
Confirmed that this issue is resolved with pex v2.1.79 ๐Ÿ’ช
โค๏ธ 1
e

enough-analyst-54434

04/13/2022, 6:19 PM
Excellent. Thanks for checking Dave.
9 Views