https://pantsbuild.org/ logo
Join Slack
Powered by
I may meet a framework specific interop bug with p...
# general
p
I may meet a framework specific interop bug with pants repo Essentially I try to put a ray.io code into a PEX, and the workers failed to start (cannot find the library it depends)
Copy code
ModuleNotFoundError: No module named 'ray'
Traceback (most recent call last):
  File "~/.cache/pants/named_caches/pex_root/installed_wheels/a0d5c63d05cf2c837e5e8a0f410c1075f296d621/ray-1.0.1-cp38-cp38-macosx_10_13_x86_64.whl/ray/workers/default_worker.py", line 7, in <module>
    import ray
👋 2
h
Is this a 
zipsafe=False
pex? Which version of Pants are you on?
e
@plain-sundown-25537 you got hit by a variant of https://github.com/pantsbuild/pex/issues/1200. Your requirements.txt conflicts with your constraints.txt. Since you're using the Pip legacy resolver (consult the warning message when you run pants for upgrade instructions), the resolve failure is silent. To fix, just re-gen your constraints file.
👍 1
p
It is on pants 2.3
e
Yes, but you can still upgrade the Pip resolver without changing the Pants version:
Copy code
$ ./pants package ray_demo
17:27:50.05 [WARN] /home/jsirois/.cache/pants/setup/bootstrap-Linux-x86_64/2.3.0_py38/lib/python3.8/site-packages/pants/util/memo.py:123: DeprecationWarning: DEPRECATED: defaulting to the legacy pip resolver will be removed in version 2.4.0.dev0.
  In Pants 2.4.0, Pants will default to using pip's new resolver. The legacy resolver will be removed in Pants 2.5.

Set `[python-setup].resolver_version = 'pip-2020-resolver` to prepare for the default changing. Refer to <https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-2-2020|https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-2-2020> for more information on the new resolver.
  result = func(*args, **kwargs)
17:27:50.14 [INFO] Wrote dist/ray_demo/ray_demo.pex
p
Thanks @enough-analyst-54434 - I think I have updated the resovler by adding 
resolver_version = 'pip-2020-resolver'
for a normal venv - if we do
Copy code
pip3.7 install ray
python
> import ray
> ray.init()
Then ray server spins up fine. However, doing that in pants , it hangs
Copy code
./pants repl ray_demo
> import ray
I suspect the problem is much deeper, by inspecting the log - the workers complains about missing 
ray
package. Then if I manually install 
ray
into the global env, the workers at least started up
the global dependency helps the workers to bootstrap
e
Have you tried setting the pex_binary execution_mode to venv?
p
yup it is 
venv
already
e
Hrm, ok. I'll try to poke at this later tonight. There is almost nothing different between a PEX venv and a hand built one.
Oh wait. @plain-sundown-25537 is your only problem with repl / run? Does executing the PEX built by the package goal work fine?
p
both havning problem - let me update my test a bit more!
e
If you could file an issue that lays out the various exact ways to repro the various issues you hit using your example repo, that would be helpful.
p
Thanks!
3 Views
Open in Slack
PreviousNext
Powered by