https://pantsbuild.org/ logo
#general
Title
# general
p

plain-sundown-25537

04/04/2021, 4:49 AM
I may meet a framework specific interop bug with pants repo Essentially I try to put a ray.io code into a PEX, and the workers failed to start (cannot find the library it depends)
Copy code
ModuleNotFoundError: No module named 'ray'
Traceback (most recent call last):
  File "~/.cache/pants/named_caches/pex_root/installed_wheels/a0d5c63d05cf2c837e5e8a0f410c1075f296d621/ray-1.0.1-cp38-cp38-macosx_10_13_x86_64.whl/ray/workers/default_worker.py", line 7, in <module>
    import ray
👋 2
h

happy-kitchen-89482

04/04/2021, 7:23 AM
Is this a
zipsafe=False
pex? Which version of Pants are you on?
e

enough-analyst-54434

04/04/2021, 5:07 PM
@plain-sundown-25537 you got hit by a variant of https://github.com/pantsbuild/pex/issues/1200. Your requirements.txt conflicts with your constraints.txt. Since you're using the Pip legacy resolver (consult the warning message when you run pants for upgrade instructions), the resolve failure is silent. To fix, just re-gen your constraints file.
👍 1
p

plain-sundown-25537

04/04/2021, 5:26 PM
It is on pants 2.3
e

enough-analyst-54434

04/04/2021, 5:29 PM
Yes, but you can still upgrade the Pip resolver without changing the Pants version:
Copy code
$ ./pants package ray_demo
17:27:50.05 [WARN] /home/jsirois/.cache/pants/setup/bootstrap-Linux-x86_64/2.3.0_py38/lib/python3.8/site-packages/pants/util/memo.py:123: DeprecationWarning: DEPRECATED: defaulting to the legacy pip resolver will be removed in version 2.4.0.dev0.
  In Pants 2.4.0, Pants will default to using pip's new resolver. The legacy resolver will be removed in Pants 2.5.

Set `[python-setup].resolver_version = 'pip-2020-resolver` to prepare for the default changing. Refer to <https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-2-2020|https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-2-2020> for more information on the new resolver.
  result = func(*args, **kwargs)
17:27:50.14 [INFO] Wrote dist/ray_demo/ray_demo.pex
p

plain-sundown-25537

04/04/2021, 5:58 PM
Thanks @enough-analyst-54434 - I think I have updated the resovler by adding
resolver_version = 'pip-2020-resolver'
for a normal venv - if we do
Copy code
pip3.7 install ray
python
> import ray
> ray.init()
Then ray server spins up fine. However, doing that in pants , it hangs
Copy code
./pants repl ray_demo
> import ray
I suspect the problem is much deeper, by inspecting the log - the workers complains about missing
ray
package. Then if I manually install
ray
into the global env, the workers at least started up
the global dependency helps the workers to bootstrap
e

enough-analyst-54434

04/04/2021, 6:39 PM
Have you tried setting the pex_binary execution_mode to venv?
p

plain-sundown-25537

04/04/2021, 6:53 PM
yup it is
venv
already
e

enough-analyst-54434

04/04/2021, 7:08 PM
Hrm, ok. I'll try to poke at this later tonight. There is almost nothing different between a PEX venv and a hand built one.
Oh wait. @plain-sundown-25537 is your only problem with repl / run? Does executing the PEX built by the package goal work fine?
p

plain-sundown-25537

04/04/2021, 7:18 PM
both havning problem - let me update my test a bit more!
e

enough-analyst-54434

04/04/2021, 7:19 PM
If you could file an issue that lays out the various exact ways to repro the various issues you hit using your example repo, that would be helpful.
p

plain-sundown-25537

04/05/2021, 2:26 AM
Thanks!
3 Views