Hi is there any way to get the equivalent of `pex binary` s Pants #general

Hi, is there any way to get the equivalent of `pex...

plain-author-67175

12/02/2024, 8:24 PM

Hi, is there any way to get the equivalent of `pex_binary`'s

execution_mode="venv"

and

venv_site_packages_copies=True

with

pants run path/to/some/file.py

? I have some source file that depends on

torch

and it looks like the 3rd-party libraries are explicitly looking for files in the virtual environment.

plain-author-67175

12/02/2024, 8:25 PM

path/to/some/file.py

is available as a

python_source

broad-processor-92400

12/04/2024, 10:05 PM

I think there may not be, so building/running a pex may be the most reliable way to get that behaviour

plain-author-67175

12/05/2024, 2:00 AM

OK. Thanks.

plain-author-67175

12/05/2024, 2:27 AM

It would be nice if the pex settings were available as global options for

run

goal. We have a bunch of ad-hoc / utility scripts that people run locally for various experiments and updating the BUILD file every time is a tad inconvenient for the end-users who aren’t software developers, just ML folks.

plain-author-67175

12/05/2024, 2:28 AM

I was hoping that I could at least create a

run-local

CLI alias with some global pex settings predefined.

broad-processor-92400

12/05/2024, 2:47 AM

Ah, good thinking! maybe you can do this with https://www.pantsbuild.org/stable/reference/subsystems/pex-cli#global_args

plain-author-67175

12/05/2024, 3:09 AM

Oh, nice! Does that work with

run

too? The doc doesn’t state that explicitly.

broad-processor-92400

12/05/2024, 3:15 AM

it should work with any invocation of pex, including the ones used to configure what's executed by

run

... but you'll be poking at implementation details here, where the exact commands and invocations may change from version to version (of pants)

plain-author-67175

12/05/2024, 3:15 AM

I see. Will give it a shot. Thanks.

plain-author-67175

12/06/2024, 2:11 AM

Welp, didn't work. I tried

Copy code

pants --keep-sandboxes=on_failure --pex-cli-global-args="--venv --venv-copies" run //path/to/script.py

and

Copy code

pants --keep-sandboxes=on_failure --pex-cli-global-args="--layout=packed --pre-install-wheels --venv --venv-copies" run //path/to/script.py

no luck. The sandbox has the following entries (changed the particular names to <name>)

Copy code

.python-build-standalone -> /home/<user>/.cache/pants/named_caches/python_build_standalone/
<package>.pex_pex_shim.sh*
<package>.pex_bin_python_shim.sh*
src/
.cache/
<package>.pex/
../
__run.sh*
./

pants

pex

didn't fail, but my script failed as before.

6 Views

Open in Slack

Previous Next