I have a venv pex that needs to call an application via subp Pants #general

I have a venv pex that needs to call an applicatio...

ripe-gigabyte-88964

04/27/2023, 6:27 PM

I have a venv pex that needs to call an application via subprocess that is also included in the pex. I see the

VIRTUAL_ENV

environment variable is set and also that the

PATH

is inheriting from my terminal. Is there a way I can easily include the

VIRTUAL_ENV

path in my

PATH

variable or do I need to manually export the venv first?

enough-analyst-54434

04/27/2023, 6:36 PM

How did you create the venv PEX? With pex you'd say

--venv append|prepend

when building it.

enough-analyst-54434

04/27/2023, 6:38 PM

Copy code

$ pex cowsay --venv prepend -o cowsay.pex
$ ./cowsay.pex -c 'import subprocess, sys; subprocess.check_call(["cowsay", *sys.argv[1:]])' 'Moo!'
  ____
| Moo! |
  ====
    \
     \
       ^__^
       (oo)\_______
       (__)\       )\/\
           ||----w |
           ||     ||

ripe-gigabyte-88964

04/27/2023, 6:39 PM

Copy code

dagster_pex_request = await Get(
            VenvPexRequest,
            PexRequest(
                output_filename="dagster.pex",
                internal_only=True,
                requirements=pex_requirements,
                inject_env={"DAGSTER_ENVIRONMENT": "LOCAL_DEV"},
                interpreter_constraints=InterpreterConstraints(
                    python_setup.resolves_to_interpreter_constraints[dagster_subsystem.resolve]
                ),
                main=EntryPoint("dagster"),
            ),
        )
        dagster_pex = await Get(VenvPex, VenvPexRequest, dagster_pex_request)

Building it as part of a plugin

enough-analyst-54434

04/27/2023, 6:40 PM

IIRC internal_only is the rabbit hole to go down, but just grepping "--venv" probably works.

enough-analyst-54434

04/27/2023, 6:41 PM

Out of luck:

Copy code

$ git grep -C10 '"\--venv"' src/python/pants/backend/python/util_rules/pex.py
src/python/pants/backend/python/util_rules/pex.py-    # named caches store might be pruned at any time. To guard against that case we introduce a shim
src/python/pants/backend/python/util_rules/pex.py-    # bash script that checks to see if the `pex` venv script exists in the PEX_ROOT and re-creates
src/python/pants/backend/python/util_rules/pex.py-    # the PEX_ROOT venv if not. Using the shim script to run Python tools gets us down to the ~1ms
src/python/pants/backend/python/util_rules/pex.py-    # of overhead we currently enjoy.
src/python/pants/backend/python/util_rules/pex.py-
src/python/pants/backend/python/util_rules/pex.py-    pex_request = request.pex_request
src/python/pants/backend/python/util_rules/pex.py-    seeded_venv_request = dataclasses.replace(
src/python/pants/backend/python/util_rules/pex.py-        pex_request,
src/python/pants/backend/python/util_rules/pex.py-        additional_args=pex_request.additional_args
src/python/pants/backend/python/util_rules/pex.py-        + (
src/python/pants/backend/python/util_rules/pex.py:            "--venv",
src/python/pants/backend/python/util_rules/pex.py-            "--seed",
src/python/pants/backend/python/util_rules/pex.py-            "verbose",
src/python/pants/backend/python/util_rules/pex.py-            pex_environment.venv_site_packages_copies_option(
src/python/pants/backend/python/util_rules/pex.py-                use_copies=request.site_packages_copies
src/python/pants/backend/python/util_rules/pex.py-            ),
src/python/pants/backend/python/util_rules/pex.py-        ),
src/python/pants/backend/python/util_rules/pex.py-    )
src/python/pants/backend/python/util_rules/pex.py-    venv_pex_result = await Get(BuildPexResult, PexRequest, seeded_venv_request)
src/python/pants/backend/python/util_rules/pex.py-    # Pex verbose --seed mode outputs the absolute path of the PEX executable as well as the
src/python/pants/backend/python/util_rules/pex.py-    # absolute path of the PEX_ROOT.  In the --venv case this is the `pex` script in the venv root

enough-analyst-54434

04/27/2023, 6:42 PM

So Pants passes

--venv

but not

--venv prepend

--venv append

enough-analyst-54434

04/27/2023, 6:42 PM

So looks like you need to break out a precursor PR that either just always does that or else plumbs a parameter to toggle it.

ripe-gigabyte-88964

04/27/2023, 7:16 PM

Should I create an issue along with the PR?

ripe-gigabyte-88964

04/27/2023, 7:39 PM

As a temp workaround passing

"PEX_VENV_BIN_PATH": "prepend"

as an env var to my process worked!

enough-analyst-54434

04/27/2023, 8:14 PM

Aha, I forgot I added that. Well, that's probably the right thing to stick with.

2 Views

Open in Slack

Previous Next