https://pantsbuild.org/ logo
#general
Title
# general
h

high-yak-85899

02/20/2023, 5:05 AM
Is there any way to activate the virtual environment a test is run in for that test execution? I'm dealing with a bunch of horrible things that Flink does that seem to only work if the python interpreter corresponds to one built with a virtual environment. It's running commands from a jvm and bash script like
python -m <some things it expects to be on path>
. When the test is executing,
which python
just points to the system install at
/usr/bin/python
.
h

high-yak-85899

02/20/2023, 5:15 AM
Hmm, when I do
subprocess.check_output(['which', 'python'])
it points back to system
e

enough-analyst-54434

02/20/2023, 5:17 AM
Well, there are a few things getting mixed up here. If you want
PATH
adjustment, that's one thing. Your example claimed
sys.path
was the issue (-m ...)
IOW:
sys.executable -m ...
should work fine.
h

high-yak-85899

02/20/2023, 5:18 AM
Here is the specific part that is unhappy. And yeah, I'm trying to play some
PYTHONPATH
games to get things happy currently.
e

enough-analyst-54434

02/20/2023, 5:18 AM
So, have you tried exporting
python=sys.executable
? The script you point to succumbs to that.
Just need to hook early enough. Maybe conftest hook. I have 0 clues about flink.
h

high-yak-85899

02/20/2023, 5:24 AM
How do you mean export? By environment variable or something else?
e

enough-analyst-54434

02/20/2023, 5:25 AM
Yes, env var. Read the ~1st line of the script you pointed to.
h

high-yak-85899

02/20/2023, 5:32 AM
Yeah, mostly good things and then a few things like this that are so broken out of the box. Unfortunately that env var setting didn't affect things and I'm pretty sure it was high up enough
It does work fine from the terminal so I must be doing something slightly wrong in test set up. I'll keep poking. Thanks for the ideas!
e

enough-analyst-54434

02/20/2023, 5:41 AM
Ok, and, at the end of the day, don't forget the
--venv prepend
thing. I think that could be a safe, global, change to all Pants VenvPex setups; so might be worth bugging & fixing.
You could try it and run Pants from sources via your favorite technique to test.
h

high-yak-85899

02/20/2023, 5:45 AM
I think I found a suitable workaround for now. Setting
python
envvar didn't work, but setting
PYTHONPATH
to where my source code and deps are did. So basically copying what they do in https://github.com/apache/flink/blob/af681c63821c027bc7a233560f7765b686d0d244/flink-python/pyflink/bin/pyflink-udf-runner.sh#L28
h

happy-kitchen-89482

02/20/2023, 2:24 PM
Just checking that you're aware of
--keep-sandboxes=always
which lets you play in the sandbox the test ran in
👍 1
7 Views