https://pantsbuild.org/ logo
Join Slack
Powered by
Got v2 working locally (hoooray), but for the life...
# general
n
Got v2 working locally (hoooray), but for the life of me I can't figure out what's wrong on Jenkins. The Dockerfile handles v1. End of the stack dump is this, and I can't even really tell which file it's not finding. Happens during a ./pants binary call to build a single pex.
Untitled
👀 1
🎉 1
h
Yay! We finished yesterday rewriting the docs at https://pants.readme.io/docs/welcome-to-pants. Hope those help, and please let us know if you have any suggestions! Hm, which Pants version is this? I’ve never seen this where it doesn’t say what file can’t be found..That error message is from the underlying OS
Ohhh, I suspect that we can’t find the “file” 
python
a
if we could repro this with process_executor, we could potentially get better info, but those tools aren’t embedded in the pants pex and it’s not easy to get to them. i think this error might happen if the output file wasn’t generated. process_executor would let us at least reproduce this error. i think we publish process_executor to s3. we could use that to repro the error here
i thought so too! but then i looked at the PATH!
h
@numerous-fall-96475 where is your Python 3.7 interpreter located, or any Python interpreter in general?
☝️ 1
a
do we need to use the python exe in the command line here? or could we execute the pex directly?
h
do we need to use the python exe in the command line here?
Yes, I believe this is necessary. See a comment in 
hermetic_pex.py
lines 43-37 for why we use the generic 
python
name rather than something more specific like 
python3.7
a
that comment mentions remote execution, which isn't relevant here i thought?
i'm just trying to figure out how to bisect the issue
h
Even in a world where remote execution didn’t exist, we would still use the generic 
python
name because Pants does not know which single Python interpreter to use, nor does it know the absolute path to that interpreter. We leave it up to the OS + Pex to handle all binary selection. We might be able to get away with 
/usr/bin/python
, but I think we had issues with each OS handling that differently. For example, macOS will in its next release not even ship with Python. We won’t know if users installed their 
python
bin in 
/usr/bin/python
or somewhere else.
n
It's running on a modified nvidia/cuda image build ontop of 18.04, and it installs python3 with apt. So I imagine python would be in /usr/bin/ unless the upstream cuda image did somethink really funky.
👍 1
a
i was under the impression that executing a pex file without the 
python
prefix would select the correct interpreter automatically. i actually know that’s not true, because i recently had to make the change to several ci jobs to add it. but i don’t know why there’s a difference in behavior
n
So I should be running python3 ./pants binary blah instead?
a
no no not for pants
this is purely relevant for pants calling pex internally
👍 1
sorry for the noise
👍 1
breaking out the pex discussion into #pex, sorry for being more confusing
i think process_executor can be used here to repro the issue. if you make a new issue with the info from this thread and ping this thread when you’ve made it we should be able to investigate it todayish
just the info that seems necessary. the log + your second most recent comment would be the relevant info i think
n
Forgive my ignorance here but is process_executor a pants specific flag? A different binary? Something else?
a
it is a different binary that allows debugging process executions like this. the v2 engine relies on a serializable remote process execution API and process_executor is the top-level binary to test that. we also have fs_util which allows modifying and inspecting the pants virtual file store. basically if you make a ticket i’m thinking that i could provide a set of commands for you to run after downloading the process_executor binary to reproduce the issue
oh wait!!!! try with 
--no-process-execution-cleanup-local-dirs
, look at the output for a directory path “preserved local process execution dir at”
and then let us know the contents of that directory
i can be more clear but it would be helpful at this point to make an issue. sorry to keep spamming the thread :) i know it’s confusing
n
I'll get this image build local and shell into it to poke around. Will have something for you later tonight although I have meetings until about 5 pm EST.
a
no problem!!! we totally understand busy schedules. i’ll see if i can jump in earlier than that given that you’ve already provided some quite nice info
w
hey @numerous-fall-96475: sorry for the trouble. based on the command and the failure, this seems like the biggest open question: https://pantsbuild.slack.com/archives/C046T6T9U/p1588781541409300?thread_ts=1588781192.403400&cid=C046T6T9U
because the error is indicating that there is no 
python
in that PATH, afaict.
👍 1
h
In particular, running 
which -a python
in the CI job would help
w
and note that it doesn’t need to be “the” python you’re expecting the run to actually use: once the 
pex
has started, it will locate an appropriate one in the PATH to match the constraint.
👍 1
n
Haven't forgotten about you guys. Meetings are done. Although having some trouble gaining access to the environments needed to provide a real apples to apples comparison on the which call.
no alias for python / python2 is not on the docker image, only python3
w
yea, that should be fine. pex will bootstrap with either of them, as long as “something” is at 
python
but you’re saying that your python3 install is not aliased to 
python
?
n
that's right
w
ah. well… that would do it, heh. cc @hundreds-father-404
h
Hm, that is indeed the issue
n
Which is very surprising, given that it's ubuntu 18.04 under the hood
👍 1
w
well, that’s your workaround i think.
n
But thank you for the helm
help*
a
please feel free to ping us in the future about this sort of thing!!! glad we could help
h
You’re welcome! Two solutions here, as far as I can tell: 1) Add a symlink such that 
python
shows up on your PATH 2) Modify the 
interpreter_search_paths
option to point to a binary named 
python
, if you have one. See https://pants.readme.io/v1.27/docs/python-interpreter-compatibility#changing-the-interpreter-search-path.
👖 1
a
thank you for closing the loop @hundreds-father-404
❤️ 1
w
@hundreds-father-404: the interpreter search paths should probably already find things with alternative names?
but i wonder if it’s possible to give pex an insanely permissive shebang that would bootstrap under ~anything it can find, so that we can avoid the 
python ./pex
in favor of 
#!/env/please/just/any python*
…i’ll take that question elsewhere.
👍 2
a
#!/usr/bin/env python would allow us to revert several command lines i had to edit for MLWF
w
won’t match 
python3
, which was the bug here.
👍 1
Open in Slack
PreviousNext
Powered by