https://pantsbuild.org/ logo
Join Slack
Powered by
Oook, I think this is enough PyOxidizer functional...
# development
a
Oook, I think this is enough PyOxidizer functionality to enable alpha-quality distribution. Feel free to take a look at the PR https://github.com/pantsbuild/pants/pull/16484 (but I suspect most of the review will need to be from @enough-analyst-54434 because 
ox.py
is utter utter madness)
e
What problem is that actually trying to solve? Is it that the Pants binary is now dual purpose and sometimes needs to be Pants and sometimes Python?
If so, could that just be inverted and run as Python by default but the pants wrapper script says PANTS_BE_PANTS=PLEASE ?
h
Is it that the Pants binary is now dual purpose and sometimes needs to be Pants and sometimes Python?
I know we said to do this project incrementally, but do we need to? We're on 2.15.0.dev0 today -- that's several releases to iron out kinks Imo, the important thing is that the 
./pants
script can continue to support pre-2.15
a
I’m answering the question
@enough-analyst-54434 the point here is that we can distribute the frozen binary without a Python interpreter, but we still need to be able to pull down plugin packages from PyPI without an actual Python interpreter hanging around.
The plugin bootstrapper in Pants uses 
sys.executable
to find the python interpreter that Pants is running under, and it invokes 
pex
with that interpreter. In a pyoxed binary, the way the Python interpreter is through the binary itself
and then, pex itself needs to be able to run pip, venv, and 
python -c
and the only Python interpreter you can guarantee is the Pants binary
e
I completely understand that the python is baked in the binary,
I'm asking if the main of the binary can be default just be python
a
we do not have the Python command line parser available to us
I spoke with Nick Coghlan about that, and he’d be willing to help me deliver that for a future CPython release
e
Cotcha.
Ok, so its a crippled Python.
Makes sense.
a
(really productive discussion over the weekend. If the work comes out of it, it completely justifies the weekend away)
it’s a crippled Python that can support Pex downloading packages from Pip into a venv
e
Yeah. From here as an outsider to this whle dev process, I assumed pyoxidizer could deliver a working python (in addition to the fact your rust code was in there and potentially owned main). That assumption was wrong.
a
Yeah, the Python is fully workable except for not having access to the command line parser
e
That sure is too bad, but thank you. Now oxy makes sense.
a
Right. When I was speaking with Nick, who wrote 
runpy
, he says that access to the raw 
argv
would make a lot of sense, at a minimum
e
This is great for not-python backends but disappointing for python since this means Pex still has the find 1 valid python interpreter to get bootstrapped with as a problem users will hit.
I was thinking we were guaranteed and we could always use the pyoxided binary as a valid python, guaranteed.
a
It’s not so much of an issue, because Python development environments should have a Python available
e
Well, history would disagree. Mac and Debian have been issues
Oh well, this is progress!
Ok. hopefully my 1 comment is easily swatted down or else easily leads to less insanity.
a
it’s no worse than Pants is currently, if you have a working Python
Our JVM support is miles ahead in that regard
e
Yeah, someone was nice enough to make downlaoding pre-packaged jvms easy IIUC!
Ok, afaict LGTM. I'm not sure how you figured it all out - that had to be gnarly! But it would be nice to drop the insane stuff / using the pyoxidized binary as a sys.executable for other Python processes on purpose at some point.
❤️ 1
a
I’ll explain the necessity in the PR for the paper trail, but basically, I was logging the argv for every invocation of the pants binary to a file with a hardcoded path, so I could see all of the executions that Pex made
and then figuring out what each invocation actually did
e
Ok. It would be good to also document why you don't just ditch that hardcoded use of sys.executable we specify elsewhere or else why ditching that didn't work, etc. Its basically alot of torture whose justification is not currently clear.
a
Yup.
I’ve explained over here: https://github.com/pantsbuild/pants/pull/16484#discussion_r956334210 — if you think this belongs in-code too, I can add that.
Open in Slack
PreviousNext
Powered by