https://pantsbuild.org/ logo
Join Slack
Powered by
Why does the `find_python` rule in `system_binarie...
# development
b
Why does the 
find_python
rule in 
system_binaries
not just use 
sys.executable
? For 
pants
it'd be the standlone, for 
./pants
it'd be whatever the script found.
I'm very confused by all the Pythons floating around and how they are resolved. This Python I don't quite understand 😕
e
Well, the rule takes a PythonBootstrap - that defines legal places to look for Python. The sys.executable may not be legal. So that's the obvious thing. As to whether that restriction is important, I'm not sure. But it could be that the 
./pants
bootstrap script finds a bogus Mac system interpreter that happens to work to bootstrap Pants but then would fail for running user code through Pex. That's my guess.
The graveyard of Mac system interpreter issues is full but always expanding.
😂 1
The 
pants
sys.executable is certainly bogus for building sdists! So it helps save PBS bootstraps for getting used for things they should not be used for.
It really truly causes unending confusion that Pants rules are written in Python. I think its best to always pretend they're written in COBOL.
You cannot cheat and use the runtime.
b
When is this used to run user code though? Wouldn't we still need to filter the possible Pythons down to the relevant ICs?
I'm guessing this is only used to run the pex command that runs to find ICs, which isn't user code
e
Follow the breadcrumb, grep says 2 places ...
How do you know though that Pex interpreter discovery doesn;t require a fully functioning interpreter?
b
Does it?
e
b
PBS is "fully-functioning" is it not? Sans sdists which I know PEX will never depend on
e
It is not fully funtioning! Parts of distutils do not work in PBS. Basically, you can take it from here I think, just dig and make sure - but truly the COBOL thing is really the way to think about things. The runtime really should not be used - period.
b
(The 
./pants
issue remains however)
Ok I think I get it. The Mac example was helpful
e
Ok - but the COBOL point should be even more helpful. Any use of the current rule runtime to do anything other than run rule code is not a good idea. It should never be used for a Process is, I think, always safe to say.
p
Now I’m imagining BUILD files w/ COBOL divisions, sections, and paragraphs… oh boilerplate hell! (yeah, I know it’s an illustration to help in not thinking of BUILD as python but as a DSL)
e
It's not actually. It's to help with not thinking about rule code as Python. It is Python, but it's not real Python, or should not be treated that way. More to the point, whatever the rule code language was, say COBOL, we should never try to run a COBOL Process using the Pants COBOL runtime. The Pants configuration language is another matter entirely. For that, pretend BUILD is toml.
p
ah yes. The rules are more of a DSL as well.
I ran up against that quite a bit when first learning to write rules… And I purposely broke some of the pants conventions in a couple of places, using some pythonic code with comments explaining why it was ok to not use a Get at that point… 😛
b
So in thinking about de-coupling Pants' reliance on system Python, theoretically would 
pyoxy
work as a Python exe for pex's sake?
Aww hell, let's find out 😛
e
PyOxy == PBS
b
Oh I was gonna go even harder and use the one-file
e
Pex does not currently guard its sys.executable - unlike how Pants must. IOW, Pex will use sys.executable to do building if sys.executable meets ICs, etc. And so that is trouble.
b
Hmm not following. Put another way? This is just for the case that we're running a PEX and no ICs were baked in
e
Running a PEX or the Pex PEX?
b
Certainly the latter. I think sometimes the former. Oh damn I see
e
I'd really slow way the heck down and try to understand things.
You can run an experiment, succeed, and in the end be wrong here.
b
OK I'm relatively certain it is only the latter.
e
Pex works with any Python; so you can just pyenv ~random, then run the Pex PEX with that; then find out the real Pythons you need, then pyenv those. You are fully correct and divorced from the runtime. You have potentially wasted the initial rando pyenv bootstrap if that interpreter is not used by user code. That, at least, gets you fully correct. Later you can figure out how to optimize the case where rando gets never used again.
b
That's.... extremely fair in terms of piecemeal development
e
But I would expend some effort towards thinking about if the optimization is even structurally possible. I myself am not sure.
b
I suspect it is, albeit using some other python. I don't think it's possible solely using pyenv
PBS/`pyoxy` seem like prime candidates for the decoupling
e
Doesn't Pants have modeling for ICs? So rule code could know, for example, Python 3.7 is in-range for user code?
b
Yes it does, however not all rule code is running in the context of ICs.
e
If not, that sounds like a bug.
b
Not all rule code is user code
I did find an example where it isn't PEX-related though. 
GunzipBinary
e
So, there is either a default IC or one or more explicit ones, right?
b
Not for 
GunzipBinary
. That should be usable without enabling the Python backend 😐
e
At any rate, I work poorly in your mode. I think you have enough nudges to spin on this.
b
Yeah certainly. I think best path forward is user-code-via-pyenv. Then untangling the Pants internals
3 Views
Open in Slack
PreviousNext
Powered by