https://pantsbuild.org/ logo
Join Slack
Powered by
<@UQ1678BE1> how to handle Python 2 vs. Python 3.
# plugins
h
@jolly-midnight-72759 how to handle Python 2 vs. Python 3.
First, I strongly recommend getting third party dependencies working as a precursor. It’s going to be important that the 
requirements.pex
and 
jupyter.pex
use the same interpreter constraints. This is key to avoid a runtime error where you have “missing wheels”. In your v1 plugin, it sounds like it was manual to tell Pants whether to use Py2 or Py3. I recommend making this automatic; the tool can do that hard work for you. I recommend using the same approach used by 
ipython.py
. 
PexFromTargetsRequest.for_requirements()
will compute the final compatibility for all your input targets and their dependencies when merged. Then, you can pipe this into the 
PexRequest
for creating Jupyter. https://github.com/pantsbuild/pants/blob/c98ee3696818bc9945968ba94a14180d9466a6e7/src/python/pants/backend/python/goals/repl.py#L61-L69 and line 83 Make sure that the Jupyter hardcoded requirements work with both Py2 and Py3. If you want the versions to be different, use that trick I showed you this week to have conditional versions, the whole 
foo==13 ; sys_version>3
thing
🤔 1
j
I'll tackle the 3rdparty first then grep this.
It is getting more and more like a repl. 🤔
h
Yeah, it basically is a repl. I expect the code to look nearly identical to the IPython rule once the dust all settles
j
My 
requirements_pex_request
does not have a 
interpreter_contraints
like the code in 
pants.backend.python.goals.repl
.
h
can you update the gist?
j
Copy code
dir(requirements_pex_request)
21:07:04.25 [INFO] ['__abstractmethods__', '__annotations__', '__args__', '__await__', '__class__', '__dataclass_fields__', '__dataclass_params__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__extra__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__next_in_mro__', '__orig_bases__', '__origin__', '__parameters__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__slots__', '__str__', '__subclasshook__', '__tree_hash__', '__weakref__', '_abc_cache', '_abc_generic_negative_cache', '_abc_generic_negative_cache_version', '_abc_registry', '_gorg', '_is_frozen', '_validate_input', '_validate_input_type', '_validate_output_type', 'input', 'input_type', 'output_type', 'parse_input_and_output_types']

dir(requirements_pex_request.input)
21:07:04.25 [INFO] ['__annotations__', '__class__', '__dataclass_fields__', '__dataclass_params__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_is_frozen', 'additional_args', 'additional_inputs', 'additional_requirements', 'additional_sources', 'addresses', 'description', 'direct_deps_only', 'entry_point', 'for_requirements', 'hardcoded_interpreter_constraints', 'include_source_files', 'internal_only', 'output_filename', 'platforms']
I'll update the gist...
h
Thanks! So the reason it doesn’t have 
interpreter_constraints
is that you are saying 
await Get(Pex, PexFromTargetsRequest)
. This will return a 
Pex
object. Normally, that’s what you want, so that you actually create the Pex file. But here, we’re gonna do something weird, that we instead say 
await Get(PexRequest, PexFromTargetsRequest)
. This will get us back a 
PexRequest
object, which has an 
interpreter_constraints
field. When you say 
await Get(Pex, PexFromTargetsRequest)
, what really happens is the engine goes from 
PexFromTargetsRequest -> PexRequest -> Pex
, and does that middle step for you. Here, we care about the value in the middle step, so we explicitly ask for it. Then, once you have that intermediate 
PexRequest
object, you’ll need to say 
await Get(Pex, PexRequest, my_pex_request_object_from_earlier)
👍🏽 1
j
Why does it have the second 
PexFromTargetsRequest
? i.e. 
Get(PexRequest, PexFromTargetsRequest, PexFromTargetsRequest.for_requirements(...),)
h
I’m not seeing that. Where at? In 
repl.py
, for the 
create_ipython_repl_request
rule, I’m seeing an 
await Get(Get(PexRequest, PexFromTargetsRequest, PexFromTargetsRequest.for_requirements(...))
at the top so that we get the intermediate 
PexRequest
. Then we set up right after the 
Get(Pex, PexRequest)
for it. And then another 
Get(Pex, PexRequest)
, but this time it’s for ipython, rather than requirements
j
variable name question. I think you are saying 
Get(Pex, PexRequest)
returns a pex, not a pex_request. So I should rename 
jupyter_pex_request
to something else?
h
variable name question.
The variable names are kind of a mess here. We use the word “request” way too much. Sorry about that To clarify, 
await Get(Pex, PexRequest)
will give back a 
Pex
object. It will pause execution of Python and cause the engine to trigger a subprocess to build the Pex. Note the 
await
. If we leave off the 
await
, and only have 
Get(Pex, PexRequest)
, then we only have a 
Get
object. Nothing actually gets executed yet, until we say 
await
later. We’ve been calling this a “request”, which is super confusing.
j
Yep. I won't lie that that is confusing.
But I've looked at code I've written over a month ago and get confused, too.
h
It is very confusing. It made sense until we added the type 
PexRequest
hah Maybe something like 
ipython_pex_get
is a better name than 
ipython_pex_request
3 Views
Open in Slack
PreviousNext
Powered by