https://pantsbuild.org/ logo
#development
Title
# development
b

bitter-ability-32190

06/24/2022, 3:00 PM
From within a PEX, I want to find the entry_point of a console_script.
from pex.finders import get_entry_point_from_console_script
gets me most of the way there, but I need to also give it the list of distributions. Pex grabs this with
dists = list(self.activate())
on a
PEX
object, however mimicking that goes šŸ’„
Copy code
from pex.pex import PEX
dists=PEX().activate()
->
Copy code
FileNotFoundError: [Errno 2] No such file or directory: '-c/PEX-INFO'
Before I go on, thought I'd ask here to make sure I'm not about to do something stupid šŸ™‚
āœ… 1
For more info, this is for running
debugpy
and having it execute a
console_script
. Basically I have
pex
->
debugpy.pex
(which acts like the
python
interpreter) ->
-c "...my code to find and execute the script"
Looks like both
pex
and
debugpy
use
runpy
, so we're all still in one big happy env
Oh ho ho, setting
pex=os.environ.get("PEX')
gets me somewhere! Although I only see the distribution for
debugpy
and not any of the other deps I'm shoehorning in via
PEX_PATH
print(os.environ.get('PEX_PATH', None));
is
None
so I guess that checks. I'm guessing PEX is scrubbing itself by the time my code is executing
"--no-strip-pex-env"
wins me a coke
Copy code
# NB: Use PEX itself to execute the console_script
(
    "import os;"
    + "from pex.pex import PEX;"
    + f"PEX(pex=os.environ['PEX']).execute_script('{value.name}')"
),
Ah doesn't work for venv PEXs seemingly
Aha I can reach into pexs vendored dir for pkg_resources
e

enough-analyst-54434

06/26/2022, 4:05 AM
It would be great if you don't rely on any Pex API (it supports only the CLI as an API) except for the PEX env var. Best is to just depend on setuptools directly and use its API. Pex vendoring setuptools will go away.
b

bitter-ability-32190

06/26/2022, 10:58 AM
I agree, however this is a stopgap until debugpy drops 3.7 support. Happy to hear suggestions for alternatives though šŸ™ƒ
This code executes in a PEX with user reqs so: ā€¢ Trying to add a req might unintentionally cause a dep difference ā€¢ Pants code isn't handled for that for tools Which makes this challenging. Additionally this is just to resolve a console script, so an entirely new process seems like A LOT of overhead šŸ˜ž
e

enough-analyst-54434

06/26/2022, 1:41 PM
Ok, the context helps. What does 3.7 have to do with this? Pants supports user code that requires as old as 2.7 and your extra context implies this must work mixed in to user code.
From what I can tell, you're too worried about perf. If the context is debugging, which is slow by definition, a separate Process to predetermine the console script module is fine.
Oh, so Pants offers debugging, but only for a subset of the user code it supports, 3.6+ now, 3.7+ later. 2.7 and 3.5 don't get this feature. Makes sense.
b

bitter-ability-32190

06/26/2022, 1:54 PM
ā€¢ 3.7 because 3.8 has importlib.metadata in stdlib ā€¢ Yeah the underlying support library is 3.7+ ā€¢ Not perf focused per-se, but the gymnastics to add code outside of what PEX is running won't be easy on Pants šŸ˜ž
And yeah I just opened an issue to early-error if the target ICs don't jive with debugpy's 3.7+
If the current hack is too hacky we can file an issue and swing around and do it right after the provisional support is added
e

enough-analyst-54434

06/26/2022, 2:04 PM
FWIW you're fighting a generic unsolved issue. The current status quo is to use PEX_PATH to adjoin a tool PEX to a user requirement PEX. This allows adding deps to the tool PEX that may conflict with user code and 1st on sys.path wins. For the case of finding console script entry points, this typically shaky arrangement is solid.
b

bitter-ability-32190

06/26/2022, 2:09 PM
I'm using that currently, the technical details make it hard to add additional deps easily. Also I dont want our supplemental dep to "win" so order matters
I'm thinking extra process might not be too bad. It's there an easy way to ask PEX this?
e

enough-analyst-54434

06/26/2022, 2:22 PM
Copy code
jsirois@siroisdesign:~$ pex pex -c pex -o pex.pex
jsirois@siroisdesign:~$ ./pex.pex cowsay -c cowsay -o cowsay.pex
jsirois@siroisdesign:~$ PEX_SCRIPT=pex-tools ./pex.pex cowsay.pex info -i2
{
  "bootstrap_hash": "2fe5b6193b3a13f029607461b1403422c82eb36e",
  "build_properties": {
    "pex_version": "2.1.92"
  },
  "code_hash": "1c57aa9488b00dac83e854a24dff9d6320daa013",
  "distributions": {
    "cowsay-5.0-py2.py3-none-any.whl": "9d9be34e631f7691e63cfbeb4b0f9add9b480da561e2b6d7421fc4e192a6d320"
  },
  "emit_warnings": true,
  "entry_point": "cowsay.main:cli",
  "ignore_errors": false,
  "includes_tools": false,
  "inherit_path": "false",
  "interpreter_constraints": [],
  "pex_hash": "d73b21ed0785f13e73b6b2272b6def7a0163224f",
  "pex_path": null,
  "requirements": [
    "cowsay"
  ],
  "strip_pex_env": true,
  "venv": false,
  "venv_bin_path": "false",
  "venv_copies": false,
  "venv_site_packages_copies": false,
  "pex_root": "/home/jsirois/.pex"
}
You can use the
"entry_point"
value to get the module of the console script.
The 1st two lines of that example set up things Pants already has, the Pex PEX and the user PEX.
b

bitter-ability-32190

06/26/2022, 3:48 PM
I'll find the pants incantation for this šŸ‘€
AH I see, we're close. How can I resolve an arbitrary console script inside the PEX? I don't need
entry_point
I need a mapping of console_script name to entry_points šŸ¤”
e

enough-analyst-54434

06/26/2022, 5:27 PM
You can't. So why can't you do this with a bespoke PEX that just contains setuptools + your code using it and PEX_PATH in the user requirements?
If you're just reading metadata, your setuptools tromping theirs (if they depend on it too) isn't really a thing. No user code is imported at all.
b

bitter-ability-32190

06/26/2022, 5:29 PM
I can, just seems like a lot of legwork to resolve a console_script šŸ™ˆ ā€¢ We have to pick a version of
setuptools
and make sure we use hashes and the other stuff to ensure safety for our users ā€¢ Have to make sure the
PEX_PATH
uses the correct ordering ā€¢ Involves yet another PEX
e

enough-analyst-54434

06/26/2022, 5:30 PM
Seems par for the course to me.
Pretend this isn't Python. How would you do it in Java? You'd need a tool.
b

bitter-ability-32190

06/26/2022, 5:39 PM
I'm tempted to leave the code as-is. as bad as it is to assume its super simple
alternatively I think the next best thing is folding
importlib_metadata
into the debugpy PEX and conventionally add it last. Pants isnt set up to support this easily though (hashes and whatnot would be handcrafted)
e

enough-analyst-54434

06/26/2022, 5:40 PM
Yeah, that sounds fine. If Pex breaks you, you know at least it is expected.
āœ… 1
b

bitter-ability-32190

06/26/2022, 5:41 PM
I'm betting by the time PEX drops vendored setuptools, debugpy will have dropped py3.7 support