https://pantsbuild.org/ logo
Powered by
r

rhythmic-morning-87313

05/02/2022, 9:05 AM
Q22: My projects inside the mono repo expose their own package entrypoints to detect its own features along with plugins (just like pytest plugins). Previously I could make them to recognize self-exported entrypoints because they ran as an editable installation. It seems that pants/pex does not make editable installations but just sets up python's path/environment to recognize the dependencies archived as pex and the source roots as the package import paths. This means that there is no package metadata available (as in my Q19) when my projects run via pants. How could I expose self-exported entrypoints to pants-bootstrapped Python programs?
So this answer tells me that I need to make a kind of circular dependency between 
python_sources
and 
python_distribution
to self-reference entrypoints as package metadata. I'm trying that now.
hmm... making dependencies like:
Copy code
python_distribution("dist") -> ":lib"
pex_binary("cli") -> ":lib", ":dist"
python_sources("lib")
makes 
./pants run pkgpath:cli
to fail with missing 3rd party dependencies, which was fine until I add 
cli -> :dist
dependency relationship.
image.png
it began to suddenly fail with:
Copy code
pex.environment.ResolveError: Failed to resolve requirements from PEX environment @ /home/joongi/.cache/pants/named_caches/pex_root/unzipped_pexes/8a8630b19664717a8aabe2dc76c0019704a83f40.
Needed cp310-cp310-manylinux_2_31_aarch64 compatible dependencies for:
 1: click>=7.1.2
    Required by:
      FingerprintedDistribution(distribution=backend.ai-cli 22.9.0.dev0 (/home/joongi/.cache/pants/named_caches/pex_root/installed_wheels/b90dd7e1886232c3c399cadb17c6baf5ecb3c9bd66694b059307004cdf0412ee/backend.ai_cli-22.9.0.dev0-py3-none-any.whl), fingerprint='5eaa7f063131fd1d7d1dbd6b495d91f1293d77aaa08bd1c1cddef7a4d669d30b')
    But this pex had no 'click' distributions.
if I comment out L31 in the above screenshot, it works fine
(though i don't get the desired results)
i also don't want put explicit dependencies from 
cli
to 
:dist
targets of other custom-entrypoint-exposing first party packages, because it is intended to detect and import them optionally.
...and the number of 
cli
-compatible packages may increase/decrease arbitrarily in the future.
for example, 
maanger
pkg exposes:
for example, previously, if i (editable-)install 
backend.ai-cli
and 
backend.ai-manager
in the same venv, i could run: 
<http://backend.ai|backend.ai> mgr start-server
if I (editable-)install 
backend.ai-cli
and 
backend.ai-agent
in another venv, then inside it i could run 
<http://backend.ai|backend.ai> ag ...
commands
i'd like to know how to achieve the same thing with pants-based monorepo
with the monorepo approach, i could just run all 
<http://backend.ai|backend.ai> xxx
variants in the unified venv/pex-env.
even if I add 
:dist
to 
python_sources
instaed of 
pex_binary
, the sudden 3rd-party resolution failure occurs in the same way
So there are two ways, either: 1. Fix the dependency resolution failures when adding 
:dist
as explicit dependency of the run targets. 2. Write a custom editable installation rule....
https://github.com/pantsbuild/pex/issues/196 there was an issue, but i think this discussion would matter much more in pants, not pex.
(though implementation may involve pex as well)
this comes closer to what i need, but still it does not cover custom entrypoints other than console_scripts.
good to know 
PEX_SCRIPT
trick
my usecase other than console scripts is like:
I decided to write a custom entrypoint scanner which integrates with existing 
pkg_resources
and scans BUILD files through the source directories, because it seems to take too much time for me to wait until 
pex
to support editable installations and/or 
pants
to support entrypoint metadata injection...
image.png
wrote a helper function like this...
Powered by