https://pantsbuild.org/ logo
#general
Title
# general
a

able-shampoo-44547

07/09/2022, 1:46 PM
I'm stuck again this time at a new error,
Copy code
pex.environment.ResolveError: Failed to resolve requirements from PEX environment @ /home/russell/.cache/pants/named_caches/pex_root/unzipped_pexes/65af0cca88bb416d7230e8e933478380260e9e31.
Needed cp39-cp39-manylinux_2_33_x86_64 compatible dependencies for:
 1: Sphinx
    Required by:
      sphinxcontrib-cldomain 0.14
    But this pex had no ProjectName(raw='Sphinx', normalized='sphinx') distributions.
 2: setuptools
    Required by:
      sphinxcontrib-cldomain 0.14
    But this pex had no ProjectName(raw='setuptools', normalized='setuptools') distributions.
I have verified that my python_distribution correctly generates with dependencies on these 2 libraries in it's metadata, and the libraries show up as dependencies
Copy code
$ ./pants dependencies --transitive tests:tests0
//:pyproject
//:sphinxcontrib-cldomain
//LICENSE:cldomain_files
//README.rst:cldomain_files
//src/sphinxcontrib/__init__.py:../../lib
//src/sphinxcontrib/cldomain/__init__.py:../../../lib
//src/sphinxcontrib/cldomain/cldomain.py:../../../lib
//src/sphinxcontrib/hyperspec/__init__.py:../../../lib
//src/sphinxcontrib/hyperspec/hyperspec.py:../../../lib
3rdparty/python#Sphinx
3rdparty/python#setuptools
3rdparty/python/requirements.txt
src/sphinxcontrib/cldomain/cldomain.lisp:lisp_lib
src/sphinxcontrib/cldomain/package.lisp:lisp_lib
src/sphinxcontrib/cldomain/test.lisp:lisp_lib
src/sphinxcontrib/cldomain/version.lisp-expr:lisp_lib
tests/test_cldomain.py:tests0
tests/test_hyperspec.py:tests0
but i get the
ResolveError
as above when i run
./pants tests tests:tests0
the code i'm working off is https://github.com/russell/sphinxcontrib-cldomain/tree/013c10294fb63c5371cbef92ff6999a9845aee32
👀 1
I'm thinking that a
Project
is a
*.pex
directory in the execution process execution. but i haven't figured out how to map a 3rd party dependency into a pex file yet
looking in my
local_dists.pex/PEX-INFO
i can only see the library
sphinxcontrib-cldomain
looking in my
requirements.pex/PEX-INFO
i see the expected
"requirements": ["Sphinx", "setuptools"],
and the associated dependencies they have
ah ha, so i also see
"--pex-path", "pytest.pex:requirements.pex:local_dists.pex"
so i think pex is being detected. I'm not familiar with pex so i'll start debugging it next i guess
h

happy-kitchen-89482

07/09/2022, 2:48 PM
I don't see a
pants
script or
pants.toml
in that repo you linked to?
e

enough-analyst-54434

07/09/2022, 2:50 PM
One thing to sort out 1st, you appear to be stomping the
sphinxcontrib
namespace package with
src/sphinxcontrib/__init__.py
. You need to delete that file. I see evidence for
sphinxcontrib
being a namespace package by all the same-prefixed project names in your lock: https://github.com/russell/sphinxcontrib-cldomain/blob/013c10294fb63c5371cbef92ff6999a9845aee32/3rdparty/python/default.lock#L654
👀 1
Or, for old school ns package support, like this project: https://github.com/sphinx-contrib/napoleon/blob/master/sphinxcontrib/__init__.py
After clearing that up 1st, it will be worth taking stock of the issues that still remain. That's a pretty fundamental one that is not Pants specific.
a

able-shampoo-44547

07/09/2022, 3:53 PM
I don't see a
pants
script or
pants.toml
in that repo you linked to?
it's on a branch, i linked the commit because of that, but the branch is called
build-tooling
@enough-analyst-54434 i added back the namespace support stuff, but it didn't seem to effect pants
e

enough-analyst-54434

07/09/2022, 3:58 PM
Ok, great. Now, what command do you run to get the OP error?
a

able-shampoo-44547

07/09/2022, 3:58 PM
./pants -ldebug test tests:tests0
and i just did a test, i copied the dependencies from the requirements.pex to the local_dist.pex and that fixed the error. the local dists pex only has this,
Copy code
{"bootstrap_hash": "c73b47f34b37d54ae038f1957cbeb175a5db8ff2", "build_properties": {"pex_version": "2.1.90"}, "code_hash": "0daa6940842ebea72601f860fa1fa42ecbbfe0db", "distributions": {"sphinxcontrib_cldomain-0.14-py37.py38.py39-none-any.whl": "1ff3a7d9ef09c93ef5da278f34d5bc587620b25f99187e4dc1347412d97bbc01", }, "emit_warnings": false, "ignore_errors": false, "includes_tools": false, "inherit_path": "false", "interpreter_constraints": [], "pex_hash": "c6e9c7eb2afe58e1116e36df30eee4cf753b0018", "pex_path": null, "requirements": ["sphinxcontrib-cldomain==0.14"], "strip_pex_env": true, "venv": false, "venv_bin_path": "false", "venv_copies": false, "venv_site_packages_copies": false}
so it's missing all the actual dependencies of the library
so i'm guessing that a pex must declare all dependencies? but they aren't being calculated for my library for some reason
i wonder if this bug is related https://github.com/pantsbuild/pants/issues/15801
e

enough-analyst-54434

07/09/2022, 4:02 PM
Before pointing you off to an issue, because I think this is known / there is a workaround: Do your tests need to depend on
//:sphinxcontrib-cldomain
(i.e.: do they rely on distribution metadata), or do they just test the production code. If the latter, this can probably be fixed by letting dep inference do its job. If the former, I can dig up an issue link and explain more what you need to do.
Ah, yeah - you found the correct issue.
a

able-shampoo-44547

07/09/2022, 4:04 PM
ok,that sounds like a plan, so replace the dependencies with source ones, that will work i think, i'll give it a go
e

enough-analyst-54434

07/09/2022, 4:04 PM
But also, the thing is you should be able to do less!
Right now you manually specify deps but Pants should be able to infer all those such that the only target needing explicit deps is your
python_distribution
target.
a

able-shampoo-44547

07/09/2022, 4:05 PM
i think moving my code to
src/
is going to trip me up, i'll have to modify the python path too
e

enough-analyst-54434

07/09/2022, 4:06 PM
You shouldn't have to move your code to
src/
on Pants account. Presumably you're doing that for other reasons?
And unless you're developing in-repo Pants plugins, there is no python path altering you need to do for Pants, Once it knows your source roots, it knows your python path.
If you're still having trouble here in 30 minutes or so, I can take a go at getting
./pants test tests::
working https://github.com/russell/sphinxcontrib-cldomain/tree/bulid-tooling
a

able-shampoo-44547

07/09/2022, 4:10 PM
i moved it because i was struggling to figure out how to declare what files were included in a module. Python packaging changed a lot since i last worked on this library
i should be fine, i think what you propose will work, thanks for the help
e

enough-analyst-54434

07/09/2022, 4:10 PM
a

able-shampoo-44547

07/09/2022, 4:11 PM
yep, i started to move to the pypackage.toml , but it took a bit for me to figure out what parts should be declared in pants and what parts in the python side
e

enough-analyst-54434

07/09/2022, 4:12 PM
Ok, just speak up if you continue to hit issues. Do know, that Pants can just generate setup.py for you. You may have reasons to use a manual setup, but if you don't that may save alot of pain.
👍 1
a

able-shampoo-44547

07/09/2022, 4:29 PM
OK, i read up on namespace packages. removed the legacy configuration, and things are working now. both tests and packaging, amazing
e

enough-analyst-54434

07/09/2022, 4:36 PM
Ok, great. Pants can definitely work with legacy ns packages, so if you need to revisit and hit issues, let us know. The short of it though, I think, is you also need to tell setup.py / setup.cfg about legacy ns packages.
7 Views