https://pantsbuild.org/ logo
Join Slack
Powered by
I'm stuck again this time at a new error, ```pex....
# general
a
I'm stuck again this time at a new error,
Copy code
pex.environment.ResolveError: Failed to resolve requirements from PEX environment @ /home/russell/.cache/pants/named_caches/pex_root/unzipped_pexes/65af0cca88bb416d7230e8e933478380260e9e31.
Needed cp39-cp39-manylinux_2_33_x86_64 compatible dependencies for:
 1: Sphinx
    Required by:
      sphinxcontrib-cldomain 0.14
    But this pex had no ProjectName(raw='Sphinx', normalized='sphinx') distributions.
 2: setuptools
    Required by:
      sphinxcontrib-cldomain 0.14
    But this pex had no ProjectName(raw='setuptools', normalized='setuptools') distributions.
I have verified that my python_distribution correctly generates with dependencies on these 2 libraries in it's metadata, and the libraries show up as dependencies
Copy code
$ ./pants dependencies --transitive tests:tests0
//:pyproject
//:sphinxcontrib-cldomain
//LICENSE:cldomain_files
//README.rst:cldomain_files
//src/sphinxcontrib/__init__.py:../../lib
//src/sphinxcontrib/cldomain/__init__.py:../../../lib
//src/sphinxcontrib/cldomain/cldomain.py:../../../lib
//src/sphinxcontrib/hyperspec/__init__.py:../../../lib
//src/sphinxcontrib/hyperspec/hyperspec.py:../../../lib
3rdparty/python#Sphinx
3rdparty/python#setuptools
3rdparty/python/requirements.txt
src/sphinxcontrib/cldomain/cldomain.lisp:lisp_lib
src/sphinxcontrib/cldomain/package.lisp:lisp_lib
src/sphinxcontrib/cldomain/test.lisp:lisp_lib
src/sphinxcontrib/cldomain/version.lisp-expr:lisp_lib
tests/test_cldomain.py:tests0
tests/test_hyperspec.py:tests0
but i get the 
ResolveError
as above when i run 
./pants tests tests:tests0
the code i'm working off is https://github.com/russell/sphinxcontrib-cldomain/tree/013c10294fb63c5371cbef92ff6999a9845aee32
👀 1
I'm thinking that a 
Project
is a 
*.pex
directory in the execution process execution. but i haven't figured out how to map a 3rd party dependency into a pex file yet
looking in my 
local_dists.pex/PEX-INFO
i can only see the library 
sphinxcontrib-cldomain
looking in my 
requirements.pex/PEX-INFO
i see the expected 
"requirements": ["Sphinx", "setuptools"],
and the associated dependencies they have
ah ha, so i also see 
"--pex-path", "pytest.pex:requirements.pex:local_dists.pex"
so i think pex is being detected. I'm not familiar with pex so i'll start debugging it next i guess
h
I don't see a 
pants
script or 
pants.toml
in that repo you linked to?
e
One thing to sort out 1st, you appear to be stomping the 
sphinxcontrib
namespace package with 
src/sphinxcontrib/__init__.py
. You need to delete that file. I see evidence for 
sphinxcontrib
being a namespace package by all the same-prefixed project names in your lock: https://github.com/russell/sphinxcontrib-cldomain/blob/013c10294fb63c5371cbef92ff6999a9845aee32/3rdparty/python/default.lock#L654
👀 1
Or, for old school ns package support, like this project: https://github.com/sphinx-contrib/napoleon/blob/master/sphinxcontrib/__init__.py
After clearing that up 1st, it will be worth taking stock of the issues that still remain. That's a pretty fundamental one that is not Pants specific.
a
I don't see a 
pants
script or 
pants.toml
in that repo you linked to?
it's on a branch, i linked the commit because of that, but the branch is called 
build-tooling
@enough-analyst-54434 i added back the namespace support stuff, but it didn't seem to effect pants
e
Ok, great. Now, what command do you run to get the OP error?
a
./pants -ldebug test tests:tests0
and i just did a test, i copied the dependencies from the requirements.pex to the local_dist.pex and that fixed the error. the local dists pex only has this,
Copy code
{"bootstrap_hash": "c73b47f34b37d54ae038f1957cbeb175a5db8ff2", "build_properties": {"pex_version": "2.1.90"}, "code_hash": "0daa6940842ebea72601f860fa1fa42ecbbfe0db", "distributions": {"sphinxcontrib_cldomain-0.14-py37.py38.py39-none-any.whl": "1ff3a7d9ef09c93ef5da278f34d5bc587620b25f99187e4dc1347412d97bbc01", }, "emit_warnings": false, "ignore_errors": false, "includes_tools": false, "inherit_path": "false", "interpreter_constraints": [], "pex_hash": "c6e9c7eb2afe58e1116e36df30eee4cf753b0018", "pex_path": null, "requirements": ["sphinxcontrib-cldomain==0.14"], "strip_pex_env": true, "venv": false, "venv_bin_path": "false", "venv_copies": false, "venv_site_packages_copies": false}
so it's missing all the actual dependencies of the library
so i'm guessing that a pex must declare all dependencies? but they aren't being calculated for my library for some reason
i wonder if this bug is related https://github.com/pantsbuild/pants/issues/15801
e
Before pointing you off to an issue, because I think this is known / there is a workaround: Do your tests need to depend on 
//:sphinxcontrib-cldomain
(i.e.: do they rely on distribution metadata), or do they just test the production code. If the latter, this can probably be fixed by letting dep inference do its job. If the former, I can dig up an issue link and explain more what you need to do.
Ah, yeah - you found the correct issue.
a
ok,that sounds like a plan, so replace the dependencies with source ones, that will work i think, i'll give it a go
e
But also, the thing is you should be able to do less!
Right now you manually specify deps but Pants should be able to infer all those such that the only target needing explicit deps is your 
python_distribution
target.
a
i think moving my code to 
src/
is going to trip me up, i'll have to modify the python path too
e
You shouldn't have to move your code to 
src/
on Pants account. Presumably you're doing that for other reasons?
And unless you're developing in-repo Pants plugins, there is no python path altering you need to do for Pants, Once it knows your source roots, it knows your python path.
If you're still having trouble here in 30 minutes or so, I can take a go at getting 
./pants test tests::
working https://github.com/russell/sphinxcontrib-cldomain/tree/bulid-tooling
a
i moved it because i was struggling to figure out how to declare what files were included in a module. Python packaging changed a lot since i last worked on this library
i should be fine, i think what you propose will work, thanks for the help
e
a
yep, i started to move to the pypackage.toml , but it took a bit for me to figure out what parts should be declared in pants and what parts in the python side
e
Ok, just speak up if you continue to hit issues. Do know, that Pants can just generate setup.py for you. You may have reasons to use a manual setup, but if you don't that may save alot of pain.
👍 1
a
OK, i read up on namespace packages. removed the legacy configuration, and things are working now. both tests and packaging, amazing
e
Ok, great. Pants can definitely work with legacy ns packages, so if you need to revisit and hit issues, let us know. The short of it though, I think, is you also need to tell setup.py / setup.cfg about legacy ns packages.
7 Views
Open in Slack
PreviousNext
Powered by