https://pantsbuild.org/ logo
b

big-xylophone-43403

05/21/2021, 7:20 PM
Hi, I’m seeing a weird behavior when running the
package
goal in Pants 2.4.0 🧵
I’m using this interpreter setting -
Copy code
[python-setup]
interpreter_constraints = [">=3.7,<=3.8.5"]
interpreter_search_paths = ["<PYENV_LOCAL>", "<PYENV>", "<PATH>"]
when running
./pants package ::
, I’m getting a
ERROR: No matching distribution found for setuptools<54.0,>=50.3.0
For some reason, Pants is select my system python when building the setuptools.pex -
Copy code
ERROR: Could not find a version that satisfies the requirement setuptools<54.0,>=50.3.0
ERROR: No matching distribution found for setuptools<54.0,>=50.3.0
pid 59496 -> /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
h

happy-kitchen-89482

05/21/2021, 7:44 PM
Hmmm that is strange. It should exclude 2.7 due to interpreter constraints. @hundreds-father-404 any ideas?
Can you run with
-ldebug
?
and post a gist of the output (after checking that it's sanitized for anything you don't want public)
h

hundreds-father-404

05/21/2021, 7:52 PM
First, could you please run
./pants --backend-packages=pants.backend.python.mixed_interpreter_constraints py-constraints path/to:python_distribution_tgt
? That will confirm this isn't coming from somewhere in your repo and it's a Pants issue
b

big-xylophone-43403

05/21/2021, 7:56 PM
I get a
Copy code
14:55:47.50 [WARN] No Python files/targets matched for the `py-constraints` goal. All target types with Python interpreter constraints: pex_binary, python_library, python_tests
for my distribution target
h

hundreds-father-404

05/21/2021, 8:03 PM
Ah ha. Sorry about that. @happy-kitchen-89482 do you agree it's sensible for
python_distribution
target to have
interpreter_constraints
, like we now have for
pex_binary
and
python_awslambda
? I think it will still take the transitive closure of your constraints, not just what you set on that distribution But that way you can set additional constraints + use
py-constraints
goal
b

big-xylophone-43403

05/21/2021, 8:03 PM
Here is the debug from the package goal
h

hundreds-father-404

05/21/2021, 8:03 PM
I think it will still take the transitive closure of your constraints
Hm, although that does seem desirable to let you override constraints for binary types...and that's how v1 did things iirc
h

happy-kitchen-89482

05/21/2021, 8:35 PM
I'm not sure if
python_distribution
having constraints is pertinent here, it seems to me that the global constraints should still apply in this case?
h

hundreds-father-404

05/21/2021, 8:39 PM
@big-xylophone-43403 how about
./pants --backend-packages=pants.backend.python.mixed_interpreter_constraints py-constraints ::
?
We indeed set the interpreter constraints by looking at the transitive closure of the `python_distribution`: https://github.com/pantsbuild/pants/blob/16965761365856597602f5cf7796cdf51a2e86f7/src/python/pants/backend/python/goals/setup_py.py#L366-L370 So if that
py-constraints
command doesn't come back showing Py2 is somehow being set as a constraint, then it's unclear what's going on
b

big-xylophone-43403

05/21/2021, 9:00 PM
Yeah, I don’t see any py2 constraints -
Copy code
Final merged constraints: CPython<=3.8.5,>=3.7
  (These are the constraints used if you were to depend on all of the input
  files/targets together, even though they may end up never being used together in
  the real world. Consider using a more precise query or running `./pants py-
  constraints --summary`.)
CPython<=3.8.5,>=3.7
  pants-plugins/internal_plugins/packaging/__init__.py
  pants-plugins/internal_plugins/packaging/register.py
  src/python/gh/core/__init__.py... ..
👍 1
👀 1
@hundreds-father-404 Is there a work around for this?
h

hundreds-father-404

05/24/2021, 8:22 PM
Ah thanks for bumping this. Was this working previously, e.g. 2.3.0? Pants isn't doing what I'd expect and I'm trying to understand why / when that broke
b

big-xylophone-43403

05/24/2021, 8:57 PM
I’m not sure. I’ve only tried 2.4.0 and 2.4.1. I can try
2.3.0
and see if it still breaks
h

hundreds-father-404

05/24/2021, 8:59 PM
That'd be great if you have a moment
b

big-xylophone-43403

05/24/2021, 9:19 PM
I’m getting the same error with Pants
2.3.0
and the
>=3.7,<=3.8.5
interpreter constraints
h

hundreds-father-404

05/24/2021, 9:22 PM
Thanks, Deep. On 2.3.0, can you please try setting in `pants.toml`:
Copy code
[setuptools]
interpreter_constraints == [">=3.7,<=3.8.5"]
? That option went away in 2.4.0, but I want to see if that fixes it and gives you a workaround
b

big-xylophone-43403

05/24/2021, 9:26 PM
that worked
🙌 1
h

hundreds-father-404

05/24/2021, 9:27 PM
Great! See this option to ignore the deprecation warning if it's bothersome: https://www.pantsbuild.org/v2.3/docs/upgrade-tips#ignore-deprecation-messages-with-ignore_pants_warnings I will look into why this isn't working how we want in the meantime so that you can use Pants 2.4 and 2.5. Are you able to share the repository by chance? It's okay if over DM or you need to scrub some things
b

big-xylophone-43403

06/07/2021, 10:16 PM
Hey Eric, do you have an update on this? Pants 2.3.0 mostly works but I would like to be on 2.4.0 to use the default module mapping functionality
h

hundreds-father-404

06/08/2021, 8:34 PM
Hey Deep, thanks for following up. I unfortunately did not have a chance to look into it and am heading out for vacation the rest of the week Would you be willing to make a GitHub issue with what's going on, e.g. if you have a run with
-ldebug
? Want to make sure it doesn't get lost in this thread
Hey @big-xylophone-43403, what does
./pants dependencies path/to:python_distribution_tgt
show? I want to double check there's output, particularly first-party dependencies like
python_library
targets
Also, could you please try adding this to the BUILD file where your
python_distribution
is:
Copy code
python_library(name="debug", sources=[])
Then add
:debug
to the
dependencies
field of your
python_distribution
. Remove
[setuptools].interpreter_constraints
from
pants.toml
and run
./pants package
- does it work?
b

big-xylophone-43403

06/16/2021, 2:16 PM
Hey @hundreds-father-404, sorry for the delayed response. The
./pants dependencies path/to:python_distribution_tgt
correctly show all first-party python files.
h

hundreds-father-404

06/16/2021, 2:50 PM
b

big-xylophone-43403

06/16/2021, 3:24 PM
Ah, I see the issue. We have a repo with multiple distribution targets. One of the distribution target config was missing the
dependencies=[]
field 🤦 Sorry for the false alarm. The CI process executes the package goal for targets that changed since our main branch. And we only ran into the issue when there were changes to the target with missing dependency declaration. Sorry for the false alarm. I confirm everything works with
2.4.0
h

hundreds-father-404

06/16/2021, 3:26 PM
Yay! Then that does confirm this is broken exactly how I expected. Thanks for highlighting that! It's weird to have a Python_distribution with no dependencies, but it is legal, and we need it to work properly
For posterity, the fix for this edge case you discovered: https://github.com/pantsbuild/pants/pull/12207
👍 1
3 Views