https://pantsbuild.org/ logo
Join Slack
Powered by
Hi all. Currently I am exploring Pants and I am en...
# general
f
Hi all. Currently I am exploring Pants and I am enjoying it so far 🙂 I am facing something that I am still not sure if it is an misconfiguration on my side or an issue. Description I have a repo where the 
requirements.txt
only has one dependency listed. This dependency lives in a private repository. After configuring the 
.pants.rc
to set the index URL with the creds, when I run 
./pants generate-lockfiles
it returns a list of the dependency versions available (since I haven’t constraint that in the 
requirements.txt
). However, I noticed it doesn’t list the most recent ones. So I constraint the version to the most recent listed in the output of pants (let’s call it 
version A
) and was able to generate the lockfile. However, when tried to constrain to the version that was released after that one (let’s call it 
version B
), it couldn’t find a match distribution. I was inspecting the 
setup.py
files in both distributions and found this change:
Copy code
```
# version A
    'python_requires': '>=3.7.0,<3.8.0',

# version B
    'python_requires': '>=3.7.1,<3.8.0',
```
which aligns to the diff between the `pyproject.toml`s.
Copy code
```
# version A
python = "~3.7.0"

# version B
python = "~3.7.1"
```
In the 
pants.toml
I found that I was able to generate the lockfile for 
version A
if 
[python].interpreter_constraints = ["==3.7.*"]
. But if changes to 
[python].interpreter_constraints = [">=3.7,<4"]
for example it can’t find matching distributions for the dependency. Has anyone faced something similar? Thanks in advance 🙂 EDIT: I am using 
pex
as the 
[python].lockfile_generator
✅ 1
h
Hi there! Welcome! What Pants version are you using? If 2.11, what is 
[python].lockfile_generator
set to?
f
Hi Eric. Yes, 2.11 with 
[python].lockfile_generator = "pex"
👍 1
h
Interesting. It is plausible this is an edge case we have not encountered with Pex lockfile generation yet - thank you for reporting! It does sound like interpreter constraints are at play here. Good debugging to find that. I imagine you couldn't share that distribution for us to try to reproduce?
The next step I'd probably try to do is take Pants out of the equation and run directly via Pex. I personally use 
pipx install pex
, then it would be something like
Copy code
pex3 lock create --style=universal --resolver-version=pip-2020-resolver --interpreter-constraint='CPython==3.7.*' --repo=$foo -o debug.lock my-requirement
Not sure on the 
--repo
part, depends on how you're using 
[python-repos]
whether you use 
--repo
vs 
--index
Copy code
-f PATH/URL, --find-links PATH/URL, --repo PATH/URL
                        Additional repository path (directory or URL) to look for requirements.
  -i URL, --index URL, --index-url URL
                        Additional cheeseshop indices to use to satisfy requirements.
e
The answer is contained in the OP. Some dependency in the resolve says `'python_requires': '>=3.7.0,<3.8.0',`in its setup.py; so you can never generate a lock for any python outside that range, since that one dep, at least, does not support it.
@flat-summer-8204 the fact you posted the question means that answer must not be making sense however. So, I'm not sure how to better word things. You might have to ask more pointed questions.
How's this?: If your Pants configuration says: 
[python].interpreter_constraints = [">=3.7,<4"]
then that says, you need lockfiles to work with all Pythons convered by that configuration, for example, Python 3.7.4, Python 3.9.10 and Python 3.11b1 (and all the rest <4). SInce the resolve of your requirements runs into a distribution A that itself says it only works with Python in the 3.7 range, that's a conflict. Pants cannot satisfy your request for a lockfile that works with all Pythons covered by 
>=3.7,<4
because of that distribution A that has a narrower constraint.
➕ 1
@hundreds-father-404 were you seeing something I wasn't there? I can't see how we'd ever support this.
Basically, I'm not seeing the edge case, This seems like it should be a straight up failure and everything is working as it should.
f
thank you all for the quick relies. Adding some more details on my findings: Constraining the interpreter to a version I know I don’t have locally (
==3.7.1
), from the output I could see the possible interpreters:
Copy code
python3.6 CPython==3.6.12
python3.7 CPython==3.7.11
python3.7 CPython==3.7.7
python3.9 CPython==3.9.7
python3.8 CPython==3.8.9
Changed the constraint to 
3.7.11
and it was possible to generate the lockfile either for 
version A
and 
version B
So this seems to suggest there is something going on indeed in the constraints relaxations on 
pex
side. @hundreds-father-404 will follow your suggestion and run directly via 
pex
, thanks. @enough-analyst-54434 my apologies, I am not sure if I followed. Both versions 
A
and 
B
have ranges on 
python_requires
that should match interpreter constraint`["==3.7.*"]]` for any 
>=3.7.1
, they overlap. Unless I am missing something.
e
@flat-summer-8204 I think I still have done a poor job of making this clear. The issue will have nothing to do with local interpreters. The lock process Pants uses only considers 2 things: 1.) What are the interpreter constraints you specify, via Pants configuration. - The lock process will follow the rule that all dependencies locked must work with every interpreter on any machine implied by these constraints. 2.) What is the compatibility of the distribution currently being looked at when attempting to lock (say "requests" or "A"). It must be compatible with each and every interpreter implied by 1. So, If your Pants specified repo interpreter constraints are 
==3.7.*
*(That says all locked dependencies must work for all possible Python 3.7 interpreters), and the lock process comes across a needed dependency "A" that only works for 
>=3.7.1
, then that dependency conflicts with criteria #1. You said the repo should work for 
==3.7.*
and 
3.7.0
is not a subset of 
>=3.7.1
since 
==3.7.*
contains 
3.7.0
and 
>=3.7.1
does not. In other words, the lock fails since Pants (Pex really), sees it cannot meet the demand you placed in 1 of creating a lock that works for every possible `==3.7.` interpreter. It fails for the 
3.7.0
case in particular for dependency "A".
Does that make sense?
f
Hi @enough-analyst-54434 . Thanks for the effort on making it clearer. I think I was not clear on the cases I ran as well. Please find the summary below: Use Case 1 
requirements.txt
used to generate the lockfile
Copy code
dep-a ==4.2.31 #
On 
dep-a
v 4.2.31 , in the 
setup.py
, 
'python_requires': '>=3.7.0,<3.8.0'
Results: I was playing just changing the 
interpreter_constraints
✅ lockfile generated, ❌ fails to generate the lockfile with error: 
ERROR: No matching distribution found for dep-a==4.2.31
1. If 
pants.toml.[python].interpreter_constraints = ["==3.7.11"]
, ✅ 2. If 
pants.toml.[python].interpreter_constraints = ["==3.7.*"]
, ✅ 3. If 
pants.toml.[python].interpreter_constraints = [">=3.7.11"]
, ❌ Use Case 2 
requirements.txt
 used to generate the lockfile
Copy code
dep-a ==4.2.32
On 
dep-a
v 4.2.32 , in the 
setup.py
, 
'python_requires': '>=3.7.*1*,<3.8.0'
Results: I was playing just changing the 
interpreter_constraints
✅ lockfile generated, ❌ fails to generate the lockfile with error: 
ERROR: No matching distribution found for dep-a==4.2.32
1. If 
pants.toml.[python].interpreter_constraints = ["==3.7.11"]
, ✅ 2. If 
pants.toml.[python].interpreter_constraints = ["==3.7.*"]
, ❌ 3. If 
pants.toml.[python].interpreter_constraints = [">=3.7.11"]
, ❌ So based on the that, why does it have different results when setting 
interpreter_constraints
to 
"==3.7.11"
and 
">=3.7.11"
? I agree, seems to be a 
pex
issue indeed. I will create an issue there to iterate over this.
e
There is no Pex issue here, this is operating exactly as should be expected. Thanks for the details, I'll respond with details as well a bit later, but you may be missing the fact that 
>=3.7.11
includes all Python 3.8 interpreters, all Python 3.9 interpreters, all Python 3.10 interpreters, etc ... and none of those work with 
dep-a
.
f
Thanks a lot @enough-analyst-54434 for your reply, I understand now my problem was on NOT set the upper boundary simple smile Adding 
<3.8
to the use cases above, Use Case 1 3. If 
pants.toml.[python].interpreter_constraints = [">=3.7.11,<3.8"]
, ✅ Use Case 2 2. If 
pants.toml.[python].interpreter_constraints = ["==3.7.*"]
, ❌ , as expected because it considers 3.7.0 as you pointed out. 3. If 
pants.toml.[python].interpreter_constraints = [">=3.7.11,<3.8"]
, ✅
❤️ 1
Open in Slack
PreviousNext
Powered by