https://pantsbuild.org/ logo
#general
Title
# general
s

stocky-helmet-22655

12/07/2022, 5:12 PM
Hi all. I’ve got an issue with Pants claiming it cannot find artifacts that pipenv finds just fine. I’ve got a simplified example here: https://github.com/nikwotton/Broken-Pants-Demo - when I run
pipenv install
it works fine but when I run
./pants generate-lockfiles
it claims numpy cannot be found
e

enough-analyst-54434

12/07/2022, 5:13 PM
So what are your interpreter constraints?
Oh, sorry = thanks for the example repo, just a sec...
Yeah, thats the infidelity. You limit to ==3.7.* in your Pipfile but not in pants.toml
You want to add this: https://www.pantsbuild.org/docs/reference-python#interpreter_constraints
Copy code
[python]
interpreter_constraints = "==3.7.*"
s

stocky-helmet-22655

12/07/2022, 5:16 PM
Is there documentation somewhere on what Pants automatically takes from a pipfile and what it doesn’t? It seems arbitrary to me…
e

enough-analyst-54434

12/07/2022, 5:16 PM
Pants is trying to create a lock for >=3.7 by default and probably failing trying to get a 3.11 or 3.12 wheel
Does Pants take anything from Pipfile at all? That is a new feature to me.
I'm guessing it only takes dependencies.
s

stocky-helmet-22655

12/07/2022, 5:17 PM
*Pipfile.lock
That’s what I mean - it takes dependency constraints but not interpreter constraints for example
e

enough-analyst-54434

12/07/2022, 5:17 PM
Sure - I think this was a recent addition by a new contributor. SO maybe pile on and chip in a fix.
At the least check out the code / file an issue. Likely just an incomplete / not fully polished new feature.
s

stocky-helmet-22655

12/07/2022, 5:18 PM
Fair enough
I’m new to Pants so I didn’t realize that was a recent addition - from the documentation it seemed like it’d been there a while
Thanks for the help!
e

enough-analyst-54434

12/07/2022, 5:19 PM
I guess it depends on the defintion of new. Aug 22: https://github.com/pantsbuild/pants/pull/10654
s

stocky-helmet-22655

12/07/2022, 5:21 PM
hmm, I added the line you suggested and I’m getting the same error actually
e

enough-analyst-54434

12/07/2022, 5:21 PM
Just a sec, let me repro with your repo. Thanks again for making that.
👍 1
s

stocky-helmet-22655

12/07/2022, 5:24 PM
actually, I pushed another commit you may want to pull - if I remove the python requirement from the pipfile it still fails
e

enough-analyst-54434

12/07/2022, 5:34 PM
So our docs apparently lie? Agreed global ICs don't work, this does:
Copy code
$ git diff
diff --git a/pants.toml b/pants.toml
index f343697..893d76f 100644
--- a/pants.toml
+++ b/pants.toml
@@ -8,6 +8,7 @@ backend_packages = [
 # This will become the default in Pants 2.15.
 tailor_pex_binary_targets = false
 enable_resolves = true
+#interpreter_constraints = "==3.7.*"

 [anonymous-telemetry]
 enabled = true
@@ -15,3 +16,6 @@ repo_id = "ED3CCABC-6067-47DD-A959-353BC3F02836"

 [python.resolves]
 My_Resolve = "3rdparty/python/my_resolve.lock"
+
+[python.resolves_to_interpreter_constraints]
+My_Resolve = ["==3.7.*"]
If I just set the global ICs and run with
-ldebug
I see
... "--interpreter-constraint", "CPython<4,>=3.7", ...
in the output - IOW Pants picks the default ICs it ships with and not the global ICs you define. Seems like either an implementation bug or victory by declaring docs buggy and fixing those.
s

stocky-helmet-22655

12/07/2022, 5:46 PM
It works with numpy….I can still reproduce the issue with other dependencies though 😞 Pushed an example of it breaking with pandas - same issue,
pipenv install
works but
./pants generate-lockfiles
fails
Maybe it needs a specific X in 3.7.X ?
yep, got it working by specifying 3.7.5…
e

enough-analyst-54434

12/07/2022, 6:29 PM
So the 1st check is does the failing thing have a more specific Python requirement than you? We've had folks get stomped on the package wanting, say,
>=3.7.1
and them saying, like you,
==3.7.*
@stocky-helmet-22655 can you confirm this is the issue with this last one?
If it is, Pipfile had a bug.
It should have failed too.
Since 3.7.0 meets 3.7 but pandas says no to 3.7.0.
It wants
>=3.7.1
Pex will keep you honest.
I'm guessing when you tell Pipfile 3.7 it just confirms the current Python is any 3.7 and resolves with that. As long as it wasn't 3.7.0, it works, but the resulting lock is a lie since it won't work on another machine with 3.7.0
I mean the lockfile isn't a lie, but the success creating it is a lie. Little do you know 10 months later it will be broken for some poor sod still using 3.7.0.
@stocky-helmet-22655 I hope that all makes sense.
Trying to create a lockfile for any Python and not just the exact current one is insane. Pex participates in this insanity and tries to be correct. Nonetheless, this general style of lockfile, though convenient, is hairy.
s

stocky-helmet-22655

12/07/2022, 7:08 PM
Sorry, was in a meeting
So the 1st check is does the failing thing have a more specific Python requirement than you? We’ve had folks get stomped on the package wanting, say,
>=3.7.1
and them saying, like you,
==3.7.*
TBH it’s not fully clear to me how to tell what the failing thing’s python requirements are - I don’t see one in the Pipfile.lock for example. Apologies if there’s an obvious answer, I’m still a little new to Python in general 😬 Ah, I see it’s on pypi.org
can you confirm this is the issue with this last one?
I can confirm that setting a more specific version does fix the issue, if that’s what you’re asking. It’s unclear whether the version matters though or just that it be specific - In docker 3.7.5 works and on my machine directly 3.7.15 works. Those just happen to be versions I have installed, I haven’t tried others yet
It wants
>=3.7.1
Pex will keep you honest.
Ahhhh, that makes sense I think. To parrot back what you’re saying, Pex is stricter in version requirements than Pipenv - it’s not that pants should be succeeding but that pipenv should’ve been failing because 3.7.0 does not work. Either failing, or marking in its lockfile the >=3.7.1 requirement, which it didn’t do. Given all of this information, it seems using Pex through Pants will be a boon to my codebase because it forces more exact versioning, which I am definitely in favor of
e

enough-analyst-54434

12/07/2022, 7:13 PM
Ok, yeah. It sounds like you understand what's going on here. That's what I was hoping for. You can make your own judgements with that knowledge.
s

stocky-helmet-22655

12/07/2022, 7:16 PM
Big thanks for the help!
4 Views