https://pantsbuild.org/ logo
Join Slack
Powered by
OK, interpreter constraints conundrum, for real:
# development
h
OK, interpreter constraints conundrum, for real:
The use case is a repo that needs to build dists and run tests on several different interpreters, including non-CPython. For the sake of simplicity let's assume just two: 
CPython==3.8.*
and 
PyPy==3.7.*
The question is, what should the default repo-wide interpreter_constraints be
There seem to be two options, and neither works
1) 
interpreter_constraints=["CPython==3.8.*", "PyPy==3.7.*"]
. These are ORed, which is fine, but as soon as you try to select one via more specific interpreter_constraints on a target (e.g., via parametrization) we try to AND the more specific constraints with both of these, and one will fail because we can't AND across interpreter types (we check for this and fail)
2) 
interpreter_constraints=[]
. I.e., no global constraints at all, and let the target constraints determine the interpreter. The problem here is that the ANDing leads to the merged interpreter constraints being 
[]
themselves, i.e., no constraints at all.
2) May simply be a bug? In the ANDing we don't handle the empty constraint case correctly?
a
you need to work through each runtime environment individually
So each test environment only calls the targets that are compatible with that test environment
so you do 
CPython==3.8.* AND (target interpreter constraints)
then 
PyPy==etc etc etc
.
h
Who is "you" in that sentence?
Pants needs to be modified to do this?
Trying to confirm my sense that there is no way to do this today without fixing Pants somehow
a
Yes, I think Pants needs a way to handle the terms in the repo-wide constraints on a termwise basis, because any target-level constraint will be disjoint with the alternate interpreters
we don’t handle 
*
as an interpreter constraint, do we?
h
I think that would be interpreted as 
CPython==*
?
h
I think @witty-crayon-22786’s work on https://github.com/pantsbuild/pants/pull/15373 is relevant. My understanding is we stop doing that wildly complex merging of ICs algorithm and just use the target's directly. (Except validating it's compatible w/ deps)
w
i think that 1) above is probably a bug? there is clearly a way to solve that case
because we support 
OR
, you shouldn’t need to parametrize every single target (…although that would be a workaround i think?)… you should be able to rely on 
OR
here
um, yea: #15373 is relevant. but probably primarily because we will switch from executing the 
AND
s to doing `contains`… so it’s only relevant if contains does the right thing for: 
["CPython==3.8.*", "PyPy==3.7.*"].contains(["CPython==3.8.*"])
so… yea. it seems like both 1 and 2 could be fixed.
i empathize =)
🤣 2
(…although probably a ubikey)
h
Goddamn it
I had set that to not emit unless pressed for an extended length of time
w
it looks like the “AND only allows identical interpreters” bit goes back to https://github.com/pantsbuild/pants/pull/9265 … but it seems like you can trivially always have the AND of two interpreters fail to match? i.e., don’t generate a clause there. EDIT: … except i guess that there is no “clause that doesn’t match” to be generated.
@happy-kitchen-89482: given that we’re going to remove most/all merging with #15373 in a few weeks, it might be easiest to fix the “empty constraints” case
but i need to switch off of this to some BSP stuff, because #15373 already ran over
h
Yeah, going to fix that now
Go switch
w
@happy-kitchen-89482: mm, i just realized that i made a change similar to the empty constraints change in #15373: 
IC.merge
filters out empty sets: https://github.com/pantsbuild/pants/blob/8e986aa84e98a76761c2e1a93af0ec34e41ddda4/[…]thon/pants/backend/python/util_rules/interpreter_constraints.py … but not 
merge_constraints
, which is what most codepaths use
so beware differences on very recent main
h
Will pull, thanks
Am indeed making changes to 
merge_constraints
So could technically revert yours
w
fire away
h
Open in Slack
PreviousNext
Powered by