https://pantsbuild.org/ logo
#development
Title
# development
h

happy-kitchen-89482

05/11/2022, 7:13 PM
OK, interpreter constraints conundrum, for real:
The use case is a repo that needs to build dists and run tests on several different interpreters, including non-CPython. For the sake of simplicity let's assume just two:
CPython==3.8.*
and
PyPy==3.7.*
The question is, what should the default repo-wide interpreter_constraints be
There seem to be two options, and neither works
1)
interpreter_constraints=["CPython==3.8.*", "PyPy==3.7.*"]
. These are ORed, which is fine, but as soon as you try to select one via more specific interpreter_constraints on a target (e.g., via parametrization) we try to AND the more specific constraints with both of these, and one will fail because we can't AND across interpreter types (we check for this and fail)
2)
interpreter_constraints=[]
. I.e., no global constraints at all, and let the target constraints determine the interpreter. The problem here is that the ANDing leads to the merged interpreter constraints being
[]
themselves, i.e., no constraints at all.
2) May simply be a bug? In the ANDing we don't handle the empty constraint case correctly?
a

ancient-vegetable-10556

05/11/2022, 7:18 PM
you need to work through each runtime environment individually
So each test environment only calls the targets that are compatible with that test environment
so you do
CPython==3.8.* AND (target interpreter constraints)
then
PyPy==etc etc etc
.
h

happy-kitchen-89482

05/11/2022, 7:20 PM
Who is "you" in that sentence?
Pants needs to be modified to do this?
Trying to confirm my sense that there is no way to do this today without fixing Pants somehow
a

ancient-vegetable-10556

05/11/2022, 7:22 PM
Yes, I think Pants needs a way to handle the terms in the repo-wide constraints on a termwise basis, because any target-level constraint will be disjoint with the alternate interpreters
we don’t handle
*
as an interpreter constraint, do we?
h

happy-kitchen-89482

05/11/2022, 7:26 PM
I think that would be interpreted as
CPython==*
?
h

hundreds-father-404

05/11/2022, 7:35 PM
I think @witty-crayon-22786’s work on https://github.com/pantsbuild/pants/pull/15373 is relevant. My understanding is we stop doing that wildly complex merging of ICs algorithm and just use the target's directly. (Except validating it's compatible w/ deps)
w

witty-crayon-22786

05/11/2022, 7:35 PM
i think that 1) above is probably a bug? there is clearly a way to solve that case
because we support
OR
, you shouldn’t need to parametrize every single target (…although that would be a workaround i think?)… you should be able to rely on
OR
here
um, yea: #15373 is relevant. but probably primarily because we will switch from executing the
AND
s to doing `contains`… so it’s only relevant if contains does the right thing for:
["CPython==3.8.*", "PyPy==3.7.*"].contains(["CPython==3.8.*"])
so… yea. it seems like both 1 and 2 could be fixed.
i empathize =)
🤣 2
(…although probably a ubikey)
h

happy-kitchen-89482

05/11/2022, 8:09 PM
Goddamn it
I had set that to not emit unless pressed for an extended length of time
w

witty-crayon-22786

05/11/2022, 8:15 PM
it looks like the “AND only allows identical interpreters” bit goes back to https://github.com/pantsbuild/pants/pull/9265 … but it seems like you can trivially always have the AND of two interpreters fail to match? i.e., don’t generate a clause there. EDIT: … except i guess that there is no “clause that doesn’t match” to be generated.
@happy-kitchen-89482: given that we’re going to remove most/all merging with #15373 in a few weeks, it might be easiest to fix the “empty constraints” case
but i need to switch off of this to some BSP stuff, because #15373 already ran over
h

happy-kitchen-89482

05/11/2022, 8:19 PM
Yeah, going to fix that now
Go switch
w

witty-crayon-22786

05/11/2022, 9:14 PM
@happy-kitchen-89482: mm, i just realized that i made a change similar to the empty constraints change in #15373:
IC.merge
filters out empty sets: https://github.com/pantsbuild/pants/blob/8e986aa84e98a76761c2e1a93af0ec34e41ddda4/[…]thon/pants/backend/python/util_rules/interpreter_constraints.py … but not
merge_constraints
, which is what most codepaths use
so beware differences on very recent main
h

happy-kitchen-89482

05/11/2022, 9:29 PM
Will pull, thanks
Am indeed making changes to
merge_constraints
So could technically revert yours
w

witty-crayon-22786

05/11/2022, 9:30 PM
fire away
h

happy-kitchen-89482

05/11/2022, 11:03 PM