https://pantsbuild.org/ logo
#general
Title
# general
p

proud-dentist-22844

12/22/2022, 3:44 PM
I'm adjusting
interpreter_constraints
. The key piece I'm questioning is in this BUILD file: https://github.com/StackStorm/st2/pull/5850/files#diff-a1de578ad6ec792e46e84af2ab04188980ed060c25f673c018cf0cf4bd142d93 I had interpreter constraints in
pants.toml
for the
pants-plugins
resolve without applying them here. I assumed that the
[python.resolves_to_interpreter_constraints].pants-plugins
would automatically be applied to anything that used that resolve. But, the default comes from
[python].interpreter_constraints
instead, which is not compatible with that resolve. Was my assumption off base? Why do I need to duplicate the constraints in both places? Could we change how default
interpreter_constraints
get calculated so it uses
[python.resolves_to_interpreter_constraints].<resolve>
first before falling back to
[python].interpreter_constraints
? Is there a feature here that I'm misunderstanding?
h

hundreds-father-404

12/22/2022, 3:46 PM
cc @happy-kitchen-89482, relates to our convo yesterday about interpreter constraints
The main idea you're hitting is that currently targets can have ICs that are a subset of their resolve. Thing about a resolve w/ ICs 3.7-3.9. You may want to run your tests three times, with ==3.7, ==3.8, and ==3.9
p

proud-dentist-22844

12/22/2022, 4:04 PM
That's true. I had my default constraints set up like that (
==3.6.*
and
==3.8.*
) but that messed up building one wheel for all of those. So I switched to
>=3.6,<3.9
. the pants-plugins resolve is the exception - I don't care much about the constraints for it, I just want it compatible with pants. (Aside: a shortcut constraints keyword similar to the
pants_requirements
target that looks up its value based on the version of pants in pants.toml would be really nice. That would simplify the pants-plugins resolve config so I don't have to manually track what pants requires) But, back to your point about testing multiple versions: we run tests (for code in our default resolve) on 3.6 and 3.8. It's not clear how I'll accomplish that (once pants can run our tests). For CI, I'm going to only install the target version of python and maybe manipulate the search paths to ensure the version of python I want is selected. On my local machine, that will be more awkward as I have installs of lots of different versions of python including every minor (not patch) version from 3.4 to 3.11 plus 2.7. So, a more standardized way to specify which interpreter(s) to use for tests would be excellent.
c

curved-television-6568

12/22/2022, 4:34 PM
isn’t the interpreter_constraints a list value? and multiple values `OR`’d together? if so, we could use that list as your “test matrix”. If you just want a single run, use a single value:
[">=3.6,<3.9"]
, but if you want to test with multiple versions go with multiple ICs instead:
["==3.6.*", "==3.7.*", ">=3.8,<3.10", …]
e

enough-analyst-54434

12/22/2022, 4:39 PM
Oh my ... consider the OR broken, even though it is supported. That is a legacy holdover that needs to go away.
🤪 1
Addressing
tox -p -epy3{6,8,9}
behavior is probably best addressed 1st class comprehensively with fresh eyes.
p

proud-dentist-22844

12/22/2022, 5:03 PM
If you just want a single run, use a single value: [">=3.6,<3.9"], but if you want to test with multiple versions go with multiple ICs instead: ["==3.6.*", "==3.7.*", ">=3.8,<3.10", …]
Yes. That doesn't work. I need my wheel to have one IC, but I want to test with multiple. So, would I have to parametrize all of my python_tests to use the list instead of the single constraint?
👍 1
The key thing in this thread:
Could we change how default
interpreter_constraints
for targets get calculated so it uses
[python.resolves_to_interpreter_constraints].<resolve>
first before falling back to
[python].interpreter_constraints
?
h

hundreds-father-404

12/22/2022, 5:08 PM
Oh that's a good idea. I think it's feasible, although requires some refactoring because IC resolution now needs access to another field on the target. Feasible, but boilerplatey
h

happy-kitchen-89482

12/22/2022, 10:16 PM
As Eric alluded, I am rethinking the complexity around ICs (and, to a lesser extent, resolves), so this is great feedback.
1
p

proud-dentist-22844

12/28/2022, 2:40 PM
@happy-kitchen-89482 here's someone else who was confused by the separate IC config in
pants.toml
vs
BUILD
. https://github.com/pantsbuild/pants/issues/16566#issuecomment-1366593885
My request---pull the default IC for a resolve from
[python.resolves_to_interpreter_constraints]
---helps minimize config duplication, allowing me to not add ICs in
BUILD
files.
1
h

happy-kitchen-89482

12/28/2022, 5:04 PM
Thanks, yes, ICs are a hot mess right now. I think I will start by documenting that they must be specified explicitly, then enforcing that they must be.
Default constraints make no sense
2
p

proud-dentist-22844

12/28/2022, 5:48 PM
@happy-kitchen-89482 Are you thinking of removing
[python].interpreter_constraints
and
[python.resolves_to_interpreter_constraints]
from
pants.toml
? And just collecting the relevant info from all the BUILD files? That could work too.
If someone wants to apply a default, they can easily do
__defaults__(all={"interpreter_constraings": ...})
instead of adding it to
pants.toml
e

enough-analyst-54434

12/28/2022, 5:58 PM
But, IIRC not all targets currently support ICs. Just
pex_binary
?
So what's the IC for a test if that's right, or a library
p

proud-dentist-22844

12/28/2022, 5:58 PM
python_test*
and
python_source*
and
python_distribution
and friends support it.
e

enough-analyst-54434

12/28/2022, 5:58 PM
Ok then, could work.
Really, imo, the fact that pants.toml and BUILD are two different things is weird. We should have multiple toml and 0 BUILD or multiple BUILD and 0 toml.
👆 1
h

hundreds-father-404

12/28/2022, 6:01 PM
I think Benjy's (very tentative!) proposal is instead to get rid of the field and have ICs come from the target's resolve. If you're not using resolves, ICs come from an option and you can only have one set of ICs in the repo This aligns with a proposal both Stu and I advocate for, named interpreter constraints: https://github.com/pantsbuild/pants/issues/12652. One of my main motivations is it is currently too easy for a dev to add new ICs to their repo without an admin knowing. That typically results in lots of new complexity, so it should be less easy to define new unique ICs
p

proud-dentist-22844

12/28/2022, 6:03 PM
@enough-analyst-54434 I think pushing more into BUILD files is the way to go, though I don't think everything will end up there. Some things are really settings for pants itself, and have nothing to do with BUILD metadata. Things like anonymous telemetry, or cli colors. So, I think there will continue to be a separation between the two types of config files, but the dividing line could be brighter/clearer on what to find in each.
e

enough-analyst-54434

12/28/2022, 6:03 PM
Why can't build metadata be in toml. Why two formats?
p

proud-dentist-22844

12/28/2022, 6:04 PM
I can see tying ICs to the resolve. That makes sense the way I was initially trying to use it.
👍 1
e

enough-analyst-54434

12/28/2022, 6:04 PM
Or vice versa
It's just apparently a hazing ritual for a new user afaict
p

proud-dentist-22844

12/28/2022, 6:08 PM
Why can't build metadata be in toml. Why two formats?
Ah. true. The main pants config file could also be python-esque. TOML is a lot more restrictive, so I wouldn't want it in BUILD files where the expressive syntax of python is very useful. And having something with a more limited syntax, like toml, to define the version of pants to use is helpful because you can more easily scrape settings like that with bash+grep+etc.
h

happy-kitchen-89482

12/28/2022, 6:09 PM
Yeah, pants.toml existing at all, rather than as a top-level BUILD file, is a continuation of historical accident
I'm not 100% sure what shape the simplifying of ICs will take, but I do know that it will be A) explicit and B) make it easier to support common cases without confusion, even at the expense of making it hard to support uncommon ones (as long as harder != impossible)
👍 1
p

proud-dentist-22844

12/28/2022, 6:11 PM
Very nice. I look forward to developments here 🙂
h

happy-kitchen-89482

12/28/2022, 6:14 PM
Will be airing ideas (and grievances...) in public 🙂
😄 1
7 Views