I've been looking again into using lockfiles, and ...
# development
a
I've been looking again into using lockfiles, and well, it'd be quite a painful process to convert all our
BUILD
files to a loop rather than the
parametrize
for interpreter constraints we use now, since that's in a macro that sets
__defaults__
. There's this ticket, but no activity on it, is this still a 2023 priority? I've been looking at the suggestions there, not sure I like any of them, tbh. The context manager one is a bit weird, it'd require just indenting all the file if you want to use
__defaults__
(or some deeper magic in how that's handled). The
**parametrize_group
is also not great, since you can't use defaults with it. The
parameterize_group
kwarg to
parametrize
might break people's stuff, so it can't go in immediately...
c
Thanks for the feedback, I don’t think there’s been any more work on that ticket since those comments unfortunately.
Do you have any other concerns about the
**parametrize_group
approach? I can make it work in a backwards compatible way. How would you want to use
__defaults__
with it..?
If you want one of the parametrizations to use a value from
__defaults__
, I guess you just omit that field from the group.. 😉
looking at this again, I don’t think any of the suggestions are really 100% yet.. but points in a general direction that could be desirable to go for.
would be helpful if you’d want to comment on the ticket with your use case and concerns 🙂
posted yet another idea onto that ticket just now… 😜
a
Sorry, I got distracted by work, then... life. Read your suggestion now, I guess
parametrize
doesn't accept a mix of kwargs and args? I guess with some magic, this can be made to be backwards compatible. As for
**parametrize_group
, if that's just syntactic sugar, and not some magic to get it to work, I have no objection. What we do is we have a function call at the top of our
BUILD
files that looks like this:
Copy code
set_interpreter_constraints(supported_versions=["3_7", "3_10"], build_version="3_10")
That will set the
__defaults__
for
interpreter_constraints
for tests and sources to 3.7 and 3.10 (tests parametrized, so it runs for both) and the pex_binary to 3.10. If we couldn't use
__defaults__
in a sensible and predictable manner, we couldn't use this, and literally every
BUILD
file we have uses it, and we only override in probably 10 (out of 300+)
c
Thanks. So the feedback here is to make the new parametrize grouping be supported also from
__defaults__
, which makes sense now. So you could do:
Copy code
__defaults__({python_sources: [parametrize("3_7", interpreter_constraints="3.7", ...), parametrize("3_10", ...)]})
The implication being, that you'd need two calls to
__defaults__
if you want to provide both grouped parametrizations and regular field values to a particular target. In general terms:
Copy code
__defaults__({TARGET_TYPE(s): KWARGS_DICT | ARGS_LIST, ...})
a
To be honest, if the
parametrized_group
was the syntactic sugar that... I don't remember who, mentioned, I wouldn't have any problem. I just think it should be obvious what that generates and be just an alternative to a well documented way of doing it 🙂
c
There's no issue with backwards compatibility here,
parametrize
does support mixing args and kwargs (I'm pretty sure). But, targets do currently not accept args, only kwargs, so opening up the args slot for targets to treat parametrize values as args differently is a new possibility. I'm no longer sure the
**parametrize_group
is a great idea (in light of the new args based one), which I'm currently in favor of..
I think it was me.. inspired of what Stu wrote early on..
either way, whatever option we go with, will have to be backwards compatible.
a
Yeah, so a keyword arg to
parametrize
won't work.
c
a parametrize as a field value would work as before, only when passed as a target positional arg would it be treated differently
I don't see an issue with it, do you have an example showing what doesn't work?
ah, sorry I may have misread your comment... yea, merely a keyword arg to parametrize while using it as a field value won't work, correct.
that's why really like the parametrize as a positional arg, as that's new and doesn't interfere with current use
a
Yeah, that's what I was gonna say, that should work if you cannot combine kwargs and args now
👍 1
Btw, not related to this, but the docs about parametrize are very misleading
There's literally an example about doing _this_: https://www.pantsbuild.org/docs/targets#parametrizing-targets
The implication being that it will actually work, via some other mechanism
👀 1
c
😅 I've not looked at them too much.. you're encouraged to improve them to make them more approachable/correct.
a
I mean, it doesn't say it will work, but if it's an example in the docs, you sort of expect it to do something useful 🙂
c
I'm looking at it, but the issue is not immediately apparent to me...
it does say it will create a cartesian product if you have multiple parametrized fields...
a
Maybe that's just me, but I assume most people will have a
resolves_to_interpreter_constraints
, and that's why they want different resolves.
c
so, this example:
Copy code
# Creates four targets:
#
#    example:tests@interpreter_constraints=py2,resolve=lock-a
#    example:tests@interpreter_constraints=py2,resolve=lock-b
#    example:tests@interpreter_constraints=py3,resolve=lock-a
#    example:tests@interpreter_constraints=py3,resolve=lock-b

python_test(
    name="tests",
    source="tests.py",
    interpreter_constraints=parametrize(py2=["==2.7.*"], py3=[">=3.6"]),
    resolve=parametrize("lock-a", "lock-b"),
)
shows you'll have both resolves for both ICs in the commented part.. so it's not an example of what we're trying to solve here.. (which is a missing feature..)
is that the misleading part, perhaps?
a
Yeah, but I think the most common usecase for this is what I mentioned. When I read that, after knowing resolves_to_interpreter_constraints exists, I thought "oh, they must be filtering by that"
c
that's good feedback, I don't make the same conclusion so it didn't occur to me.. maybe having this hole filled and documented will help avoid that pitfall for future readers
a
I'll probably make that my first PR for pants, trying to find something that's obvious that you intend for it to be the full cartesian product.
❤️ 1
Definitely not tonight, but I'll try to get it done next week. And, well, I will probably implement some version of this as a plugin in our repo, but that'll be two weeks from now, I get a day to work on whatever I want (basically) at work every other week.
c
yea, I wouldn’t expect anything tonight. Not expecting anything, you do as much as you feel compelled to at your own time. We’re happy for it whenever that is. Thanks for the feedback. I’m actually inclined to take a closer look at implementing this right now.
this is the kind of problems I enjoy tackling… 😉
a
I think @flat-zoo-31952 might also have struggled with the current way you have to tackle this.
👀 1
And, we're very heavily invested in pants, we're trying to work on sponsoring pants, since... that's easier, heh, but we're definitely not opposed to contributing to the code. I think we just managed to squeeze by without any contributions so far, heh.
c
heh.. yea that sounds great.
a
(we're working on sponsoring pants, not trying to, didn't want to edit it as you were replying, heh)
👍 1
c
that was how I read it 😅