https://pantsbuild.org/ logo
Join Slack
Powered by
Next question up about parametrize (and maybe this...
# general
h
Next question up about parametrize (and maybe this is a try-and-see-what-happens), what gets picked up as the default for dependency inference?
Seems like neither gets picked as the default. Is there a way to set one as the default?
The parametrize option is great, but I don't want to have to go around to every dependent and specify the build file just to accomodate a singular unique case in one place.
c
so when you parametrize a field value, that target you added the parametrization to cease to exist, and the parametrized versions takes it place. So any references you may have to the original target needs to be adjusted to one of the parametrized versions. Does that make sense?
h
Yup, it totally does.
c
when you parametrize resolves (which is the most common case, afaik) for instance, the dep inference does this for you behind the scenes, but also requires all code you interact with to be in the same resolve(s)
h
I was just hoping I can do something quick like "the first becomes the default for inference"
c
I’m not sure I follow.. when/how does this matter?
oh, you mean you have a regular file a, that imports b which has been parametrized over dependencies..
so question is which target to pick that owns b, as there’s now more than one.
h
Yeah, exactly that
Most times, we're going to want the case that includes all the dependencies and only a few we want the limited set
c
that’s interesting, as I think there’s errors built into pants about multiple file owners that covers exactly this (disallows it)
h
But this doesn't get used in a lot of places so there's not too much issue if there's a little verbosity here
c
that predates parametrizations
hmm… I’ve no experience with this, would take some 🤔
h
Yeah, what happens currently is dependency inference just says "hey both of these parametrizations exist, we don't know which one you want"
c
guess there’s currently no way to guide the dep inference here, so an explicit dep is the only way I see right now
h
Which is certainly the safer behavior
Yeah, agreed. Just wanted to make sure I wasn't missing some hook
c
I guess you could do a little BUILD file target trickery to hide the parametrization one level to deflect it impacting all dependents too much
h
I'll defer the magic for now since this isn't used in a lot of places for now.
oh, another dev pain point: you also have to go turn off dependency inference for that import even if you've populated the build file correctly
✅ 1
l
Ah, jumping in late here but I am a bit confused, and would like to learn. You have two files 
a.py
and 
b.py
and a has an 
import b
in it. In your BUILD file you have a 
python_source
for 
b
that has 
resolve=parametrize(...)
and a python_source for 
a
that has no 
resolve
specified explicitly, and you are getting an error that it can't figure out which target 
a
depends on?
oh ... it is 
dependencies=parametrize(...)
...
h
Not quite. Here's what a minimal example of what I have at the directory level
Copy code
src/
  - a.py
  - b.py
  - file1.txt
  - file2.txt
I want 
a.py
to be parametrized as having either 1) just 
file1.txt
or 2) 
file1.txt
and 
file2.txt
, so I make a build file like this
Copy code
resource(name="file1", source="file1.txt")
resource(name="file2", source="file2.txt")
python_sources(overrides={"a.py": {"dependencies": parametrize(one=[":file1"], both=[":file1", ":file2"])}})
Then, in 
b.py
, I have an 
import a
statement.
So, now I have two forms of 
a.py
successfully; one that has one file and one that has both.
Dependency inference rightfully complains when it sees 
import a
because there are two potential targets to pick and a resultant ambiguity.
So, my first question was is there a way to say "always pick this parametrization"
Then the second question was, can I avoid adding the 
no infer dep
comments since I've updated my build files to manually call out the dependency anyway.
l
right, and dep inference would not struggle if it was the 
resolve
that is parametrized because there is special machinery to keep those worlds separate.
h
Yup. So, I'm fine being verbose in the BUILD files and adding the comment directive for now because this import happens in two places right now. So it's not a burden for the benefit added of having the clear parametrized targets.
l
isn't the issue is that the 
dependencies
here usually means dependencies in addition to those that were inferred? so just supplying them in the BUILD file does not imply that inference should not be done.
h
Yeah, that's probably what's happening. I think what I typically expect to happen is "ah I can't infer this. Is there anything manually declared that covers it?"
And if not, then error out
But it doesn't seem like there's a way for me to make inference happy once I've parametrized my original source file.
l
Stepping slightly back, kind of wondering if there is a prettier way for you to do it. You have basically an optional dependency. Ultimately you are trying to make two different packages distributions - like a lite one and a fully-featured one, right?
h
Exactly that
The lite actually gets packaged, stored, and shipped around so having minimal dependencies ensures it doesn't change unnecessarily.
Whereas the full version is more for dev where that implication doesn't really matter
parametrize felt like the right tool to have multiple forms of my dependency
l
Then can you exclude the dependency in the lite one at packaging time?
h
Hmm, true I didn't think about that
l
Like when making your 
python_distribution
or lambda or pex_binary
You can add a 
!!big_optional_file
h
Although, it's a little more challenging because there's a huge number of files rather than just the two I gave the example of.
l
If they are accessed through a single gateway you could exclude that. Like if they were referenced from a python file. 
big_files_for_local_dev.py
h
They unfortunately are not
l
oh I see, so scattered throughout the codebase there is the possibility of files depending on minimal and full resources.
Interesting, food for thought. Thanks for sharing
h
Not so much scattered. We're just picking off pieces of a larger directory tree
Open in Slack
PreviousNext
Powered by