https://pantsbuild.org/ logo
#development
Title
# development
p

proud-dentist-22844

11/18/2022, 5:32 AM
@curved-television-6568 and @careful-address-89803 : Here is a first pass at allowing the python dep inference to use the new dependency rules feature to disambiguate possible deps.
<https://github.com/pantsbuild/pants/compare/main...cognifloyd:pants:py-infer-dep-rules|https://github.com/pantsbuild/pants/compare/main...cognifloyd:pants:py-infer-dep-rules>
I'm not sure why slack is messing up that link
@curved-television-6568 what do you think of working up the dep rules like this @careful-address-89803 you refactored python dep inference recently. Can you look at this? Is there a better place to disambiguate inferred deps?
Note: I haven't tested this. I was sick today, and all I had with me was my phone. So I did this on my phone.
😮 2
c

careful-address-89803

11/18/2022, 6:28 AM
Will have a look this weekend. My refactor was mostly to surface the inference decision-making to the debug goal. I remember most of the ambiguous detection happening inside of maybe_warn_of_ambiguous_dependency_inference. I'll have to read up on the dependency rules feature. One thought of further direction that occurs to me is that we may also want to hook into the code that looks for possible sources of a dependency (I think that's only for 3rd party at the moment, looking in other resolves)
p

proud-dentist-22844

11/18/2022, 6:59 AM
Yeah. I wonder about surfacing info in the debug goal about targets that got filtered out based on dependency rules. My commit does not do that.
c

curved-television-6568

11/18/2022, 12:33 PM
interesting (and 🤯 you typed this up on your phone!) I’ll take a closer look tonight. 🙂
c

careful-address-89803

11/20/2022, 7:17 PM
Coming at this from a debug point of view, I think there are 2 distinct situations: 1. dep would be ambiguous, but visibility rules disambiguate 2. dep is findable, but visibility rules discard, resulting in Unowned I'm using those situations, and the ability to get the debug info, to guide my thinking on this. Depending on the ethos of Dependency Rules, there are a few places I see we could inject the filtering. I think this MR views it as way of refining the resolved Owners of an import. This does limit the actions it can do, really only to modifying the PythonModuleOwners by removing ambiguous owners and trying to identify an Unambiguous owner. We can slide this into
_get_imports_info
instead, for a similar result (like you indicated). I think a different view would be to view the dependency rules as culling the targets which could become Owners. We would then move the logic into the
Get(..., PythonModuleOwners)
or somewhere deeper in that. There is already some filtering done in
FirstPartyPythonModuleMapping.providers_for_module
based on the resolve. It's a bit different in that it either searches in one resolve or all of them, rather than actually culling the available mappings. This view doesn't readily offer debug information, but could be provided through something similar to the way 3rd party deps have suggestions for other resolves that contain them. I haven't actually dug into this part of the codebase, tho. A couple edge cases I can see: • need to discard unambiguous but blocked owners • explicit dep on blocked owner
👍 1
p

proud-dentist-22844

11/24/2022, 4:44 AM
Draft PR with that branch.
Coming at this from a debug point of view, I think there are 2 distinct situations:
1. dep would be ambiguous, but visibility rules disambiguate
2. dep is findable, but visibility rules discard, resulting in Unowned
I landed on the first option. My thinking is documented in the PR description. I looked at modifying
Get(..., PythonModuleOwners)
, but that looks like a dangerous path to modify, because that seems like it would be a lot less performant as the request would be a lot more specific (and less reusable). The
PythonModuleOwnersRequest
would have to get the relevant details about both sides of the dependency (the dependent and the dependency) to be able to effectively filter the valid owners. I guess the way to approach implementing your second option, "dep is findable", we would need an additional
Get(..., FilteredPythonModuleOwners)
that reuses the existing
Get(..., PythonModuleOwners)
rule, but then filters the full set.
👍 1
c

curved-television-6568

11/24/2022, 4:58 AM
opt 1. makes sense to me 🙂
c

careful-address-89803

11/24/2022, 5:39 PM
I thought both of those situations would be results of dependency rules, I don't think we can have only one of them. Imagine the case where there are 2 deps, 1 allowed and 1 forbidden. To perform situation 1, we'd remove the forbidden dep and get the allowed dep. But if the allowed dep is deleted by the user, we should still disallow the forbidden dep and have no deps left. I'll have a look at the PR, probably over the weekend.