#17039 Python dependency inference: add/emit debugging information
Issue created by
lilatomic
Is your feature request related to a problem? Please describe.
There currently isn't any information emitted about what dependencies were inferred. If dependency inference fails, developers don't have much to pinpoint the error.
For concrete usecases:
• I ran into an issue where dependent files weren't pulled into a test PEX. This was due to some shenanigans I had with pants source roots, where the imports were resolved from the repository root, not the closest parent source root.
• I ran into an issue where guarded imports weren't pulled into a test PEX. I had the incorrect module path, but the PEX built fine because they were weak imports
Describe the solution you'd like
A good first pass would be having the ability to output,
• for every file, the imports found and whether they were resolved.
• the map of 3rd party dependencies to modules
• the map of 1st party targets to modules
This would help developers confirm that the imports were found and understand if the targets of the import were found or ignored. For the example, with the second usecase, the imports found I would expect something like the following to let me know that it was deliberate and acceptable that there wasn't an error not resolving the imports:
[
{
"path": "//folder/file.py",
"imports": [
{
"import": "a.b.c",
"weak": true,
"resolved": false
}
]
}
]
Describe alternatives you've considered
The current output is just checkpointing. With
-ltrace
, these are what is printed (many messages elided):
11:31:51.66 [DEBUG] Completed: Find all targets in the project
11:31:51.66 [DEBUG] Completed: Find all Python targets in project
11:31:51.66 [DEBUG] Completed: Creating map of third party targets to Python modules
11:31:51.67 [DEBUG] Completed: Creating map of first party Python targets to Python modules
11:31:51.67 [TRACE] Completed: Inferring Python dependencies by analyzing source
11:31:51.67 [TRACE] Completed: Inferring Python dependencies by analyzing source
Additional context
rel:
#13283 ; this requests information on dependency inference, that requests graphing the normal dependencies. Discussion in that mentions graphing at different scopes, this mentions a scope below what could ordinarily be graphed by that.
pantsbuild/pants