https://pantsbuild.org/ logo
Join Slack
Powered by
I'm trying to think of a way to support "system-pr...
# development
f
I'm trying to think of a way to support "system-provided" dependencies in Python inference. As it stands I don't think I could write a plugin that supports this because module mapping only supports first and (pip-based) third party modules. I think to support this I'd just need to add a 
@union
called 
PythonModuleMapping
and add first party and third party mapping types to that union, so I can write a plugin that adds to that union membership. Does that sound right?
👀 1
inb4 "this will violate hermeticism." I know it will, and i'm working on that. In the future I hope this could open up other ways python deps could be provided like via conda or docker or something
Actually there's already a union around first party sources so maybe I could hack that and hope that it being semantically wrong doesn't cause any problems 😅
f
having a union is similar to how Go dep inference works
f
Yeah the Python one too, but it's defined for first party only (to handle generated code mostly, from what I can tell) https://github.com/pantsbuild/pants/blob/989c5d0fcac1af2aa28d2847eb3767c044c19dc0/[…]thon/pants/backend/python/dependency_inference/module_mapper.py
I hacked it and it "basically" works, but I'd need to dig deeper to see what trying to run things like 
pants test
might do
h
Where would these system-provided deps be configured?
f
Probably as targets somewhere... I haven't fully thought out how that would work, but I think eventually this could be wired in with environments
In my case (RPM) I was thinking of naming the packages and then setting up a process to call 
rpm -q
to get info about the package to do some rudimentary attempt at getting caching right on those, even if we can't guarantee hermeticism
but it's something I'll need to experiment with to figure out. Conda is definitely not on my personal radar, but it was just a use case I could see for having another union there
h
It seems like this is a property of an interpreter?
Like, this is a generalization of the concept of system provided - today we determine those per interpreter version, but it should be per actual interpreter instance?
f
I think it’s technically the site configuration of the interpreter, which I’m not sure is 1:1 with the interpreter, but it may be close enough, especially given that you’re really giving up hermeticity by consuming these deps.
Created an issue to think about and work on this a bit more https://github.com/pantsbuild/pants/issues/19552
Open in Slack
PreviousNext
Powered by