I want to get people's 🧠 on what I'm working on which relates to (this post in general and this post above) I have a C extension module written in C++ which is being used heavily by other Python code. The file itself uses

pybind11

. Ideally, at any point where something depends on this module, it would get inferred. I think that's possible if I code it so the target is a code generator (because it kind of is). This kind of ignores a lot of hermeticity, but oh well. The two problems I face are

pybind11

. (I think I can wire up the codegenerator to invoke

setup.py

much like the

package

rule for

python_distribution

.) It's actually very trivial to add a

HTTPFileTarget(Target)

plus plumbing to Pants (and I want to upstream that). But the problem is that the existing machinery we'd want to re-use for building the sandbox filters the transitive dependencies to be "sources that likely make sense". Since

pybind11

has no

source

, I'll silently get nothing showing up in the sandbox. This smells of a larger problem though. Pants has tons of machinery revolving things with a "source". Codegen, generator targets, etc... But unless your chain of targets/rules/fields starts with a sources field, you don't get to participate (and in this case, my target gets silently filtered). One issue is that a dependency is just a target address, and so you need to know how to get it onto disk. If there was a way to take any target and ask make a request which gets it onto disk then you can have anything depend on anything.

(saw this, but am catching up on a few things before reading)

I'll probably document my hacks, and we can work on un-hacking and then upstreaming them.