https://pantsbuild.org/ logo
#development
Title
# development
b

bitter-ability-32190

04/11/2022, 1:45 PM
I want to get people's šŸ§  on what I'm working on which relates to (this post in general and this post above) I have a C extension module written in C++ which is being used heavily by other Python code. The file itself uses
pybind11
. Ideally, at any point where something depends on this module, it would get inferred. I think that's possible if I code it so the target is a code generator (because it kind of is). This kind of ignores a lot of hermeticity, but oh well. The two problems I face are
pybind11
. (I think I can wire up the codegenerator to invoke
setup.py
much like the
package
rule for
python_distribution
.) It's actually very trivial to add a
HTTPFileTarget(Target)
plus plumbing to Pants (and I want to upstream that). But the problem is that the existing machinery we'd want to re-use for building the sandbox filters the transitive dependencies to be "sources that likely make sense". Since
pybind11
has no
source
, I'll silently get nothing showing up in the sandbox. This smells of a larger problem though. Pants has tons of machinery revolving things with a "source". Codegen, generator targets, etc... But unless your chain of targets/rules/fields starts with a sources field, you don't get to participate (and in this case, my target gets silently filtered). One issue is that a dependency is just a target address, and so you need to know how to get it onto disk. If there was a way to take any target and ask make a request which gets it onto disk then you can have anything depend on anything.
Funny enough, this is hackable šŸ˜‚ ā€¢ Make
source
field
OptionalSingleSourceField
and omit ā€¢ Declare a generator which generates a
FileSourceField
ā€¢ Provide a rule which downloads the file and produces a
GeneratedSources
This is however, an abuse of the system IMO.
šŸ‘ 1
w

witty-crayon-22786

04/11/2022, 6:30 PM
(saw this, but am catching up on a few things before reading)
b

bitter-ability-32190

04/11/2022, 6:47 PM
I'll probably document my hacks, and we can work on un-hacking and then upstreaming them.
One interesting thing here too, I guess there's a difference between "build time dependencies" (e.g.
pybind11
) and runtime dependencies. But I suppose for
python_distribution
everything is a build-time dep. And the runtime deps is all calculated install-side by installing the wheel.
Poking at source is fun!
SpecialCasedDependencies
šŸ¤”