Is there a way to make a "shadow target" which wou...
# pluginsb
bitter-ability-32190
06/13/2022, 7:56 PMIs there a way to make a "shadow target" which would allow me to re-use existing rule logic?
Specifically I want to be able to infer Python import dependencies from a file I'm generating using another target. Currently the Python Infer rule expects a
PythonSourceFieldPythonSourceFieldf
fast-nail-55400
06/13/2022, 7:58 PMYou can subclass
PythonSourceFieldPythonSourceField👍 1
b
bitter-ability-32190
06/13/2022, 7:59 PMRight, but my thing isn't a Python source, so the parser will choke and die.
f
fast-nail-55400
06/13/2022, 7:59 PMYou could also try writing a codegen rule that targets
PythonSourceFieldb
bitter-ability-32190
06/13/2022, 8:00 PMYeah I was kinda thinking that too, but does inference work for generated targets?!
bitter-ability-32190
06/13/2022, 8:01 PMAnd even still, the full inference rule for Python looks for other fields like
interpreter_constraintsbitter-ability-32190
06/13/2022, 8:04 PMIt seems I want a full synthetic
python_sourcebitter-ability-32190
06/13/2022, 8:41 PMFull disclosure, I'm looking at adding a full Jupyter Notebook plugin (first internally, then upstream) and need to parse my imports. The notebook file is just JSON, but concatenating the code blocks gives me Python.
There'd need to be discussions, since technically notebooks are multi-language, but for the PoC it's all Python baby
bitter-ability-32190
06/13/2022, 8:42 PMIt'd be easy to do in-Pants because I can modify the inference rules to take the contents instead of the source. But I can't do that in my plugin outside-of-Pants
h
hundreds-father-404
06/13/2022, 11:31 PMWould it make sense to modify Pants's code itself to be more plugin-friendly? I think we're open to changes like that 🙂
b
bitter-ability-32190
06/14/2022, 12:00 AMYeah. That might be a separate discussion though 🤔
bitter-ability-32190
06/14/2022, 2:51 PMSo for now, I know it's only Python notebooks, so I'm going to imitate the built-in Python dep inference code.
When I upstream, we can bend the dep inference rules to operate on content and then we can invoke the dep inference on the generated content
bitter-ability-32190
06/14/2022, 2:52 PM@witty-crayon-22786 for visibility. I think this maybe smells of a problem we might face in other places
f
fast-nail-55400
06/14/2022, 2:53 PMit might be tangential, but https://github.com/pantsbuild/pants-jupyter-plugin may be of interest to you
b
bitter-ability-32190
06/14/2022, 2:54 PMYeah, I think that is somewhat of a "poor man's" solution to the problem. I think it can definitely live harmoniously for "out-of-repo" notebooks, but nothing beats first-class in-repo notebook support
w
witty-crayon-22786
06/14/2022, 5:27 PMAnd even still, the full inference rule for Python looks for other fields likeit needs to here as well… your notebook target probably still needs that field🤔interpreter_constraints
b
bitter-ability-32190
06/14/2022, 5:28 PMYeah I added that and the resolve field
w
witty-crayon-22786
06/14/2022, 5:28 PMit seems like multiple levels of codegen would get one step closer. but it’s true that dependency inference won’t run on generated code currently, which means we require plugins to pre-specify the deps of the code that they will generate. mostly intentional, because we don’t want e.g. protobuf to have to run to calculate dependencies.
6 Views