https://pantsbuild.org/ logo
Join Slack
Powered by
Howdy everyone. Is there an obvious way I’m missin...
# plugins
p
Howdy everyone. Is there an obvious way I’m missing to “hook into” pants command execution to do some wrapper logic any time some does 
pants run
?
The use case is: observability into end user interactions with pants.
We’re looking to answer Qs like: How are users using our monorepo and what kind of issues are they experiencing?
With such a hook I’ll be able to log all commands to a centralized location — like a metric somewhere into New Relic or an RDS server I’ll stick somewhere, etc. The details (where to log, how to obtain token to the logging destination, etc.) can be part of a plugin
I just don’t see if there’s a way to trigger a plugin on a generic 
pants run
command
f
Pants has a monitoring subsystem. Search "streaming workunit handler" in the Pants source.
Plugins can hook into that to get essentially a trace graph.
It is not well-documented,
It will also give your plugin access to Pants' counters and histogram metrics. (The same ones output by the 
--stats-log
option.)
For example, there are counters covering cache hits & misses (among other observed topics).
So for example, your plugin could what for the parent workunit generated when the 
run
goal runs for a target and extract metadata from it.
If you need additional metadata in the work units, please feel free to submit a PR to Pants to add that data. It should be straightforward to do so; any Pants rule type which implements either 
EngineAwareParameter
or 
EngineAwareReturnType
has a 
metadata
method which can return a dict with arbitrary data to add to the work units generated.
👀 1
g
You could probably resurrect the anonymous telemetry plugin as an in-repo plugin and modify for your needs? Removed here https://github.com/pantsbuild/pants/pull/18952. Some parts still live, so you probably only need what was removed there as a single rule.
👀 1
p
Thank you both! Sorry for the reply — I’m still looking at your suggestions. I’ll sure to be back with updates
@fast-nail-55400 a more targeted Q for you based on what you described about workunit logger — do you see it being possible to grab the output of a python process inside the plugin? Asking because I see that 
pants.log
and 
run-tracker
logs don’t have any output from python interpreter. All python-script logs only go to stdout+stderr to the screen, nothing to file. I’m getting the feeling I’ll hit a dead end if my goal is to have a plugin that will “intercept” output from python interpreter
I have a thread about this in #C046T6T9U (since I wanted to find out about pants logging as a whole) but here are the screenshot from that conversation — they show that python exceptions are not logged to file
f
@fast-nail-55400 a more targeted Q for you based on what you described about workunit logger — do you see it being possible to grab the output of a python process inside the plugin? Asking because I see that 
pants.log
and 
run-tracker
logs don’t have any output from python interpreter. All python-script logs only go to stdout+stderr to the screen, nothing to file. I’m getting the feeling I’ll hit a dead end if my goal is to have a plugin that will “intercept” output from python interpreter
Do you mean the output from the execution of a 
Process
? (whether or not it was Python)
I just want to clarify between output from Pants rule code and output from executing what happens to be a Python script (for example, 
pex
executions)
p
I mean grabbing execution of a process, yes.
I just want to clarify between output from Pants rule code and output from executing what happens to be a Python script (for example, 
pex
executions)
I see pants outputs “output from Pants” both to screen and pants.log+run-tracker but “output from executing what happens to be a Python script (for example, pex executions)” only goes to to screen, there are no log files for it
2 Views
Open in Slack
PreviousNext
Powered by