https://pantsbuild.org/ logo
Join Slack
Powered by
This is kind of related to my <investigations> int...
# general
p
This is kind of related to my investigations into having a plugin for monitoring pants commands but also a generic logging Q: what is logdir for? From what I see all logs are controlled with pants_workdir (default is 
.pants.d
): • 
pants.log
as a kind of “top-level” log • 
run-tracker
as detailed log of each `pants`run eg if I set
Copy code
[GLOBAL]
pants_workdir = "workdir"
logdir = "logs"
logs
is always empty. Is it still true that logdir is not wired up to anything?
Related: I found workunit-logger but the JSON files it creates are empty
Bigger-picture: I’m trying to log everything pants outputs to screen to file so I can set up “log shipping” to a central place for analysis. Not sure if workunit-logger accomplishes that but trying it out
b
Searching the code does seem to imply the global 
logdir
option isn't used for anything
👌 1
😢 1
p
yeah and workunit-logger logs empty files (containing just 
[]
):
here’s 
pants version -ldebug
lol. Lots of screen output, empty json
b
Can you try a goal that actually runs a process (e.g. 
pants run ...
or 
pants test ...
, even if that process isn't successful)
👀 1
p
ok here’s one but maybe it’s a “premature” failure, let me get something else
h
Try also running with 
-ldebug
to generate more verbosity
👀 1
b
(filed https://github.com/pantsbuild/pants/issues/21612 about the 
logdir
option doing nothing, thanks for raising)
👍 1
p
oh hey yay
What use case do workunit logs solve? It looks like they have some kind of timing information. I’m looking to do “log shipping” for errors. Are run-tracker logs my best bet for tracking the kinds of errors our monorepo users are hitting?
The reason I started looking around even knowing about 
workdir/pants.log
and 
workdir/run-tracker
logs is because those logs don’t have everything. “everything” in this case is all stdout+stderr output to screen
b
yeah, workunit logs are focused on things like performance analysis. The "your configuration is wrong" errors indeed are mostly stdout/stderr, AIUI
p
Thanks for your help so far @broad-processor-92400 @happy-kitchen-89482 I just wanted to wrap up this thread with an opinion from you guys — Where do you envision the boundary being of where pants’ responsibility for logging errors stop and python interpreter’s begin? So far I see I can use run-tracker logs to log (to file) pants-specific errors (ie “your config is wrong” type errors) but not errors from python scripts themselves. Here’s a clear example: when a script is ran via 
pants run src/package_a/failing_script.py
and `raise`s an error output to stdout looks great but the log files are silent
Which way to do envision this behavior leaning long-term? 1. letting python scripts themselves specify the location / “sink” where to log? 2. have pants intercept (for lack of a better) python interpreter and add a stdout+stderr redirect to pants.log? Just trying to get “the feels” of which way I should go in this “monorepo observability” project I am tasked with
I looked into (2) a bit and am convinced that short of a kernel there’s no way to add another stdout redirect to “mirror” exactly what being output to terminal on Macs. Short of a clean redirect, I am faced with the only solution I see in pants: monitor both 
pants.log
(logs the command that was ran) and 
run-tracker
(logs the output) dirs and “stick” the 2 logs together (then ship those logs off the Macs to somewhere I can store and comb thru them later). Doing this is something I’m trying to avoid at all costs due to the huge PITA it’s going to be to write and maintain
b
Hm, I think there's two things to distinguish: 1. pants's own internal logging/printing (e.g. the various 
[INFO]
/ 
[ERROR]
etc. lines that pants was printing, and errors about configuration) 2. your user code arbitrary logging/printing (e.g. that exception caused by 
failing_script.py
itself directly raising it) Are you wanting to syphon off both of these into an external system for analysis? You could play games like a wrapper script around pants that runs something like 
pants "$@" |& tee logs.txt
or whatever... but I think this'll start interfering with, for instance, control codes that interact with the terminal (sounds like you're aware of this, though)
p
Are you wanting to syphon off both of these into an external system for analysis?
yes exactly this Reading the issue 9988 you linked to, I still think it only concerns (1) pants output only. I’m looking to grab the response from python itself (2)
And yeah I think getting into the wrapper scripts territory will be unwieldy, disruptive to our monorepo users, and high-maintenance
Open in Slack
PreviousNext
Powered by