https://pantsbuild.org/ logo
Join Slack
Powered by
I've finally gotten my log-streaming up to date wi...
# development
g
I've finally gotten my log-streaming up to date with latest Rust and fixed a bunch of bugs with it. However; I'm finding a literal scale issue. The way this is implemented simply adds a few lines of log output to each swim lane, pulled directly from the process output. That's slightly problematic on larger machines, the swimlanes easily overflow the available vertical space. For example; my new laptop has 20 cores. When all are working and showing f.ex. 6 lines of logs we get 140 (!!) lines of output - twice the number of lines I have in my default terminal. This makes the output completely illegible. I'm planning to hide this behind a flag (
--dynamic-ui-log-streaming=false
) and make the number of lines configurable (
--dynamic-ui-log-streaming-lines=1
). However, I'm wondering if we can find some better "default" heuristics that give good results OOTB while avoiding walls of text. It works OK with 1-3 lines, but multi-line logs are quite terrible still. Any thoughts? cc @dry-orange-3870 who asked about this a while back, and @witty-crayon-22786 + @curved-television-6568 who have expressed interest in this feature elsewhere.
I'm wary of anything that leads to opt-in/opt-out at the "shipped" level of code, as it's hard to predict where this'd be useful to an end-user. Some users might want it for Pex while others only want it for Docker. I was thinking of tying it to the "level" used by the Workunits, but it seems quite coarse as well -- and also makes it hard for users to opt-in. One maybe-not-stupid option would be a 
--dynamic-ui-log-streaming-filter=[<rex>]
which would filter based on... command line? label? The latter is much easier to implement, but might be quite complex to configure.
c
how about having a cap on how many swimlanes to provide these extra lines for?
for me, I think having it on just one would be good enough for most cases… as it’s the straggler long running task that’s alone in the end that’s most interesting
g
Hmm, that's not a bad option. Definitely something that's easy to add. The indicatif output already goes heavy-hitters at the top iirc so if one starts from those it'd show for "long-running" tasks by default
🙌 1
Maybe a balancing flag that's nice for users either way -- one line for every task, 10 for the top tasks, whatever you want. Some goals go much wider too...
c
hmm.. interesting, yea 👀
g
So one could do 
pants  --dynamic-ui-log-streaming-lines=20 --dynamic-ui-log-streaming-topn=2 package ::
Could even do 
topn=auto
and compute the cap from terminal height, now that I think about it.
Definitely going to incorporate that idea anyways, it seems like a good and sensible flag to have in general.
👍 1
d
I don't have anything helpful to add to this discussion, but I just wanted to say thank you for your work on this!
w
you would probably also reduce the amount of output considerably if you waited for some configurable threshold of runtime to elapse before logging anything
4 Views
Open in Slack
PreviousNext
Powered by