https://pantsbuild.org/ logo
b

bitter-ability-32190

06/01/2022, 7:45 PM
Anyone have some thoughts on how to make the test output blob more legible?
Copy code
✓ path/to/my/project/and/my/module/test_that_my_module_works.py:tests succeeded in 54.06s (cached locally).
At the very least I'm thinking we could apply colors to a few select atoms of data here (like the test name and the process metadata source) Maybe
succeeded
and
failed
could also be color at least to break the monocolor line. (The check or X on the waaaay left doesn't help split the line visually)
h

high-yak-85899

06/01/2022, 7:46 PM
I really like bazel's way of doing this
They cut down on the clutter by not reporting the individual status of all tests and only break it down for failed tests
Haven't had time to dig through the plugin to see how feasible it is for me to do a pull request, but I do find this especially noisy for our build systems that run everything.
b

bitter-ability-32190

06/01/2022, 7:48 PM
In this case I think the output is hardwired into Pants'
test
goal handling, so it'd be a change to Pants itself...... Although making this customizable outside of Pants (via a config or pluggable) is quite interesting
h

high-yak-85899

06/01/2022, 7:48 PM
The benefits I'm aware of for seeing success output is 1) verifying no false positive where your test wasn't actually collected/executed 2) seeing how long the test took to know if interfaces are being mocked properly
The idea of separating test execution/report generation and report rendering/visualization is something I've seen and worked with in other tools so there's some precedent there.
w

witty-crayon-22786

06/01/2022, 7:57 PM
formatting changes should be very straightforward: https://github.com/pantsbuild/pants/blob/c0ba9ffeeb98e9ec17ad4bf69f98470ef113a1d5/src/python/pants/core/goals/test.py#L523-L545 … most of the effort would probably be around finding good defaults, but more options (or a threshold on test count before summarizing, etc) could work too.
👍 2
b

bitter-ability-32190

06/01/2022, 9:12 PM
@witty-crayon-22786 what do you think about the feasibility and "does-it-make-sense"ability for making the output pluggable?
FWIW I originally started editing
_format_test_summary
before making this post because I thought it was worth debate from the community what the ouptut should look like
w

witty-crayon-22786

06/01/2022, 9:17 PM
not sure about pluggability… for a case like this, structured output feels like a more generally useful capability. e.g. json.
probably a question of how many folks who would write a plugin would end up writing json with their plugin =P
i generally think that plugins make sense when you cannot anticipate usecases. but this seems like a relatively bounded domain, where we could improve the output to catch 95% of cases without plugins (and leave the rest to structured output)
👍 1
b

bitter-ability-32190

06/01/2022, 9:21 PM
A third facet here is optionability. Like @high-yak-85899’s ability to "turn off" reporting the passed tests individually
w

witty-crayon-22786

06/01/2022, 9:23 PM
yea. i feel like that could be a threshold with a default (more than 50 tests defaults to a summary)
🤔 2
h

high-yak-85899

06/01/2022, 9:28 PM
Yeah something to capture the "if I have to scroll to see all the output, it's too much" sentiment. Totally on board with a setting like that.
👍 1
h

hundreds-father-404

06/01/2022, 10:25 PM
I like that idea. My fear the whole time has been "I'm trying to run a single test -- did it run??"
h

high-yak-85899

06/01/2022, 10:27 PM
Yeah definitely. I think Stu's idea captures that fine. If I'm developing locally, I'll be running just that test and feel confident Pants isn't skipping it because it will show the full output. In practice, are people regularly auditing this test output against what is expected?
w

witty-crayon-22786

06/01/2022, 10:50 PM
having said that, summarizing also moves the problem to a degree… need to decide what a good summary looks like. i can imagine summary plugins, although whether they needed to be thirdparty is another question
b

bitter-ability-32190

06/01/2022, 10:54 PM
FWIW the OP asked about splitting a single line, visually. So let's not forget that 😂
h

high-yak-85899

06/01/2022, 10:55 PM
I personally find it legible. Just overbearing sometimes.
w

witty-crayon-22786

06/01/2022, 10:57 PM
@bitter-ability-32190: yea, sorry. i suspect that that would be uncontroversial… both color and indentation would be great.
👍 1
see https://github.com/pantsbuild/pants/issues/13588#issuecomment-991863653 for the last discussion of that topic i think
2 Views