https://pantsbuild.org/ logo
Join Slack
Powered by
Does anyone here use `experimental_test_shell_comm...
# general
f
Does anyone here use 
experimental_test_shell_command
? Is there anything you would consider essential to implement or fix for stabilizing it as just 
test_shell_command
?
d
We're using it! I don't think there are any blockers. I have some suggestions for improvement, though. 1. The difference between the 
tools
attribute on 
experimental_test_shell_command
vs. 
runnable_dependencies
- why aren't they the same attribute? The former takes the name of binaries whereas the latter takes addresses to 
system_binaries
. (there are some other inconsistencies, like 
path_env_modify
that can be confusing). 2. It would be helpful to easily capture the output of tests (e.g. if your command outputs logs or coverage... or in our case screenshots from cypress tests).
f
> 1. The difference between the 
tools
attribute on 
experimental_test_shell_command
vs. 
runnable_dependencies
- why aren't they the same attribute? The former takes the name of binaries whereas the latter takes addresses to 
system_binaries
. (there are some other inconsistencies, like 
path_env_modify
that can be confusing). The shell backend treats 
tools
and 
runnable_dependencies
fields as distinct entities. I just never added 
runnable_dependencies
to the 
experimental_test_shell_command
target type. (And finding out points like that is part of why I'm asking the stabilization question.) As for why both exist, not sure, probably just written at different times. Side note: 
path_env_modify
exists mainly for the in-workspace execution support when downstream tools (like Bazel) incorporate 
PATH
into their own cache key. In those cases, an always-changing 
PATH
means the downstream tool may never cache results since the tool's cache key changes for every Pants invocation.
👍🏻 1
2. It would be helpful to easily capture the output of tests (e.g. if your command outputs logs or coverage... or in our case screenshots from cypress tests).
I agree. That wasn't an initial use case when I first wrote the proof of concept for 
experimental_test_shell_command
, but sounds immensely useful. (It should be easy enough to implement as well since the shell backend rules use the same helper to setup both 
shell_command
and 
experimental_test_shell_command
executions so I just need to add the fields. It may need a little work to make sure the results can be used with other targets.)
Should the 
experimental_test_shell_command
just be folded into 
shell_command
then? I.e.., make 
shell_command
targets runnable as tests via some sort of 
test_goal=True
field (with a better name of course)?
That said, 
experimental_test_shell_command
exists as a target type because it fits the pattern established by 
run_shell_command
being the way to integrate with the 
run
goal.
The main DX concern is the UX for how to extract the captured results from the 
experimental_test_shell_command
execution.
Opened https://github.com/pantsbuild/pants/issues/21981 for the test output capture feature.
Opened https://github.com/pantsbuild/pants/issues/21982 for stabilization discussion.
d
Thanks @fast-nail-55400!
Yeah, I'm a little confused if it's better to be based off of run_shell_command or shell_command. I always have to look up the difference between the two. But in our case, the goal would be to capture logs/etc and expose them within the workspace for a human to review them. I was actually just looking into how to do this via 
experimental_test_shell_command
. I was trying to do something like:
Copy code
cp -r {chroot}/path/to/cypress/screenshots cypress/screenshots
as part of the 
cmd
, but it doesn't look like 
{chroot}
or 
$CHROOT
are being populated. Is that a documentation bug or should it work? We do something similar in a 
run_shell_command
where it does work (and maybe I should be using the local environment feature instead? but that's a bigger change because I'm setting up all the node stuff inside of the pants sandbox). I can move these comments into the gh issue instead, too 🙂
f
Yeah better on the issue.
experimental_test_shell_command
runs in the execution sandbox so the current directory is not the repository.
so should be functioning correctly. let's discuss on the issue for UX for how specifying capture should work. The 
test
goal already supports capture and materializing the captured outputs, assuming you are satisfied with that support.
Open in Slack
PreviousNext
Powered by