#18082 Stabilising `experimental_shell_command` and friends New discussion created by chrisjrn After a bunch of work, we're now in a position where the contents of

shell_command.py

are ready for their forever home in Pants. This ticket describes the things that need to be done, along with initial proposals for what we do to accomplish these. These are subject to discussion. Note that

experimental_wrap_as_*

, while written largely to support

experimental_shell_command

(

esc

) and

experimental_run_in_sandbox

(

eris

), are not subject to this stabilisation work, since there are known deficiencies with the current approach. Moving and renaming

experimental_run_in_sandbox

eris

solves the technical problem of not being able to use build scripts written in Pants-supported languages as ad-hoc steps in Pants, along with the secondary goal of allowing 3rd-party tools to be integrated into a Pants installation in an ad-hoc fashion. It also fulfils a philosophical goal of Pants: code quality is better if it's validated and tested. Currently, we have a great answer for production code, but not for code that is run in the build itself. With

eris

, these ad-hoc steps can be quality-checked by other Pants goals (test/lint/fmt/fix/check). Such validation is not possible with

esc

as it is currently designed. We should encourage using code that can be quality-checked over code that cannot. PROPOSAL:

eris

is documented the preferred way to run user-supplied code as a build step, whenever it is possible, not

esc

.

esc

should be introduced only for cases where Pants does not provide any support. See future work for an important realisation on this topic. PROPOSAL: Rename and move

experimental_run_in_sandbox

from the

shell

subsystem. The proposed new subsystem name is

pants.backend.adhoc

, and the goal be named

adhoc_tool

. I'm proposing the name

adhoc

over

generic

, because I feel it describes the use cases for these targets: one-off cases where a plugin does not exist, or would not be suitable. To me,

generic

, invokes the idea of re-use, which doesn't really apply here. On the other hand,

adhoc

invokes the idea of "this is a thing that is being done instead of a plugin". Happy to bikeshed this one. Renaming

experimental_shell_command

PROPOSAL: Rename

experimental_shell_command

to

shell_command

. Once

shell_source

is runnable through

eris

,

esc

will only be the preferred option for trivial shell commands. At this point,

shell_command

is the obvious name for the thing that executes a line of shell script PROPOSAL: Rename

experimental_test_shell_command

to

shell_command_test

This is in line with nomenclature for other targets. Removing

experimental_run_shell_command

PROPOSAL: Delete

experimental_run_shell_command

(

ersc

), and make

shell_command

runnable. Now that

esc

supports

workdir

, and

tools

is optional for

esc

, there is negligible difference between

esc

and

ersc

from an interface perspective. Making

esc

runnable would reduce the number of targets (and would be in line with the goal of making more targets

runnable

, for use in

eris

). It would also open up the possibility of

esc

being used as the

runnable

for

eris

, which is important (see Future work). Future work This list addresses possibilities that are enabled by following the above proposal. Enable

shell

usage through

eris

Note that the

shell

backend currently does not implement

RunRequest

on

shell_source

targets. However, if

RunRequest

were implemented, then

shell_source

targets could be executed using

eris

, which would make it possible to quality-check those build steps. With this in mind, we should be encouraging

eris(runnable=some_shell_source)

as the preferred way to execute ad-hoc shell code in all but the most trivial of cases. Allow

eris

to run multiple `runnable`s in sequence in the same sandbox If

shell_command

becomes a

runnable

target (that could optionally be supplied to

eris

), then it would be possible to string together several

runnable

steps in the same

sandbox

if you needed to compose the results of multiple ad-hoc steps. Consensus/decision making. I'll leave this discussion open for 48 hours to allow for initial thoughts and to see if consensus emerges. If some alternate viewpoints emerge, I'll try to find consensus, and allow 24 hours of cooling off to see if we've actually met consensus. Ideally the contentious decisions policy won't come into play, but if we don't achieve consensus, then that's what it's there for. pantsbuild/pants