ancient-vegetable-10556
02/01/2022, 6:49 PMRunRequest
and related infrastructure at the moment — is there a particular blocker around refactoring it to use a Process
object rather than specifying raw args? Currently constructing a valid JVM process is complicated, so I’d love to make use of the JvmProcess
infrastructure I used previously. (pinging @witty-crayon-22786 in particular since he’s seen the JvmProcess
work thus far)witty-crayon-22786
02/01/2022, 6:56 PMProcess
, it will likely need to be an InteractiveProcess
ancient-vegetable-10556
02/01/2022, 6:57 PMInteractiveProcess
from a Process
, most of the timejunit.py
witty-crayon-22786
02/01/2022, 6:58 PM@rule
authors constructing the InteractiveProcess
, to push considering those constraints to themancient-vegetable-10556
02/01/2022, 6:59 PMrun
executing things in a chroot workspace? Currently our JVM bootstrapper needs to ln
a JAVA_HOME
directory, which is done inline with running the relevant executable, but the chroot approach would probably involve some sort of two-step that we don’t currently have a good model forwitty-crayon-22786
02/01/2022, 9:58 PMancient-vegetable-10556
02/01/2022, 9:59 PMRunRequest
, but Process
doesn’t have a good model for this eitherwitty-crayon-22786
02/01/2022, 9:59 PMln
was mostly supposed to support providing a stable location for the JDK to be referenced from within a command. if there is another way to accomplish that, then a chroot might not be necessary.ancient-vegetable-10556
02/01/2022, 10:01 PMRunRequest
so that we can run the preparation codewitty-crayon-22786
02/01/2022, 10:01 PMrun
is mostly that the CWD may not be equal to your temporary directoryancient-vegetable-10556
02/01/2022, 10:02 PMjava_home
come from then?witty-crayon-22786
02/01/2022, 10:03 PMancient-vegetable-10556
02/01/2022, 10:04 PMwitty-crayon-22786
02/01/2022, 10:04 PMancient-vegetable-10556
02/01/2022, 10:05 PMwitty-crayon-22786
02/01/2022, 10:09 PMrun
?Process
has been written using relative paths, absolutizing them is challenging without its supportancient-vegetable-10556
02/01/2022, 11:37 PMwitty-crayon-22786
02/01/2022, 11:38 PMancient-vegetable-10556
02/01/2022, 11:40 PMwitty-crayon-22786
02/01/2022, 11:40 PMjava_home/bin
becomes {chroot}/java_home/bin
, etc.ancient-vegetable-10556
02/01/2022, 11:44 PMwitty-crayon-22786
02/01/2022, 11:44 PM{chroot}
template variablesrc/python/pants/backend/python/goals/run_pex_binary.py
for comparison… it basically prepends {chroot}
to everything relevantancient-vegetable-10556
02/01/2022, 11:45 PMwitty-crayon-22786
02/01/2022, 11:46 PMcoursier java-home
can run anywhere, and will emit an absolute path to a JDKancient-vegetable-10556
02/01/2022, 11:46 PMwitty-crayon-22786
02/01/2022, 11:47 PMancient-vegetable-10556
02/01/2022, 11:47 PMwitty-crayon-22786
02/01/2022, 11:48 PMancient-vegetable-10556
02/01/2022, 11:48 PMRunRequest
will make a thing that can run, but it downloads a JDK every time, so when I come back to this, I’ll need to figure out why that’s the case.witty-crayon-22786
02/02/2022, 1:03 AMancient-vegetable-10556
02/02/2022, 4:00 PMRunRequest
before I go back and make it efficientWould it make sense to pre-populate a JDK into the right place and dump that into the workspace? Or should we be trying to make the cache mechanism work more generally?Copy codeValueError: InteractiveProcess requested setup of append-only caches and also requested to run in the workspace. These options are incompatible since setting up append-only caches would modify the workspace.
witty-crayon-22786
02/02/2022, 5:43 PMancient-vegetable-10556
02/02/2022, 5:44 PMcd
over to the temporary directory and do things therewitty-crayon-22786
02/02/2022, 5:44 PMancient-vegetable-10556
02/02/2022, 5:45 PMwitty-crayon-22786
02/02/2022, 5:45 PM{chroot}
templating and temporary directory creation could move inside of InteractiveProcess running?ancient-vegetable-10556
02/02/2022, 5:49 PM{chroot}
templating at all, by running
cd `dirname $0`
in the root InteractiveProcess
witty-crayon-22786
02/02/2022, 5:49 PM./pants run $target -- a/relative/arg for/my/process
…should workancient-vegetable-10556
02/02/2022, 5:50 PMrun
machinery is delicate and I don’t yet understand it 🙂witty-crayon-22786
02/02/2022, 5:52 PM{chroot}
templating is necessary if relative paths will be used, afaict
2. the {chroot}
templating and temporary directory creation moving into InteractiveProcess
(maybe as prework?) would likely clarify all of this a lotancient-vegetable-10556
02/02/2022, 5:54 PMwitty-crayon-22786
02/02/2022, 5:55 PMrun_in_workspace=True
.ancient-vegetable-10556
02/02/2022, 5:55 PMwitty-crayon-22786
02/02/2022, 5:56 PMancient-vegetable-10556
02/02/2022, 5:56 PMwitty-crayon-22786
02/02/2022, 5:56 PMimmutable_inputs
, append_only_caches
, etc on InteractiveProcess
, then adjusting the interactive process runner code to use prepare_workdir
would go a long wayancient-vegetable-10556
02/03/2022, 5:00 PMrun
to work more generally (Java first, then Scala)witty-crayon-22786
02/03/2022, 6:38 PM