famous-river-94971
01/05/2023, 7:36 PMexperimental_run_shell_commands
were identified as changed:
./pants --changed-since=HEAD --changed-dependees=transitive --filter-tag-regex='^cdk$' list
aws/projects/project_1:cdk
aws/projects/project_2:cdk
But if I try to run
them, I get an error:
./pants --changed-since=HEAD --changed-dependees=transitive --filter-tag-regex='^cdk$' run
12:32:43.46 [ERROR] 1 Exception encountered:
TooManyTargetsException: The `run` goal only works with one valid target, but was given multiple valid targets:
* aws/projects/project_1:cdk
* aws/projects/project_2:cdk
Please select one of these targets to run.
I can hack something with xargs
like this:
export PANTS_CONCURRENT=True && ./pants --changed-since=HEAD --changed-dependees=transitive --filter-tag-regex='^cdk$' list | xargs -L1 -P 2 ./pants run
but I was wondering if there's a "better"/more pants
-y way to do this?parallel
would be better than xargs
because it batches up output as if commands were run sequentially.bitter-ability-32190
01/05/2023, 7:56 PMfamous-river-94971
01/05/2023, 7:57 PMInteractive ProcessesMy command doesn't need to take any input - does that mean it's non-interactive and I should be using a different method other than
experimental_run_shell_command
?bitter-ability-32190
01/05/2023, 8:00 PMrun
.
What do your processes do? AFAIK we don't have a mechanism to have the user request multiple processes run in parallel on-demand and without caching 🤔famous-river-94971
01/05/2023, 8:02 PMcdk
(AWS CDK infrastructure-as-code tool) to run a command (diff
or deploy
infrastructure). That command needs the Python context from Pants since the CDK code itself is defined in Python.BUILD
file:
experimental_run_shell_command(
name="cdk",
tags=["cdk"],
command="../../scripts/cdk-deploy.sh",
dependencies=["aws/projects/project_1:project_1"],
workdir="aws/projects/project_1",
)
and then the script (simplified) looks like this:
#! /bin/bash
source "$SCRIPT_DIR/../../dist/export/python/virtualenvs/cdk_dependencies/3.8.16/bin/activate"
export PYTHONPATH="$SCRIPT_DIR/../projects:$PYTHONPATH"
npx -y cdk synth
pants export
to get the virtualenv and set the PYTHONPATH myself. There might be a better way.npx
(NPM execute) to run it. Sadly, I can't just "run a Python file"bitter-ability-32190
01/05/2023, 8:05 PMpackage
command. There's a deploy
one as well. That'd involve a plugin today, but could also be extendedfamous-river-94971
01/05/2023, 8:06 PMdeploy
and I thought it was helm-specific.package
at all for thisbitter-ability-32190
01/05/2023, 8:06 PMfamous-river-94971
01/05/2023, 8:10 PMexperimental-deploy
is how I'd plug in? https://www.pantsbuild.org/docs/reference-experimental-deploy (not really any docs, but maybe I can reference the Helm impl for an example)bitter-ability-32190
01/05/2023, 8:10 PMexperimental_shell_publish_command
which: can run in parallel, as part of publish
CC @ancient-vegetable-10556 /@happy-kitchen-89482 /@witty-crayon-22786 while we're splashing at shell
stufffamous-river-94971
01/05/2023, 8:11 PMrun
any arbitrary script
in the package.json
with a concurrency flag. It's super nice!
yarn lerna run --since '' --concurrency 10 cdk -- deploy '**'
bitter-ability-32190
01/05/2023, 8:12 PMpublish
not package
. brain fog got me there)ancient-vegetable-10556
01/05/2023, 8:15 PMbitter-ability-32190
01/05/2023, 8:18 PMancient-vegetable-10556
01/05/2023, 8:20 PMexperimental_shell_command(name="a", command="first_command")
experimental_shell_command(name="b", command="second_command")
experimental_run_shell_command(name="c", dependencies=[":a", ":b"], command="/bin/true")
and then ./pants run path/to:c
first_command
and second_command
can be run inside the sandboxbitter-ability-32190
01/05/2023, 8:20 PMancient-vegetable-10556
01/05/2023, 8:21 PMfamous-river-94971
01/05/2023, 8:22 PMancient-vegetable-10556
01/05/2023, 8:23 PMfamous-river-94971
01/05/2023, 8:26 PMparallel
is probably safer in that respect?ancient-vegetable-10556
01/05/2023, 8:29 PMfamous-river-94971
01/05/2023, 8:30 PMancient-vegetable-10556
01/05/2023, 8:31 PMfamous-river-94971
01/05/2023, 8:32 PMbazel
before pants
and pants
was so much more understandable for me!bazel
tutorial video and I was like... https://media.giphy.com/media/wYyTHMm50f4Dm/giphy.gif▾
bitter-ability-32190
01/05/2023, 8:33 PMbusy-vase-39202
01/05/2023, 10:17 PMfamous-river-94971
01/05/2023, 10:31 PMbusy-vase-39202
01/05/2023, 10:49 PMbitter-ability-32190
01/07/2023, 12:37 PM