breezy-mouse-20493
04/29/2023, 2:46 AMshell_command
. I have a python project, with python tests, and inside the BUILD file for said tests I've added
python_tests(...)
shell_command(
command="./just_run_touch_foo.sh",
tools=["bash", "touch"],
execution_dependencies=[":scripts"],
output_directories=["."],
workdir=".",
description="this just runs `touch foo"
)
shell_sources(
name="scripts"
)
What should I expect to happen here? Hoping to see a file foo
sitting in the directory after the tests run. (Or, at least, inside the sandbox while tests run. So far I have seen neither.)
At the current time, my attempt at running the shell command above results in
19:40:00.77 [WARN] No applicable files or targets matched. The `test` goal works with these target types:
* experimental_test_shell_command
* python_test
* shunit2_test
However, you only specified target arguments with these target types:
* shell_command
I'm pretty sure I don't want to run experimental_test_shell_command
, as this actually runs a test inside bash script. What I want eventually is to download some canonical dataset that will be used in my unit tests...broad-processor-92400
04/29/2023, 3:16 AMshell_command
is designed as a target for generating files for other targets to use: you will need a dependency from the Python test target to the shell command one, so that pants knows the files are needed.pants export-codegen path/to:target
. Dependencies are the right way to integrate it with other targets.
(That said, for downloading a public file specifically, another option might be: `file(name=“`…`”, source=http_source(…
))` https://www.pantsbuild.org/docs/reference-file)breezy-mouse-20493
04/29/2023, 4:26 AMfile
target... Is there any way I can send an authentication section in a header, like what you'd do with curl?broad-processor-92400
04/29/2023, 6:57 AMcurved-television-6568
04/29/2023, 12:04 PMurl_handler
in that case.. https://github.com/pantsbuild/pants/tree/dcdd1bcc5c7b2f4992658fce28e469c3bc294bd9/src/python/pants/backend/url_handlersbreezy-mouse-20493
05/01/2023, 5:03 PMbroad-processor-92400
05/01/2023, 10:19 PMbreezy-mouse-20493
05/01/2023, 10:29 PMwitty-crayon-22786
05/01/2023, 10:42 PMcurl
is a decent option: you’d only want to ensure that you included some version information and/or `shasum`ing of the downloaded file in your script.breezy-mouse-20493
05/01/2023, 11:24 PMshell_command
that I can refer to? As noted above, I'm seeing an error "No applicable files or targets matched."
I think it's because I'm doing something wrong with the BUILD file for this directory. I'm on Pants 2.16.0rc1broad-processor-92400
05/01/2023, 11:27 PMpants run ...
? You'll need to use pants export-codegen ...
to validate/debug it, and set dependencies in appropriate targets to use it for tests, e.g.
shell_command(
name="data",
command="curl ... && shasum ...",
tools=["curl", "shasum"],
output_file=["some_file.txt"]
)
python_tests(
...,
dependencies=[":data"]
)
This will mean that running those tests knows that it needs to run the data command and materialize the files within the testing sandboxbreezy-mouse-20493
05/01/2023, 11:28 PMcommand=
option?That sounds like you still might be usingbtw, I'm using
pants test ::
broad-processor-92400
05/01/2023, 11:30 PMshell_command
target describes how to generate some files for other targets to use.pants test ::
?breezy-mouse-20493
05/01/2023, 11:35 PMwitty-crayon-22786
05/01/2023, 11:35 PMbreezy-mouse-20493
05/01/2023, 11:36 PMpython_tests(
name="tests",
dependencies=[":data"]
)
files(
name="data",
sources=["test-data/*.h5"]
)
shell_command(
command="./just_run_touch_file.sh",
tools=["bash", "touch"],
execution_dependencies=[":scripts"],
output_directories=["."],
workdir=".",
description="run a shell command",
dependencies=[":scripts"],
)
shell_sources(
name="scripts"
)
and here is the script:
#!/usr/bin/bash
echo "hello"
touch foo
I don't see either "hello" nor the file foo
broad-processor-92400
05/01/2023, 11:37 PMshell_command(..., log_output=True)
to help with debugging. You might also need to change the script slightly (e.g. add a comment # 1
that you can just increment to bust the cache for now)breezy-mouse-20493
05/01/2023, 11:38 PMbroad-processor-92400
05/01/2023, 11:39 PMfoo
will, theoretically, be placed into the same directory as the BUILD
file, in the test sandbox. You can peek into those sandboxes with https://www.pantsbuild.org/docs/reference-global#keep_sandboxes. For instance, if the tests are failing pants --keep-sandboxes=on_failure test ::
and the cd/exploring the directory that is logged.adhoc_tool
, which is similar to shell_command
but not exactly what you're doing... so might not be super easy to generalise from for this initial learning curve.output_directory=["."]
might not be doing the expected thing. One way to eliminate that variable would be using output_files=["foo"]
instead, for now.)breezy-mouse-20493
05/01/2023, 11:49 PM