https://pantsbuild.org/ logo
#general
Title
# general
h

happy-fish-67882

11/23/2020, 11:49 PM
Hey guys, using pants v1, I'd like to add a shell script test to ensure it passes basic tests... how would my BUILD file look like ? I would like to use
bash -n
<scriptname> ? Also, any way I can do more tests like validate output of running flags for the script on the command line?
w

witty-crayon-22786

11/23/2020, 11:56 PM
hey @happy-fish-67882!
in both v1 and v2, doing something like that would currently require writing a plugin
there isn’t “BUILD file only” syntax to do it.
h

happy-fish-67882

11/23/2020, 11:57 PM
any pointers on how to get started ?
w

witty-crayon-22786

11/23/2020, 11:57 PM
h

happy-fish-67882

11/23/2020, 11:57 PM
Need v1 for now
w

witty-crayon-22786

11/23/2020, 11:58 PM
the docs for v1 are still around, but we would recommend trying to get on v2 before writing the plugin, as things have changed fundamentally. one sec. lemme find.
you’d want two pages linked from here: https://v1.pantsbuild.org/dev.html “Developing a Plugin” and “Developing a Task”.
h

happy-fish-67882

11/23/2020, 11:59 PM
Cool! The problem we have is that v2 doesn't seem to support what we need for other languages
h

hundreds-father-404

11/23/2020, 11:59 PM
I’d like to add a shell script test to ensure it passes basic test
Might it be possible to write a light-weight Python wrapper that calls the relevant bash commands with
subprocess.run()
, then running that via
./pants test
and a
python_tests()
target?
w

witty-crayon-22786

11/23/2020, 11:59 PM
@happy-fish-67882: ah, yea… which languages?
@hundreds-father-404: maybe… but in v1,
test
didn’t depend on
binary
, so your binary might not be available
would need a plugin to force that
i think that in the short/medium term, “i’d like to do something simple with a shell script” will be addressed in v2 by https://github.com/pantsbuild/pants/issues/3734
h

happy-fish-67882

11/24/2020, 12:01 AM
For v2 -- node and scala don't seem supported
w

witty-crayon-22786

11/24/2020, 12:02 AM
got it. yea, that’s correct. JVM support is pretty high on the priority list to be added to 2.0.0: node might be a little while longer.
h

hundreds-father-404

11/24/2020, 12:03 AM
but in v1, test didn’t depend on binary, so your binary might not be available
True. But in v1, expressing that type of dependency between
test
and
binary
is non-trivial, right? That was a prime reason we implemented the v2 engine in the first place Martin, how feasible would it be to have an outer bash script that calls several Pants commands to achieve what you’re hoping for? Very clunky indeed, but I don’t want to lead you astray with a v1 plugin if you don’t need it. They’re harder to write and not as well documented
w

witty-crayon-22786

11/24/2020, 12:03 AM
@happy-fish-67882: do you know if you are using “source dependencies” for node?
h

happy-fish-67882

11/24/2020, 12:05 AM
@hundreds-father-404 yeah it's feasible... we could have it execute into some CI system on build
@witty-crayon-22786 can you elaborate, another dev also not sure what you mean on source deps
h

hundreds-father-404

11/24/2020, 12:07 AM
For v2 -- node and scala don’t seem supported
Are you using Python too? It’s possible to set it up so that the Python portion of your repo uses Pants 2.0+, but the rest of your repo still uses 1.30 Not sure if the functionality you’re looking for is around Scala/Node, though. If so, probably not helpful. https://www.pantsbuild.org/docs/how-to-upgrade-pants-2-0#step-2-setup-jvm-node-and-go-support-if-relevant
👌 1
h

happy-fish-67882

11/24/2020, 12:07 AM
@hundreds-father-404 but what pants command would those be to do what I'd like to wrt bash script inclusion/testing though
h

happy-fish-67882

11/24/2020, 12:20 AM
I don't believe we're using it @witty-crayon-22786
w

witty-crayon-22786

11/24/2020, 12:21 AM
ok, thanks.
h

happy-fish-67882

11/24/2020, 12:21 AM
We could have that test run in a python script, yeah
h

hundreds-father-404

11/24/2020, 12:23 AM
Cool, so then you can avoid writing a plugin for now, regardless of v1 or v2.
but what pants command would those be to do what I’d like to wrt bash script inclusion/testing though
This is the part we gotta figure out. I’m trying to understand your workflow. Are you hoping to do something like this?
Copy code
$ ./pants package path/to:tgt
$ result=$(dist/result.pex arg1 --arg2)
$ assert result == foo
h

happy-fish-67882

11/24/2020, 6:37 PM
Nice! And yeah I'm trying to understand pants 🙂 I guess we'll meet in the center at some point lol
@hundreds-father-404 What would result.pex be?
and I'd like to run bash -n <myscript.sh>
h

hundreds-father-404

11/24/2020, 6:40 PM
The result of running
./pants bundle
or
./pants binary
. Both of those got consolidated into
./pants package
in 2.0 Generally, I’m trying to see if this is what you’re hoping to do: build a binary with Pants, then run a script that does something with that built artifact, such as testing that it works correctly. Something like this: https://www.pantsbuild.org/v2.2/docs/python-test-goal#depending-on-packages
Are you able to share what
<myscript.sh>
does?
h

happy-fish-67882

11/24/2020, 6:42 PM
I can tell you it installs ansible and deps, deploys a kubernetes cluster all by itself, including a custom application. It's a couple of thousand lines
Goal would be to flag through pants if there's any syntax errors in it, and possibly run some args on the script to validate output
💯 1
h

hundreds-father-404

11/24/2020, 6:44 PM
Okay, thanks. I think Stu and I might have been a little confused then with mentioning
./pants binary
- sounds like a red herring. Sorry about that Taking a step back, you have
<myscript.sh>
, and the goal is to use Pants to test that
<myscript.sh>
works properly. Is that right? You are not trying to rewrite
<myscript.sh>
, only test is.
h

happy-fish-67882

11/24/2020, 6:45 PM
correct
That's why I was thinking bash -n <script> but it may be limited
I'd prefer <myscript.sh> --arg1 value1 --arg2 value2 --arg3 value3 and that produces output, validate the output is equal to an expected string kind of thing
h

hundreds-father-404

11/24/2020, 6:47 PM
Cool, so then, I would recommend that idea of writing a Python test that calls
myscript.sh
Copy code
import subprocess

def test_script():
  result = subprocess.run(["myscript.sh", "arg1"], stdout=suprocess.PIPE, ...)
  assert "foo" in result.stdout.decode()
Then, you could use a
python_tests()
target to have Pants always run these tests. How does that sound?
h

happy-fish-67882

11/24/2020, 6:47 PM
Sounds about right yeah
so the python code could be put straight inside the BUILD file, right ?
since it's python
or does it need to be called a better way maybe
h

hundreds-father-404

11/24/2020, 6:49 PM
Awesome. So that will allow you to test the script without needing a plugin, nor needing to upgrade to v2 Pants if aren’t yet able to. The Python code will go into an actual Python file, which can be placed anywhere you want. For example,
<build root>/test_myscript.py
, or some people like to have a
build-support
folder for things like this. Wherever you define that Python file, you’ll have a BUILD file with a target like this in it:
Copy code
python_tests(name="tests")
since it’s python
It is Python, but extremely limited Python. For example, import statements are banned. BUILD files are nothing more than a declarative way to describe metadata for some code - they’re intentionally very limited You use macros and/or plugins to add new functionality to Pants
h

happy-fish-67882

11/24/2020, 6:51 PM
ok cool!
Makes lots of sense
Will try that and let you know if there are issues with it 🙂 Thank you very much for the help and responsiveness, it's awesome
❤️ 1
h

hundreds-father-404

11/24/2020, 6:52 PM
Cool, let us know how that goes.
Goal would be to flag through pants if there’s any syntax errors in it,
Have you heard of Shellcheck? Our example plugin repo has an implementation of it. If you’re interested, we can help wire it up into your repo - mostly just copy pasta 🙂 Shellcheck has saved me so much headache We also have implemented a unit testing framework for bash called shunit2
h

happy-fish-67882

11/24/2020, 6:54 PM
Haven't heard before, what's the url ?
for these 2
h

hundreds-father-404

11/24/2020, 6:57 PM
Couple links that might be helpful: + Excellent blog on how to use `subprocess.run`: https://codecalamity.com/run-subprocess-run/ + shellcheck: https://github.com/koalaman/shellcheck + overview of Pants plugins, particularly how they’re loaded https://www.pantsbuild.org/v2.0/docs/plugins-overview + example plugin repo: https://github.com/pantsbuild/example-plugin For the last two, I wouldn’t worry about them yet until you’ve tried out Shellcheck and confirm you want to use it in your project. No sense in seeing how to write a plugin until you know you want the underlying functionality
h

happy-fish-67882

11/24/2020, 7:14 PM
oh wow, shellcheck is amazing -- thanks for that