https://pantsbuild.org/ logo
#general
Title
# general
r

rich-kite-32423

12/27/2022, 9:42 PM
Stuck on a pytest issue in this repo: https://github.com/BruceEckel/RethinkingObjects The
pytest.ini
in the root directory contains:
Copy code
[pytest]
minversion = 7.0.1
python_files = *.py
addopts = -rA
The
python_files = *.py
should make pytest look in all files for tests, according to https://docs.pytest.org/en/7.1.x/example/pythoncollection.html#customizing-test-collection, but it doesn't seem to be; it only seems to look in `foo_test.py`:
Copy code
bruce@groot:~/RethinkingObjects$ ./pants test ::
14:40:42.23 [ERROR] Completed: Run Pytest - test_experiments/foo_test.py:tests failed (exit code 5).
============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.0.1, pluggy-1.0.0
rootdir: /tmp/pants-sandbox-pDDAT5, configfile: pytest.ini
plugins: forked-1.4.0, cov-3.0.0, xdist-2.5.0
collected 0 items

- generated xml file: /tmp/pants-sandbox-pDDAT5/test_experiments.foo_test.py.tests.xml -
============================ no tests ran in 0.02s =============================
e

enough-analyst-54434

12/27/2022, 9:46 PM
Pants tries not to get on the way of the tools you use, but its sandboxing model necessarily gets involved with files and which ones and caching etc. So options like
python_files
are ~never going to work. Looking ...
Bad test name. I repro but then I edit:
Copy code
$ git diff
diff --git a/test_experiments/foo_test.py b/test_experiments/foo_test.py
index 8255e7f..ef0e3aa 100644
--- a/test_experiments/foo_test.py
+++ b/test_experiments/foo_test.py
@@ -1,3 +1,3 @@
-def foo2_test():
+def test_foo2():
   print("running foo_test_2")
   assert False
And:
Copy code
$ pants test ::
14:51:21.24 [ERROR] Completed: Run Pytest - test_experiments/foo_test.py:tests failed (exit code 1).
============================= test session starts ==============================
platform linux -- Python 3.10.7, pytest-7.0.1, pluggy-1.0.0
rootdir: /tmp/pants-sandbox-MCekm2, configfile: pytest.ini
plugins: forked-1.4.0, cov-3.0.0, xdist-2.5.0
collected 1 item

test_experiments/foo_test.py F                                           [100%]

=================================== FAILURES ===================================
__________________________________ test_foo2 ___________________________________

    def test_foo2():
      print("running foo_test_2")
>     assert False
E     assert False

test_experiments/foo_test.py:3: AssertionError
----------------------------- Captured stdout call -----------------------------
running foo_test_2
- generated xml file: /tmp/pants-sandbox-MCekm2/test_experiments.foo_test.py.tests.xml -
=========================== short test summary info ============================
FAILED test_experiments/foo_test.py::test_foo2 - assert False
============================== 1 failed in 0.03s ===============================



āœ• test_experiments/foo_test.py:tests failed in 0.20s.
FWIW, pytest collects 0 tests too, Pants aside:
Copy code
$ pex pytest -cpytest -- test_experiments/foo_test.py
============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.10.6, pytest-7.2.0, pluggy-1.0.0
rootdir: /home/jsirois/dev/BruceEckel/RethinkingObjects, configfile: pytest.ini
collected 0 items

============================================================================================= no tests ran in 0.00s ==============================================================================================
r

rich-kite-32423

12/27/2022, 10:50 PM
OK, this is really strange. I have also switched to just using pytest (without Pants). First I hid the
pytest.ini
file. Pytest does not appear to be recursing the directories. To test this I put a couple of test files at the top level:
e

enough-analyst-54434

12/27/2022, 10:51 PM
To be clear, the issue wasn't file collection here, it was test function collection. Your naming convention did not match what pytest expected for a test function.
r

rich-kite-32423

12/27/2022, 10:53 PM
f1_test.py
and
test_f1.py
. It finds these but only registers test functions if they start with the word "test", not end with "test" as the Pytest documentation seems to say. The output is:
Copy code
bruce@groot:~/RethinkingObjects$ pytest
================================================= test session starts ==================================================
platform linux -- Python 3.10.6, pytest-7.2.0, pluggy-1.0.0
rootdir: /home/bruce/RethinkingObjects
collected 3 items

f1_test.py F                                                                                                     [ 33%]
test_f1.py F                                                                                                     [ 66%]
test_f2.py .                                                                                                     [100%]

======================================================= FAILURES =======================================================
______________________________________________________ test_f1_b _______________________________________________________

    def test_f1_b():
>     assert False
E     assert False

f1_test.py:5: AssertionError
_______________________________________________________ test_f1 ________________________________________________________

    def test_f1():
>     assert False
E     assert False

test_f1.py:5: AssertionError
=============================================== short test summary info ================================================
FAILED f1_test.py::test_f1_b - assert False
FAILED test_f1.py::test_f1 - assert False
============================================= 2 failed, 1 passed in 0.02s ==============================================
These have been added to https://github.com/BruceEckel/RethinkingObjects
e

enough-analyst-54434

12/27/2022, 10:53 PM
Correct, it sounds like you understand the issue now.
r

rich-kite-32423

12/27/2022, 10:55 PM
I don't, actually. The Pytest docs say that pytest will recurse subdirectories and that the word "test" can be at the beginning or ending of the function. (I'll go re-look at them now).
r

rich-kite-32423

12/27/2022, 10:57 PM
Just saw that: "test-prefixed..."
So for the file names, it doesn't care whether "test" is at the beginning or the end, but for function names it wants "test" at the beginning. And according to https://docs.pytest.org/en/7.1.x/example/pythoncollection.html#changing-naming-conventions I should also be able to change what a test function name looks like. I will now go experiment with that.
e

enough-analyst-54434

12/27/2022, 11:01 PM
Yup.
r

rich-kite-32423

12/27/2022, 11:02 PM
Thanks
And Pants does seem to work fine with the different function naming for tests.
e

enough-analyst-54434

12/27/2022, 11:11 PM
Excellent.
r

rich-kite-32423

12/27/2022, 11:45 PM
Or at least it seemed to, briefly, but now it seems to work with pytest but not with pants. Can't tell what's happening.
./pants test ::
doesn't produce any results.
e

enough-analyst-54434

12/27/2022, 11:59 PM
The
python_tests
target has its own naming conventions; you're violating them. You want to carefully spell out
sources=[...]
for
python_tests
targets and remove certain
python_sources
targets that would otherwise catch things like
a.py
. You are bucking conventions and being forced to learn what the defaults are in essence. If you're going to do that you have to expect to do a lot of reading and explicit configuration, or get lucky and have someone chime in and save you effort like I've done here
See here: https://www.pantsbuild.org/docs/reference-python_tests#codesourcescode Same for any other target you may be bucking conventions with.
h

happy-kitchen-89482

12/28/2022, 4:57 PM
To elaborate: What happens when you run ./pants test ::, say, is that Pants gathers up all the test files, and runs pytest on them. So pytest config doesn't even come into play until Pants finds the test files and executes pytest on them (by default in one process per test file)
How does Pants know which are test files? It looks for python_tests() targets in BUILD files. You'll notice that those usually have no sources= stanza. That is because that target type's default sources glob is usually correct. You can see what that glob is here
If you want test files that don't match that glob, no problem: you'll have to enumerate them in the python_tests target's sources=[] field and you'll have to subtract them from that directory's python_sources target (if any), because that target's default sources is "all .py files except those that match the python_tests target's default globs.
Once you do that, you can run ./pants test path/to/file.py or ./pants test path/:: or whatever, and Pants will run pytest on each of your test files, and apply your pytest.ini when it does so. So at that point, the python_files = *.py in pytest.ini will tell pytest not to ignore the file Pants passed to it, even though it's not a _test.py file, and python_functions = *_main will do what you expect
As you can see from this example, Pants's defaults are designed to work well with conventions, in this case Pytest's conventions. You can buck them, but at a cost of more boilerplate.
But am I understanding correctly that the idiom you're implementing is that test functions live in the same files as the code they're testing? So that a file is both a source file and a test file?
r

rich-kite-32423

12/28/2022, 6:36 PM
Yes, these are the example files exactly as they appear in the book. So each file will contain both regular code, and then the automated tests that have function names ending with
_main
. The
pytest.ini
file which does this is:
Copy code
[pytest]
minversion = 7.0.1
python_files = *.py  # Discover tests in every python file
python_functions = *_main
addopts = -rA
This works correctly with Pytest and came from following the Pytest docs. (In those docs they just seem to consider it normal configuration options, and not "bucking conventions," so I was a bit surprised at that reference). I'm confused, though --- are you saying I shouldn't/cannot use
pytest.ini
at all, and must do all the configuration inside the
BUILD
file? And it sounds like that configuration looks like this:
Copy code
python_tests(sources = ["*.py"])
Also, I'm not sure what "subtracting" looks like, or how to know which files to subtract (especially as the goal is to be able to put tests in any
.py
file).
Sorry for my ignorance here. I'm still testing the waters with Pants, and Pytest seems very focused on flexibility and configurability so I didn't have the impression I was coloring outside the lines in any way.
e

enough-analyst-54434

12/28/2022, 6:54 PM
I will say you will be sending your readers down a rocky road when they step away from your book out into the wild. Since the convention you're bucking is pervasive, the readers may get a false impression of what works.
Basically ~100% of projects stick to those conventions as near as I can tell, Pants aside.
So I guess it depends on what your teaching goals are/ what you expect the audience to be.
IOW you can do what you're doing with pytest (and Pants) but I think ~no one does.
I'm technically not working today but I'll have time later to send you a PR for your repo if you haven't figured it out
r

rich-kite-32423

12/28/2022, 7:01 PM
I easily did it in pytest using the
pytest.ini
file shown above. I am indeed doing something different here, and will explain it in the book (which is not a beginner book).
e

enough-analyst-54434

12/28/2022, 7:01 PM
Ah, ok.
h

happy-kitchen-89482

12/28/2022, 7:13 PM
Well, if every file can be a test file then
python_tests(sources = ["*.py"])
is correct and subtraction from python_sources is moot, since you'd be subtracting everything, so you would just omit the
python_sources()
entirely.
But then I'm not sure if dep inference works when the dep target is in a
python_tests()
, if your examples have complex dependencies across different directories?
But easy enough to test
One impedance mismatch between pants and what you're doing is that Pants assumes that a file is either source (to be depended on by other sources, and eventually bundled into a pex or a wheel or whatever) or it is a test, but in your case a file is both
So we don't have the best modeling for that.
But that could be ironed out with a plugin
If it came to it
e

enough-analyst-54434

12/28/2022, 10:20 PM
Ok, dual target "ownership" works as far as I tested it: https://github.com/BruceEckel/RethinkingObjects/pull/1
@rich-kite-32423 read through that diff, play with it, maybe read some more pantsbuild.org docs and see if that doesn't make sense / explain more about how you model non-standard things in Pants.
I had to add an explicit dep from a
pex_binary
target there on a "lib"
python_sources
target that I wouldn't normally have had to. That's almost certainly due to the dual-target ownership. I think you'll have hell to pay with dep inference not generally working in this sort of setup because of the dual ownership.
If you're willing to repeat yourself and add explicit deps as needed, you may be fine though. You just are playing in Pants v1 / Bazel / Buck land at that point and losing some of the power.
h

happy-kitchen-89482

12/28/2022, 10:30 PM
Well, if this is all for book examples, the issues may not come into play at all? So this may be fine in practice
r

rich-kite-32423

12/29/2022, 4:24 PM
Thank you! I will study this and get back to you.
6 Views