https://pantsbuild.org/ logo
Join Slack
Powered by
is it possible that Pants overrides the tests disc...
# general
f
is it possible that Pants overrides the tests discovered by 
pytest
when using the 
--ignore
option (of 
pytest
)? Or does one need to pass the path to the directory differently when run with Pants?
Make a test fail:
Copy code
diff --git a/helloworld/greet/greeting_test.py b/helloworld/greet/greeting_test.py
index 8d43373..2a564f0 100644
--- a/helloworld/greet/greeting_test.py
+++ b/helloworld/greet/greeting_test.py
@@ -5,5 +5,6 @@ from helloworld.greet.greeting import Greeter
 
 
 def test_greeter() -> None:
+    assert False
     greeter = Greeter(translations={"hello": {"es": "hola"}})
     assert greeter.greet("test") == "Hola, test!"
Checkout https://github.com/pantsbuild/example-python.git, run
Copy code
$ pytest --ignore="helloworld/greet"
======================================================================================================== test session starts ========================================================================================================
platform darwin -- Python 3.9.12, pytest-7.0.1, pluggy-1.0.0
rootdir: /Users/user/code/PantsBuild/example-python
collected 3 items                                                                                                                                                                                                                   

helloworld/translator/translator_test.py ...                                                                                                                                                                                  [100%]

========================================================================================================= 3 passed in 0.02s =========================================================================================================
Then run with Pants:
Copy code
$ ./pants test :: --test-force -- --ignore="helloworld/greet"
./pants test :: --test-force -- --ignore="helloworld/greet"                             
11:25:13.17 [INFO] Completed: Run Pytest - helloworld/translator/translator_test.py:tests succeeded.
11:25:13.27 [ERROR] Completed: Run Pytest - helloworld/greet/greeting_test.py:tests failed (exit code 1).
============================= test session starts ==============================
platform darwin -- Python 3.8.2, pytest-7.0.1, pluggy-1.0.0
rootdir: /private/var/folders/k0/3nssm8v15r5b08k1k1wyhs2r0000gp/T/process-executionOeZbEu
plugins: cov-3.0.0
collected 1 item

helloworld/greet/greeting_test.py F                                      [100%]

=================================== FAILURES ===================================
_________________________________ test_greeter _________________________________

    def test_greeter() -> None:
>       assert False
E       assert False

helloworld/greet/greeting_test.py:8: AssertionError
- generated xml file: /private/var/folders/k0/3nssm8v15r5b08k1k1wyhs2r0000gp/T/process-executionOeZbEu/helloworld.greet.greeting_test.py.tests.xml -
=========================== short test summary info ============================
FAILED helloworld/greet/greeting_test.py::test_greeter - assert False
============================== 1 failed in 0.24s ===============================



✓ helloworld/translator/translator_test.py:tests succeeded in 1.12s.
✕ helloworld/greet/greeting_test.py:tests failed in 1.24s.
The tests in the 
greet
directory are still run. I expect them to be ignored.
b
This might be that case where tools have to choose between excluding based on a flag and being given a file on the CLI. Usually tools treat the CLI file as winning. It's why 
black
has force exclude (so when your editor tells black to format CLI file doesn't win).
You can use a pants ignore spec to have it be ignored in pants if that works for you
f
Right. So when Pants collected all the tests as per it's configuration and passed each individual test to pytest, it's too late to ask to ignore the directories
@hundreds-father-404 if you could confirm the above, do you think it would be helpful if I extend the docs https://www.pantsbuild.org/docs/python-test-goal#passing-arguments-to-pytest to let users know that 
--ignore
(and other options) may interfere with how Pants gets the tests to run?
h
f
yep, I’ve seen this and it does help
h
do you think it would be helpful if I extend the docs
maybe a small "warning" info box? Saying that 
--ignore
doesn't work. Instead use 
skip_test
field
👍 1
f
however, if there is a root level glob 
tests/**/*
and underneath there is a directory I’d like quickly to exclude, I’d need to create a 
BUILD
file in that directory?
h
Ah, you would use 
overrides
Copy code
python_tests(
   sources=["tests/**/*_test.py"],
   overrides={
      "tests/dir/*_test.py": {"skip_tests": True},
  },
)
f
oh this is superb, thank you!
❤️ 1
b
maybe a small "warning" info box? Saying that 
--ignore
doesn't work. Instead use 
skip_test
field
If we do this, we shouldn't word it so strongly. At some point 
pytest
might prefer the flag ignore over specified files, and we don't control the pytest version.
🙌 1
👍 1
3 Views
Open in Slack
PreviousNext
Powered by