https://pantsbuild.org/ logo
#general
Title
# general
f

fresh-cat-90827

05/27/2022, 10:30 AM
is it possible that Pants overrides the tests discovered by
pytest
when using the
--ignore
option (of
pytest
)? Or does one need to pass the path to the directory differently when run with Pants?
Make a test fail:
Copy code
diff --git a/helloworld/greet/greeting_test.py b/helloworld/greet/greeting_test.py
index 8d43373..2a564f0 100644
--- a/helloworld/greet/greeting_test.py
+++ b/helloworld/greet/greeting_test.py
@@ -5,5 +5,6 @@ from helloworld.greet.greeting import Greeter
 
 
 def test_greeter() -> None:
+    assert False
     greeter = Greeter(translations={"hello": {"es": "hola"}})
     assert greeter.greet("test") == "Hola, test!"
Checkout https://github.com/pantsbuild/example-python.git, run
Copy code
$ pytest --ignore="helloworld/greet"
======================================================================================================== test session starts ========================================================================================================
platform darwin -- Python 3.9.12, pytest-7.0.1, pluggy-1.0.0
rootdir: /Users/user/code/PantsBuild/example-python
collected 3 items                                                                                                                                                                                                                   

helloworld/translator/translator_test.py ...                                                                                                                                                                                  [100%]

========================================================================================================= 3 passed in 0.02s =========================================================================================================
Then run with Pants:
Copy code
$ ./pants test :: --test-force -- --ignore="helloworld/greet"
./pants test :: --test-force -- --ignore="helloworld/greet"                             
11:25:13.17 [INFO] Completed: Run Pytest - helloworld/translator/translator_test.py:tests succeeded.
11:25:13.27 [ERROR] Completed: Run Pytest - helloworld/greet/greeting_test.py:tests failed (exit code 1).
============================= test session starts ==============================
platform darwin -- Python 3.8.2, pytest-7.0.1, pluggy-1.0.0
rootdir: /private/var/folders/k0/3nssm8v15r5b08k1k1wyhs2r0000gp/T/process-executionOeZbEu
plugins: cov-3.0.0
collected 1 item

helloworld/greet/greeting_test.py F                                      [100%]

=================================== FAILURES ===================================
_________________________________ test_greeter _________________________________

    def test_greeter() -> None:
>       assert False
E       assert False

helloworld/greet/greeting_test.py:8: AssertionError
- generated xml file: /private/var/folders/k0/3nssm8v15r5b08k1k1wyhs2r0000gp/T/process-executionOeZbEu/helloworld.greet.greeting_test.py.tests.xml -
=========================== short test summary info ============================
FAILED helloworld/greet/greeting_test.py::test_greeter - assert False
============================== 1 failed in 0.24s ===============================



✓ helloworld/translator/translator_test.py:tests succeeded in 1.12s.
✕ helloworld/greet/greeting_test.py:tests failed in 1.24s.
The tests in the
greet
directory are still run. I expect them to be ignored.
b

bitter-ability-32190

05/27/2022, 11:47 AM
This might be that case where tools have to choose between excluding based on a flag and being given a file on the CLI. Usually tools treat the CLI file as winning. It's why
black
has force exclude (so when your editor tells black to format CLI file doesn't win).
You can use a pants ignore spec to have it be ignored in pants if that works for you
f

fresh-cat-90827

05/27/2022, 11:53 AM
Right. So when Pants collected all the tests as per it's configuration and passed each individual test to pytest, it's too late to ask to ignore the directories
@hundreds-father-404 if you could confirm the above, do you think it would be helpful if I extend the docs https://www.pantsbuild.org/docs/python-test-goal#passing-arguments-to-pytest to let users know that
--ignore
(and other options) may interfere with how Pants gets the tests to run?
h

hundreds-father-404

05/27/2022, 1:59 PM
f

fresh-cat-90827

05/27/2022, 1:59 PM
yep, I’ve seen this and it does help
h

hundreds-father-404

05/27/2022, 2:00 PM
do you think it would be helpful if I extend the docs
maybe a small "warning" info box? Saying that
--ignore
doesn't work. Instead use
skip_test
field
👍 1
f

fresh-cat-90827

05/27/2022, 2:00 PM
however, if there is a root level glob
tests/**/*
and underneath there is a directory I’d like quickly to exclude, I’d need to create a
BUILD
file in that directory?
h

hundreds-father-404

05/27/2022, 2:19 PM
Ah, you would use
overrides
Copy code
python_tests(
   sources=["tests/**/*_test.py"],
   overrides={
      "tests/dir/*_test.py": {"skip_tests": True},
  },
)
f

fresh-cat-90827

05/27/2022, 2:22 PM
oh this is superb, thank you!
❤️ 1
b

bitter-ability-32190

05/27/2022, 2:42 PM
maybe a small "warning" info box? Saying that
--ignore
doesn't work. Instead use
skip_test
field
If we do this, we shouldn't word it so strongly. At some point
pytest
might prefer the flag ignore over specified files, and we don't control the pytest version.
🙌 1
👍 1
3 Views