https://pantsbuild.org/ logo
#general
Title
# general
h

high-yak-85899

01/20/2022, 10:26 PM
I have a collect_ignore_glob callout in my
conftest.py
to not run modules called
*_manual_test.py
. I know my
conftest.py
is getting picked up properly because I can see it with
./pants dependencies
calls. Are there any special things to get conftest to apply other than making sure it's a dependency like that? The docs make it seem like it should just work ™️ after that.
e

enough-analyst-54434

01/20/2022, 10:28 PM
It should just work after that. I have less confidence about how pytest hooks declared in conftest.py like you have interact with the way we sandbox and call pytest though. Does your plugin hook not appear to fire?
h

high-yak-85899

01/20/2022, 10:32 PM
Yeah, it's collecting tests like
my_manual_test.py
and fails which makes sense because it's not intended to work.
I can obviously get around this by just renaming those files, but was wondering if I was close.
e

enough-analyst-54434

01/20/2022, 10:35 PM
OK. Speak up if you need more help The combination of: 1. ... and fails which makes sense because it's not intended to work. 2. I can obviously get around this by just renaming those files ... Don't make sense to me. The 1st sounds like a-ok as expected, the second sounds like not ok - need to workaround, If you're good to go though, I'm happy to be left confused.
h

high-yak-85899

01/20/2022, 10:35 PM
My
pytest.ini
file got picked up happily but maybe pants is collecting tests before
pytest
gets a chance to say what should be collected?
e

enough-analyst-54434

01/20/2022, 10:37 PM
Pants definitely collect files before running anything_._ It then invokes pytest against all those files: https://github.com/pantsbuild/pants/blob/main/src/python/pants/backend/python/goals/pytest_runner.py#L309
h

high-yak-85899

01/20/2022, 10:37 PM
The manual tests usually need some files or something special on one person's machine so they can run it. With those files missing, the test fails. We prevent them from being collected by pytest normally with that conftest flag. Since pytest collects tests like
*_test.py
, and that happens to match
*_manual_test.py
, I can just rename my few
*_manual_test.py
looking files to something like
*_test_manual.py
so pytest never collects it.
e

enough-analyst-54434

01/20/2022, 10:38 PM
Its probably the latter,
pytest .... all your files explicitly listed here as positional args
that defeats your hook?
h

high-yak-85899

01/20/2022, 10:39 PM
Yeah, I would think so. Maybe that's a bug? Idk. This is actually the first I am learning of this behavior in our repo and I already didn't like it. Felt like a lot of indirection.
e

enough-analyst-54434

01/20/2022, 10:40 PM
Does
@pytest.mark.skipif(function_to_determine_file_presence, reason="You don;t have required files.")
work? Presumably that's alot of churn to add though and your conftest hook approach is more transparent to end-user-tests.
h

high-yak-85899

01/20/2022, 10:42 PM
Yup, that probably also works, though maybe you see a little more clutter if you have a lot of skipped tests.
Actually, seems like the pants output is a little misleading. It shows output as "succeeded" when the test is skipped.
e

enough-analyst-54434

01/20/2022, 10:48 PM
In general Pants is dumb about the nature of the subprocesses it executes at that layer, Exit code 0 - success, non-zero failure.
h

high-yak-85899

01/20/2022, 10:49 PM
Got it. Just has me wondering about maintaining insights on false positives for our CI chain.
e

enough-analyst-54434

01/20/2022, 10:50 PM
Usually CI setups can ingest those and turn them into reports or take other actions on them.
h

hundreds-father-404

01/20/2022, 11:30 PM
@high-yak-85899 there is also the
skip_tests
option: https://www.pantsbuild.org/docs/existing-repositories#3-set-up-tests
h

high-yak-85899

01/20/2022, 11:32 PM
That feels a little overreaching. They are colocated with other things that work fine. pytest skipif markers (which we have already been using) seems like a good path. But that's a good option to keep in mind if I run into a real blocker.
👍 1
Maybe this could be worth a bug. Just found a different instance of this. I had
python_files = *_test.py
in
pytest.ini
but because
python_tests
already has a default glob pattern to include
test_*.py
, some files I didn't intend to have picked up were being executed. So, while maybe not a bug, it does seem important/restricting that collection happens without regard for what is in conftest.py or pytest.ini.
I think I found another case where
Exit code 0 - success, non-zero failure.
is a problem. We have custom fixtures set up with our
conftest.py
and properly marked in the test file. As expected, that test shouldn't run without the proper marker applied with a run time flag. But, because pants collects them first and then doesn't run them, we get an exit code because no tests are run. A silly way to fix this is to put something like this in the file that has a marked test so that something runs and prevents the error from being thrown.
Copy code
class DontLetPantsFail(unittest.TestCase):
    def test_nothing(self):
        pass
6 Views