https://pantsbuild.org/ logo
Join Slack
Powered by
pytest question: I have a subset of `python_test` ...
# general
r
pytest question: I have a subset of 
python_test
targets which I would like to run with additional pytest arguments. 
python_test
target does not have any fields for pytest arguments, and I think that makes sense to me. I can add pytest arguments via cli args or pants.toml. I would like to codify the mapping between 
python_test
targets and pytest arguments. I can create a 
pants.jupyter_tests.toml
and override both 
tag
and 
pytest.args
. This works but might lead to a proliferation of 
.toml
files if we have lots of different sets of tests which require specific arguments. Am I missing another better way to configure my project?
Immediate use case is using nbmake plugin to test a set up jupyter notebooks. I need to activate the plugin by adding a 
--nbmake
pytest argument.
b
I'm not sure the target API has support for this. You might be able to get away with it using a 
conftest.py
that mucks with pytest/that plugin
h
+1 to trying conftest.py
r
Ah - ok! That sounds like a good solution. Thanks
h
Because we run one test-per-target, a field w/ additional CLI args is reasonable & trivial to implement. I'm not sure if it's feature bloat, but it could be worth considering if conftest.py isn't gonna work out
➕ 1
e
FWIW, Pants v1 supported this for junit_tests IIRC, presumably driven by need.
👀 1
👍 1
r
To close the loop, it looks like pytest supports a 
PYTEST_ADDOPTS
environment variable for including extra cli options. https://docs.pytest.org/en/7.1.x/example/simple.html . I'll probably create something like a 
nb_test
macro and set 
PYTEST_ADDOPTS=--nbmake
for my specific use case. That should be functionally equivalent to an 
args
field in the 
pytest
target.
h
In conftest.py?
Or literally by setting PYTEST_ADOPTS on the cli?
r
h
ah, but won't that set them for all tests in the repo?
Oh, in the target!
NM, sorry, brain fog
r
No worries!
Copy code
PYTEST_ADDOPTS_VAR = "PYTEST_ADDOPTS"

def nbmake_test(**kwargs):
    extra_env_vars = kwargs.pop("extra_env_vars", {})
    pytest_addopts = extra_env_vars.pop(PYTEST_ADDOPTS_VAR, "")
    pytest_addopts += " --nbmake"
    extra_env_vars[PYTEST_ADDOPTS_VAR] = pytest_addopts
    python_test(**kwargs)
This works nice. But looks like we weren't expecting to run 
pytest
against files without`.py` extension.
Copy code
InvalidFieldException: The 'source' field in target src/python/<>:<>_nb_test can only contain files that end in one of ['', '.py'], but it had these files: ['src/python/<>/<>.ipynb'].
h
Hm, quick workaround for that is a bit hacky. Use the Target API to define a new target type like 
notebook_test
, where all the fields are the same as 
python_test
except for subclassing 
PythonSourceField
and setting 
expected_file_extensions = (".ipynb",)
https://www.pantsbuild.org/docs/target-api-concepts and the 2 pages after it
r
Cool - thanks for the pointer
h
Would you expect 
.ipynb
to work with 
python_source
and 
python_test
? That has many implications like that MyPy will run on those notebooks. Given that notebooks tend to be experimental, we figured in the past they are best handled via dedicated targets
r
I agree with your thinking there. We want to configure some integration tests validating that notebook examples can run to completion. We don't treat 
.ipynb
as python source internally. We don't have 
.ipynb
unit tests with assertions or include them in linting. Trying to put some sanity checks around keeping the notebooks maintained but not sure how deep of a rabbit-hole we want to go down treating it as first class source code.
👍 1
h
Btw, you can use the Target API to make this more fleshed out without writing a single Rules API line. For example, the 
skip_tests
field defaults to 
False
. You can subclass it and have it default to 
True
👍 1
Open in Slack
PreviousNext
Powered by