https://pantsbuild.org/ logo
Join Slack
Powered by
Hi all, I have an issue with testing. (disclaimer,...
# general
f
Hi all, I have an issue with testing. (disclaimer, I'm quite new to python and not an expert in pytest config at all 😛) I recently added a dependencies to my stack (addok). It has a plugin for its own test suites, but the plugin is loaded and maybe some other stuff. It adds an automatic tear down which cannot work (it requires a connection to local redis). Here is a bit of command output :
Copy code
09:41:42.58 [ERROR] Completed: Run Pytest - external-data/hw_datasets/tests/datasets/dvf/test_processor.py:../../../hw_datasets_tests - failed (exit code 1).
Loaded local config from /home/maurin/.cache/pants/named_caches/pex_root/venvs/s/c5ca3b9a/venv/lib/python3.11/site-packages/addok/config/test.py
Loaded plugins:
addok.shell==1.1.2, addok.http.base==1.1.2, addok.batch==1.1.2, addok.pairs==1.1.2, addok.fuzzy==1.1.2, addok.autocomplete==1.1.2
============================= test session starts ==============================
platform linux -- Python 3.11.4, pytest-7.0.1, pluggy-1.0.0
rootdir: external-data/hw_datasets/tests, configfile: pytest.ini
plugins: xdist-2.5.0, mock-3.12.0, addok-1.1.2, cov-3.0.0, forked-1.6.0, anyio-4.4.0
collected 1 item
And the bottom of the stacktrace is :
Copy code
def pytest_runtest_teardown(item, nextitem):
    from addok import db, ds
    from addok.config import config as addok_config

    assert db.DB.connection_pool.connection_kwargs["db"] == 14
    db.DB.flushdb()
    if addok_config.DOCUMENT_STORE == ds.RedisStore:
        assert ds._DB.connection_pool.connection_kwargs["db"] == 15
        ds._DB.flushdb()
So I'm in the teardown setup by addok (in the addok plugin). When I execute pytest on my project manually everything runs fine, but when it's through the 
pants test ::
command, it run inside the pex built for testing and it loads this addok plugin. Is there a way to avoid that ? (I add my pants build config in the thread) Thanks in advance for the help 🙂
✅ 1
Here is my pants build file :
Copy code
poetry_requirements(
    name="poetry",
    resolve="hw_datasets_resolver",
)

resources(name="hw_datasets_resources",
          sources=["hw_datasets/resources/*",
            "hw_datasets/datasets/**/*.ipynb",
            "hw_datasets/datasets/**/*.json",
            "hw_datasets/datasets/**/*.yaml"
          ]
          )

resources(name="test_resources",
          sources=["tests/resources/*"],
          dependencies=[":hw_datasets_resources"]
          )

python_sources(name="test_sources",
              sources=["tests/**/*.py"],
              resolve="hw_datasets_resolver",
              )

python_tests(name="hw_datasets_tests",
            sources=["tests/**/test_*.py", "!**/addok/*"],
            dependencies=[":test_resources", ":test_sources", "!:poetry"],
            resolve="hw_datasets_resolver",
            )
So, after all that, it clarified a bit the situation in my head and I've been able to find a way to configure pytest to ignore the addok plugin. For those who look for the answer, I had to create a 
pytest.ini
file in my tests forlder with the following content :
Copy code
[pytest]
addopts = -p no:addok
You can deactivate any plugin from pytest this way.
✅ 1
However, I would be interested to know if there's a way to not include some dependencies in the test through pants ? I tried to add a 
!addok:poetry
in the dependencies of my hw_datasets_tests target but it doesn't help.
c
perhaps using transitive exclude 
!!addok:poetry
?
f
Oh nice, I'll try that, I didn't found this way to exclude in the doc I've found
Copy code
The file or directory 'addok' does not exist on disk in the workspace, so the address 'addok:poetry' from the `dependencies` field from the target external-data/hw_datasets/tests/steps/test_downloader.py:../../hw_datasets_tests cannot be resolved.
It doesn't find the dependency, but it looks to be looking for a file or directory. I tried with wildcard too but no success either.
c
I didn't found this way to exclude in the doc I've found
see note towards end of this section https://www.pantsbuild.org/2.21/docs/using-pants/key-concepts/targets-and-build-files#dependencies-field 😉
I don't know your project well enough, but if you run 
pants dependencies some/path/file.py
you can see what that file is pulling in. Not sure where the 
addok
is coming from. I guess it's a requirement from your 
pyproject.toml
file, and if so, there's likely some code that imports from it, otherwise it shouldn't be in the sandbox..?
you can also try 
pants paths --from=':poetry#addok' --to=some/test_file.py
to find out the link, if any.. (guessing on the target addresses there..)
2 Views
Open in Slack
PreviousNext
Powered by