Hello Back from vacation I m looking at trying to upgrade pa Pants #general

Hello! Back from vacation. I'm looking at trying t...

worried-painter-31382

08/27/2023, 6:07 PM

Hello! Back from vacation. I'm looking at trying to upgrade pants to 2.16 from 2.15, but have hit a snag in pytest plugin loading and the new

install_from_resolve

. We have custom in-repo plugins loaded via

addopt

arguments in setup.cfg and a top level

conftest.py

that imports the plugin code to help dependency inference. With the upgrade however, the plugins seem to be loaded multiple times. Anyone experienced anything similiar?

👋 2

worried-painter-31382

08/27/2023, 6:08 PM

Copy code

20:03:50.45 [ERROR] Completed: Run Pytest - amra/atlas/operations/acceptance_tests/test_generate_machine_readable_profiler_4_reports.py:tests and 41 other files - failed (exit code 1).
Partition: default
...
"/home/tobiasn/.cache/pants/named_caches/pex_root/venvs/s/ceb221d6/venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 704, in consider_conftest
    self.register(conftestmodule, name=conftestmodule.__file__)
  File "/home/tobiasn/.cache/pants/named_caches/pex_root/venvs/s/ceb221d6/venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 463, in register
    ret: Optional[str] = super().register(plugin, name)
  File "/home/tobiasn/.cache/pants/named_caches/pex_root/venvs/s/ceb221d6/venv/lib/python3.10/site-packages/pluggy/manager.py", line 104, in register
    raise ValueError(
ValueError: Plugin already registered: /tmp/pants-sandbox-GxxjnS/./amra/conftest.py=<module 'amra.conftest' from '/tmp/pants-sandbox-GxxjnS/./amra/conftest.py'>
<lists of all plugins>

worried-painter-31382

08/27/2023, 6:17 PM

Additionally it seems to be happening depending on what files end up in the sandbox of a batch

worried-painter-31382

08/27/2023, 6:25 PM

Okey so it is def something to do with what files the sandbox ends up containing, got my hands on a __run.sh that reproduces so will drop 1 file at a time to suss it out

worried-painter-31382

08/27/2023, 6:34 PM

Seems as long as there's multiple input files, pytest attempts to load the same plugin multiple times.

worried-painter-31382

08/27/2023, 6:47 PM

Replacing arguments in

__run.sh

--trace-config

yields

Copy code

rootdir: /tmp/pants-sandbox-GxxjnS, configfile: setup.cfg
plugins: cov-2.10.1, anyio-3.5.0, xdist-2.1.0, asyncio-0.17.2, timeout-2.1.0, forked-1.1.3, alembic-verify-0.1.4
asyncio: mode=strict
collecting ... PLUGIN registered: <module 'amra.conftest' from '/tmp/pants-sandbox-GxxjnS/./amra/conftest.py'>
collected 0 items / 1 error                                                                                                                                                                                

================================================================================================== ERRORS ==================================================================================================
______________________________________________________________________________________ ERROR collecting test session _______________________________________________________________________________________
/home/tobiasn/.cache/pants/named_caches/pex_root/venvs/s/ceb221d6/venv/lib/python3.10/site-packages/_pytest/runner.py:338: in from_call
    result: Optional[TResult] = func()
/home/tobiasn/.cache/pants/named_caches/pex_root/venvs/s/ceb221d6/venv/lib/python3.10/site-packages/_pytest/runner.py:369: in <lambda>
    call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
/home/tobiasn/.cache/pants/named_caches/pex_root/venvs/s/ceb221d6/venv/lib/python3.10/site-packages/_pytest/main.py:723: in collect
    for x in self._collectfile(pkginit):
/home/tobiasn/.cache/pants/named_caches/pex_root/venvs/s/ceb221d6/venv/lib/python3.10/site-packages/_pytest/main.py:575: in _collectfile
    ihook = self.gethookproxy(fspath)
/home/tobiasn/.cache/pants/named_caches/pex_root/venvs/s/ceb221d6/venv/lib/python3.10/site-packages/_pytest/main.py:539: in gethookproxy
    my_conftestmodules = pm._getconftestmodules(
/home/tobiasn/.cache/pants/named_caches/pex_root/venvs/s/ceb221d6/venv/lib/python3.10/site-packages/_pytest/config/__init__.py:575: in _getconftestmodules
    mod = self._importconftest(conftestpath, importmode, rootpath)
/home/tobiasn/.cache/pants/named_caches/pex_root/venvs/s/ceb221d6/venv/lib/python3.10/site-packages/_pytest/config/__init__.py:620: in _importconftest
    assert mod not in mods
E   AssertionError
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================================= 1 error in 0.15s =============================================================================================

worried-painter-31382

08/27/2023, 6:49 PM

no idea what that means but first guess is somehow the venv in which pytest is invoked has already, somehow, encountered the conftest file

worried-painter-31382

08/27/2023, 7:00 PM

setup.cfg

containing

addopts

section pointing to plugins, when those are loaded all parent conftests are also found..?

worried-painter-31382

08/27/2023, 7:02 PM

removing the entries

Copy code

addopts =
    -p amra.<snip>
    -p amra.<snip>
    -p amra.<snip>

causes the tests to run, anyhow

broad-processor-92400

08/28/2023, 1:01 AM

FWIW, we configure pytest with:

Copy code

[pytest]
install_from_resolve = "default"
requirements = [
  "pytest",
  "pytest-asyncio",
...
]

And have no

addopts = -p ...

settings, like your last message suggests And don't have

worried-painter-31382

08/28/2023, 7:06 AM

Moving from

addopts = -p ....

pytest_plugins = (...)

inside the top-level conftest seem to work, but I dont particularily understand why.

3 Views

Open in Slack

Previous Next