Trying to upgrade to `2 8 0` and I m running into some troub Pants #general

Trying to upgrade to `2.8.0` and I'm running into ...

dazzling-diamond-4749

11/18/2021, 6:14 AM

Trying to upgrade to

2.8.0

and I'm running into some trouble with conftest My conftest would complain about missing module, if I run

./pants test path/to/my/test.py:tests

But when I run the same target in

./pants repl path/to/my/test.py:tests

and import the same missing module, python is able to find the module.

dazzling-diamond-4749

11/18/2021, 6:16 AM

conftest is located in

project/test/python/package/conftest.py

and the BUILD looks like

Copy code

python_test_utils()

python_tests(
    name="tests0",
    dependencies=[
        stuff...
    ],
)

the test is in

project/test/python/package/sub_pkg/test.py

the BUILD looks like

Copy code

python_tests(
    name="tests",
)

hundreds-father-404

11/18/2021, 6:31 AM

You might need to put some of those dependencies onto the python_test_utils target. If they are deps that belong to the conftest.py, they belong there I'm really sorry about how this particular change breaks our deprecation policy 😞 the modeling has been broken since pants 1 of claiming that conftest.py is a test itself, and it was going to keep making more and more problems, especially when we roll out multiple lockfiles. We spent two days trying to find alternatives that wouldn't break things, and this was the least offensive

dazzling-diamond-4749

11/18/2021, 6:51 AM

I see, but couldn't context infer those deps? Like how

python_sources

can infer deps?

dazzling-diamond-4749

11/18/2021, 7:09 AM

hmm, adding deps didn't help

Copy code

➜  pynest git:(rmeng/pants28-round2) ✗ ./pants test ml/cbml/test/python/cbml/util/test_sql_parser.py:tests
23:07:00.84 [ERROR] Completed: Run Pytest - ml/cbml/test/python/cbml/util/test_sql_parser.py:tests failed (exit code 4).
ImportError while loading conftest '/private/var/folders/28/6tcr0h8j1b3485bqvp8pwr6m0000gp/T/process-executionmerCJe/ml/cbml/test/python/cbml/conftest.py'.
ml/cbml/test/python/cbml/conftest.py:17: in <module>
    from cbml.dataio.loading import load_sql
E   ModuleNotFoundError: No module named 'cbml.dataio'



𐄂 ml/cbml/test/python/cbml/util/test_sql_parser.py:tests failed.

➜  pynest git:(rmeng/pants28-round2) ✗ ./pants filedeps --transitive ml/cbml/test/python/cbml/util/test_sql_parser.py:tests | grep ml/cbml/src/python/cbml/dataio
ml/cbml/src/python/cbml/dataio/BUILD
ml/cbml/src/python/cbml/dataio/emr.py
ml/cbml/src/python/cbml/dataio/loading.py
ml/cbml/src/python/cbml/dataio/s3.py
ml/cbml/src/python/cbml/dataio/snowflake.py
ml/cbml/src/python/cbml/dataio/tarutil.py

➜  pynest git:(rmeng/pants28-round2) ✗ ./pants filedeps --transitive ml/cbml/test/python/cbml:utils | grep ml/cbml/src/python/cbml/dataio
ml/cbml/src/python/cbml/dataio/BUILD
ml/cbml/src/python/cbml/dataio/emr.py
ml/cbml/src/python/cbml/dataio/loading.py
ml/cbml/src/python/cbml/dataio/s3.py
ml/cbml/src/python/cbml/dataio/snowflake.py
ml/cbml/src/python/cbml/dataio/tarutil.py

Maybe conftest loading has some quirks? Like run before path is setup?

dazzling-diamond-4749

11/18/2021, 7:11 AM

ohhh, the

__init__.py

isn't getting picked up some how, but it's empty anyways

hundreds-father-404

11/18/2021, 8:29 AM

Hmmm init.py shouldn't have had any changes...are things working now?

hundreds-father-404

11/18/2021, 8:30 AM

And yes, conftest.py does have dependency inference. Certain dependencies might not be inferrable tho, like runtime dependencies like Pytest-django

dazzling-diamond-4749

11/18/2021, 7:27 PM

No 😞 It's still missing those modules even filedeps are correct

hundreds-father-404

11/18/2021, 7:30 PM

Hmmm. How about we try this workaround to help with debugging? It is still legal (albeit deprecated) in 2.8 to explicitly put

conftest.py

in a

python_tests

target. We only changed the default For now, comment out the

python_test_utils

target and explicitly set

sources=["conftest.py", "*_test.py"]

on your

python_tests

target (or whatever the relevant globs are)

dazzling-diamond-4749

11/18/2021, 9:00 PM

w88888, https://www.pantsbuild.org/docs/reference-python_test_utils#codesourcescode we use

test_*

. Maybe it's

python_test_utils

picking up tests causing pants to get confused

dazzling-diamond-4749

11/18/2021, 9:00 PM

let me try using python sources first

dazzling-diamond-4749

11/18/2021, 10:06 PM

ok, I've tried

Copy code

python_test_utils(
    sources=["conftest.py"],
)

python_tests(
    name="tests0",
    dependencies=[
        stuff...
    ],
)

and

Copy code

python_tests(
    name="tests0",
    sources=["conftest.py", "test_*"],
    dependencies=[
        stuff...
    ],
)

neither worked 😞

hundreds-father-404

11/18/2021, 10:07 PM

Hmm, that's really weird. I'm out sick today, but maybe would you be interested in pair programming for a few minutes tomorrow? I'd really love to understand this and unblock you from 2.8

🙌 1

🍵 1

dazzling-diamond-4749

11/18/2021, 10:08 PM

really appreciate the help. And yes, tomorrow works, as long as you are feeling better

❤️ 1

4 Views

Open in Slack

Previous Next