https://pantsbuild.org/ logo
#general
Title
# general
c

cold-vr-15232

10/23/2022, 11:58 AM
What is the good way to structure a python monorepo so that mypy works well with test code? How should the roots be setup? How should the test utils be configured? The examples I found are for apps without tests, and I have issues with all variants that I tried. For example, if "tests" is not a package, mypy wants unique names of the files across all applications. If it is a package, then there are import problems. This is the structure I have
Copy code
apps
├── demo_app
│   ├── BUILD
│   ├── pyproject.toml
│   ├── src
│   │   └── demo_app
│   │       ├── BUILD
│   │       ├── __init__.py
│   │       ├── main.py
│   │       └── model.py
│   └── tests
│           ├── BUILD
│           ├── test_demo_app_main.py
│           └── util.py
and the roots are
Copy code
apps/demo_app/src
apps/demo_app/tests
I guess the question is related to @wide-midnight-78598’s one about more complete examples of how to setup a more complicated repo. While it's interesting to get through the process of figuring it out oneself, it is a lot of wasted time if everyone has to do that, and it might scare off newcomers.
👍 1
e

enough-analyst-54434

10/23/2022, 12:39 PM
What are the import errors you see?
c

cold-vr-15232

10/23/2022, 12:41 PM
It's a bit difficult to show everything here, I'll try to put together a demo repo
w

wide-midnight-78598

10/23/2022, 12:53 PM
@cold-vr-15232 If you can show the import errors from
mypy
- as I mentioned in another thread, like 90-95% of my mypy problems were solved with init.py's or using namespace packages. Not suggesting blindly that will work here, but it's been my Pants mypy go-to
While it's interesting to get through the process of figuring it out oneself, it is a lot of wasted time if everyone has to do that, and it might scare off newcomers
I agree with this sentiment - personally, I want to figure out how to solve these kinda things so that when they inevitably crop up again, I know where to start. But, if it's something that only crops up once every few months, then 🤷 The other side is - as interesting as it may be, fiddling with a build system isn't revenue-generating...
b

bitter-ability-32190

10/23/2022, 1:22 PM
FWIW the pants repo and our work repo use test files next to prod code ( E.g.
test_foo.py
next to
foo.py
) and we don't have any issues
w

wide-midnight-78598

10/23/2022, 1:29 PM
I use both - unit tests are placed alongside code, integration/e2e code is separated
c

cold-vr-15232

10/23/2022, 1:30 PM
Ok, I think I figured it out. The problem was that either mypy or pytest could not find files or found conflicting files. And I got it fixed by using different package names under src and tests, for example "src/app1" and "tests/app1_test". Not the prettiest, but works now. I haven't tried having the test code in the same place. I suppose pants recognizes that and doesn't package test file in the binary? How would you handle test utilities?
b

bitter-ability-32190

10/23/2022, 1:41 PM
Pants'dependency graph is precise. Unless you're importing test code, it'll never be packaged. Try it out!
./pants dependencies <target or file>
c

cold-vr-15232

10/23/2022, 2:00 PM
Then maybe I will switch to using tests alongside code too. Even integration tests could live under src, I suppose. Thanks! (A downside is that the whole project is now dependent on pants 🙂 )
😈 2
w

wide-midnight-78598

10/23/2022, 2:04 PM
pytest
(and I think
mypy
and other tools) will happily run against tests next to code. That's how I've been running everything in the pre-pants years.
But, maybe some tools will need to be globbed away or something... hmm
I suppose pants recognizes that and doesn't package test file in the binary
I think I missed this - in your
BUILD
, if you have
python_sources
and
python_tests
- you're fine. As Josh mentioned, unless you actively import test code (either in your
.py
files or your
dependencies
field in BUILD), it won't/shouldn't be packaged
8 Views