https://pantsbuild.org/ logo
Join Slack
Powered by
New user! I’m trying to get my monorepo’s pytest s...
# general
p
New user! I’m trying to get my monorepo’s pytest suite to run via 
./pants test
Adding BUILD files a little at a time since I hit a bug when `tailor`ing. the whole project, so initially I have things configured to run just one test file. When I run 
./pants filter --target-type=python_test ::
I see the output I expect since this is the directory where I added a BUILD file with python_test(…):
Copy code
sundial_utils/tests/sundial_utils/analytics:analytics
But if I run 
./pants test
the test file inside that analytics subdirectory (named 
test_rando.py
) is not found, despite the fact that its name conforms to the glob pattern described in the “Unmatched globs” error message:
Copy code
Unmatched globs from sundial_utils/src/sundial_utils/analytics:analytics's `sources` field: ["sundial_utils/src/sundial_utils/analytics/*_test.py", "sundial_utils/src/sundial_utils/analytics/test_*.py", "sundial_utils/src/sundial_utils/analytics/tests.py"]
Not sure what to try to get this working.
w
p
No, it’s tracked in git
One thing that seems a bit “off” about that error message (or maybe it’s expected) is that all the paths listed are from the monorepo root instead of from the subproject root. My repo structure is like this, with parallel structure between the 
src
and 
tests
directories for each project in the monorepo:
Copy code
repo
  projectA
    src
      projectA
        ... packages ...
    tests
      projectA
        ... packages ...
  projectB
    src
      projectB
        ... packages ...
    tests
      projectB
        ... packages ...
  projectC
    src
      projectC
        ... packages ...
    tests
      projectC
        ... packages ...
So, lots of nesting and use of subdirectories for namespacing imports. My pants.toml looks like this to configure every 
src
and 
tests
directory as roots:
Copy code
[GLOBAL]
pants_version = "2.10.0"
use_deprecated_python_macros = false

backend_packages = [
    "pants.backend.python"
]

[source]
root_patterns = [
    "src",
    "tests"
]
[anonymous-telemetry]
enabled = false
The reason I mention the layout is that the “unmatched globs” error message shows paths from the monorepo root not the source roots that I configured.
(Thank you and @fast-nail-55400 for jumping right in to help, by the way!)
w
ah, yea: complete path names are expected… “namespacing is good, let’s do more of that!“. but relatedly, Pants doesn’t currently support running from a subdirectory: if it did, messages might look different.
h
Hmm, can you post the BUILD file in question?
w
he did in the other thread: sorry: it’s split.
replying here:
so: note that source globs are not recursive. if you have a definition like
Copy code
python_tests(
    name="sundial_utils_tests"
)
at the “root” of something, there would need to be files in the same directory in order for it to do anything
h
Your root_patterns look right to me, and in any case, that shouldn't cause this kind of problem, possibly other kinds...
p
w
getting 
tailor
working (https://github.com/pantsbuild/pants/issues/15276) is probably the best path, because manually creating BUILD files is… so last year 😃
h
Am I reading correctly that there is a 
python_tests()
in the source root and then also a 
python_tests()
in a subdirectory of that source root?
p
yeah, that’s what I’ve got.
h
Are there test .py files directly in that source root?
Wondering what the purpose of that
Copy code
python_tests(
    name="sundial_utils_tests"
)
is
I can't offhand think of why that should cause problems, but you never know
w
@plain-monkey-49051: are you able to share the first few lines of the problematic generated 
BUILD
file from https://github.com/pantsbuild/pants/issues/15276 ?
…actually, i’ll just respond there. sorry for the delay.
p
Regarding the BUILD files generated by tailor they were all just one-liners with 
python_requirements()
w
@plain-monkey-49051: you should have had 
python_sources
and 
python_tests
targets generated: in the common case, 0 hand edits should be necessary
p
@happy-kitchen-89482 no there were no py files at that root. And if I delete it I don’t get the “unmatched globs” error… but I don’t see any sign that the test ran either. Just a silent return.
h
The unmatched globs part makes sense, that target wasn't matching anything.
✅ 1
p
@witty-crayon-22786 I could try running tailor again and see if the bug recurs.
h
And what was your full pants command?
Just to be sure I'm on the same page
p
./pants test
h
from the repo root?
You do have to give it some targets to run on
w
@happy-kitchen-89482: we should probably both respond on https://github.com/pantsbuild/pants/issues/15276… it has more context.
h
./pants test
alone indeed does nothing (perhaps unintuitively)
@witty-crayon-22786 that seems like a separate issue?
What happens if you 
./pants test path/to/foo_test.py
?
p
If I give it a path from repo root I get a ModuleNotFound on the import. That does’t happen if I just execute 
pytest
from the root: the test suite runs fine.
Does Pants read from pyproject.toml to get things like the pythonpath? It’s acting as if not.
[noting that we’ve now migrated to a new issue and thanking you both for helping me understand and fix the first one] 🙏
w
p
Oh I see the problem with the ModuleNotFound: I’m using pytest with 
import-mode = "importlib"
which is only supported by pytest >= 7.0 and it looks like Pants is running an older version.
@witty-crayon-22786’s intuition was right: the tailor issue was a couple of requirements.txt files that were not ready for prime time: https://github.com/pantsbuild/pants/issues/15276#issuecomment-1113871160. For some reason after having tailor ignore those paths even the ModuleNotFound issue mentioned above went away and I can get a passing test run. Thanks and have a good weekend! 🙌
w
Ah, awesome. Yea, the 
python_requirements
macro creates targets for the 
requirements.txt
, and then dependency inference picks those up automatically.
Have a good weekend!
Open in Slack
PreviousNext
Powered by