https://pantsbuild.org/ logo
#general
Title
# general
p

plain-monkey-49051

04/29/2022, 10:21 PM
New user! I’m trying to get my monorepo’s pytest suite to run via
./pants test
Adding BUILD files a little at a time since I hit a bug when `tailor`ing. the whole project, so initially I have things configured to run just one test file. When I run
./pants filter --target-type=python_test ::
I see the output I expect since this is the directory where I added a BUILD file with python_test(…):
Copy code
sundial_utils/tests/sundial_utils/analytics:analytics
But if I run
./pants test
the test file inside that analytics subdirectory (named
test_rando.py
) is not found, despite the fact that its name conforms to the glob pattern described in the “Unmatched globs” error message:
Copy code
Unmatched globs from sundial_utils/src/sundial_utils/analytics:analytics's `sources` field: ["sundial_utils/src/sundial_utils/analytics/*_test.py", "sundial_utils/src/sundial_utils/analytics/test_*.py", "sundial_utils/src/sundial_utils/analytics/tests.py"]
Not sure what to try to get this working.
w

witty-crayon-22786

04/29/2022, 10:32 PM
p

plain-monkey-49051

04/29/2022, 10:46 PM
No, it’s tracked in git
One thing that seems a bit “off” about that error message (or maybe it’s expected) is that all the paths listed are from the monorepo root instead of from the subproject root. My repo structure is like this, with parallel structure between the
src
and
tests
directories for each project in the monorepo:
Copy code
repo
  projectA
    src
      projectA
        ... packages ...
    tests
      projectA
        ... packages ...
  projectB
    src
      projectB
        ... packages ...
    tests
      projectB
        ... packages ...
  projectC
    src
      projectC
        ... packages ...
    tests
      projectC
        ... packages ...
So, lots of nesting and use of subdirectories for namespacing imports. My pants.toml looks like this to configure every
src
and
tests
directory as roots:
Copy code
[GLOBAL]
pants_version = "2.10.0"
use_deprecated_python_macros = false

backend_packages = [
    "pants.backend.python"
]

[source]
root_patterns = [
    "src",
    "tests"
]
[anonymous-telemetry]
enabled = false
The reason I mention the layout is that the “unmatched globs” error message shows paths from the monorepo root not the source roots that I configured.
(Thank you and @fast-nail-55400 for jumping right in to help, by the way!)
w

witty-crayon-22786

04/29/2022, 11:08 PM
ah, yea: complete path names are expected… “namespacing is good, let’s do more of that!“. but relatedly, Pants doesn’t currently support running from a subdirectory: if it did, messages might look different.
h

happy-kitchen-89482

04/29/2022, 11:10 PM
Hmm, can you post the BUILD file in question?
w

witty-crayon-22786

04/29/2022, 11:10 PM
he did in the other thread: sorry: it’s split.
replying here:
so: note that source globs are not recursive. if you have a definition like
Copy code
python_tests(
    name="sundial_utils_tests"
)
at the “root” of something, there would need to be files in the same directory in order for it to do anything
h

happy-kitchen-89482

04/29/2022, 11:10 PM
Your root_patterns look right to me, and in any case, that shouldn't cause this kind of problem, possibly other kinds...
w

witty-crayon-22786

04/29/2022, 11:11 PM
getting
tailor
working (https://github.com/pantsbuild/pants/issues/15276) is probably the best path, because manually creating BUILD files is… so last year 😃
h

happy-kitchen-89482

04/29/2022, 11:12 PM
Am I reading correctly that there is a
python_tests()
in the source root and then also a
python_tests()
in a subdirectory of that source root?
p

plain-monkey-49051

04/29/2022, 11:13 PM
yeah, that’s what I’ve got.
h

happy-kitchen-89482

04/29/2022, 11:14 PM
Are there test .py files directly in that source root?
Wondering what the purpose of that
Copy code
python_tests(
    name="sundial_utils_tests"
)
is
I can't offhand think of why that should cause problems, but you never know
w

witty-crayon-22786

04/29/2022, 11:14 PM
@plain-monkey-49051: are you able to share the first few lines of the problematic generated
BUILD
file from https://github.com/pantsbuild/pants/issues/15276 ?
…actually, i’ll just respond there. sorry for the delay.
p

plain-monkey-49051

04/29/2022, 11:16 PM
Regarding the BUILD files generated by tailor they were all just one-liners with
python_requirements()
w

witty-crayon-22786

04/29/2022, 11:19 PM
@plain-monkey-49051: you should have had
python_sources
and
python_tests
targets generated: in the common case, 0 hand edits should be necessary
p

plain-monkey-49051

04/29/2022, 11:22 PM
@happy-kitchen-89482 no there were no py files at that root. And if I delete it I don’t get the “unmatched globs” error… but I don’t see any sign that the test ran either. Just a silent return.
h

happy-kitchen-89482

04/29/2022, 11:23 PM
The unmatched globs part makes sense, that target wasn't matching anything.
1
p

plain-monkey-49051

04/29/2022, 11:23 PM
@witty-crayon-22786 I could try running tailor again and see if the bug recurs.
h

happy-kitchen-89482

04/29/2022, 11:23 PM
And what was your full pants command?
Just to be sure I'm on the same page
p

plain-monkey-49051

04/29/2022, 11:23 PM
./pants test
h

happy-kitchen-89482

04/29/2022, 11:24 PM
from the repo root?
You do have to give it some targets to run on
w

witty-crayon-22786

04/29/2022, 11:25 PM
@happy-kitchen-89482: we should probably both respond on https://github.com/pantsbuild/pants/issues/15276… it has more context.
h

happy-kitchen-89482

04/29/2022, 11:25 PM
./pants test
alone indeed does nothing (perhaps unintuitively)
@witty-crayon-22786 that seems like a separate issue?
What happens if you
./pants test path/to/foo_test.py
?
p

plain-monkey-49051

04/29/2022, 11:29 PM
If I give it a path from repo root I get a ModuleNotFound on the import. That does’t happen if I just execute
pytest
from the root: the test suite runs fine.
Does Pants read from pyproject.toml to get things like the pythonpath? It’s acting as if not.
[noting that we’ve now migrated to a new issue and thanking you both for helping me understand and fix the first one] 🙏
w

witty-crayon-22786

04/29/2022, 11:36 PM
p

plain-monkey-49051

04/29/2022, 11:36 PM
Oh I see the problem with the ModuleNotFound: I’m using pytest with
import-mode = "importlib"
which is only supported by pytest >= 7.0 and it looks like Pants is running an older version.
@witty-crayon-22786’s intuition was right: the tailor issue was a couple of requirements.txt files that were not ready for prime time: https://github.com/pantsbuild/pants/issues/15276#issuecomment-1113871160. For some reason after having tailor ignore those paths even the ModuleNotFound issue mentioned above went away and I can get a passing test run. Thanks and have a good weekend! 🙌
w

witty-crayon-22786

04/30/2022, 12:30 AM
Ah, awesome. Yea, the
python_requirements
macro creates targets for the
requirements.txt
, and then dependency inference picks those up automatically.
Have a good weekend!