Hello it’s me again 😄, so after some tinkering with the settings and removing `[source].root_patterns = ["src"]`it looks like Pants is trying to build PEX-files for my tests (YEAY 🙌 ) when I run

./pants test src/core/tests/test_companies.py

for example. Question #1: In the example-django repo, the tests were placed in a

models_test.py

. When I have nested testing folders for submodules of a module, what is the best way to formulate a BUILD file inside of the module? Do I adjust the

core

BUILD file with

sources=["tests/**/test_*.py"]

or should I not worry at all and let tailor create all the BUILD-files? Question #2: I’m getting an error which I think tells me that I need to configure

root_patterns

correctly. The error is:

Exception: String("Can only merge Directories with no duplicates, but found 2 duplicate entries in :\n\n`pyproject.toml` 1.) file digest=9d7d5d9df3e4ae5a4842a380891776c10adfc04ffdf0d48728db8ff4e7a05943 size=1006 ... 2.) file digest=55e6c945374e7d8f58b1d90e4dc00dee93a906850e953db2512938ead8b84fef size=994

I’m gonna post an example of our project-structure inside the thread here to not spam #general too much. The question is: How do I setup

source-roots

correctly? I’ve read the documentation about source-roots a couple of times now, but I’m not really sure what’s the optimal way. Maybe this error is again me missing an error inside of our

pyproject.toml

, but I hope it’s not 😅 .

Generally you would have a

BUILD

file in each module’s directory with

python_sources()

(and

python_tests(name="tests")

as applicable. Note: If you run

./pants tailor

, then Pants will write out these

BUILD

files for you automatically.

Here’s a simplified example of the directory structure:

src/ - This is the django root folder
  api/
  core/
    models/
      tests/
        test_companies.py - company unit-tests
      companies.py
    tests/
      test_companies.py - company integration-tests
    admin.py
    apps.py
    tasks.py
  another_module1/
  another_module2/

I’d like to be able to import like this

from core.models import Company

. In the past we adjusted the

__init__.py

inside of the

core/models

folder to look like this:

__all__ = [
    # companies.py
    "Company",
]

from .companies import (
    Company
)

As I understand Pants, this adjustment could be a thing of the past. But that’s a question for another time. As always: Help is highly appreciated 💚

Do you have multiple

pyproject.toml

at all?

No, I think that’s the roots reading

.

and

src/../

I assume that venv is not checked into source control. Can you add

src/venv/**

to the --pants-ignore option?

I did add

"/src/venv/**",

to the

pants_ignore

array but it didn’t change the outcome of the command.

The maintainers with more experience with Pants/Python should be online once work week starts again (so tomorrow).

Here are the contents of my

pyproject.toml

if that does matter to somebody: https://pastebin.com/QUz5bT35

It shouldn't be a problem to have a source root inside another source root. For a given file, pants will take the longest prefix that matches

Here is the complete stacktrace:

❯ ./pants test src/core/tests/test_companies.py --print-stacktrace
07:36:56.15 [INFO] Initialization options changed: reinitializing scheduler...
07:36:56.81 [INFO] Scheduler initialized.
07:37:00.77 [ERROR] 1 Exception encountered:

Engine traceback:
  in select
  in pants.core.goals.test.run_tests
  in pants.backend.python.goals.pytest_runner.run_python_test (src/core/tests/test_companies.py:../tests)
  in pants.backend.python.goals.pytest_runner.setup_pytest_for_target
Traceback (no traceback):
  <pants native internals>
Exception: String("Can only merge Directories with no duplicates, but found 2 duplicate entries in :\n\n`pyproject.toml`: 1.) file digest=9d7d5d9df3e4ae5a4842a380891776c10adfc04ffdf0d48728db8ff4e7a05943 size=1006:\n\n[tool.black]\nline-length = 100\ntarget-version = [ \"py38\",]\ninclude = \"\\\\.pyi?$\"\nforce-exclude = \"/(\\n    .git\\n  | .hg\\n  | .eggs\\n  | .mypy_cache\\n  | .tox\\n  | .venv\\n  | .venv\\n  | _build\\n  | buck-out\\n  | build\\n  | dist\\n  | migrations\\n)/\\n\"\n\n[tool.isort]\nprofile = \"black\"\nline_length = 100\nsections = [ \"FUTURE\", \"STDLIB\", \"THIRDPARTY\", \"FIRSTPARTY\", \"LOCALFOLDER\",]\nno_lines_before = \"LOCALFOLDER\"\nskip_glob = [ \"*/migrations/*.py\",]\nsrc_paths = [ \"src\", \".\",]\nknown_thirdparty = [ \"django\",]\n\n[tool.coverage.run]\nomit = [ \"*/tests/*\", \"*/migrations/*\", \"*/prototypes/*\", \"*/site-packages/*\", \"*/test_*\", \"scripts/*\", \"docker/*\", \"config/*\", \"pytest.pex/*\",]\nrelative_files = true\n\n[tool.coverage.report]\nprecision = 2\n\n[tool.pytest.ini_options]\npython_files = \"tests.py test_*.py *_tests.py\"\nDJANGO_SETTINGS_MODULE = \"config.settings_testing\"\nmarkers = [ \"slow: marks tests as slow (deselect with '-m \\\"not slow\\\"')\", \"jwt_scope: sets required scope for api test\",]\nenv = [ \"ENVIRONMENT=test\",]\n\n\n`pyproject.toml`: 2.) file digest=55e6c945374e7d8f58b1d90e4dc00dee93a906850e953db2512938ead8b84fef size=994:\n\n[tool.black]\nline-length = 100\ntarget-version = ['py38']\ninclude = '\\.pyi?$'\nforce-exclude = \"\"\"\n/(\n    .git\n  | .hg\n  | .eggs\n  | .mypy_cache\n  | .tox\n  | .venv\n  | .venv\n  | _build\n  | buck-out\n  | build\n  | dist\n  | migrations\n)/\n\"\"\"\n\n[tool.isort]\nprofile = \"black\"\nline_length = 100\nsections = [\"FUTURE\", \"STDLIB\", \"THIRDPARTY\", \"FIRSTPARTY\", \"LOCALFOLDER\"]\nno_lines_before = \"LOCALFOLDER\"\nskip_glob = [\"*/migrations/*.py\"]\nsrc_paths = [\"src\", \".\"]\nknown_thirdparty = [\"django\"]\n\n[tool.coverage.run]\nomit = [\n    \"*/tests/*\",\n    \"*/migrations/*\",\n    \"*/prototypes/*\",\n    \"*/site-packages/*\",\n    \"*/test_*\",\n    \"scripts/*\",\n    \"docker/*\",\n    \"config/*\",\n]\n\n[tool.coverage.report]\nprecision = 2\n\n[tool.pytest.ini_options]\npython_files = \"tests.py test_*.py *_tests.py\"\nDJANGO_SETTINGS_MODULE= \"config.settings_testing\"\nmarkers = [\n    \"slow: marks tests as slow (deselect with '-m \\\"not slow\\\"')\",\n    \"jwt_scope: sets required scope for api test\",\n]\nenv = [\n    \"ENVIRONMENT=test\",\n]\n")

Does that section exist in both pyproject.tomls?

I think I only have one pyproject.toml 😅

Oh, in that case something truly silly is happening 🙂

It really feels like I‘m pushing the wrong buttons 😅. Thank you very much!

Extremely weird that these have different digests and sizes...

That’s what I was confused about as well. When I replaced the

\\n

with new-lines both files seemed identical.

When I look at the error above, I see, e.g., that one file has

target-version = [ \"py38\",]

and the other has

target-version = ['py38']

They are actually single quotes, I’m gonna try changing to double-quotes to see if that helps 😄

Yes, we detect that

pyproject.toml

is relevant to

pytest

runs by looking for a

tool.pytest

table in the file

Which in turn explains why regexing the file could be a problem, because it contains a regex itself (?)

One with single quotes, one with double quotes

Does this somehow parse the file?

No, that just fetches the file content

Could this somehow be related to the Snapshot mechanism? As we’re seeing two different files = actual content vs. maybe snapshot content

No, Snapshot just grabs file content

I just checked again:

❯ find . -name 'pyproject.toml' | xargs wc -l
      44 ./pyproject.toml
      13 ./src/venv/lib/python3.9/site-packages/spacy/tests/package/pyproject.toml
      57 total

OK

Sure I can try

Thanks!