I m running into ``` ResolveError No owning targets could be Pants #general

I'm running into ```... ResolveError: No owning t...

quaint-gold-40000

07/26/2021, 1:13 PM

I'm running into

Copy code

...

ResolveError: No owning targets could be found for the file `dir/package-a/tests/conftest.py`.

Please check that there is a BUILD file in the parent directory dir/package-a/tests with a target whose `sources` field includes the file. See <https://www.pantsbuild.org/v2.6/docs/targets> for more information on target definitions.

quaint-gold-40000

07/26/2021, 1:13 PM

Full output:

Copy code

~/Repo develop $ ./pants test "dir/package-a::"
15:00:52.67 [WARN] Unmatched globs from dir/package-a's `sources` field: ["dir/package-a/*.py", "dir/package-a/*.pyi"], excludes: ["dir/package-a/*_test.py", "dir/package-a/*_test.pyi", "dir/package-a/conftest.py", "dir/package-a/test_*.py", "dir/package-a/test_*.pyi", "dir/package-a/tests.py", "dir/package-a/tests.pyi"]

Do the file(s) exist? If so, check if the file(s) are in your `.gitignore` or the global `pants_ignore` option, which may result in Pants not being able to see the file(s) even though they exist on disk. Refer to <https://www.pantsbuild.org/v2.6/docs/troubleshooting#pants-cannot-find-a-file-in-your-project>.
15:00:52.73 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
  File "/home/user/.cache/pants/setup/bootstrap-Linux-x86_64/2.6.0rc2_py39/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 234, in _run_inner
    return self._perform_run(goals)
  File "/home/user/.cache/pants/setup/bootstrap-Linux-x86_64/2.6.0rc2_py39/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 173, in _perform_run
    return self._perform_run_body(goals, poll=False)
  File "/home/user/.cache/pants/setup/bootstrap-Linux-x86_64/2.6.0rc2_py39/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 190, in _perform_run_body
    return self.graph_session.run_goal_rules(
  File "/home/user/.cache/pants/setup/bootstrap-Linux-x86_64/2.6.0rc2_py39/lib/python3.9/site-packages/pants/init/engine_initializer.py", line 135, in run_goal_rules
    exit_code = self.scheduler_session.run_goal_rule(
  File "/home/user/.cache/pants/setup/bootstrap-Linux-x86_64/2.6.0rc2_py39/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 533, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/home/user/.cache/pants/setup/bootstrap-Linux-x86_64/2.6.0rc2_py39/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 501, in _raise_on_error
    raise ExecutionError(

Exception message: 1 Exception encountered:

  ResolveError: No owning targets could be found for the file `dir/package-a/tests/conftest.py`.

Please check that there is a BUILD file in the parent directory dir/package-a/tests with a target whose `sources` field includes the file. See <https://www.pantsbuild.org/v2.6/docs/targets> for more information on target definitions.

You may want to run `./pants tailor` to autogenerate your BUILD files. See <https://www.pantsbuild.org/v2.6/docs/create-initial-build-files>.



(Use --print-stacktrace for more error details and/or --no-process-execution-local-cleanup to inspect chroots and/or -ldebug for more logs. See <https://www.pantsbuild.org/v2.6/docs/troubleshooting> for common issues. Consider reaching out for help: <https://www.pantsbuild.org/v2.6/docs/getting-help.>)

quaint-gold-40000

07/26/2021, 1:15 PM

Context:

Copy code

dir/
  package-a/
    src/
      ...
    tests/
      conftest.py
      test_something.py
    BUILD
pants.toml

That BUILD file:

Copy code

python_library()

python_tests(name="tests", sources=["tests/test_*.py"])

fast-nail-55400

07/26/2021, 1:19 PM

The exclusion of

conftest.py

from

python_libary

(and its assignment to

python_tests

) appears to only work if it is in the same directory as the

python_tests

target. See https://github.com/pantsbuild/pants/blob/ddc3c3d791178a610a9b8988cb74ac072bdac9c0/src/python/pants/backend/python/target_types.py#L412 where it does not have a

**

. Try moving the

python_tests

target to a

BUILD

file in

dir/package-a/tests

and it should work.

fast-nail-55400

07/26/2021, 1:20 PM

Invoking the test as `./pants test "dir/package-a::"`should still work since

::

in a spec will find all targets deeper in the directory tree.

quaint-gold-40000

07/26/2021, 1:23 PM

Thanks! So I'll have two BUILD files: one with

python_library()

and another with

python_tests()

? Just wondering whether a link between the code and tests is then somehow lost?

quaint-gold-40000

07/26/2021, 1:32 PM

Running the tests can't find the package-a package though...

fast-nail-55400

07/26/2021, 1:46 PM

Dependency inference should find dependencies, but I am unsure how it operates in this case.

fast-nail-55400

07/26/2021, 1:47 PM

This is probably an edge case. One of the other maintainers will have more knowledge, and should be able to help later in the day once they are online.

happy-kitchen-89482

07/26/2021, 6:22 PM

If I'm reading this correctly, the BUILD file is in the parent directory of

src

and

tests

. The default sources for the

python_library()

are non-recursive

*.py

(minus test-related files), so no files in your

src

directory are currently covered by a target (unless there's a BUILD file in

src

that you didn't mention), and neither is

conftest.py

, since you provided explicit sources in your

python_tests

target that don't include

conftest.py

happy-kitchen-89482

07/26/2021, 6:23 PM

Possibly the defaults for targets should be recursive, but that is debatable, and anyway they aren't right now...

happy-kitchen-89482

07/26/2021, 6:24 PM

If you delete your BUILD file and run

./pants tailor dir/package_a

it will generate BUILD files that should just work.

👍 1

happy-kitchen-89482

07/26/2021, 6:25 PM

But if you prefer to have that single high-level BUILD file then you can modify the

sources

manually

happy-kitchen-89482

07/26/2021, 6:26 PM

You'll probably want

Copy code

python_library(sources=['src/**/*.py'])

python_tests(name="tests", sources=["tests/**/*.py"])

Then

conftest.py

will be captured by the

python_tests

target.

happy-kitchen-89482

07/26/2021, 6:27 PM

And each target will glob recursively too

happy-kitchen-89482

07/26/2021, 6:27 PM

I'm assuming that all code under

tests

is test-related and no code under

src

is, so you don't need to worry about separating tests from non-tests by looking at a

test_

prefix of

_test

suffix and so on...

happy-kitchen-89482

07/26/2021, 6:28 PM

Let us know here if this solves the issue!

hundreds-father-404

07/26/2021, 6:49 PM

that glob will bite you the moment you add a test util to tests/, as pants will try to run Pytest on it and it will error So you'd have to be strict about all test utils living in the src/ folder. Instead, this glob should do it

Copy code

python_library(sources=['**/*.py', '!**/*_test.py', '!**/conftest.py'])
python_tests(name="tests", sources='**/*_test.py', '**/conftest.py'])

witty-crayon-22786

07/26/2021, 7:28 PM

but yea, as @happy-kitchen-89482 mentioned, using

tailor

is recommended rather than maintaining disjoint globs: see https://pantsbuild.slack.com/archives/C046T6T9U/p1627323890454200?thread_ts=1627305207.450800&cid=C046T6T9U

👍 1

witty-crayon-22786

07/26/2021, 7:30 PM

see https://www.pantsbuild.org/docs/create-initial-build-files

👍 1

quaint-gold-40000

07/27/2021, 8:08 AM

This got it working:

You'll probably want

python_library(sources=['src/**/*.py'])

python_tests(name="tests", sources=["tests/**/*.py"])

Then
conftest.py
will be captured by the
python_tests
target.

But dependency inference doesn't work for PyYAML (

import yaml

quaint-gold-40000

07/27/2021, 8:26 AM

Trying to add the dependency manually (for a library without an

src

dir):

Copy code

python_library(
    sources=["**/*.py"],
    dependencies=["mymodule/foo.py:PyYAML"],
)

yields

Copy code

ResolveError: The file or directory 'mymodule/fo.py' does not exist on disk in the workspace, so the address 'mymodule/fo.py:PyYAML' cannot be resolved.

EDIT: figured this out (more below)

quaint-gold-40000

07/27/2021, 8:35 AM

It's not clear to me from the docs whether I need to have a requirements.txt or not. It seemed that dependency inference would take care of that, and that a requirements.txt would optionally add version constraints.

quaint-gold-40000

07/27/2021, 8:40 AM

With a requirements.txt and BUILD with

python_requirements

in place,

./pants dependencies dir/package-b/module.py

does finally list PyYAML and the tests run.

quaint-gold-40000

07/27/2021, 8:43 AM

Import statements inside functions are not detected by dependency inference?

quaint-gold-40000

07/27/2021, 8:54 AM

I misunderstood how to manually set dependencies.

python_requirements()

creates new "3rd party" targets, which I need to reference.

quaint-gold-40000

07/27/2021, 9:03 AM

I had to kill pantsd for a new dependency added to requirements.txt to be picked up.

quaint-gold-40000

07/27/2021, 9:09 AM

Still have the issue with package-a (with

src

dir) that the tests fail to import _package_a_ (in conftest.py).

Copy code

E   ModuleNotFoundError: No module named 'package_a'

quaint-gold-40000

07/27/2021, 9:17 AM

Copy code

$ ./pants dependencies "dir/package-a/tests/conftest.py"
dir/package-a/src/package_a/database.py:../../package-a
dir:Sphinx

quaint-gold-40000

07/27/2021, 9:22 AM

Now trying autogenerated BUILD files:

If you delete your BUILD file and run
./pants tailor dir/package_a
it will generate BUILD files that should just work.

quaint-gold-40000

07/27/2021, 9:23 AM

./pants tailor dir/package_a

generates BUILD files everywhere, not just in dir/package-a. Is that expected?

quaint-gold-40000

07/27/2021, 9:30 AM

When running the tests for package-a, package_a can be imported now.

Copy code

$ ./pants list "::"
dir
dir:PyYAML
dir:Sphinx
dir:requirements.txt
dir:setuptools
dir/package_b
dir/package_b/package_b
dir/package_b/tests
dir/package-a/src/package_a
dir/package-a/tests

quaint-gold-40000

07/27/2021, 10:33 AM

Tests/code that uses _pkg_resources_ or _importlib_metadata_ fail, very likely because the packages under test aren't being installed. Is there any way to get that working?

witty-crayon-22786

07/27/2021, 5:57 PM

I had to kill pantsd for a new dependency added to requirements.txt to be picked up.

this is an unfortunate bug, sorry about that. 1) you shouldn’t need the `requirements.txt`: the poetry

pyproject.toml

is an alternative to a requirements file (although point 2 still applies). 2) there is supposed to be a hack in place to restart

pantsd

for

requirements.txt

pyproject.toml

changes to work around this bug, but it only matches files named

3rdparty/**/requirements.txt

requirements.txt

(at the root): you can add config like the following to the

[GLOBAL]

section of

pants.toml

to work around it:

Copy code

pantsd_invalidation_globs.add = [
  "/the/path/to/my/requirements.txt",
]

happy-kitchen-89482

07/27/2021, 10:48 PM

Sounds like we could improve the docs a bit, to clarify that

requirements.txt

or Poetry stanzas in

pyproject.toml

are how you specify the universe of possible 3rd-party dependencies, from which dep inference selects.

👍 1

happy-kitchen-89482

07/27/2021, 10:49 PM

Also re

./pants tailor dir/package_a
generates BUILD files everywhere, not just in dir/package-a. Is that expected?

Which version of Pants are you on?

hundreds-father-404

07/27/2021, 10:49 PM

cc @steep-breakfast-98857, who we talked about looking into that and having a tool tip about multipe pyproject.tomls in the same project

happy-kitchen-89482

07/27/2021, 10:52 PM

Oh, looks like support for running

tailor

on just a subdir was not cherry-picked into 2.6.x, so it will only be available in 2.7.0

happy-kitchen-89482

07/27/2021, 10:52 PM

The first dev release of which will happen this week

quaint-gold-40000

07/28/2021, 9:25 AM

I'm on Pants 2.6.0rc2

happy-kitchen-89482

07/30/2021, 5:38 PM

Yeah, alas, will have to wait for 2.7.x

14 Views

Open in Slack

Previous Next