https://pantsbuild.org/ logo
#general
Title
# general
q

quaint-gold-40000

07/26/2021, 1:13 PM
I'm running into
Copy code
...

ResolveError: No owning targets could be found for the file `dir/package-a/tests/conftest.py`.

Please check that there is a BUILD file in the parent directory dir/package-a/tests with a target whose `sources` field includes the file. See <https://www.pantsbuild.org/v2.6/docs/targets> for more information on target definitions.
Full output:
Copy code
~/Repo develop $ ./pants test "dir/package-a::"
15:00:52.67 [WARN] Unmatched globs from dir/package-a's `sources` field: ["dir/package-a/*.py", "dir/package-a/*.pyi"], excludes: ["dir/package-a/*_test.py", "dir/package-a/*_test.pyi", "dir/package-a/conftest.py", "dir/package-a/test_*.py", "dir/package-a/test_*.pyi", "dir/package-a/tests.py", "dir/package-a/tests.pyi"]

Do the file(s) exist? If so, check if the file(s) are in your `.gitignore` or the global `pants_ignore` option, which may result in Pants not being able to see the file(s) even though they exist on disk. Refer to <https://www.pantsbuild.org/v2.6/docs/troubleshooting#pants-cannot-find-a-file-in-your-project>.
15:00:52.73 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
  File "/home/user/.cache/pants/setup/bootstrap-Linux-x86_64/2.6.0rc2_py39/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 234, in _run_inner
    return self._perform_run(goals)
  File "/home/user/.cache/pants/setup/bootstrap-Linux-x86_64/2.6.0rc2_py39/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 173, in _perform_run
    return self._perform_run_body(goals, poll=False)
  File "/home/user/.cache/pants/setup/bootstrap-Linux-x86_64/2.6.0rc2_py39/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 190, in _perform_run_body
    return self.graph_session.run_goal_rules(
  File "/home/user/.cache/pants/setup/bootstrap-Linux-x86_64/2.6.0rc2_py39/lib/python3.9/site-packages/pants/init/engine_initializer.py", line 135, in run_goal_rules
    exit_code = self.scheduler_session.run_goal_rule(
  File "/home/user/.cache/pants/setup/bootstrap-Linux-x86_64/2.6.0rc2_py39/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 533, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/home/user/.cache/pants/setup/bootstrap-Linux-x86_64/2.6.0rc2_py39/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 501, in _raise_on_error
    raise ExecutionError(

Exception message: 1 Exception encountered:

  ResolveError: No owning targets could be found for the file `dir/package-a/tests/conftest.py`.

Please check that there is a BUILD file in the parent directory dir/package-a/tests with a target whose `sources` field includes the file. See <https://www.pantsbuild.org/v2.6/docs/targets> for more information on target definitions.

You may want to run `./pants tailor` to autogenerate your BUILD files. See <https://www.pantsbuild.org/v2.6/docs/create-initial-build-files>.



(Use --print-stacktrace for more error details and/or --no-process-execution-local-cleanup to inspect chroots and/or -ldebug for more logs. See <https://www.pantsbuild.org/v2.6/docs/troubleshooting> for common issues. Consider reaching out for help: <https://www.pantsbuild.org/v2.6/docs/getting-help.>)
Context:
Copy code
dir/
  package-a/
    src/
      ...
    tests/
      conftest.py
      test_something.py
    BUILD
pants.toml
That BUILD file:
Copy code
python_library()

python_tests(name="tests", sources=["tests/test_*.py"])
f

fast-nail-55400

07/26/2021, 1:19 PM
The exclusion of
conftest.py
from
python_libary
(and its assignment to
python_tests
) appears to only work if it is in the same directory as the
python_tests
target. See https://github.com/pantsbuild/pants/blob/ddc3c3d791178a610a9b8988cb74ac072bdac9c0/src/python/pants/backend/python/target_types.py#L412 where it does not have a
**
. Try moving the
python_tests
target to a
BUILD
file in
dir/package-a/tests
and it should work.
Invoking the test as `./pants test "dir/package-a::"`should still work since
::
in a spec will find all targets deeper in the directory tree.
q

quaint-gold-40000

07/26/2021, 1:23 PM
Thanks! So I'll have two BUILD files: one with
python_library()
and another with
python_tests()
? Just wondering whether a link between the code and tests is then somehow lost?
Running the tests can't find the package-a package though...
f

fast-nail-55400

07/26/2021, 1:46 PM
Dependency inference should find dependencies, but I am unsure how it operates in this case.
This is probably an edge case. One of the other maintainers will have more knowledge, and should be able to help later in the day once they are online.
h

happy-kitchen-89482

07/26/2021, 6:22 PM
If I'm reading this correctly, the BUILD file is in the parent directory of
src
and
tests
. The default sources for the
python_library()
are non-recursive
*.py
(minus test-related files), so no files in your
src
directory are currently covered by a target (unless there's a BUILD file in
src
that you didn't mention), and neither is
conftest.py
, since you provided explicit sources in your
python_tests
target that don't include
conftest.py
Possibly the defaults for targets should be recursive, but that is debatable, and anyway they aren't right now...
If you delete your BUILD file and run
./pants tailor dir/package_a
it will generate BUILD files that should just work.
đź‘Ť 1
But if you prefer to have that single high-level BUILD file then you can modify the
sources
manually
You'll probably want
Copy code
python_library(sources=['src/**/*.py'])

python_tests(name="tests", sources=["tests/**/*.py"])
Then
conftest.py
will be captured by the
python_tests
target.
And each target will glob recursively too
I'm assuming that all code under
tests
is test-related and no code under
src
is, so you don't need to worry about separating tests from non-tests by looking at a
test_
prefix of
_test
suffix and so on...
Let us know here if this solves the issue!
h

hundreds-father-404

07/26/2021, 6:49 PM
that glob will bite you the moment you add a test util to tests/, as pants will try to run Pytest on it and it will error So you'd have to be strict about all test utils living in the src/ folder. Instead, this glob should do it
Copy code
python_library(sources=['**/*.py', '!**/*_test.py', '!**/conftest.py'])
python_tests(name="tests", sources='**/*_test.py', '**/conftest.py'])
w

witty-crayon-22786

07/26/2021, 7:28 PM
but yea, as @happy-kitchen-89482 mentioned, using
tailor
is recommended rather than maintaining disjoint globs: see https://pantsbuild.slack.com/archives/C046T6T9U/p1627323890454200?thread_ts=1627305207.450800&amp;cid=C046T6T9U
đź‘Ť 1
q

quaint-gold-40000

07/27/2021, 8:08 AM
This got it working:
You'll probably want
python_library(sources=['src/**/*.py'])
python_tests(name="tests", sources=["tests/**/*.py"])
Then 
conftest.py
 will be captured by the 
python_tests
 target.
But dependency inference doesn't work for PyYAML (
import yaml
).
Trying to add the dependency manually (for a library without an
src
dir):
Copy code
python_library(
    sources=["**/*.py"],
    dependencies=["mymodule/foo.py:PyYAML"],
)
yields
Copy code
ResolveError: The file or directory 'mymodule/fo.py' does not exist on disk in the workspace, so the address 'mymodule/fo.py:PyYAML' cannot be resolved.
EDIT: figured this out (more below)
It's not clear to me from the docs whether I need to have a requirements.txt or not. It seemed that dependency inference would take care of that, and that a requirements.txt would optionally add version constraints.
With a requirements.txt and BUILD with
python_requirements
in place,
./pants dependencies dir/package-b/module.py
does finally list PyYAML and the tests run.
Import statements inside functions are not detected by dependency inference?
I misunderstood how to manually set dependencies.
python_requirements()
creates new "3rd party" targets, which I need to reference.
I had to kill pantsd for a new dependency added to requirements.txt to be picked up.
Still have the issue with package-a (with
src
dir) that the tests fail to import _package_a_ (in conftest.py).
Copy code
E   ModuleNotFoundError: No module named 'package_a'
Copy code
$ ./pants dependencies "dir/package-a/tests/conftest.py"
dir/package-a/src/package_a/database.py:../../package-a
dir:Sphinx
Now trying autogenerated BUILD files:
If you delete your BUILD file and run 
./pants tailor dir/package_a
 it will generate BUILD files that should just work.
./pants tailor dir/package_a
generates BUILD files everywhere, not just in dir/package-a. Is that expected?
When running the tests for package-a, package_a can be imported now.
Copy code
$ ./pants list "::"
dir
dir:PyYAML
dir:Sphinx
dir:requirements.txt
dir:setuptools
dir/package_b
dir/package_b/package_b
dir/package_b/tests
dir/package-a/src/package_a
dir/package-a/tests
Tests/code that uses _pkg_resources_ or _importlib_metadata_ fail, very likely because the packages under test aren't being installed. Is there any way to get that working?
w

witty-crayon-22786

07/27/2021, 5:57 PM
I had to kill pantsd for a new dependency added to requirements.txt to be picked up.
this is an unfortunate bug, sorry about that. 1) you shouldn’t need the `requirements.txt`: the poetry
pyproject.toml
is an alternative to a requirements file (although point 2 still applies). 2) there is supposed to be a hack in place to restart
pantsd
for
requirements.txt
/
pyproject.toml
changes to work around this bug, but it only matches files named
3rdparty/**/requirements.txt
or
requirements.txt
(at the root): you can add config like the following to the
[GLOBAL]
section of
pants.toml
to work around it:
Copy code
pantsd_invalidation_globs.add = [
  "/the/path/to/my/requirements.txt",
]
h

happy-kitchen-89482

07/27/2021, 10:48 PM
Sounds like we could improve the docs a bit, to clarify that
requirements.txt
or Poetry stanzas in
pyproject.toml
are how you specify the universe of possible 3rd-party dependencies, from which dep inference selects.
đź‘Ť 1
Also re
./pants tailor dir/package_a
 generates BUILD files everywhere, not just in dir/package-a. Is that expected?
Which version of Pants are you on?
h

hundreds-father-404

07/27/2021, 10:49 PM
cc @steep-breakfast-98857, who we talked about looking into that and having a tool tip about multipe pyproject.tomls in the same project
h

happy-kitchen-89482

07/27/2021, 10:52 PM
Oh, looks like support for running
tailor
on just a subdir was not cherry-picked into 2.6.x, so it will only be available in 2.7.0
The first dev release of which will happen this week
q

quaint-gold-40000

07/28/2021, 9:25 AM
I'm on Pants 2.6.0rc2
h

happy-kitchen-89482

07/30/2021, 5:38 PM
Yeah, alas, will have to wait for 2.7.x
14 Views