https://pantsbuild.org/ logo
Join Slack
Powered by
So I'm having some issues with nested source roots...
# general
f
So I'm having some issues with nested source roots. 🧵
The easiest way to demonstrate might be against the example-python repo. Why does this change to pants.toml cause the inference failure?
Copy code
--- a/pants.toml
+++ b/pants.toml
@@ -19,7 +19,7 @@ repo_id = "3B1D361B-E9F1-49A8-B761-03DCC41FD58E"

 [source]
 # The Python source root is the repo root. See <https://www.pantsbuild.org/docs/source-roots>.
-root_patterns = ["/"]
+root_patterns = ["/", "helloworld/"]
Copy code
ACAM-M-RWLH:example-python acam$ pants dependents helloworld/greet/greeting.py
23:07:27.68 [INFO] Initializing scheduler...
23:07:27.70 [INFO] Initializing Nailgun pool for 20 processes...
23:07:29.65 [INFO] Scheduler initialized.
23:07:29.74 [WARN] Pants cannot infer owners for the following imports in the target helloworld/greet/greeting_test.py:tests:

  * helloworld.greet.greeting.Greeter (line: 4)
I would have expected, since 
"/"
still exists as a source root, that the dependency inference should still be able to find 
helloworld.greet.greeting.Greeter
? Changing the 
ambiguity_resolution
setting doesn't make a difference either...
The problem is that I have a large monorepo with a common structure like:
Copy code
├── foo
│ ├── app.py
│ └── test
│     ├── base.py
│     └── test_foo
│         └── test_foo.py
├── bar
│ ├── app.py
│ └── test
│     ├── base.py
│     └── test_bar
│         └── test_bar.py
├── baz
│ ├── app.py
│ └── test
│     ├── base.py
│     └── test_baz
│         └── test_baz.py
Some of the 
test_{*}.py
scripts do imports like 
from base ...
while others do 
from test.base ...
. Therefore, I'm trying to set it up so e.g. both 
foo/
and 
foo/test/
are valid source roots. This seems to confuse Pants -- I get errors like:
Copy code
23:31:43.93 [WARN] Pants cannot infer owners for the following imports in the target foo/test/test_foo/test_foo.py:tests:

  * test.base.Foobar (line: 4)

If you do not expect an import to be inferrable, add `# pants: no-infer-dep` to the import line. Otherwise, see <https://www.pantsbuild.org/2.25/docs/using-pants/troubleshooting-common-issues#import-errors-and-missing-dependencies> for common problems.
I probably should fix up all the imports to be consistent but that would be a real slog 😞
s
> Some of the 
test_{*}.py
scripts do imports like 
from base ...
while others do 
from test.base ...
. Therefore, I'm trying to set it up so e.g. both 
foo/
and 
foo/test/
are valid source roots. IMHO it's a very confusing practice and you should fix your imports to be consistent and use single source root
👍 1
e
Having been in this spot before, my perspective is that this is due to poor design of python's import system, and that restricting yourself to a more well-defined usage (ie. only using relative imports or fully-qualified imports from a top-level source root) will lead to much clearer code in the future. Its a pain now, but I hope it will feel worth it in the future to have put the effort in
👍 1
f
I am most of the way through this! 🥵
👍 1
e
Good luck!
f
Are there any known bugs here btw? I'm getting some inference errors that seem like false positives.
But I'll keep digging, maybe I just missed something.
e
One thing I found a bit confusing at first was that you need to set your source root_patterns to point to directories that contain source roots. eg. if you have 
src/python/foo
and 
src/python/bar
, then you have 
source.root_patterns=["/src/python"]
and you can do 
from foo import util
or `from bar.base import main_func`etc. Maybe that's related to what you're seeing?
f
Yeah I already have my source roots all configured. I'm actually using a combination of 
marker_filenames
and 
root_patterns
.
👍 1
Markers are convenient since each of our subprojects has a top-level "PKGBUILD" file (we have a custom build system with packaging based on Arch Linux's pacman).
👍 1
17 Views
Open in Slack
PreviousNext
Powered by