I'm observing behavior that I don't understand/can...
# general
g
I'm observing behavior that I don't understand/can't explain. I am hoping someone can shed light if this is a bug or expected behavior. I created a sample repo of the issue here. I don't understand why pants can't infer that two separate modules that depend on a 3rd party library (i.e. click) with different root patterns. I've read the docs referred to in the error message but I'm still confused. Primarily I don't understand why I would go down a particular path.
Copy code
$ pants dependencies --transitive utility/one/bin/main.py
08:33:13.80 [INFO] Initializing scheduler...
08:33:16.22 [INFO] Scheduler initialized.
08:33:16.27 [WARN] The target utility/one/bin/main.py imports `click`, but Pants cannot safely infer a dependency because more than one target owns this module, so it is ambiguous which to use: ['utility/one:reqs#click', 'utility/some/long/subdir/two:reqs#click'].

Please explicitly include the dependency you want in the `dependencies` field of utility/one/bin/main.py, or ignore the ones you do not want by prefixing with `!` or `!!` so that one or no targets are left.

Alternatively, you can remove the ambiguity by deleting/changing some of the targets so that only 1 target owns this module. Refer to <https://www.pantsbuild.org/v2.19/docs/troubleshooting#import-errors-and-missing-dependencies>.
08:33:16.27 [WARN] Pants cannot infer owners for the following imports in the target utility/one/bin/main.py:

  * click (line: 1)

If you do not expect an import to be inferrable, add `# pants: no-infer-dep` to the import line. Otherwise, see <https://www.pantsbuild.org/v2.19/docs/troubleshooting#import-errors-and-missing-dependencies> for common problems.
This question is tied to a greater fear with adoption for my use case. I have about 100 pyproject.toml with their own
poetry.lock
files currently without any build tools. I'm attempting to adopt pants, but I fear that because of the complexity of inter dependencies (1st party) and then overlapping/conflicting 3rd party dependencies it will be challenging to get everything tied to the "right" resolve (lockfile) where I can build everything without specifying all 3rd party dependencies across the monorepo explicitly in
BUILD
files.
b
Thanks for the sample repo - I think the issue that you have there is that both requirements targets belong to the same resolve (the only one), so Pants can’t figure out which declaration of the requirement should take precedence. The dependencies can also be inferred from
pyproject.toml
directly so I don’t think you necessarily need to use
poetry.lock
or
requirements.txt
as the source for the dependencies: https://www.pantsbuild.org/2.18/reference/targets/poetry_requirements When I migrated my own monorepo to Pants, we had about ~15 subprojects managed using Poetry which we unified into a single resolve; that was a pain at first but definitely worth the effort. For you, you may want to think about which subprojects need to share code with each other (if at all) and use that to drive the strategy for how to split resolves/lockfiles.
g
Thanks, Krishnan. I am still in a proof of concept phase of the project so I'm exploring all options. I've tried out the -
poetry_requirements()
and now I'm testing out what it would look like if we exported requirements.txt from poetry in the interim with
python_requirements
explicitly. If I have a predictable pattern for python_requirements (or poetry_requirements) am I able to write some meta programming to set it up? I saw
__defaults__
and wasn't sure if I can use that. For example, Initially I want all of the BUILD files living in the same directory as pyproject.toml to have this:
Copy code
poetry_requirements(
    name="poetry",
    resolve="<dynamically-driven-value>",
)
In my case the
<dynamically-driven-value>
is really just the name of the directory that contains the BUILD file.
I also want to be able to set the default resolve for all children
python_sources
and
python_tests
I'm assuming I can use the __defaults__
b
I admittedly haven't tried dynamically generating resolves before, so not sure on that part
g
This worked:
Copy code
diff --git a/utility/one/BUILD b/utility/one/BUILD
index b8519ce..994a1a8 100644
--- a/utility/one/BUILD
+++ b/utility/one/BUILD
@@ -1,3 +1,7 @@
+__defaults__({
+  (python_sources, python_tests, python_requirements, pex_binary): dict(resolve="one")
+})
+
 python_requirements(
     name="reqs",
 )
c
> In my case the
<dynamically-driven-value>
is really just the name of the directory that contains the BUILD file. you can use
build_file_dir()
for this, it's a
PurePath
object:
Copy code
resolve=build_file_dir().name,
❤️ 1