https://pantsbuild.org/ logo
#general
Title
# general
r

rapid-bird-79300

01/19/2023, 5:18 PM
Hey Pants, looking to integrate a project in an existing monorepo containing a single monolith. I'd like to keep these project separate at the dependency level too. Is it possible to define something like "for this source root, always use these thirdparty dependencies"?. I'm currently facing errors like this
Copy code
pants.backend.python.dependency_inference.rules.UnownedDependencyError: Pants cannot infer owners for the following imports in the target app/monolith/something/models.py:

  * psycopg2 (line: 8)
This makes sense because I added a new pyproject.toml for the new project but curious if there a way around this without impacting the monolith dependencies.
hmm but that requires us to maintain an additional lock file, seems a bit distruptive as we only use poetry.lock files
h

happy-kitchen-89482

01/19/2023, 5:36 PM
I'm confused - aren't you asking for multiple lockfiles?
r

rapid-bird-79300

01/19/2023, 5:37 PM
I guess we are, but does this mean I have to generate a lock file for the monolith. We had not done that before
unless it's compatible with poetry lock. I guess we can say
Copy code
[resolves]
new-project = "src/python/new-project/poetry.lock"
then the monolith will just use default resolves and not get conflicted?
if we have visibility rules, the monoliuth and this project would be restricted on cross dependencies
h

happy-kitchen-89482

01/19/2023, 6:02 PM
You can have multiple poetry-generated lockfiles, I believe
And in 2.16 I added a feature that resolves ambiguity in dep inference by source root
❤️ 1
🙌 1
Which I think is exactly what you need here
🎯 1
Well, actually, maybe you don't even need that
if your new project declares that it uses "resolve2" then dep inference will use that
r

rapid-bird-79300

01/19/2023, 6:04 PM
It definintely sounds like that would help get this working out-of-the-box
with no additional configuration
h

happy-kitchen-89482

01/19/2023, 6:04 PM
So the question is, do you want two resolves, or do you want the project to share 3rdparty deps
r

rapid-bird-79300

01/19/2023, 6:05 PM
the new-project is an internal tool that should never share any dependencies with the monolith
any changes to new-project should not impact the monolith
So trying the following:
Copy code
enable_resolves = true
resolves = """
{
  "monolith": "poetry.lock",
  "test-selection": "src/python/test-selection-api/poetry.lock"
}
"""
seeing this conflict:
Copy code
11:09:22.51 [WARN] The target app/monolith/something.py imports `psycopg2`, but Pants cannot safely infer a dependency because more than one target owns this module, so it is ambiguous which to use: ['//:poetry#psycopg2-binary', 'src/python/test-selection-api:poetry#psycopg2'].

Please explicitly include the dependency you want in the `dependencies` field of app/monolith/something.py, or ignore the ones you do not want by prefixing with `!` or `!!` so that one or no targets are left.

Alternatively, you can remove the ambiguity by deleting/changing some of the targets so that only 1 target owns this module. Refer to <https://www.pantsbuild.org/v2.14/docs/troubleshooting#import-errors-and-missing-dependencies>.
11:09:22.51 [WARN] The target app/monolith/something.py imports `psycopg2`, but Pants cannot safely infer a dependency because more than one target owns this module, so it is ambiguous which to use: ['//:poetry#psycopg2-binary', 'src/python/test-selection-api:poetry#psycopg2'].
monolith has the following build defaults:
Copy code
__defaults__(all=dict(resolve="monolith"))
okay think I got it working
I did the following: 1. Enable resolves for the project
Copy code
[python]
enable_resolves = true
resolves = """
{
  "python-default": "default.lock",
  "test-selection": "src/python/test-selection/test-selection.lock"
}
"""
I added both
python-default
and
test-selection
. The first
python-default
is a place holder to make pants happy (since the default "3rdparty/python/default.lock" does not exists anyway, it shouldn't matter) 2. Set the build defaults for the test-selection project to use
test-selection
resolves This is important because you must mark
poetry_requirements
to their correct resolve. For example
Copy code
poetry_requirements(
    name="poetry",
    resolve="test-selection"
)
This is important to do priori to generating the resolve lock. 3. Generate a lock file for the new project
Copy code
./pants generate-lockfiles --resolve=test-selection
After doing all the above, the monolith can now infer dependencies as it did before. This means we can work on a new project without impact the monolith
now I'm curious if step 3 is even necessary
maybe a good thing to do for performance of pants runs in CI but it seems like it's not really necessary to get it working. it is necessary for new-project but not necessary for the monolith default
h

happy-kitchen-89482

01/19/2023, 8:02 PM
Yes, you don't even need my new feature, because you actually do want separate resolves
🙌 1
You have a poetry lockfile for one resolve and a pex lockfile for the other, but that's fine
1
r

rapid-bird-79300

01/19/2023, 10:26 PM
yeah this absolutely amazing, thank you so much for the help!