https://pantsbuild.org/ logo
#general
Title
# general
c

curved-microphone-39455

12/14/2022, 7:16 PM
Hey there! I have made good progress, but one thing still not working, I have 3 Libraries in Python ( I can run all test/lint/fmt for all libraries) and now I am trying to add a Project that use those 3 Libraries (local dependencies), All projects use Poetry but when I run the tests for the Project, it won't detect my dependencies. my
project/BUILD
looks like this
Copy code
python_sources(
    name="project",
    dependencies = [
        "src/libraries/lib1:poetry",
        "src/libraries/lib2:poetry",
        "src/libraries/lib3:poetry",
    ]
)

poetry_requirements(
    name="poetry",
)
and all lib looks like this
Copy code
poetry_requirements(
    name="poetry",
    module_mapping={
        [...] # multiple modules mapping 
    },
    overrides={
        [...] # multiple overrides
    }
)
it looks like it getting the local dependencies since its throwing errors from inside the library
Copy code
src/libraries/lib1/lib1/v2/enums/base_format.py:5: in <module>
    import yaml
E   ModuleNotFoundError: No module named 'yaml'
but if I run the tests of
lib1
everything is ok. Local Dependencies are already declared as Editable in the
pyproject.toml
w

witty-crayon-22786

12/14/2022, 8:12 PM
hey @curved-microphone-39455! which version of Pants is this?
c

curved-microphone-39455

12/14/2022, 8:13 PM
Copy code
pants_version = "2.14.0"
w

witty-crayon-22786

12/14/2022, 8:13 PM
are you seeing any warnings about “unowned imports”? you are likely getting one for yaml in this case
or perhaps some warnings about “ambiguous imports”?
c

curved-microphone-39455

12/14/2022, 8:15 PM
yes
there is tons of those
w

witty-crayon-22786

12/14/2022, 8:16 PM
in this case, it sounds like your
poetry_requirements
likely have some overlapping dependencies, so you are probably getting ambiguity warnings
c

curved-microphone-39455

12/14/2022, 8:16 PM
and those
w

witty-crayon-22786

12/14/2022, 8:16 PM
yea. so those will need to be resolved in order for things to be inferred. right now there are N different places to get
yaml
one way to accomplish that in this case is to extract all of the common thirdparty requirements into a new file.
c

curved-microphone-39455

12/14/2022, 8:19 PM
Do I need to remove those common dependencies from my
pyproject.toml
? for each individual project
w

witty-crayon-22786

12/14/2022, 8:19 PM
that, or not use the
poetry_requirements
helper, yea…
c

curved-microphone-39455

12/14/2022, 8:23 PM
the problem is, we already use Poetry in all python projects and our devs use Poetry to generate venv for each individual project to debug and code, in the optic of keeping poetry to detect the requirements, what would be your alternative even if it is a little more hassle
w

witty-crayon-22786

12/14/2022, 8:28 PM
could you extract the common dependencies into another poetry project?
c

curved-microphone-39455

12/14/2022, 8:30 PM
like creating a
/src/common/unit-tests
/src/common/runtime
that are empty project with a simple
pyproject.toml
that contains stuff like pytest, flask, etc, and my
src/libraries
and
/src/services
would have a local dependencies with Poetry to those projects ?
w

witty-crayon-22786

12/14/2022, 8:31 PM
yea.
c

curved-microphone-39455

12/14/2022, 8:31 PM
so my devs could simply do the
poetry install
from a
src/libraries
and the venv creation should be good
I don't hate the idea, I'll try this! Thanks a lot
I try to find a compromise between changing everything to a monorepo and the happiness of my devs by keeping the original workflow
b

broad-processor-92400

12/14/2022, 8:44 PM
We handled a similar migration from many-poetries to pants by having a global pyproject.toml that had path dependencies to each subproject, so that we didn’t have to manually duplicate the global list of dependencies (and have progressively moved to not use/remove the project pyproject.tomls at all, as we switched all required functionality into pants). This does force pants to be using everything the same versions of all dependencies across all projects, though.
🙌 1
👍 1
c

curved-microphone-39455

12/14/2022, 8:49 PM
I guess we will have the same scenario then!
w

witty-crayon-22786

12/14/2022, 8:51 PM
This does force pants to be using everything the same versions of all dependencies across all projects, though.
right: if you actually need multiple versions of some dependencies, you’d need to use multiple `resolve`s in Pants, which is mentioned here. but if you can avoid that, you’re better off.
1
c

curved-microphone-39455

12/14/2022, 9:34 PM
is there a command to see the ambiguity ?
4 Views