Sanity check - I've got two modules - snowflake-connector-python and snowflake-sqlalchemy - that both map to the name "snowflake". I wasn't able to use python_requirements/module_mapping to express that; I found that only one of those could be mapped to "snowflake", with the last one winning. But I was able to do what's pasted below - that seem like the best approach?

python_requirement(
    name="snowflake",
    requirements=[
        "snowflake-connector-python==2.7.2",
        "snowflake-sqlalchemy==1.3.2"
    ],
    modules=["snowflake"]
)

Oh that's interesting. So both packages get unpacked into a

snowflake

directory in the venv? And they just agree not to step on each others toes? Last I checked

poetry

does this as well with

poetry-core

going into

poetry/core

.

Right - the toplevel.txt file in both wheels has a value of "snowflake"

Without thinking of the technical feasibility, seems like something that should be supported by Pants. In the meantime just want to double-check your posted workaround works for you?

It does work. I'll file an issue too.

FWIW there are some (admittedly rare) cases that actually map a package to multiple modules. I think logically speaking it makes sense to map packages to modules (as that's semantically what packages are/do). But you bring up a very interesting point that the reverse might make more sense in this context. "These packages provide these modules" vs "These modules are provided by these packages" @hundreds-father-404 might have thoughts as well

FWIW this is canonical for projects with namespace packages. The very purpose being to support distribution in multiple artifacts.

Ok so admittedly much less rare 😛

Especially since historically the big3 cloud manager tools all used this extensively. Each had 10s to near 100s of dists contributing to aws, azure, google_cloud? namespace packages.

That does make me realize that the module mapper allows for qualified module names: https://github.com/pantsbuild/pants/blob/main/src/python/pants/backend/python/dependency_inference/default_module_mapping.py

Yeah, that's the way to go for sure. A more pointed link: https://github.com/pantsbuild/pants/blob/0aa7f290ed390ab3ceba0c44e4b38354d7a41d3d/[…]s/backend/python/dependency_inference/default_module_mapping.py

I have that link in my clipboard, but for whatever reason didn't paste it 😐

Yeah that indeed is what is intended. For Pants's own distributions, we set

pantsbuild.pants

to be

pants

module whereas

pantsbuild.pants.testuil

is

pants.testutil

- our dep inference code tries the most precise thing first, then falls back to the general one

And as for distribution packages that map to multiple module names, there are some! E.g.,

setuptools

provides

setuptools

,

easy_install

and

pkg_resources

, and

pymongo

provides

bson

,

gridfs

and

pymongo

@bitter-ability-32190 Both of those packages contain namespace_packages.txt and top_level.txt files that only contain "snowflake".