#23399 Using Pants-built python_distribution wheel without resolve parameterization New discussion created by HeroCC Hi Pants peeps! I have a question on the interaction between

python_requirement

and

python_distribution

. We’re trying to model an internal Python library as a Pants-built wheel, then consume that wheel from another project in the same repo without first uploading it to an external package registry. The important constraint: the consuming project should own its dependency versions through its own resolve. The library wheel should behave like a normal pip package artifact whose metadata participates in dependency resolution, rather than forcing the consumer to use the library project’s resolve. Extra Details Simplified tree:

repo/
    libs/
      dd_internal_pyspark/
        src/python/
          PANTSBUILD
          pyproject.toml
          dd_internal_pyspark/
            PANTSBUILD
            __init__.py
            dd_pyspark.py
            dd_spark_session.py

    subprojects/
      pyspark_smoke_testing/
        PANTSBUILD
        3rdparty/python/
          PANTSBUILD # <---- this is where we include dd-internal-pyspark
          lockfile.json
        src/python/pyspark_smoke_testing/app/pyspark_job/
          PANTSBUILD
          pyspark_smoke_testing_job_writer.py
      some_other_pyspark_task/
        PANTSBUILD
...

We have separate resolves:

[python.resolves]
dd_internal_pyspark = "libs/dd_internal_pyspark/3rdparty/python/lockfile.json"
pyspark_smoke_testing = "subprojects/pyspark_smoke_testing/3rdparty/python/lockfile.json"
some_other_pyspark_task = "subprojects/..."

The relevant macro is basically: def wheel(distribution_name: str, version: str, **kwargs): kwargs["sdist"] = False kwargs["wheel"] = True kwargs["name"] = distribution_name kwargs["generate_setup"] = kwargs.get("generate_setup", True) resources(name="wheel_resources", sources=["pyproject.toml", "*.md"]) kwargs["dependencies"] = kwargs.get("dependencies", []) + [":wheel_resources"] python_distribution( output_path=f"wheels/{distribution_name}/{version}", provides=python_artifact( name=distribution_name, version=version, ), **kwargs, ) The consumer currently declares the library as a normal third-party requirement: python_requirement( name="dd-internal-pyspark", # we package the library in one PR, merge it, then update this version in a second PR requirements=["dd-internal-pyspark~=2.0.0"], ) python_requirement( name="pyspark", requirements=["pyspark"], ) python_requirement( name="numpy", requirements=["numpy>=2.3.0"], ) # other libraries... But our goal is to consume the locally built libs/dd_internal_pyspark/src/python:dd-internal-pyspark wheel instead of going back and forth between a registry. Something like this python_requirement( name="dd-internal-pyspark", requirements=[ "dd-internal-pyspark @ libs/dd_internal_pyspark/src/python:dd-internal-pyspark", ], ) We also tried including it directly as a

dependency

on the python_sources, but this does not give us the behavior we want. Pants treats the distribution’s source/dependency closure as part of the target graph, so all targets need to share the same resolve. That makes the library’s own resolve leak into the consumer. We do not want to solve this by parametrizing the library over every consumer resolve. We want the consumer resolve to remain authoritative, e.g. pyspark_smoke_testing should decide its numpy, pyspark, etc. versions, and the local wheel should participate like a regular package artifact. TLDR Is there a Pants-native way to say: build this python_distribution target, then use the resulting wheel as the artifact for a python_requirement in another resolve? We’re trying to understand what model Pants expects here, especially for monorepos with internal Python packages where consumers should retain independent resolves. pantsbuild/pants