Hello 👋 We’re using

pants

to publish python packages as

.whl

files and I’m wondering how do I publish a package with optional dependencies, like with extras_require in setuptools? Since

pants

infers 3rd-party dependencies or pulls them from

python_requirement

targets specified in

dependencies

of a

python_source

target, I assume this would have to be specified in the

python_source

target? It’s almost more like I need to subtract dependencies from the published

python_distribution

than adding extra dependencies. I apologize if documentation for this already exists, but I cannot find it.

I have a feeling optional dependencies is a lacking feature, tbh.

To be more specific, suppose I have a package named

my_package

and sub-module

my_package.dataframes

uses

pandas

but other sub-modules do not. Dependents of

my_package

should be able to use

my_package

without necessarily using

pandas

too. Those dependents that do want to use

my_package.dataframes

should specify a dependency on

my_package[pandas]

yea, so optional “extras”

Right, and to clarify what I mean by “subtract”, in

pants

,

pandas

would need to be specified as a

python_requirement

, it would need to be included as a dependency in the

python_sources

for this package and it would need to be installed in the

resolve

, but that dependency needs to get subtracted from the published

.whl

I guess you’d want to mark it as an extras dependency: https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#optional-dependencies

Right, I linked that same link in my 1st post, but how do I make it not a default dependency in the

python_distrubtion

? Does including it in

extras_require

automatically remove it from default? Extending the example I described above, my

BUILD

file should look something like:

python_requirement(
    name="pandas",
    requirements=["pandas"],
)

python_sources(
    name="lib",
    sources=["src/**/*.py"],
    dependencies=[":pandas"],
)

python_tests(
    name="tests",
    sources=["test/**/*.py"]
)

python_distribution(
    name="dist",
    dependencies=[":lib"],
    sdist=False,
    provides=setup_py(
        name="my_package",
        version="1.2.3",
    )
)

As this is.

pants package

will create a

.whl

file that has a dependency on

pandas

. I understand that I can add

extra_requires={"pandas": ["pandas"]}

to

setup_py

, but does that remove

pandas

from the default dependencies of

my_package

? Do I need to pass

install_requires=[]

to

setup_py

?

Sorry, I overlooked that you had a link in your original post, but it I read it just as you meant it, and my reply is the same, still:

I have a feeling optional dependencies is a lacking feature, tbh.

Ah, so I can do this?

python_distribution(
    name="dist",
    dependencies=[":lib", "!pandas"],
    sdist=False,
    provides=setup_py(
        name="my_package",
        version="1.2.3",
        extras_require={"pandas": ["pandas"]}
    )
)

yea, (but with the double

!!

), hopefully 😜

Yeah, unfortunately today generating setup.py is an all-or-nothing proposition, unless you write a plugin.

You can write a plugin that adds the require_extra keyword, can't you? https://www.pantsbuild.org/docs/plugins-setup-py

That part is easy, but you also need to exclude these requirements from the regular install_requires

makes sense .. I am working on a proposal for modifying requirements post-inference (we had a discussion on GH). That would address this problem too?

Curious, Stefan, is that GH discussion one of the above issues?

I don't think it is @late-advantage-75311, the discussion is here https://github.com/pantsbuild/pants/discussions/20022