Hi everyone! I'm trying to figure out how to exclude a transitive dependency. I'm depending on a python library called talon and it then depends on a library called scipy. First thing is when i call dependencies with --dependencies-transitive flag, it only shows one layer of transitive dependencies, which is talon. It doesn't the entire dependency tree.

./pants dependencies sailebot/referral:referral --dependencies-transitive

So i'm able to exclude talon fine, but i want to exclude a dependency that talon depends on. This works, to exclude talon

python_awslambda(
    name="referral",
    runtime="python3.7",
    handler="referral_lambda.py:lambda_handler",
    dependencies=["//:reqs#psycopg2-binary","!!//:reqs#talon"],
)

But this doesn't work

python_awslambda(
    name="referral",
    runtime="python3.7",
    handler="referral_lambda.py:lambda_handler",
    dependencies=["//:reqs#psycopg2-binary","!!//:reqs#scipy"],
)

You cannot exclude transitive third party Python dependencies with Pants, Pex or Pip (or in the Python world generally that I'm aware of).

Thanks for the response @enough-analyst-54434! It is a lambda function, and pulling in scipy doubles the size and puts it over the size limit. The function we are using within talon, doesn't use scipy. So we want to pull in talon, but not scipy. We haven't been successful at finding a replacement for talon I'm able to exclude talon, so you can exclude 3rd party that you directly depend on within your monorepo, just not a 3rd party that a 3rd party depends on? project-a (lambda) --project-b (library) ----talon (3rd party referenced in requirements.txt) --------scipy (3rd party not referenced in requirements.txt)

Correct, you cannot exclude a transitive 3rdparty dependency.

I appreciate your time in responding. I'll give it a try. Thanks!

Examples here: + https://pip.pypa.io/en/stable/topics/vcs-support/ + https://www.pantsbuild.org/docs/python-third-party-dependencies#version-control-requirements

That worked! As I was wrapping up, i ran across this in the setup.py within talon. https://github.com/mailgun/talon/blob/master/setup.py

class InstallCommand(install):
    user_options = install.user_options + [
        ('no-ml', None, "Don't install without Machine Learning modules."),
    ]
    ...
    for not_required in ["numpy", "scipy", "scikit-learn==0.24.1"]:
                dist.install_requires.remove(not_required)

This is what i needed. Is there a way to specify no-ml within pants?

Hrm , they are doing that in an entirely non-standard way 😕 The standard way is to provide extras, and then you'd say

talon[ml]

as a requirement to get ML deps but they would be left off otherwise. All Python tools speak extras. As it is .. I don't think so. You could with Pex, but I believe Pants gets in the way here. With Pex / Pip you use a requirements file that passes the options as documented here: https://pip.pypa.io/en/stable/reference/requirements-file-format/#per-requirement-options