I have a package “data” that has a requirements.txt with torch >= 1.8, and a binary “train” that has a requirements.txt with torch == 1.10.1 and depends on data. In my BUILD I have to explicitly do:

pex_binary(
    name="train",
    entry_point="train.py",
    dependencies=[
        "src/data:data",
        "!!src/data:torch",
        "examples/ml_example:torch"
    ]
)

and I still get `The target examples/ml_example/train.py imports

torch.nn.functional

, but Pants cannot safely infer a dependency because more than one target owns this module, so it is ambiguous which to use: ['examples/ml_example:torch', 'src/data:torch']` . If I get rid of

"!!src/data:torch"

pants says:

More than one direct requirement is satisfied by torch 1.10.1:
0. torch
1. torch>=1.10

My goal is to allow varying level of strictness - data should support a wide variety of torches, while other people can do what they want. Is there a better way than doing

"!!src/data:torch"

?

So that warning is happening for the target

examples/ml_example/train.py

. You are disambiguating what version of

torch

to use with the

pex_binary

that depends on

examples/ml_example/train.py

, but

train.py

still is using Torch and Pants wants to infer a dep and doesn't know what to do Instead, you could do this in

examples/ml_example/BUILD

python_sources(
  overrides={
    "train.py": {"dependencies": ["examples/ml_example:torch"]},
  },
)

Then,

train.py

will be fixed to use the correct thing and Pants will ignore

src/data:data

for it because the dep is already satisfied by

"examples/ml_example:torch"

. That warning is fixed.

thank you!

You're welcome! Once you make that change, I think you might be able to remove the

"!!src/data:torch"

and

"examples/ml_example:torch"

from the

pex_binary

but I'm honestly not certain. You can use

./pants dependencies --transitive path/to:pex_binary

while debugging I think that it should be safe to remove

"examples/ml_example:torch"

if you make the above change to

examples/ml_example/train.py

because it will already be depending on

"examples/ml_example:torch"

itself But I'm unclear if you will still need to keep

"!!src/data:torch"

. That depends on if any of the deps of the

pex_binary

are transitively depending on it. If so, you'll need to use the

!!

to make sure that no matter what it doesn't get pulled in

The error goes away if I do both:

python_requirements()

python_sources(
    overrides={
        "train.py": {"dependencies": ["examples/ml_example:torch"]},
    },
)

pex_binary(name="train", entry_point="train.py", dependencies=["!!src/data:torch"])

if I don’t have the

!!

I get the double torch pip complaint, if I get rid of the overrides I get the warning

So

lint

tries to batch everything in a single run, hm..is this Pylint that's giving you issues? What you really want right now is "multiple resolves", where Pants has first-class support for part of your repo using lockfile A and another using lockfile B. When we have that, Pants will automatically partition your repo with

./pants lint ::

into a run for resolve A, another for resolve B, etc We laid a ton of groundwork for that 2-3 months ago with Pants 2.7, and it's now my main priority. We're starting with getting things working how we like for Java and Scala because they're experimental, and then going to bring to Python -- In the meantime, the best workarounds are to either: 1) use

--lint-per-file-caching

, which will run a separate process per file. Warning that this will probably be much slower, or 2) Use

skip_pylint

on targets that are giving you issues. --

I can’t seem to track it down

Try

./pants dependees src/data:data

🙂 one of my fav features of Pants