We’re having a problem with pants where we have a library (let’s say it’s named

extras

) and within that library, a file imports another module from the same library plus some 3rd party dependencies like:

from extras.foo import bar
from pyspark.sql import functions as F

pylint

correctly identifies this as

wrong-import-order

since 3rd party libraries should come first, but if we manually change the import order,

isort

incorrectly identifies both as 3rd party libraries and puts

extras

ahead of

pyspark

. It seems like

isort

does not know when something is a local import.

I suspect this is due to https://github.com/pantsbuild/pants/issues/15069 . The likely solution is to configure isort to force recognition of the firstparty package: https://pycqa.github.io/isort/docs/configuration/options.html#known-first-party

This even applies in cases where the file is inside

extras

and

isort

cannot tell that

extras

is a local dependency?

Hmm, maybe? I’m not sure. isort has a lot of heuristics. Have you tried that config change? And also, what happens if you run isort standalone outside of Pants?

It looks like

isort

is not able to properly identify the root paths. I was able to fix my problem by adding

src_paths=extras

in

.isort.cfg

, but it seems extremely cumbersome that

pants

does not do this automatically so that every time I add another project, I need to add it to

.isort.cfg

as well

Hmm, that may have incidentally fixed things somehow, but I’m not sure that’s actually correct?

src_paths

is supposed to be set to what pants calls “source roots” - package root dirs (see https://github.com/PyCQA/isort/blob/main/isort/place.py#L76-L87) , e.g., in your case the parent of extras, whereas for you

extras

is the top-level package itself?

in your case the parent of extras, whereas for you

extras

is the top-level package itself?

Not quite, the structure is more like:

extras/
    BUILD
    extras/
        __init__.py
        foo.py
models/
    model_a/
        model_a/
            train.py
            transform.py
        script.py
        config.yml
        BUILD
    model_b/
        ...

and in

pants.toml

[source]
root_patterns = [
    "/",
    "/extras/",
    "/models/model_a",
    "/models/model_b",
]

Perhaps this is all a misunderstanding of how the directory structure should be setup?

Ah OK, what are your source roots? (i.e., what does

./pants roots

show)