https://pantsbuild.org/ logo
#general
Title
# general
r

rich-london-74860

09/16/2022, 7:58 PM
We’re having a problem with pants where we have a library (let’s say it’s named
extras
) and within that library, a file imports another module from the same library plus some 3rd party dependencies like:
Copy code
from extras.foo import bar
from pyspark.sql import functions as F
pylint
correctly identifies this as
wrong-import-order
since 3rd party libraries should come first, but if we manually change the import order,
isort
incorrectly identifies both as 3rd party libraries and puts
extras
ahead of
pyspark
. It seems like
isort
does not know when something is a local import.
h

happy-kitchen-89482

09/16/2022, 9:01 PM
I suspect this is due to https://github.com/pantsbuild/pants/issues/15069 . The likely solution is to configure isort to force recognition of the firstparty package: https://pycqa.github.io/isort/docs/configuration/options.html#known-first-party
r

rich-london-74860

09/16/2022, 9:26 PM
This even applies in cases where the file is inside
extras
and
isort
cannot tell that
extras
is a local dependency?
We get this error even when we run isort for just that library
h

happy-kitchen-89482

09/17/2022, 12:45 AM
Hmm, maybe? I’m not sure. isort has a lot of heuristics. Have you tried that config change? And also, what happens if you run isort standalone outside of Pants?
r

rich-london-74860

09/20/2022, 2:36 PM
It looks like
isort
is not able to properly identify the root paths. I was able to fix my problem by adding
src_paths=extras
in
.isort.cfg
, but it seems extremely cumbersome that
pants
does not do this automatically so that every time I add another project, I need to add it to
.isort.cfg
as well
h

happy-kitchen-89482

09/20/2022, 5:02 PM
Hmm, that may have incidentally fixed things somehow, but I’m not sure that’s actually correct?
src_paths
is supposed to be set to what pants calls “source roots” - package root dirs (see https://github.com/PyCQA/isort/blob/main/isort/place.py#L76-L87) , e.g., in your case the parent of extras, whereas for you
extras
is the top-level package itself?
r

rich-london-74860

09/20/2022, 6:21 PM
in your case the parent of extras, whereas for you
extras
is the top-level package itself?
Not quite, the structure is more like:
Copy code
extras/
    BUILD
    extras/
        __init__.py
        foo.py
models/
    model_a/
        model_a/
            train.py
            transform.py
        script.py
        config.yml
        BUILD
    model_b/
        ...
and in
pants.toml
Copy code
[source]
root_patterns = [
    "/",
    "/extras/",
    "/models/model_a",
    "/models/model_b",
]
Perhaps this is all a misunderstanding of how the directory structure should be setup?
h

happy-kitchen-89482

09/20/2022, 7:12 PM
Ah OK, what are your source roots? (i.e., what does
./pants roots
show)
5 Views