stocky-yacht-20557
03/28/2023, 7:29 PMpolite-garden-50641
03/28/2023, 8:43 PMstocky-yacht-20557
03/28/2023, 8:49 PMpolite-garden-50641
03/28/2023, 8:50 PMpants lint --only=isort ::
(for example)stocky-yacht-20557
03/28/2023, 8:52 PMpolite-garden-50641
03/28/2023, 8:53 PMsrc/python/
so there might be an actual limitation or an issue with pants here if you have multiple top level projects. I am not that familiar with pants and how it works in those cases...broad-processor-92400
03/28/2023, 8:53 PMstocky-yacht-20557
03/28/2023, 8:54 PMsrc
. but it never works when they're under project/src
broad-processor-92400
03/28/2023, 8:57 PMproject/src
layout may happen to split things in a different way to src
stocky-yacht-20557
03/28/2023, 8:58 PMbroad-processor-92400
03/28/2023, 8:59 PMstocky-yacht-20557
03/28/2023, 9:00 PMbroad-processor-92400
03/28/2023, 9:02 PMpyproject.toml
[tool.isort]
...
# FIXME: <https://github.com/pantsbuild/pants/issues/15069>
# isort and/or pants doesn't seem to reliably identify first party properly (e.g. it'll tell that
# foo is, but not foo.bar, even in a single file!), so just check everything into one
# section... plus this is closer to zimports
default_section = "FIRSTPARTY"
stocky-yacht-20557
03/28/2023, 9:04 PMbroad-processor-92400
03/28/2023, 9:06 PMknown_first_party
like that example, which might be the difference...
The FIRSTPARTY workaround collapses first and third party into one section, which I'd rather not doYeah, it's not the best, but we've found the other benefits of pants to be worth it and it's better than not having import sorting at all. 🤷♂️ / 😦
stocky-yacht-20557
03/28/2023, 9:10 PMbroad-processor-92400
03/28/2023, 9:11 PM.isort.cfg
into project1
(i.e. mv .isort.cfg project1/.isort.cfg
) seems to give the right behaviour, so I wonder if this is an interaction between config file location and first party inference within isortstocky-yacht-20557
03/28/2023, 9:12 PMbroad-processor-92400
03/28/2023, 9:13 PMstocky-yacht-20557
03/28/2023, 9:13 PMsrc
directory up one level everything works correctproject1
aspect to this is just a distractionbroad-processor-92400
03/28/2023, 9:15 PM.isort.cfg
and src
is what matters: if they're adjacent, it gives good behaviour; if they're not (including if it's not exactly called src
), it gives 'bad' behaviour?stocky-yacht-20557
03/28/2023, 9:16 PMbroad-processor-92400
03/28/2023, 9:22 PMisort
outside of pants, it looks like a broken config file is just a warning and the process succeeds overall:
/Users/huon/.pyenv/versions/3.9.10/lib/python3.9/site-packages/isort/settings.py:771: UserWarning: Failed to pull configuration information from /private/tmp/pants-example-python/.isort.cfg
So, pants won't pass this through by default (since it'll only surface true failures).stocky-yacht-20557
03/28/2023, 9:26 PMbroad-processor-92400
03/28/2023, 9:26 PMknown_first_party
modules 😞stocky-yacht-20557
03/28/2023, 9:26 PMbroad-processor-92400
03/28/2023, 9:49 PMknown_first_party
might be what you need to do.
Just thinking out loud, one option to not need to specify info in multiple places would be to generate the file using info that pants knowns, e.g. add all the subdirectories of each folder in pants roots
(or something more intelligent). You might want a CI check that the file is up to date, in addition 🤔src
behaviour: https://pycqa.github.io/isort/docs/configuration/options.html#src-paths (although 15069 will still cause problems when running within pants even with it set)happy-kitchen-89482
03/28/2023, 10:00 PMpants roots
?src/
(and the repo root) as a source rootstocky-yacht-20557
03/28/2023, 10:01 PMhappy-kitchen-89482
03/28/2023, 10:02 PMsrc
, and the only thing in Pants that knows about that naming convention is the default source roots.pants roots
should show the list of dirs that are roots of your package hierarchy (i.e., the dirs you'd put on sys.path in order to import from them)stocky-yacht-20557
03/28/2023, 10:05 PMbroad-processor-92400
03/28/2023, 10:05 PMI'd still drill into this - you're seeing behavior that relies on a directory specifically named(FWIW, running, and the only thing in Pants that knows about that naming convention is the default source roots.src
isort
outside of pants depends on the src
name literally too, so there's not (only) a pants interaction here.)stocky-yacht-20557
03/28/2023, 10:16 PMbroad-processor-92400
03/28/2023, 10:23 PMpants export --resolve=isort # (get a venv with the right version of isort)
dist/export/python/virtualenvs/isort/*/bin/isort .
makes the same change to helloworld/main.py
as pants fmt ::
stocky-yacht-20557
03/28/2023, 10:28 PMsrc_paths
in the config file to e.g. project1/src
. It was my assumption that pants was providing the pants source_roots as the isort src_paths, but it seems that it is not. I'm pretty sure that's the right behavior, though.happy-kitchen-89482
03/28/2023, 10:30 PMstocky-yacht-20557
03/28/2023, 10:30 PMbroad-processor-92400
03/28/2023, 10:32 PMsrc_paths
automatically (similar to how it doesn't set known_first_party
directly), so I think that'll be why we observed the position of the .isort.cfg
file and src
directory name matteringhappy-kitchen-89482
03/28/2023, 10:54 PMallthing
isn't in the known_first_partysrc_paths=
is correct. The problem is that this won't fix all the issues with isort, e.g., when batching is used or when run incrementally on only changed files. Because then some of the source packages may not be in the sandbox at all, and so known_first_party
is really the most bullet-proof way to fix this.stocky-yacht-20557
03/28/2023, 11:59 PMhappy-kitchen-89482
03/29/2023, 7:34 PMknown_first_party
is probably the best path forward here. Unless you have a large number of first-party top-level packages? Usually people have just one, or at most a handful.stocky-yacht-20557
03/29/2023, 7:47 PMSoyep, we have a lot of known_first_party modules. there's a possible solution using pre-commit hooks to write the known_first_party field in the config file. Is there a way to get that list from pants?is probably the best path forward here. Unless you have a large number of first-party top-level packages?known_first_party
happy-kitchen-89482
03/29/2023, 8:59 PMstocky-yacht-20557
03/29/2023, 9:21 PM