https://pantsbuild.org/ logo
#general
Title
# general
a

adorable-engine-71736

11/25/2021, 4:55 PM
hi, not sure if this is the intended behaviour... when i run
./pants check my.package::
where
my.package
depends on
other.package
and
other.package
is in the same repo it pulls in the correct version of
other.package
otoh, when i run
./pants check ::
it seems to use the version of
other.package
on the filesystem - ie doesnt respect any specified versions or pull in the last published from pypi
h

happy-kitchen-89482

11/25/2021, 8:00 PM
Hi! To clarify, are you saying that the sources for
other.package
live in the same repo as
my.package
but
my.package
consumes it as a third-party requirement?
a

adorable-engine-71736

11/25/2021, 8:01 PM
yep, in the normal run of things - eg with pytest - that is how it works
h

happy-kitchen-89482

11/25/2021, 8:01 PM
./pants check ::
runs the typechecker on all the sources in the repo, so if `other.package`'s sources are in the repo, I'd expect them to be checked from source. So you're seeing the expected behavior there.
What I'm curious about is the difference with
./pants check my.package::
, I would have expected that to depend on the first party source version of
other.package
as well
Generally Pants expects to depend on sources in the repo directly, not via published dists
So you don't usually expect to specify your first-party code as a requirement
What's the reason for doing it that way?
a

adorable-engine-71736

11/25/2021, 8:03 PM
its how the repo is structured - ie to dep on published versions
rather than the versions in the repo
this is the workaround im using
Copy code
pants check $(pants roots | tr '\n' ',' | sed  's/,/:: /g' -)
the thing that seems to flag to me is that the behaviour is different
ie running
my.package::
and runing
my.package
in
::
h

happy-kitchen-89482

11/25/2021, 8:08 PM
Well I guess what's happening is that
::
globs over all sources in your repo, including
other.package
So it treats it as source
a

adorable-engine-71736

11/25/2021, 8:09 PM
yep
h

happy-kitchen-89482

11/25/2021, 8:09 PM
With
my.package::
, I would have expected dependency inference to infer a dependency from
my.package
on
other.package
, all in source
a

adorable-engine-71736

11/25/2021, 8:10 PM
not sure if my workaround posted above works actually - i have fixed the version conflict separately
h

happy-kitchen-89482

11/25/2021, 8:10 PM
But apparently
./pants check my.package::
is pulling in
other.package
from a published dist?
a

adorable-engine-71736

11/25/2021, 8:10 PM
well, im happy it doesnt in that case 😎
h

happy-kitchen-89482

11/25/2021, 8:10 PM
You have two different sources for the symbols in
other.package
, so dep inference should actually give you a warning about that
a

adorable-engine-71736

11/25/2021, 8:10 PM
yep, it works as i want - ie pulling from pypi and respecting version pins
h

happy-kitchen-89482

11/25/2021, 8:11 PM
Do you have an explicit dependency on
other.package
in `my.package`'s BUILD file?
a

adorable-engine-71736

11/25/2021, 8:11 PM
i have a workaround for the warning
the packages block out the local version
(iirc, it was a while ago when i added it)
yep - re explict deps
so i guess its my customization that is giving rise to the difference in
check
behaviour
in which case i wonder how my use case can be supported for mypy as it is for pytest (i believe at least, perhaps that works the same and i just never triggered an issue)
h

happy-kitchen-89482

11/25/2021, 9:33 PM
Ah OK so the explicit dep pulls in the published version of
other.package
in normal use, but when you reference
::
Pants interprets that as "use all the local sources"
With pytest we run each test file separately, with its own deps, so the right thing happens
with mypy, for performance reasons we run mypy on all the source files in a single process
Running mypy on each file separately would be pretty bad for performance
(until we're able to support the mypy cache at least, and even then it's probably suboptimal)
But you want to run mypy separately for each "project", it sounds like
a

adorable-engine-71736

11/26/2021, 2:42 PM
yep, that is my ideal. It defo slows things down running against each separately. The workaround i posted above speeds it back up but im not sure if its any different to just running
pants check ::
f

fast-nail-55400

11/26/2021, 5:49 PM
the linting rules do support dividing work into partitions. the mypy rules currently just use that partitioning to separate targets with conflicting interpreter constraints. I imagine the partitioning code could be made to take into account other additional criteria. maybe by source root?
h

happy-kitchen-89482

11/26/2021, 5:58 PM
As far as I can tell the workaround doesn't do what you expect. I believe we treat all the input targets as a single bucket, and then partition them by interpreter constraints
So I think that workaround is the same as
./pants check ::
Ultimately to support this kind of thing we'd need to partition in other ways, yeah
Not sure how though
"by source root" might make sense
But it wouldn't cover the case of consuming code published from the same source root
f

fast-nail-55400

11/26/2021, 6:41 PM
we could support an explicit “partition key” to be set on a target and defer to the user to set it as they want. then put targets with differing partition keys into different partitions.
and if the partition key supports some sort of templating to substitute attributes like the source root, the user could choose what precise attributes to use for partitioning
(and set a default partition key in configuration)
4 Views