Hello team :wave: I had a question about how `ins...
# general
r
Hello team 👋 I had a question about how
install_from_resolve
works for different subsystems (namely
pytest
and
mypy
) in a repository with multiple resolves. For
pytest
, isn't it necessary for
pytest
to be installed in the same resolve as the code it is testing? What happens if the
install_from_resolve
for
pytest
conflicts with the resolve for the
pytest
target (for example, they have different interpreter constraints)? Where should type stubs be installed? Should it be installed in the
install_from_resolve
for
mypy
or in the resolve for that code? What if 2 different resolves use the same package at 2 different versions (which may have necessitated creating 2 separate resolves in the first place) and therefore need type stubs at 2 different versions? Thanks in advance!
h
Hi! Yes,
pytest
is ~unique among tools in that your tests might also import from it as a library. And if they do, then the versions need to be compatible. So while you could keep
pytest
in a separate resolve, and manually ensure that it is a subset of your code’s resolve, it’s probably easier to point
pytest
to your code’s resolve via `install_from_resolve`: https://www.pantsbuild.org/dev/docs/python/overview/lockfiles#sharing-lockfiles-between-tools-and-code
And re type stubs, I believe they need to be in your code’s resolve
👌 1
r
What happens if your code has multiple resolves? If you point to the pytest in one resolve, what is used in the other resolves? What happens if different resolves need different versions of pytest?
h
Yeah, that is tricky and not handled at the moment
r
r
My code has multiple resolves and I tried to use
pytest.install_from_resolve
for one particular resolve for all resolves and this seemed to create a situations where dependencies from multiple resolves, often with conflicting versions, were installed into the same virtual environment. Suppose 2 of my resolves were named A and B. Resolve A uses
protobuf==3.7.0
and B uses
protobuf==4.25.6
. However, when running tests in resolve A, I saw
protobuf==4.25.6
installed. When I check,
google.protobuf.__path__
, I see a path in
named_caches
, which shows both directories
protobuf-3.17.2.dist-info
and
protobuf-4.25.6.dist-info
, which implies that both packages were installed and presumable 4.25.6 overwrote the install from 3.17.2. I saw this same problem with other packages (e.g.
pandas
version 1.2.4 and 1.5.3) and all other resolves other than the one used for
pytest.install_from_resolve
. Even if I wipe out all of
named_caches
and run tests in only the impacted resolve, I see the same problem. All of these problems go away if I create an additional resolve just for installing
pytest
. It appears that when using multiple resolves with conflicting versions, this is necessary.