How can I inspect the dependencies built into a PEX built as a result of running

pants test

? I have pinned

numpy==1.26.4

in my

requirements.txt

, but a test is failing with an error that indicates the test environment is using

numpy>=2

. The numpy docs indicate that this error is only raised by

numpy>=2

. The code in question transitively depends on

numpy

by way of

tensorflow

which is also pinned to

tensorflow==2.11.0

in our

requirements.txt

. Building a PEX that contains the module in question and interacting with it manually does not raise the same error.

Thanks, this gets me partway there. I can’t seem to find any of the dependencies within the PEXes contained in the sandbox tempdir however. That said, when running with that flag that the test PEX was only including

tensorflow

because

numpy

is not an explicit dependency of the code under test. By adding

import numpy as np

to explicitly indicate a dependency on

numpy

, it is built into the test PEX and the test passes. It is not clear to me why/how an error message from Numpy v2 is being raised when it is not present in the environment and the distribution of

tensorflow==2.11.0

I am using has not been updated since 2022. I will share a minimal repo that reproduces the error shortly in case it is of interest to anyone.

I can take a look, if you can do it in the next like 30 minutes, I'm going to bed soon.

3.9 which is constrained in

pants.toml

Hm

I built the repro to mimic the internal repo I encountered this with. I’m trying to upgrade to Pydantic v2 and Python 3.9 (or later) now that some of our dependencies have finally upgraded to Pydantic v2. This is why the Pants version (2.17) and Python version are not exactly up to date.

I'm on a mac, btw...

Don’t let this keep you up. I just added

import numpy as np

to the problematic module so pants can infer the dependency and that workaround should be satisfactory for now.

Ah, okay, there's an x86 mac version

Same, intel mac. I’m guessing you’re on Apple silicon? I know that TF distros are not available for M* chips for some earlier versions

Yeah, they took my Linux one away and the only reason I agreed was because it was an m3 pro 😄

lol no worries. thanks for trying

and too lazy to get my linux laptop. but, tests in pants get a pex with the requirements there, the code is copied in the sandbox

As far as I can tell Pants/PEX is not inferring that

tensorflow

requires

numpy

. Unclear why that is 🤷

That's not inferred, that comes from the package.

Sorry, that’s what I meant

We use it to stop compatible upgrades, but if you tell it not to use numpy > 2, it might give you an idea of what's causing it to use that.

The odd thing is that it doesn’t seem to be building in any version of numpy. At least it’s not listed when building the PEX.

Data science stuff is always a disaster to upgrade 🙂

Ah ok

Like when you say

pip install tensorflow

, you don't say you want

numpy

, but you get it

Makes sense. I’ll keep digging. Thanks again for the help.

Sure thing, it's my alcohol fuelled pants yak (or yak pants?) shaving night tonight 🙂

Ok I confirmed the test PEX contains Numpy v2.

tensorflow==2.11.0

requires

numpy>=1.20

as shown here which means pip will use the latest available version of

numpy

(v2+) which actually isn’t compatible. This is similar to the chaos caused by the release of the Pydantic v2 release where consumers did not pin

pydantic

to

<2

. What fun.

constraints.txt

will help you fix that.

lol, that release is the gift that keeps on giving

I'm a huge fan of pydantic, but even if I wasn't, can't really blame them

same and agreed. it’s a lesson to us all…