https://pantsbuild.org/ logo
Join Slack
Powered by
I have `resolves_to_only_binary` set for `grpcio` ...
# general
h
I have 
resolves_to_only_binary
set for 
grpcio
and 
grpcio-tools
, but I'm still seeing cpu-heavy compilation steps happening when pants is trying to install things. My lockfile correctly shows the request to not install these from source. Anything I might be missing?
The artifacts section of the lockfile also shows only wheels being picked up
Copy code
pip install --only-binary="grpcio,grpcio-tools" grpcio==1.54.2 grpcio-tools==1.48.2
outside of pants works as I would expect. Only uses wheels and no heavy compilation task happening.
b
It’s definitely those packages that are being compiled?
h
It sure looks like it from what 
htop
reports
Just having trouble extracting the full command being run
Copy code
--no-binary :all: --only-binary :none: -i <https://pypi.org/simple/> -- setuptools>=40.8.0 wheel>=0.29.0 grpcio-tools>=1.47 protobuf==3.20
Here's something interesting in one of the really long commands running
Is this possibly pants remote caching stuff since that uses a grpc interface?
The protobuf and grpcio-tools requirements do not match what we track for our resolves
f
Is this possibly pants remote caching stuff since that uses a grpc interface?
No because the remote cache code is entirely in Rust.
Using the 
tonic
and 
prost
crates.
h
Gotcha. Okay, the 
grpcio-tools
lower bound matches what's in our requirements file, but 
protobuf
does not match
Could this be coming from the 
pants_requirements
target?
I don't see 
grpc
mentioned at all in the lockfile there so I'm doubtful
But that's the only other resolve we have in place
And I doubt that could come up because this install is kicked off when we specifically say 
export --resolve=python-default
Could this be a pex thing? I can't find anything in the repo where this 
--no-binary
stuff comes up
we're on 2.15.0 btw
These seem to be the commands causing the compilation once export fires up
Untitled
Untitled
It doesn't seem like this comes from pants. I pulled down the 2.15.1 (bumped just to see if it was different) and couldn't find any mentions of these packages in the source
b
Does running with 
--keep-sandboxes=always
and jumping into the 
__run.sh
scripts reveal hints?
h
Copy code
(3.8.10) ~/devel/monorepo (no-downloaded-data)$ ./pants --keep-sandboxes=always export --no-export-symlink-python-virtualenv --resolve=python-default
15:56:17.19 [INFO] Preserving local process execution dir /tmp/pants-sandbox-kdHWQ9 for chosen_resolve.name
⠓ 40.80s chosen_resolve.name
Interrupted by user.

(3.8.10) ~/devel/monorepo (no-downloaded-data)$ ls -l /tmp/pants-sandbox-kdHWQ9/
total 4488
-rw-r--r-- 1 nathanael nathanael  522937 May 24 15:56 constraints.txt
-rwxr-xr-x 1 nathanael nathanael 4063422 May 24 15:56 pex
drwxrwxr-x 2 nathanael nathanael    4096 May 24 15:56 source_files
Doesn't look like that's in there
source_files
is empty and 
constraints.txt
is just the lockfile for our resolve we're exporting
It does seem dependent on the resolve because if I export the 
pants-plugins
resolve, the grpc building doesn't show up
I just can't make sense of why this specific pip command would show up. Those requirements don't show up anywhere in our system. The setuptools and wheels portion looks similar to https://github.com/pantsbuild/pex/blob/8b8147cd23256e7c5a141e79f9ab27549283e74f/pex/vendor/_vendored/pip/pip/_internal/pyproject.py#L128
So perhaps there's some package in our ecosystem that's overriding what we want to happen
That's exactly what's happening
We're using a modified form of 
openhtf
that is forked internally for some fixes. But there's a section like this where we're forcing pants under the hood to build grpc from source even though our first party usage doesn't allow that
2 Views
Open in Slack
PreviousNext
Powered by