https://pantsbuild.org/ logo
#general
Title
# general
e

echoing-farmer-15630

05/06/2021, 8:40 PM
OK, new question on `python_requirements`: one of our desired lambda functions uses
pydantic
which insists on lugging along its 60MB (!) of binaries. I would like to install it as
--no-binary
if possible, which supposedly gets it down to about 6mb, much nice to fit under lambda requirements. I would ordinarily, apparently, put the line
--no-binary=pydantic
in the
requirements.txt
in addition to
pydantic==1.8.1
, but I've tried a few permutations and I think pants just doesn't like that sort of thing. Is there a way to do it with
python_requirement_library
or some such?
h

happy-kitchen-89482

05/06/2021, 9:12 PM
Hmm, are you seeing errors, or is the
--no-binary
behavior just being ignored? cc @enough-analyst-54434 for thoughts.
h

hundreds-father-404

05/06/2021, 9:18 PM
It's being ignored, we discard all the
--foo
options when parsing a requirements.txt because
pkg_resources.Requirement
does not understand it, and Pex won't understand them when passed as an argument. Only works in the requirements.txt file and with pex's
-r req.txt
Part of the prework for adding per-tool lockfiles is to start respecting things like
--no-binary
and
--hash
from requirements.txt files, given that we must support
--hash
. https://docs.google.com/document/d/1If8wGowEpYaehGd_CQ8KxwFSydU1Q8UdpLr0i36NROk/edit# That prework will land in the next week if all goes to plan @echoing-farmer-15630 as part of 2.6. We can't cherry-pick to 2.5 unfortunately, it's too breaking of a change. 2.6 in general is getting an enormous revamp of 3rd party dependencies (see that google doc)
e

enough-analyst-54434

05/07/2021, 3:50 PM
More context on Eric's explanation: 1. For legacy reasons, Pants parses requirements into Requirement objects even though its now only a middle-man and could just be shuffling along requirement strings to Pex. 2. Pex ~= Pip for all things requirement related and Pip takes many options affecting requirement resolution only in requirements files (like --hash). Others, like -
-no-binary <comma-sep-packages>
that some Pip subcommands accept on the CLI, must be passed via requirements files when using Pex. Since Pants does not pass along requirements files to Pex (it could only do this in limited scenarios anyhow and in general will need to generate a synthetic requirements file), there is currently no avenue to pass this stuff. And that gets to the prework Eric mentioned.
👍 1
2 Views