https://pantsbuild.org/ logo
#general
Title
# general
h

high-yak-85899

05/03/2022, 5:22 AM
There are a number of things in our repo that still require the use of a venv. I'm curious if I could use the power of
export
to reduce our bootstrap times. This bootstrapping does something like
pip install -U -e . -r requirements.txt -c constraints.txt
. Even with a warm pip cache, this takes usually around 2 minutes or so while
./pants export
is only like 15 seconds. The missing details are that the venv written by
export
sensibly doesn't have the
-e .
install step. Anyone tried to do something like this before?
Here's my current attempt 1.
./pants export ::
2.
source <path to venv>/bin/activate
3.
python -m ensurepip --upgrade
4.
pip3 install -U -e . --no-deps
One issue at the moment is that
./pants export
breaks for the VCS entry in my
requirements.txt
file.
r

rapid-exabyte-76685

05/03/2022, 6:28 AM
2.11 has added support for VCS requirements, if you are using PEX lockfiles.
1
h

hundreds-father-404

05/03/2022, 1:17 PM
and
./pants export
now includes pip in the venv as of 2.11. (Although you may hit a bug when you're running with how interpreter constraints are chosen. Some people got it, some did not reproduce. The fix has been cherry picked and we will probably do a release this week)
h

high-yak-85899

05/03/2022, 3:45 PM
Not pex lockfiles. Just in our
requirements.txt
for right now.
It's a weird behavior where our repo is named
rtl_tools
, setup.py says the package name is
rtl_tools
, but something is wanting to interpret it as
rtl-tools
even though we do imports like
from rtl_tools.some.internal.package import module
Bumping to
2.11.0
doesn't seem to help either. I've tried every combination of how
rtl_tools
and
rtl-tools
are referenced, but nothing seems to help.
We're now seeing it for a package that doesn't have any hyphen/namespacing stuff. We're doing something like
monorepo @ <vcs url>
and yet
./pants test
is saying it can't find a distribution.
I got the
rtl_tools
case working by making sure it was included in the lockfile. I previously had to exclude it from the pip-tools generated file since it didn't make pants happy. Now that pants can handle them, including it fixed that problem (and similarly just using the pants-native lockfile generation is another valid way to fix it I think).
4 Views