https://pantsbuild.org/ logo
#general
Title
# general
t

tall-kangaroo-21898

06/06/2022, 8:15 AM
Hey, we are trying to add
firebase-admin==5.2.0
as a python dependency to our project, but when I run
./pants generate-lockfiles --resolve=python-default
it seems to hang on "Generate lockfile for python-default" (now for 1 hour), is there any way to debug this?
h

happy-kitchen-89482

06/06/2022, 9:11 AM
Hi, sorry for the trouble. Which version of Pants are you on? And are you using PEX or poetry lockfiles?
👋 1
e

elegant-carpenter-7253

06/06/2022, 9:22 AM
I managed to reproduce with
2.11.0
and pex, interpreter is
CPython==3.9.10
EDIT: Jonathan and I work in the same codebase.
h

happy-kitchen-89482

06/06/2022, 6:37 PM
Hmm, that's not good, sorry for the bad experience. Are you able to share the full input to your lockfile (which is likely your requirements.txt or similar)?
minus anything proprietary of course
c

cold-sugar-54376

06/09/2022, 7:05 PM
@elegant-carpenter-7253 @tall-kangaroo-21898 i had a similar issue. The easiest thing to do is just try and
pip install/poetry install
your requirements file as there is likely some version conflict somewhere in the requirements chain
👍 1
e

elegant-carpenter-7253

06/10/2022, 5:04 AM
I haven’t seen the reply, sorry about that. Hm, @cold-sugar-54376 but we’re not using poetry anymore, we switched to pex. @happy-kitchen-89482 I can, but there is ~50 dependencies, I’m not sure if it’s productive to go through combinations until the error persists. @tall-kangaroo-21898 Did you manage to solve/go around this issue?
t

tall-kangaroo-21898

06/10/2022, 7:08 AM
@cold-sugar-54376 I think you're right, @happy-kitchen-89482 these are the requirements (waiting to see what pip install outputs)
Copy code
aio-pika==8.0.3
aiocsv==1.2.2
aiofiles==0.8.0
aioredis[hiredis]==2.0.1
aiosmtplib==1.1.6
asgiref==3.5.2
asyncstdlib==3.10.4
bcrypt==3.2.2
click==8.1.3
cryptography==37.0.2
fakeredis==1.8
freezegun==1.2.1
furl==2.1.3
geopy[aiohttp]==2.2.0
grpcio-health-checking==1.46.3
grpcio-reflection==1.46.3
grpcio-status==1.46.3
grpcio==1.46.3
h3[numpy]==3.7.4
httpx[http2]==0.23.0
lagom==1.7.1
mongomock-motor==0.0.9
more-itertools==8.13.0
motor==3.0.0
openapi-core==0.14.2
orjson==3.6.8
phonenumbers==8.12.49
protobuf==4.21.1
pycountry==22.3.5
pydantic[email]==1.9.1
pydevd-pycharm==222.2270.35
pyjwt==2.4.0
pymongo==4.1.1
pytest-httpx==0.21.0
pytest==7.1.2
python-dateutil==2.8.2
python-rapidjson==1.6
requests==2.27.1
rfc3339-validator==0.1.4
sentry-sdk==1.5.12
setuptools==62.3.2
starlette==0.20.1
stripe==3.0.0
structlog==21.5.0
tenacity==8.0.1
timezonefinder==6.0.0
types-aiofiles==0.8.8
types-certifi==2021.10.8.2
types-freezegun==1.1.9
types-protobuf==3.19.21
types-python-dateutil==2.8.17
types-requests==2.27.29
types-setuptools==57.4.17
types-stripe==2.59.9
types-toml==0.10.7
uvicorn[standard]==0.17.6
werkzeug==2.1.2
firebase-admin==5.2.0
With pip 21.2.4, it took a long time to resolve the dependencies (I cancelled the process), upgraded to pip 22.1.2 and it worked. So with a new version of pip the requirements get resolved, not sure what this means for pants.
c

cold-sugar-54376

06/10/2022, 5:02 PM
weird. so that’s slightly different for me. my issue was that some packages had different dependency version requirements that conflicted with each other. i had to downgrade some in order to get the constraints to line up
and i was only able to test it using pip install/poetry install since my pex lockfile generator kept hanging
h

happy-kitchen-89482

06/10/2022, 5:06 PM
Pants delegates the resolves to Pex, which vendors a patched version of pip 20.3.4, so we may need to upgrade that
Sounds like you may have found a black hole in the pex resolver algorithm that they fixed at some point
Is this on Linux or macOS, or both?
c

cold-sugar-54376

06/10/2022, 5:08 PM
for me it was macos
2
h

happy-kitchen-89482

06/10/2022, 5:17 PM
Going to guess that this is the relevant pip fix: https://github.com/pypa/pip/issues/10557 (just a guess though)
Yeah, from manual experiments, seems like that's the one
@enough-analyst-54434 thoughts on vendoring a newer pip to pick up this change?
e

enough-analyst-54434

06/11/2022, 5:29 PM
Complicated, but we need to do it. We jettison support for Python 2.7 orgs (like Foursquare). It is probably possible to have split support with dual (just in time) vendoring of the newer Pip, but I haven't thought it through in detail.
h

happy-kitchen-89482

06/12/2022, 9:20 AM
Oh, ugh, dropping python 2.7 support is not ideal
or at least, I'd rather we did so on purpose and not as a byproduct of picking up some pip change
e

enough-analyst-54434

06/17/2022, 1:38 AM
t

tall-kangaroo-21898

06/22/2022, 10:18 AM
Is there a way to run pants/pex with the legacy pip resolver?
Well, I don't know how but I just tried again and there was no issue anymore
Probably something with my machine, installing the requirements works
Would be good to have an option to show more debug output for these commands
7 Views