Hello. I have a weird issue with `./pants generate...
# generalr
ripe-cpu-85141
10/21/2022, 8:34 PMHello. I have a weird issue with
./pants generate-lockfilespip install -r reqs.txtw
wide-midnight-78598
10/21/2022, 9:02 PMWhat OS and interpreter_constraints?
I ran into some numpy weirdness while generating lockfiles yesterday too, but in my case, it was the version attempted to be resolved by a newer python version (no clue even how that happened though)
r
ripe-cpu-85141
10/21/2022, 9:07 PMoh I think I found why
ripe-cpu-85141
10/21/2022, 9:10 PMI'm using snowflake-connector-python. I didn't put any extra so the installation went well. But if I put the pandas extra, it tries to install pyarrow 0.17 which wants numpy 1.14.* and cython
ripe-cpu-85141
10/21/2022, 9:11 PMI wonder if pants figured out I needed the extra and included the deps
ripe-cpu-85141
10/21/2022, 9:11 PMI guess I'm not completely clear about what's going on 🙂
c
curved-television-6568
10/21/2022, 9:19 PMI’ve found it instructive to look at the pip logs as the data for the lockfile is being compiled. While the generate lockfiles goal is running I
ps ax | grep pants…/pip.logcurved-television-6568
10/21/2022, 9:20 PM Copy code
49518 ?? S 0:00.17 /Users/x/.cache/pants/named_caches/pex_root/venvs/c9e55cc98846b062ba9676b3c1a5214512602544/b804a7d1f8bd9cd2ff386af6a48d90174070910e/bin/python -sE /Users/x/.cache/pants/named_caches/pex_root/venvs/c9e55cc98846b062ba9676b3c1a5214512602544/b804a7d1f8bd9cd2ff386af6a48d90174070910e/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --isolated -q --cache-dir /Users/x/.cache/pants/named_caches/pex_root/pip_cache --log /private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/pants-sandbox-HFUKU3/.tmp/pex-pip-log.3bwh4h5q/pip.log download --dest /private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/pants-sandbox-HFUKU3/.tmp/tmphlyaj6u0/Users.x..pyenv.versions.3.8.13.bin.python3.8 --requirement __pip_args.txt Brotli==1.0.9 Django==3.2.16 JPype1==0.7.1 J...curved-television-6568
10/21/2022, 9:20 PMNamely
/private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/pants-sandbox-HFUKU3/.tmp/pex-pip-log.3bwh4h5q/pip.logcurved-television-6568
10/21/2022, 9:21 PMrunning with increased pex verbosity is also frutiful
e
enough-analyst-54434
10/21/2022, 9:59 PM@ripe-cpu-85141 can you answer @wide-midnight-78598's question? The interpreter constraints in particular would be useful to know.
r
ripe-cpu-85141
10/22/2022, 12:55 PM@curved-television-6568 Ah good idea, I'll take a look
ripe-cpu-85141
10/22/2022, 12:56 PM@wide-midnight-78598 @enough-analyst-54434 Sorry the interpreter constraints is as follow:
CPython==3.9.*w
wide-midnight-78598
10/22/2022, 5:05 PMOh wow, yeah, if numpy 1.14 is getting lumped in, compilation makes more sense... So I guess the question is why that version is being pulled?
Are you able to share your requirements.txt and/or pants.toml/BUILD (if there is python deps specified there)?
Are you locking an old version of snowflake connector?
r
ripe-cpu-85141
10/24/2022, 2:57 PMno, pretty recent
ripe-cpu-85141
10/24/2022, 2:59 PM@wide-midnight-78598 but it seems to lock pyarrow to pyarrow 8.0
ripe-cpu-85141
10/24/2022, 3:00 PMwait no, I don't find the dependency to numpy
ripe-cpu-85141
10/24/2022, 3:16 PMwell that's actually the PyArrow dependency that triggers the compilation 😕
It requires 0.17.*
ripe-cpu-85141
10/24/2022, 4:43 PMpff, false lead. Compilation happened but that wasn't the real issue. It's the new dependency resolver that was too long. I put a minimal version on the requirements and it works better now. Thanks for the help!
5 Views