https://pantsbuild.org/ logo
Powered by
g

glamorous-accountant-97217

11/10/2022, 2:17 PM
I have the following requirement:
Copy code
python_requirement(
    name="snowflake-connector",
    requirements=["snowflake-connector-python[pandas]==2.7.11"],
    modules=["snowflake"],
)
until recently I was able to build my package with pants, but now i’m getting 
No matching distribution found for snowflake-connector-python[pandas]==2.7.11
. i don’t know what’s changed
Copy code
^[08:12:04.33 [INFO] Initializing scheduler...
08:12:04.58 [INFO] Scheduler initialized.
08:12:09.14 [INFO] Completed: Building local_dists.pex
08:13:19.53 [INFO] Completed: Building pushdown_service/service.pex with 5 requirements: fsspec[http], modin@ git+<https://github.com/devin-petersohn/modin.git@2ad8813ccf3cb263857478df951b8c7b4b7d4396>, ponder@ file:///Users/maheshv... (126 characters truncated)
08:13:19.53 [INFO] Canceled: Building pushdown_service/__main__.pex with 5 requirements: fsspec[http], modin@ git+<https://github.com/devin-petersohn/modin.git@2ad8813ccf3cb263857478df951b8c7b4b7d4396>, ponder@ file:///Users/mahesh... (127 characters truncated)
08:13:19.53 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Building pushdown_service/service.pex with 5 requirements: fsspec[http], modin@ git+<https://github.com/devin-petersohn/modin.git@2ad8813ccf3cb263857478df951b8c7b4b7d4396>, ponder@ file:///Users/maheshvashishtha/Downloads/ponder-0.0.1-py3-none-any.whl, segment-analytics-python==2.2.1, snowflake-connector-python[pandas]==2.7.11' failed with exit code 1.
stdout:

stderr:
pid 78948 -> /Users/maheshvashishtha/.cache/pants/named_caches/pex_root/venvs/c9e55cc98846b062ba9676b3c1a5214512602544/d4754ba1b307db644938eea56a5135a1b744e364/bin/python -sE /Users/maheshvashishtha/.cache/pants/named_caches/pex_root/venvs/c9e55cc98846b062ba9676b3c1a5214512602544/d4754ba1b307db644938eea56a5135a1b744e364/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --isolated -q --cache-dir /Users/maheshvashishtha/.cache/pants/named_caches/pex_root/pip_cache --log /private/var/folders/ts/7lr9hnf958b68z7qr4dvtqvh0000gn/T/pants-sandbox-LAnr5X/.tmp/pex-pip-log.l2bb1k3l/pip.log download --dest /Users/maheshvashishtha/.cache/pants/named_caches/pex_root/downloads/resolver_download.j589gq5z/Users.maheshvashishtha..pyenv.versions.3.11.0.bin.python3.11 fsspec[http] modin@ git+<https://github.com/devin-petersohn/modin.git@2ad8813ccf3cb263857478df951b8c7b4b7d4396> ponder@ file:///Users/maheshvashishtha/Downloads/ponder-0.0.1-py3-none-any.whl segment-analytics-python==2.2.1 snowflake-connector-python[pandas]==2.7.11 --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with 1 and STDERR:
WARNING: Discarding <https://files.pythonhosted.org/packages/48/41/431248b17560a85e70515462c808f3a954e8f892cba190e44cfee248cfe6/snowflake-connector-python-2.7.11.tar.gz#sha256=2e2cc1e5321cfcf1b9610b9008aac89e8abfc6fe3ee68ea5e68858abc5313d87> (from <https://pypi.org/simple/snowflake-connector-python/>) (requires-python:>=3.7). Command errored out with exit status 1: /Users/maheshvashishtha/.cache/pants/named_caches/pex_root/venvs/c9e55cc98846b062ba9676b3c1a5214512602544/d4754ba1b307db644938eea56a5135a1b744e364/bin/python /Users/maheshvashishtha/.cache/pants/named_caches/pex_root/venvs/c9e55cc98846b062ba9676b3c1a5214512602544/d4754ba1b307db644938eea56a5135a1b744e364/lib/python3.11/site-packages/pip install --ignore-installed --no-user --prefix /Users/maheshvashishtha/.cache/pants/named_caches/pex_root/pip_cache/.tmp/pip-build-env-39ee4sd7/overlay --no-warn-script-location -v --no-binary :none: --only-binary :none: -i <https://pypi.org/simple/> -- 'setuptools>=40.6.0' wheel cython 'pyarrow>=8.0.0,<8.1.0' Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement snowflake-connector-python[pandas]==2.7.11
ERROR: No matching distribution found for snowflake-connector-python[pandas]==2.7.11




Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.
e

enough-analyst-54434

11/10/2022, 2:38 PM
How long have you been using Python 3.11?
g

glamorous-accountant-97217

11/10/2022, 2:39 PM
i don’t know-- i tried using CPython<3.11 in pants.toml, but got the same error though
actually i see even with that change i’m seeing 
/Users/maheshvashishtha/.cache/pants/named_caches/pex_root/downloads/resolver_download.t2ck1v3j/Users.maheshvashishtha..pyenv.versions.3.11.0.bin.python3.11
and 
/Users/maheshvashishtha/.cache/pants/named_caches/pex_root/venvs/c9e55cc98846b062ba9676b3c1a5214512602544/d4754ba1b307db644938eea56a5135a1b744e364/bin/python /Users/maheshvashishtha/.cache/pants/named_caches/pex_root/venvs/c9e55cc98846b062ba9676b3c1a5214512602544/d4754ba1b307db644938eea56a5135a1b744e364/lib/python3.11/site-packages/pip
in the pants output. maybe it’s still using 3.11?
e

enough-analyst-54434

11/10/2022, 3:27 PM
It definitely is. What version of Pants is this?
Pretending its 2.14+, you can try this:
Copy code
curl -L <https://github.com/pantsbuild/pex/releases/download/v2.1.111/pex>
python3.11 pex "fsspec[http]" "modin@ git+<https://github.com/devin-petersohn/modin.git@2ad8813ccf3cb263857478df951b8c7b4b7d4396>" "ponder@ file:///Users/maheshvashishtha/Downloads/ponder-0.0.1-py3-none-any.whl" segment-analytics-python==2.2.1 "snowflake-connector-python[pandas]==2.7.11" --preserve-pip-download-log
That will run the same command Pants is running but preserve the pip log for you to inspect for more details about the build failure for 
snowflake-connector-python
.
g

glamorous-accountant-97217

11/10/2022, 3:37 PM
i’m on pants 2.13.0
ok, with 
python3.11 pex
i find 
ERROR: Failed building wheel for pyarrow
among other errors and it turns out pyarrow doesn’t support python 3.11: https://arrow.apache.org/docs/python/install.html
python pex
with python 3.10.6 works. how do I get pants to use python < 3.11?
e

enough-analyst-54434

11/10/2022, 3:50 PM
g

glamorous-accountant-97217

11/10/2022, 3:51 PM
I tried CP`ython<3.11"` but pants still seemed to use python 3.11. l’m trying again with 
["CPython>=3.10.*,<3.11"]
that works!
e

enough-analyst-54434

11/10/2022, 3:53 PM
So, the issue with your 1st try is our very tricky list semantics in toml. If the option is a list and you supply a string, it gets appended to the default value of the list. For interpreter constraints that means you had something like ["CPython>=3.7,<4", "CPython<3.11"] and for interpreter constraints, if there is more than 1 element in the list, we OR them.
g

glamorous-accountant-97217

11/10/2022, 3:54 PM
ok, for the snowflake-connector-python error, could pants have given a more helpful error message by default?
e

enough-analyst-54434

11/10/2022, 3:56 PM
Obviously! The question is effort and where to apply it, Pants or Pex. You should definitely file an issue if you want to see this fixed as the 1st step.
g

glamorous-accountant-97217

11/10/2022, 3:57 PM
about the interpreter constraints, why OR them?
e

enough-analyst-54434

11/10/2022, 3:58 PM
Twitter history. Perhaps there are real use cases still, not sure.
But basically, the 
,
== AND syntax is built into the Python ecosystem. The Python ecosystem does not have OR though. Twitter needed OR for I can't remember why and so some engineer added this semantic of multiple values -> OR.
That was way back in Pants v1. We broke most things in the Pants v2 transition on purpose, but this we honored for whatever reason.
4 Views
Powered by