https://pantsbuild.org/ logo
Powered by
#general
Title
# general
p

plain-carpet-73994

10/13/2021, 11:18 PM
Hi @hundreds-father-404 @witty-crayon-22786. Sorry to be a bug machine today but... 
--no-python-setup-resolve-all-constraints
got me further but then I hit what I think is a different bug so I figured I'd start a new thread. I'm getting this:
Copy code
ERROR: Cannot install numpy and opencv-python-headless==4.5.3.56 because these package versions have conflicting dependencies.
...
 The conflict is caused by:
     The user requested numpy
     opencv-python-headless 4.5.3.56 depends on numpy>=1.19.3
but that works fine on Intel with the same deps. So what gives? If you look at the 
setup.py
for opencv you find this:
Copy code
if sys.version_info[:2] >= (3, 6):
        minimum_supported_numpy = "1.13.3"
    if sys.version_info[:2] >= (3, 7):
        minimum_supported_numpy = "1.14.5"
    if sys.version_info[:2] >= (3, 8):
        minimum_supported_numpy = "1.17.3"
    if sys.version_info[:2] >= (3, 9):
        minimum_supported_numpy = "1.19.3"
So the 
1.19.3
seems like it's coming from the 
if
that checks if I'm using python 3.9 or above. pants is running with 3.9 but I'm building for 3.7. My `pants.toml`:
Copy code
[python-setup]
interpreter_constraints = ['CPython==3.7.*']
requirement_constraints = 'constraints.txt'
interpreter_search_paths = ["<PYENV>"]
so it seems like it's resolving the dependencies based on the wrong Python version.
h

hundreds-father-404

10/13/2021, 11:23 PM
Hm, you could use 
-ldebug
to see what Pants is doing. It will print the argv it uses when building the PEX, which should say which Python interpreter is being used
w

witty-crayon-22786

10/13/2021, 11:23 PM
p

plain-carpet-73994

10/13/2021, 11:26 PM
Used both flags. Output:
Copy code
/home/companion/.cache/pants/named_caches/pex_root/venvs/9014736b640bd0f495f1883ed480ed1b12d8bfdb/cc48858524bf3820a737c19c7f14d57d4a5c4208/pex --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -v --cache-dir /home/companion/.cache/pants/named_caches/pex_root --log /tmp/process-executionl0IAIg/.tmp/tmpzi_5yfjk/pip.log download --dest /tmp/process-executionl0IAIg/.tmp/tmpapmkc9l4/home.companion..pyenv.versions.3.7.10.bin.python3.7 --constraint constraints.txt numpy opencv-python-headless pydantic --index-url <https://pypi.org/simple/> --extra-index-url https://<REDACTED>@us-central1-python.pkg.dev/infrastructure-314723/python-deps/simple/ --retries 5 --timeout 15
Is that what you were looking for?
h

hundreds-father-404

10/13/2021, 11:26 PM
Yep!
home.companion..pyenv.versions.3.7.10.bin.python3.7
p

plain-carpet-73994

10/13/2021, 11:27 PM
yeah. That seems like the right answer in which case the constraint from openCV should be 
minimum_supported_numpy = "1.14.5"
and then there's no conflict. So how come there is?
Copy code
The conflict is caused by:
     The user requested numpy
     opencv-python-headless 4.5.3.56 depends on numpy>=1.19.3
And the 
numpy
in my constraints is 
numpy==1.18.5
h

hundreds-father-404

10/13/2021, 11:28 PM
So sounds like Python version isn't the issue. Something that helps when debugging is removing Pants from the equation. For example, you could install Pex: https://pex.readthedocs.io/en/v2.1.51/buildingpex.html Then run 
pex --interpreter-constraint='CPython==3.7.*' --constraint constraints.txt numpy opencv-python-headless pydantic
p

plain-carpet-73994

10/13/2021, 11:29 PM
Do I need pex or would pip do the trick? (the latter being already installed)
NVM: I have pex
Copy code
$ pex --interpreter-constraint='CPython==3.7.*' --constraint constraints.txt numpy opencv-python-headless pydantic
/home/companion/.local/lib/python3.9/site-packages/pex/tools/commands/venv.py:141: PEXWarning: Encountered collision building venv at /home/companion/.pex/venvs/short/7be0f561 from /home/companion/.pex/pip.pex/46820cb5af0dcf9295a4e7f30184cc0e9fa063dc:
1. /home/companion/.pex/venvs/425b6acb2dd2e077f462dac48fd399a48c695531/cc48858524bf3820a737c19c7f14d57d4a5c4208.6a170327f702440babb710f85ba7942a/lib/python3.7/site-packages/constraints.txt was provided by:
        /home/companion/.pex/pip.pex/46820cb5af0dcf9295a4e7f30184cc0e9fa063dc/.deps/setuptools/constraints.txt
        /home/companion/.pex/pip.pex/46820cb5af0dcf9295a4e7f30184cc0e9fa063dc/.deps/wheel/constraints.txt
  pex_warnings.warn(message)
That's interesting...
h

hundreds-father-404

10/13/2021, 11:31 PM
What's 
pex --version
? That collision is unlikely to be the issue and was fixed
p

plain-carpet-73994

10/13/2021, 11:31 PM
$ pex --version 2.1.42
How do I know I'm using the same 
pex
that pants uses?
Should I perhaps use 
/home/companion/.cache/pants/named_caches/pex_root/venvs/9014736b640bd0f495f1883ed480ed1b12d8bfdb/cc48858524bf3820a737c19c7f14d57d4a5c4208/pex
instead?
Copy code
$ /home/companion/.cache/pants/named_caches/pex_root/venvs/9014736b640bd0f495f1883ed480ed1b12d8bfdb/cc48858524bf3820a737c19c7f14d57d4a5c4208/pex --interpreter-constraint='CPython==3.7.*' --constraint constraints.txt numpy opencv-python-headless pydantic

Usage:   
  pex <command> [options]

no such option: --interpreter-constraint
Copy code
$ /home/companion/.cache/pants/named_caches/pex_root/venvs/9014736b640bd0f495f1883ed480ed1b12d8bfdb/cc48858524bf3820a737c19c7f14d57d4a5c4208/pex --version
pip 20.3.4 from /home/companion/.cache/pants/named_caches/pex_root/venvs/short/dd55de96/lib/python3.7/site-packages/pip (python 3.7)
Copy code
$ /home/companion/.cache/pants/named_caches/pex_root/venvs/9014736b640bd0f495f1883ed480ed1b12d8bfdb/cc48858524bf3820a737c19c7f14d57d4a5c4208/pex  --constraint constraints.txt numpy opencv-python-headless pydantic

Usage:   
  pex <command> [options]

no such option: --constraint
so it doesn't like either 
--constraint
or 
--interpreter-constraint
. The latter was not in the output we saw logged but the former was. Confused.
Starting with the logged command line and removing options:
Copy code
$ /home/companion/.cache/pants/named_caches/pex_root/venvs/9014736b640bd0f495f1883ed480ed1b12d8bfdb/cc48858524bf3820a737c19c7f14d57d4a5c4208/pex  download --dest /tmp --constraint constraints.txt numpy opencv-python-headless pydantic --index-url <https://pypi.org/simple/>  --retries 5 --timeout 15Looking in indexes: <https://pypi.org/simple/>, <https://_json_key_base64>:****@us-central1-python.pkg.dev/infrastructure-314723/python-deps/simple/
Collecting numpy
  Downloading numpy-1.18.5.zip (5.4 MB)
     |████████████████████████████████| 5.4 MB 813 kB/s 
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting opencv-python-headless
  Downloading opencv_python_headless-4.5.3.56-cp37-cp37m-manylinux2014_aarch64.whl (21.8 MB)
     |████████████████████████████████| 21.8 MB 6.4 MB/s 
Collecting pydantic
  Downloading pydantic-1.8.2-py3-none-any.whl (126 kB)
     |████████████████████████████████| 126 kB 33.7 MB/s 
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of numpy to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install numpy and opencv-python-headless==4.5.3.56 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested numpy
    opencv-python-headless 4.5.3.56 depends on numpy>=1.19.3

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict
Note that it used my 
--extra-index-url
even though it wasn't on the command line.
👀 1
I think I might know what's going on. pants doesn't provide a build for ARM so I built and installed my own but it was built and installed with python 3.9 and the pex executable also seems to be a 3.9 executable as a result. I think maybe I need to explicitly build for 3.7? Going to try that and will let you know.
actually, unsure how to do that...
h

hundreds-father-404

10/14/2021, 12:24 AM
Sorry, was afk. The version of Python you use to run Pants should be agnostic to what your code uses and what Pex uses. Pants installs Pex through this option: https://www.pantsbuild.org/docs/reference-download-pex-bin#section-version and the related option 
known_versions
p

plain-carpet-73994

10/14/2021, 12:54 AM
bummer. But thanks for tracking that down!
❤️ 1
7 Views
Powered by