Hi guys, I have a trouble running python targets (...
# general
e
Hi guys, I have a trouble running python targets (binary, test) with correct interpreter. It seems that even when I specify >=3.11 it still uses Python 3.9 and fails on 3.11 specific features.
Copy code
[python]
interpreter_constraints = [">=3.11"]
(Also tried explicit CPython) Part of the traceback I think it’s relevant:
Copy code
Traceback (most recent call last):
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
I’m mac. My 2 of my colleagues got the same issue (they’re on mac as well). Thanks for any help.
b
Doesn't
interpreter_constraints
only implies that the constraints are checked if they are compatible with a specific interpreter version? I think it does not imply what interpreter your target runs on.
e
I see. And is there a way how specify run interpreter version? Other issue is that when I’m building pex/wheels it fails when I specify target version Python 3.11, but is says, that there is only Python 3.9 wheel for the given package. I’m little confused here 🤦
r
You can use pyenv for a specific python version you want to use for run. As far as wheel goes for a specific package it's the responsibility of the package maintainer to publish different wheels for a different python version. If it's not available and you can't avoid using an available version, you will have to build it yourself.
This might give more insight into why some wheels can't be found. Keep in mind it's just one case but it does provide what are the things to look out for https://github.com/pantsbuild/pants/discussions/18981
e
Thank you. I tried to setup pyenv, but it’s still using the system Python.. I tried
Copy code
[python-bootstrap]
search_path = ["<PYENV>"]
and even adding path to the pyvenv bin folder for the given python version explicitly. Is there any trick how to make it work?
In parallel I’m trying to make the prebuilt wheels work, but it without much success. I’m trying to build pex file for mac (for local) and for linux (for docker). I built these wheels:
Copy code
ls 3rdparty/python/wheels/
BUILD                                              httptools-0.6.0-cp311-cp311-macosx_10_9_x86_64.whl
httptools-0.6.0-cp311-cp311-linux_x86_64.whl       httptools-0.6.0-cp311-py311-none-any.whl
I used exactly the linux target I needed, but based on @refined-addition-53644 link I used whitelabeled name.
Copy code
[python-repos]
# None of these 3 approaches worked
# repos = [
#     "%(buildroot)s/3rdparty/python/wheels"
#   ]
# find_links="%(buildroot)s/3rdparty/python/wheels"
find_links="file:///Users/xxx/workspaces/the_project/3rdparty/python/wheels"
Copy code
stderr:
No pre-built wheel was available for httptools 0.6.0.
Successfully built the wheel httptools-0.6.0-cp39-cp39-macosx_10_9_x86_64.whl from the sdist httptools-0.6.0.tar.gz but it is not compatible with the requested foreign target abbreviated platform cp311-cp311m-linux_x86_64.
You'll need to build a wheel from httptools-0.6.0.tar.gz on the foreign target platform and make it available to Pex via a `--find-links` repo or a custom `--index`.
r
I think these two issues are related as you see pants is actually building the wheel from sdist for you but for python3.9. So you might not even need to build it yourself as long as you can fix the python version issue. Just to test, can you only try
Copy code
interpreter_constraints = ["CPython==3.9.*"]
Also I think the syntax for 3.11 should be like. I think
CPython
is missing. Check the docs
Copy code
interpreter_constraints = ["CPython>=3.11,<3.12"]

# equivalent
interpreter_constraints = ["CPython==3.11.*"]
e
It works fine when using 3.9., but when using the suggested syntax for 3.11 it doesn’t work. I can see “choosing interpreter 3.11….“, so it seems that it’s loading the right one, but then it suddenly switch to the 3.9. Even when I updated pants version I saw in the logs “using CPython 3.9”, so it seems that it’s using it internally. BTW: From the provided docs:
As a shortcut, you can leave off
CPython
and just put the version specifier. For example,
==3.8
will be expanded automatically to
CPython==3.8
.
👍 1
Is it possible that pants doesn’t support newer versions of Python then 3.9?
r
Although this doesn't resolve the underlying issue, but can you try using a lockfile?
e
I’m using lockfile. Also tried without them, but still the same
r
So you are able to generate the lockfile successfully?
e
Yes, generation works fine
Hi guys, sorry for re-raising the issue, but I’m stuck on this for quite long time and we had to freeze development until I finish the migration 🤦 I still can’t make pants work with other interpreter then Python 3.9, since it’s always rollback to the system one. Even specifying in search_path only one interpreter (3.11) won’t help. I can see it’s found and used when running a binary, but then I got an error showing then 3.9 was used. See my first message in the thread. I spent hours on the internet and I’m of verge refactoring our all whole codebase back to 3.9 😅 Thanks in advance
b
Sorry for the trouble. For the original error, the stack trace provided is very generic, just suggesting that some python code is running. Could you show a bit more context for the error? For instance, what pants command are you running? Some other info that would help is the contents of pants.toml or at least the pants version
w
It’s been a long time since I’ve seen something like this, but is that your system-provided Python?
b
Also, as a bit of background, there's two interpreters in play: the version used to run pants (which is 3.9), and the version pants uses to run your code. There's a chance that these may be getting mixed up, so more context will help us work out what's up.
w
Also also, I may have missed it - but are you using
scie-pants
installed via brew or something? Or are you using the pants bash script?
e
This is my pants.toml
Copy code
[GLOBAL]
pants_version = "2.16.0"

backend_packages.add = [
  'pants.backend.python',
  'pants.backend.python.lint.docformatter',
  # "pants.backend.python.lint.autoflake",
  "pants.backend.python.lint.pyupgrade",
  'pants.backend.python.lint.black',
  'pants.backend.python.lint.flake8',
  'pants.backend.python.lint.isort',
  'pants.backend.python.typecheck.mypy',
  "pants.backend.python.lint.bandit",
  "pants.backend.docker",
  "pants.backend.shell",
  "pants.backend.shell.lint.shfmt",
  "pants.backend.shell.lint.shellcheck",
  "pants.backend.experimental.helm"
]

[source]
root_patterns = [
    "/lib/python",
    "/services/python",
    "/test/python",
]

[python]
interpreter_constraints = ["CPython>=3.11"]


enable_resolves = true
default_resolve = "python-default"

[python.resolves]
python-default = "3rdparty/python/lockfiles/default.lock"
mypy = "3rdparty/python/lockfiles/mypy.lock"


[python-repos]
find_links="%(buildroot)s/3rdparty/python/wheels"

# [python.resolves_to_no_binary]
# python-default= ["httptools"]

[python-bootstrap]
# I tried only pyenv and also specifying only explicit path to the 3.11. In other words I excluded the system Python from this search path
search_path = ["<PYENV>", "<PATH>"]

[anonymous-telemetry]
enabled = false

[python-infer]
string_imports = true

[generate-lockfiles]
diff = true


[pytest]
execution_slot_var = "PANTS_EXECUTION_SLOT"
config = "tools/pytest.ini"


[mypy]
install_from_resolve = "mypy"
config = "tools/mypy.ini"

[flake8]
config = "tools/.flake8"

[bandit]
config = "tools/bandit.yaml"

[black]
config = "tools/pyproject.toml"

[isort]
config = "tools/pyproject.toml"

[docker]
env_vars = [
  "DOCKER_BUILDKIT=0",
  "COMPOSE_DOCKER_CLI_BUILD=0"
]
Running this command
Copy code
pants --print-stacktrace run services/python/my_service:run
The target BUILD (my intention is to build it for current for local dev and linux for docker image)
Copy code
...
pex_binary(
    name="run",
    entry_point="run_uvicorn.py",
    dependencies=[":lib"],
    platforms=[
        "current",
        # "macosx-10.13-x86_64-cp-311-cp311m"
        # "linux-x86_64-cp-311-cp311m",
    ],
)
I installed via brew. I’ll try the bash script
I think
/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9
is the version mac is using. I made sure there is not other 3.9 Python version on my system. The default
python
and
python3
is pointing to 3.11.
w
scie-pants
via brew is fine - it’s what I would have recommended. Just checking, as that one comes with python3.9 for Pants to run - so that should be all good
b
Hm, that
platforms=["current"]
is suspicous. What happens if you don't provide it at all?
e
Thanks, removing the
platforms
helped to build it at my platform! 🎉 I took from some example in docs, but maybe it’s deprecated. Anyway, I still needs to be able to build it for linux for docker image (using python:3.11 base image). When I put only linux to the platforms I get this wheel error:
Copy code
08:46:33.03 [ERROR] 1 Exception encountered:

Engine traceback:
  in select
    ..
  in pants.core.goals.run.run
    `run` goal

Traceback (most recent call last):
  File "/Users/martinvo/Library/Caches/nce/2e4f5456e74f7e8b3489c9ced8c844ba1113bfbf37dcaf688859cb976f884ea0/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 623, in native_engine_generator_send
    res = rule.send(arg) if err is None else rule.throw(throw or err)
  File "/Users/martinvo/Library/Caches/nce/2e4f5456e74f7e8b3489c9ced8c844ba1113bfbf37dcaf688859cb976f884ea0/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/core/goals/run.py", line 292, in run
    request = await (
  File "/Users/martinvo/Library/Caches/nce/2e4f5456e74f7e8b3489c9ced8c844ba1113bfbf37dcaf688859cb976f884ea0/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 118, in __await__
    result = yield self
  File "/Users/martinvo/Library/Caches/nce/2e4f5456e74f7e8b3489c9ced8c844ba1113bfbf37dcaf688859cb976f884ea0/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 623, in native_engine_generator_send
    res = rule.send(arg) if err is None else rule.throw(throw or err)
  File "/Users/martinvo/Library/Caches/nce/2e4f5456e74f7e8b3489c9ced8c844ba1113bfbf37dcaf688859cb976f884ea0/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/backend/python/goals/run_pex_binary.py", line 22, in create_pex_binary_run_request
    built_pex = await Get(BuiltPackage, PexFromTargetsRequestForBuiltPackage, pex_request)
  File "/Users/martinvo/Library/Caches/nce/2e4f5456e74f7e8b3489c9ced8c844ba1113bfbf37dcaf688859cb976f884ea0/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 118, in __await__
    result = yield self
  File "/Users/martinvo/Library/Caches/nce/2e4f5456e74f7e8b3489c9ced8c844ba1113bfbf37dcaf688859cb976f884ea0/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 623, in native_engine_generator_send
    res = rule.send(arg) if err is None else rule.throw(throw or err)
  File "/Users/martinvo/Library/Caches/nce/2e4f5456e74f7e8b3489c9ced8c844ba1113bfbf37dcaf688859cb976f884ea0/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/backend/python/goals/package_pex_binary.py", line 202, in built_pacakge_for_pex_from_targets_request
    pex = await Get(Pex, PexFromTargetsRequest, pft_request)
  File "/Users/martinvo/Library/Caches/nce/2e4f5456e74f7e8b3489c9ced8c844ba1113bfbf37dcaf688859cb976f884ea0/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 118, in __await__
    result = yield self
  File "/Users/martinvo/Library/Caches/nce/2e4f5456e74f7e8b3489c9ced8c844ba1113bfbf37dcaf688859cb976f884ea0/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 623, in native_engine_generator_send
    res = rule.send(arg) if err is None else rule.throw(throw or err)
  File "/Users/martinvo/Library/Caches/nce/2e4f5456e74f7e8b3489c9ced8c844ba1113bfbf37dcaf688859cb976f884ea0/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/backend/python/util_rules/pex.py", line 800, in create_pex
    result = await Get(BuildPexResult, PexRequest, request)
  File "/Users/martinvo/Library/Caches/nce/2e4f5456e74f7e8b3489c9ced8c844ba1113bfbf37dcaf688859cb976f884ea0/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 118, in __await__
    result = yield self
  File "/Users/martinvo/Library/Caches/nce/2e4f5456e74f7e8b3489c9ced8c844ba1113bfbf37dcaf688859cb976f884ea0/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 623, in native_engine_generator_send
    res = rule.send(arg) if err is None else rule.throw(throw or err)
  File "/Users/martinvo/Library/Caches/nce/2e4f5456e74f7e8b3489c9ced8c844ba1113bfbf37dcaf688859cb976f884ea0/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/backend/python/util_rules/pex.py", line 720, in build_pex
    result = await Get(
  File "/Users/martinvo/Library/Caches/nce/2e4f5456e74f7e8b3489c9ced8c844ba1113bfbf37dcaf688859cb976f884ea0/bindings/venvs/2.16.0/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 118, in __await__
    result = yield self
pants.engine.process.ProcessExecutionFailure: Process 'Building 21 requirements for services.python.axa_service/run.pex from the 3rdparty/python/lockfiles/default.lock resolve: UnleashClient>=5.8.0, aiohttp>=3.8.4, ariadne-django>=0.3.0, ariadne>=0.20.1, cachetools>=5.3.1, django-enum-choices>=2.1.4, django-phonenumber-field[phonenumberslite]>=7.1.0, django>=4, djangorestframework>=3.14.0, orjson>=3.9.5, phonenumbers>=8.13.19, pydantic==1.10.12, python-keycloak>=3.3.0, returns>=0.19.0, sentry-sdk>=1.28.1, slack-sdk>=3.21.3, structlog>=23.1.0, tomli>=2.0.1, twilio>=8.7.0, uvicorn[standard]>=0.23.0, yarl>=1.9.2' failed with exit code 1.
stdout:

stderr:
No pre-built wheel was available for httptools 0.6.0.
Successfully built the wheel httptools-0.6.0-cp39-cp39-macosx_10_9_x86_64.whl from the sdist httptools-0.6.0.tar.gz but it is not compatible with the requested foreign target abbreviated platform cp311-cp311m-linux_x86_64.
You'll need to build a wheel from httptools-0.6.0.tar.gz on the foreign target platform and make it available to Pex via a `--find-links` repo or a custom `--index`.
I already built the linux wheel as mentioned in - https://pantsbuild.slack.com/archives/C046T6T9U/p1692882725766139?thread_ts=1692869602.927969&amp;cid=C046T6T9U
w
FWIW, what I end up doing in my pex files for cross-platform builds tends to look like this:
Copy code
pex_binary(
  name="helloworld-pex",
  entry_point="helloworld.main",
  dependencies=[
    ":libhelloworld",
  ],
  platforms=["linux-x86_64-cp-311-cp311", "macosx-13.3-arm64-cp-311-cp311",]
)
In practice, I then immediately bundle that with my python of choice into a
scie
Though, I did just recall this isn’t in Pants directly, it’s a plugin I wrote.
Copy code
scie_binary(
  name="helloworld",
  dependencies=[":helloworld-pex"],
  platforms=["linux-x86_64", "macos-aarch64"]
)
b
Okay, I'd guess
current
indeed is a bit confused, as maybe it'll be referring to the Pants-internal Python 3.9 interpreter, rather than the correct Python 3.11 one. Sorry about that. Do you have a moment to file an issue about it? For the find links, I'm not sure if the relevant find-links things will be included by default in the sandbox that pants creates. Can you use https://www.pantsbuild.org/docs/troubleshooting#debug-tip-inspect-the-sandbox-with---keep-sandboxes to look at the sandbox, e.g.
pants --keep-sandboxes=on_failure ...
, find the relveant directory and then look at the contents (including
__run.sh
in it)?
👀 1
e
This is the content of these files. Not sure what I should look for though:
Copy code
> cat __run.sh
#!/bin/bash
# This command line should execute the same process as pants did internally.
export PATH=$'/usr/bin:/bin:/usr/local/bin'
cd /private/var/folders/kp/50bqlmp100q03gpnrxdfwdwc0000gn/T/pants-sandbox-QOooEu
./find_binary.sh bash

> cat find_binary.sh
#!/usr/bin/env bash

set -euox pipefail

CHECK_FILE_ENTRIES=
for path in ${PATH//:/ }; do
    if [[ -d "${path}" ]]; then
        # Handle traditional directory PATH element.
        maybe_exe="${path}/$1"
    elif [[ -n "${CHECK_FILE_ENTRIES}" ]]; then
        # Handle PATH elements that are filenames to allow for precise selection.
        maybe_exe="${path}"
    else
        maybe_exe=
    fi
    if [[ "$1" == "${maybe_exe##*/}" && -f "${maybe_exe}" && -x "${maybe_exe}" ]]
    then
        echo "${maybe_exe}"
    fi
done

> ls
__run.sh       find_binary.sh
b
Hm, that looks like a different sandbox. Did you run with
--keep-sandboxes=always
or
--keep-sandboxes=on_failure
? The first one may output a lot of different directories for all the different bits of set-up pants does, while the second one will (hopefully) output just one, for the failing PEX build command...
e
Yeah, used always… Anyway this the output:
Copy code
> ls 3rdparty/python/
lockfiles
Run file:
Copy code
> cat __run.sh
#!/bin/bash
# This command line should execute the same process as pants did internally.
export CPPFLAGS= LC_CTYPE=UTF-8 LDFLAGS=$'-I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib' PATH=$'/Users/martinvo/.pyenv/shims:/Users/martinvo/Downloads/google-cloud-sdk/bin:/usr/local/opt/sphinx-doc/bin:/Users/martinvo/.local/bin:/Users/martinvo/.poetry/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Users/martinvo/.cargo/bin' PEX_IGNORE_RCFILES=true PEX_PYTHON=/Users/martinvo/Library/Caches/nce/2e4f5456e74f7e8b3489c9ced8c844ba1113bfbf37dcaf688859cb976f884ea0/bindings/venvs/2.16.0/bin/python PEX_ROOT=.cache/pex_root
cd /private/var/folders/kp/50bqlmp100q03gpnrxdfwdwc0000gn/T/pants-sandbox-d6uRIR
/Users/martinvo/Library/Caches/nce/2e4f5456e74f7e8b3489c9ced8c844ba1113bfbf37dcaf688859cb976f884ea0/bindings/venvs/2.16.0/bin/python ./pex --tmpdir .tmp --jobs 8 --pip-version 20.3.4-patched --python-path $'/Users/martinvo/.pyenv/versions/3.11.4/bin:/Users/martinvo/.pyenv/shims:/Users/martinvo/Downloads/google-cloud-sdk/bin:/usr/local/opt/sphinx-doc/bin:/Users/martinvo/.local/bin:/Users/martinvo/.poetry/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Users/martinvo/.cargo/bin:/usr/local/bin' --output-file services.python.axa_service/run.pex --no-emit-warnings --requirements-pex local_dists.pex --platform linux-x86_64-cp-311-cp311m --entry-point axa_service.run_uvicorn $'--sources-directory=source_files' $'UnleashClient>=5.8.0' $'aiohttp>=3.8.4' $'ariadne-django>=0.3.0' $'ariadne>=0.20.1' $'cachetools>=5.3.1' $'django-enum-choices>=2.1.4' $'django-phonenumber-field[phonenumberslite]>=7.1.0' $'django>=4' $'djangorestframework>=3.14.0' $'orjson>=3.9.5' $'phonenumbers>=8.13.19' $'pydantic==1.10.12' $'python-keycloak>=3.3.0' $'returns>=0.19.0' $'sentry-sdk>=1.28.1' $'slack-sdk>=3.21.3' $'structlog>=23.1.0' $'tomli>=2.0.1' $'twilio>=8.7.0' $'uvicorn[standard]>=0.23.0' $'yarl>=1.9.2' --lock 3rdparty/python/lockfiles/default.lock --no-pypi $'--index=<https://pypi.org/simple/>' $'--find-links=/Users/martinvo/workspaces/sure_source/3rdparty/python/wheels' --manylinux manylinux2014 --layout zipapp
I tried explicit
file:///Users/martinvo/workspaces/sure_source/3rdparty/python/wheels
value for find links and it seems to get propagated to the wheel command, but failing with the same result
b
hm, that certainly looks like it's pointing to a plausible
--find-links
. The next step here would be reproducing the problem by running
__run.sh
and then, if it does, edit it down to the minimum (e.g. progressively delete the various arguments after
.../python ./pex
alternatively, guessing and trying something: maybe this is caused by a too-old version of pip. Try editing
--pip-version 20.3.4-patched
to
--pip-version latest
?
e
Upgrading pip didn’t help. I tried latest and explicit 23.1. version. It throws the same error when the
__run.sh
is executed. I’ll try to play with this script as suggested, this should be more pleasant debugging then changing pants.toml. Thanks for getting me here. I’m thinking can be the wheel names somehow invalid so they’re not picked up? I don’t have much experience with building wheels, so I wonder I didn’t mess up something. I hoped one of these two will be matched:
Copy code
httptools-0.6.0-cp311-cp311-linux_x86_64.whl       httptools-0.6.0-cp311-py311-none-any.whl
b
yeah, either of those look like they should be matched, in theory
The pex CLI has various powers like passing
-v
several times or setting
PEX_VERBOSE=1
(or higher) for increased verbosity (I think) or an arg something like
—preserve-pip-download-log
(you’ll need to check —help to confirm), which may help with debugging
🙏 1
Hm, side-stepping again: https://pypi.org/project/httptools/#files suggests there's a lot of wheels for
httptools
, so I'm a bit surprised you're needing to build your own. I don't have specific advise or understand the mechanism, just that there's potentially something unexpected going on 🤔
e
So I got down to the minimal not working command using only failing dependency:
Copy code
pex -v httptools==0.6.0 --platform linux-x86_64-cp-311-cp311m
(note pex is alias for the python pex from the
__setup.py
) Resulting in the same error as above. This one works:
Copy code
pex -v httptools==0.6.0
But this one don’t:
Copy code
pex -v --no-index httptools==0.6.0 --find-links=/Users/martinvo/workspaces/sure_source/3rdparty/python/wheels
So it seems that the platform might be the issue. The thing is that I’m building linux on mac, so I can’t use default 🤔
b
Ah, that's useful, I wonder if this might be where stepping from
platform
to
complete_platform
might be appropriate, to give pex more specifics about what platform it needs to choose wheels for https://github.com/pantsbuild/pants/discussions/18756 walks through how we do it (after talking about lambdas)
e
@early-umbrella-56089 remove the trailing
m
in your platform strings. Python discontinued support for that in 3.8.
@wide-midnight-78598 got that right in his example. @broad-processor-92400 also got it right that PyPI has the right wheels; so you can kill your find links.
e
Thank you guys. I was able to get with complete_platform and by removing the trailing
m
from
platform
it seems I was able to create the pex file. However when running it throws an error. It seems that like some mismatch with python version again 🤔
Copy code
> python ./pex -v httptools==0.6.0 --platform linux-x86_64-cp-311-cp311
pex: Building pex :: Resolving distributions (httptools==0.6.0) :: Resolving requirements. :: Resolving for:
pex: Hashing pex: 237.1ms
pex: Isolating pex: 0.0ms
pex: Calculating compatible python versions for ['==3.11.*']: 8.9ms
pex: Building pex :: Resolving distributions (httptools==0.6.0) :: Resolving requirements. :: Calculating project names for direct requirements:
pex: Building pex :: Resolving distributions (httptools==0.6.0) :: Resolving requirements. :: Installing 1 distributions                                                                                                                                                                              pex: Building pex: 1341.6ms
pex:   Resolving distributions (httptools==0.6.0): 1341.0ms
pex:     Resolving requirements.: 1340.7ms
pex:       Resolving for:
  cp311-cp311-linux_x86_64: 1315.6ms
pex:       Calculating project names for direct requirements:
  PyPIRequirement(line=LogicalLine(raw_text='httptools==0.6.0', processed_text='httptools==0.6.0', source='<string>', start_line=1, end_line=1), requirement=Requirement(name='httptools', url=None, extras=frozenset(), specifier=<SpecifierSet('==0.6.0')>, marker=None), editable=False): 0.1ms
pex:       Installing 1 distributions: 16.0ms
pex:       Checking install: 0.2ms
WARNING: attempting to run PEX with incompatible platforms!
Running on platform macosx_13_0_x86_64-cp-3.11.4-cp311 but built for linux_x86_64-cp-311-cp311
Running PEX file at /private/var/folders/kp/50bqlmp100q03gpnrxdfwdwc0000gn/T/tmpxowh93it with args []
pex: Correcting mis-configured MACOSX_DEPLOYMENT_TARGET of 13 to 13.0 corresponding to a valid PEP-425 platform of macosx-13.0-x86_64 for PythonInterpreter('/usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11', PythonIdentity('/usr/local/opt/python@3.11/bin/python3.11', 'cp311', 'cp311', 'macosx_13_0_x86_64', (3, 11, 4))).
pex: PEX.run invoking /usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11 -s -E /private/var/folders/kp/50bqlmp100q03gpnrxdfwdwc0000gn/T/tmpxowh93it
pex: Correcting mis-configured MACOSX_DEPLOYMENT_TARGET of 13 to 13.0 corresponding to a valid PEP-425 platform of macosx-13.0-x86_64 for PythonInterpreter('/usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11', PythonIdentity('/usr/local/opt/python@3.11/bin/python3.11', 'cp311', 'cp311', 'macosx_13_0_x86_64', (3, 11, 4))).
Failed to find compatible interpreter on path /Users/martinvo/Downloads/google-cloud-sdk/bin:/usr/local/opt/sphinx-doc/bin:/Users/martinvo/.local/bin:/Users/martinvo/.poetry/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Users/martinvo/.cargo/bin:/usr/local/bin.

Examined the following interpreters:
1.)     /usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11 CPython==3.11.4
2.)    /usr/local/Cellar/python@3.10/3.10.12_1/Frameworks/Python.framework/Versions/3.10/bin/python3.10 CPython==3.10.12
3.)          /usr/local/Cellar/python@3.7/3.7.16/Frameworks/Python.framework/Versions/3.7/bin/python3.7 CPython==3.7.16
4.) /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/bin/python3.9 CPython==3.9.6

No interpreter compatible with the requested constraints was found:

  A distribution for httptools could not be resolved for /usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11.
  Found 1 distribution for httptools that do not apply:
  1.) The wheel tags for httptools 0.6.0 are cp311-cp311-manylinux1_x86_64, cp311-cp311-manylinux_2_17_x86_64, cp311-cp311-manylinux_2_5_x86_64, cp311-cp311-manylinux2014_x86_64 which do not match the supported tags of /usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11:
  cp311-cp311-macosx_13_0_x86_64
  ... 2412 more ...

  A distribution for httptools could not be resolved for /usr/local/Cellar/python@3.10/3.10.12_1/Frameworks/Python.framework/Versions/3.10/bin/python3.10.
  Found 1 distribution for httptools that do not apply:
  1.) The wheel tags for httptools 0.6.0 are cp311-cp311-manylinux1_x86_64, cp311-cp311-manylinux_2_17_x86_64, cp311-cp311-manylinux_2_5_x86_64, cp311-cp311-manylinux2014_x86_64 which do not match the supported tags of /usr/local/Cellar/python@3.10/3.10.12_1/Frameworks/Python.framework/Versions/3.10/bin/python3.10:
  cp310-cp310-macosx_13_0_x86_64
  ... 2219 more ...

  A distribution for httptools could not be resolved for /usr/local/Cellar/python@3.7/3.7.16/Frameworks/Python.framework/Versions/3.7/bin/python3.7.
  Found 1 distribution for httptools that do not apply:
  1.) The wheel tags for httptools 0.6.0 are cp311-cp311-manylinux1_x86_64, cp311-cp311-manylinux_2_17_x86_64, cp311-cp311-manylinux_2_5_x86_64, cp311-cp311-manylinux2014_x86_64 which do not match the supported tags of /usr/local/Cellar/python@3.7/3.7.16/Frameworks/Python.framework/Versions/3.7/bin/python3.7:
  cp37-cp37m-macosx_13_0_x86_64
  ... 1640 more ...

  A distribution for httptools could not be resolved for /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/bin/python3.9.
  Found 1 distribution for httptools that do not apply:
  1.) The wheel tags for httptools 0.6.0 are cp311-cp311-manylinux1_x86_64, cp311-cp311-manylinux_2_17_x86_64, cp311-cp311-manylinux_2_5_x86_64, cp311-cp311-manylinux2014_x86_64 which do not match the supported tags of /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/bin/python3.9:
  cp39-cp39-macosx_13_0_x86_64
  ... 2026 more ...
e
@early-umbrella-56089 your command builds a PEX for Linux. You should not expect that PEX to work on a Mac.
If you add a
-o some.pex
argument, then a PEX file will be successfully built. Without the
-o
an ephemeral PEX is built (which works) and immediately run (which fails).
h
FWIW I highly recommend upper-bounding your ICs:
Copy code
interpreter_constraints = ["CPython==3.11.*"]
or similar.
1
e
Yeah, good point. Anyway, this the same error I’m getting for my pants run command for this target:
Copy code
pex_binary(
    name="run",
    entry_point="run_uvicorn.py",
    dependencies=[":lib"],
    platforms=["linux-x86_64-cp-311-cp311", "macosx-13.3-arm64-cp-311-cp311"],
)
which is then used by
docker_image
target. Anyway, I’m now able to build it for my platform to run the pex locally or build it for linux so it can run in the docker target. My naive expectation was that specifying platform of my docker image + using “current” (which we found out is not current platform of machine, but platform used by pants) I can achieve that we can use one target for local runs (like testing, running the app locally) and when exporting we can use it as dependency of the docker target. Is something like this possible? Or is there any better way how to approach this?