Hi pants-people, I was invited into this slack due to https://github.com/pantsbuild/pex/issues/1477#issuecomment-941070905. After playing around a while with pex, I feel like I'm having some basic-understanding issues, since I'm running into issue after issue. My use case and assumptions: • I want to build a pex file that contains a command line utility, and distribute it within my company for pipelines (debian) and humans (3+ linux archs and Mac) to use, targeting cpython 3.8, 3.9, and tentatively 3.10 • I need fine-grained control over the dependencies, meaning that only 3rd party packages are used that were part of a successful test-suite exection (i.e., I run pex with

--no-pypi

and

--repo my_wheelhouse

) • to populate

my_wheelhouse

, I build my application as a wheel, as well as all variants of my dependencies that I will need for the platforms I plan to support (plus

setuptools

, for some reason) • since I am providing all the necessary dependencies, pex doesn't need to do run any build steps, so it doesn't matter which interpreter pex itself is installed with, or which ones it can see on the PATH Is everything ok so far? In particular the last point seems to be an issue, because my error messages change a lot depending on where pex was installed. I initially used

pipx

because it felt to me like a non-project bound build tool that I would want to install globally, and not into the virtualenv of my application. Similar to

poetry

, if that makes sense (by now, in my desperation, I have I think 4 installations of pex floating around on my system). I can try to build a reproducible example, but it would take me a while. I was hoping that someone can clear up for me how pex wants to be used instead 😃

Per the issue, what is the Pex command line you're using to create the PEX file with (key part being the list of

--platform X --platform Y ...

you use) vs the error you get when you try to run the PEX file on one of those platforms later?

for now, my platform tags are just

--platform manylinux_2_27_x86_64-cp-39-cp39

and

--platform manylinux_2_27_x86_64-cp-38-cp38

, and I get errors during build. If I provide the dependencies as positional arguments (

pex my_app setuptools greenlet ...

) I get

ERROR: Could not find a version that satisfies the requirement greenlet (from versions: none)
ERROR: No matching distribution found for greenlet
ERROR: Could not find a version that satisfies the requirement greenlet (from versions: none)
ERROR: No matching distribution found for greenlet
pid 22219 -> /home/me/.pex/venvs/4752d3a3799c096df952bdf2bf57e9caef61d7d8/4efcf8bbdd6ee48bfcee67ce2edeb12d1c553762/pex --disable-pip-version-check --no-python-version-warning --exists-action a --use-deprecated legacy-resolver --isolated -q --cache-dir /home/me/.pex --log /tmp/tmpeif168zc/pip.log download --dest /tmp/tmpi9aoh380/manylinux_2_27_x86_64-cp-39-cp39 --platform manylinux_2_27_x86_64 --implementation cp --python-version 39 --abi cp39 --only-binary :all: appdirs certifi charset-normalizer click dockerfile-parse gitdb gitpython greenlet idna pymysql python-gitlab requests-toolbelt requests semver six smmap sqlalchemy termcolor typing-extensions urllib3 setuptools my_application --no-index --find-links wheelhouse/ --retries 5 --timeout 15 exited with 1 and STDERR:
None

If I provide a requirements.txt with fixed versions, I get

ERROR: Exception:
Traceback (most recent call last):
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 223, in _main
    status = self.run(options, args)
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 180, in wrapper
    return func(self, options, args)
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_internal/commands/download.py", line 130, in run
    requirement_set = resolver.resolve(
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 170, in resolve
    requirement_set.add_requirement(req)
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_internal/req/req_set.py", line 90, in add_requirement
    if not install_req.match_markers(extras_requested):
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 290, in match_markers
    return any(
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 291, in <genexpr>
    self.markers.evaluate({'extra': extra})
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_vendor/packaging/markers.py", line 328, in evaluate
    return _evaluate_markers(self._markers, current_environment)
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_vendor/packaging/markers.py", line 244, in _evaluate_markers
    lhs_value = _get_env(environment, lhs.value)
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_vendor/packaging/markers.py", line 224, in _get_env
    raise UndefinedEnvironmentName(
pip._vendor.packaging.markers.UndefinedEnvironmentName: 'python_full_version' does not exist in evaluation environment.
ERROR: Exception:
Traceback (most recent call last):
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 223, in _main
    status = self.run(options, args)
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 180, in wrapper
    return func(self, options, args)
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_internal/commands/download.py", line 130, in run
    requirement_set = resolver.resolve(
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 170, in resolve
    requirement_set.add_requirement(req)
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_internal/req/req_set.py", line 90, in add_requirement
    if not install_req.match_markers(extras_requested):
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 290, in match_markers
    return any(
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 291, in <genexpr>
    self.markers.evaluate({'extra': extra})
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_vendor/packaging/markers.py", line 328, in evaluate
    return _evaluate_markers(self._markers, current_environment)
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_vendor/packaging/markers.py", line 244, in _evaluate_markers
    lhs_value = _get_env(environment, lhs.value)
  File "/home/me/.pex/venvs/short/da72fb94/lib/python3.8/site-packages/pip/_vendor/packaging/markers.py", line 224, in _get_env
    raise UndefinedEnvironmentName(
pip._vendor.packaging.markers.UndefinedEnvironmentName: 'python_full_version' does not exist in evaluation environment.
pid 10603 -> /home/me/.pex/venvs/4752d3a3799c096df952bdf2bf57e9caef61d7d8/4efcf8bbdd6ee48bfcee67ce2edeb12d1c553762/pex --disable-pip-version-check --no-python-version-warning --exists-action a --use-deprecated legacy-resolver --isolated -q --cache-dir /home/me/.pex --log /tmp/tmpv60fjclh/pip.log download --dest /tmp/tmpenn81s51/manylinux_2_27_x86_64-cp-39-cp39 --platform manylinux_2_27_x86_64 --implementation cp --python-version 39 --abi cp39 --only-binary :all: --requirement requirements.txt setuptools auxcli --no-index --find-links wheelhouse/ --retries 5 --timeout 15 exited with 2 and STDERR:
Failed to resolve for platform manylinux_2_27_x86_64-cp-39-cp39. Resolve requires evaluation of unknown environment marker: 'python_full_version' does not exist in evaluation environment.

Concentrating on the 1st scenario, what are the wheel names you expect Pex to find to satisfy the greenlet dependency based on your

--index

and

--find-links

settings?

greenlet-1.1.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

and

greenlet-1.1.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

I think you have enough info here actually. I've been silent but repro-ing.

I'm getting

Failed to resolve for platform linux_x86_64-cp-39-cp39. Resolve requires evaluation of unknown environment marker: 'platform_machine' does not exist in evaluation environment.

for that, from

pip._vendor.packaging.markers.UndefinedEnvironmentName: 'platform_machine' does not exist in evaluation environment.

And yeah, I have a termcolor wheel in the local repo

Thanks. Ok, let me dig into that a bit...

thanks a lot! I can help test the fix as soon as you've published the PR

You're welcome. Should be up shortly...

Thanks, I think I'll be able to manage from here on. Good luck on your trip!

Alright @millions-kilobyte-51167 Pex 2.1.52 has the fix: https://pypi.org/project/pex/2.1.52/ Let me know if this gets you down the road to the finish line or you hit other issues.

Alright, the build works now 🎉 When I run the resulting .pex with python3.9 on my path it works as expected, when I run it with 3.8 I get

/usr/bin/env: 'python3.9': No such file or directory

The output of

pex-tools my_app.pex info

is

{
  "bootstrap_hash": "3ec57293c54b7396d475ff4ae04db5ec823f6992",
  "build_properties": {
    "pex_version": "2.1.52"
  },
  "code_hash": "da39a3ee5e6b4b0d3255bfef95601890afd80709",
  "distributions": {
    "GitPython-3.1.24-py3-none-any.whl": "77355767101276ed6ceaf9ebf9606c995ce1672b",
    "PyMySQL-0.9.3-py2.py3-none-any.whl": "9cb76f040fbca52dafa4028b99b72f03189308d4",
    "SQLAlchemy-1.4.25-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "0772cc228abeac10caabc1aea12b3aa7eb917399",
    "SQLAlchemy-1.4.25-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "b560e2ff38b3d8ccf28e51db7ad83e61d428833c",
    "appdirs-1.4.4-py2.py3-none-any.whl": "b219560d4b28b017da90ce2c5484a2b6c62dfc52",
    "my_app-0.1.0-py3-none-any.whl": "6f0464af92d9ea860ecaf5c78b77106d8f18a629",
    "certifi-2021.10.8-py2.py3-none-any.whl": "ec6c8fc5615d73da818b9e3229f56b26c780d655",
    "charset_normalizer-2.0.6-py3-none-any.whl": "2d4f81e24c36da4736854f603fbc1f701e9dd144",
    "click-7.1.2-py2.py3-none-any.whl": "386b0f74344f046086c84bc4bc5c4f27f2f9a108",
    "dockerfile_parse-1.2.0-py2.py3-none-any.whl": "fcc88de95d27e7bf22517cdcff39dedaf39fed99",
    "gitdb-4.0.7-py3-none-any.whl": "cbb0a1a2c7f68bf77771d0c21546565b34ad655a",
    "greenlet-1.1.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "4493387f003343fbfedd8fd2042b7476868d2ebf",
    "greenlet-1.1.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "3973b70e466c646570645d00e65e28a8ec838eed",
    "idna-3.2-py3-none-any.whl": "f6f18646cd1fcc3b8d439bd113b7cc9ad4ebc3c0",
    "python_gitlab-2.10.1-py3-none-any.whl": "c8dfdc28961b02193158f9ba86c21cd7307100c8",
    "requests-2.26.0-py2.py3-none-any.whl": "fb70e1b8449a6408d983fb8c56f60f1e827890d8",
    "requests_toolbelt-0.9.1-py2.py3-none-any.whl": "1bbd7c07b41e671ad35e7784c56b5fa6ff9461f4",
    "semver-2.13.0-py2.py3-none-any.whl": "ccc5c528f6a241a427a8848b65617b6f4d652adc",
    "setuptools-58.2.0-py3-none-any.whl": "d025ab315362311e9bdb0d37e24d572e95f438e2",
    "six-1.16.0-py2.py3-none-any.whl": "035d7c208925c1832def39b592f3477ca36397bf",
    "smmap-4.0.0-py2.py3-none-any.whl": "3bc43d31c55385dd9f9f8ac4df4dc9575abb93f1",
    "termcolor-1.1.0-py3-none-any.whl": "ff40285b0a9a85789094b0aa04981f79386ef6f1",
    "typing_extensions-3.10.0.2-py3-none-any.whl": "e1248851ba7610196a1039644897ce611bd8536d",
    "urllib3-1.26.7-py2.py3-none-any.whl": "0468d103d09253d2e6af05983feaa1ab4d39394c"
  },
  "emit_warnings": true,
  "entry_point": "my_app.cli:main",
  "ignore_errors": false,
  "includes_tools": false,
  "inherit_path": "false",
  "interpreter_constraints": [],
  "pex_hash": "9f2024f13eb801f8fe1fda0b247ea9ed6dcc796c",
  "pex_path": null,
  "requirements": [
    "appdirs",
    "my_app",
    "certifi",
    "charset-normalizer",
    "click",
    "dockerfile-parse",
    "gitdb",
    "gitpython",
    "greenlet",
    "idna",
    "pymysql",
    "python-gitlab",
    "requests",
    "requests-toolbelt",
    "semver",
    "setuptools",
    "six",
    "smmap",
    "sqlalchemy",
    "termcolor",
    "typing-extensions",
    "urllib3"
  ],
  "strip_pex_env": true,
  "venv": false,
  "venv_bin_path": "false",
  "venv_copies": false,
  "pex_root": "/home/arecknagel/.pex"
}

Yeah, Pex will not do magic with your shebang. You probably want to specify something like

--python-shebang '/usr/bin/env python'

or

--python-shebang /usr/bin/python3

and if either of those might pick a non-applicable python like python 3.7, you'll also need to specify something like

--interpreter-constraint '>=3.8,<3.10'

as well.

ok, I think that will work out just fine

You can always run a PEX via

/usr/bin/python3.8 my.pex

and if the interpreter you picked is compatible with all

--interpreter-constraint

and

--python-path

(if any) you specified in the build, the PEX will run under that Python.

ah, perfect.

But you can't test macOS that way of course (in Linux CI for example). I curse macOS rules pertaining to vms.

Not just you