https://pantsbuild.org/ logo
Powered by
m

microscopic-refrigerator-69774

01/24/2023, 3:12 AM
Is there any proper step by step guide to make pex file platform independent? Mainly I want to run same pex file in macOS and linux?
h

high-yak-85899

01/24/2023, 3:19 AM
I don’t see how that would be possible. Aren’t the wheels that get put into them specific to the operating system?
m

microscopic-refrigerator-69774

01/24/2023, 3:23 AM
Sorry, I am a bit new to this framework and I am struggling to distribute my code for both Linux and macOS
e

enough-analyst-54434

01/24/2023, 3:54 AM
It's possible. A PEX can contain wheels for N platforms, it will activate the correct set on a given platform.
@microscopic-refrigerator-69774 this can often be non-trivial because it fundamentally requires wheels for all transitive dependencies of the PEX for all foreign platforms. Just one sdist-only dependency can scuttle things.
You'll need to either use platforms or complete_platforms in a 
pex_binary
target to achieve this. You can see how far this gets you: https://www.pantsbuild.org/docs/reference-pex_binary
@high-yak-85899 you might inspect the Pants PEX we attach to releases: https://github.com/pantsbuild/pants/releases those support many platforms in one and are hugish as a result.
m

microscopic-refrigerator-69774

01/24/2023, 4:54 AM
Thanks, @enough-analyst-54434 for the reply. Mainly I want to build my pex for Linux irrespective of the local machine environment. So for this what will be the right string to pass in platforms or complete_platforms
e

enough-analyst-54434

01/24/2023, 5:29 AM
@microscopic-refrigerator-69774 if you read the doc link above for complete_platforms, you'll find your answer, which is basically, you can't guess, you need to run a command. The Pants complete_platform doc links here: https://pex.readthedocs.io/en/latest/buildingpex.html#complete-platform and that says use 
pex3 interpreter inspect --markers --tag
. You want to do that for each Linux distribution you target. Ideally you can just run that command in the target container using docker. You want to be careful though to target the right Python interpreter. Some Linux distributions (or containers) have many. As the Pex docs call out, use 
--help
to find out more about arguments like 
--python
to help ensure you target the Python you intend.
For example:
Copy code
$ docker run -v $PWD:/share --rm -t python:3.11 bash -c 'python -mvenv pex.venv && ./pex.venv/bin/pip install -U pex && ./pex.venv/bin/pex3 interpreter inspect --python /usr/local/bin/python3.11 --markers --tags --indent 2 > /share/python3.11-complete-platform.json'
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
Collecting pex
  Downloading pex-2.1.121-py2.py3-none-any.whl (2.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.9/2.9 MB 4.9 MB/s eta 0:00:00
Installing collected packages: pex
Successfully installed pex-2.1.121
And that plopped out a complete_platform JSON file you can use in Pants:
Copy code
$ head python3.11-complete-platform.json
{
  "path": "/usr/local/bin/python3.11",
  "compatible_tags": [
    "cp311-cp311-manylinux_2_31_x86_64",
    "cp311-cp311-manylinux_2_30_x86_64",
    "cp311-cp311-manylinux_2_29_x86_64",
    "cp311-cp311-manylinux_2_28_x86_64",
    "cp311-cp311-manylinux_2_27_x86_64",
    "cp311-cp311-manylinux_2_26_x86_64",
    "cp311-cp311-manylinux_2_25_x86_64",
$ tail python3.11-complete-platform.json
    "platform_machine": "x86_64",
    "platform_python_implementation": "CPython",
    "platform_release": "5.15.79.1-microsoft-standard-WSL2",
    "platform_system": "Linux",
    "platform_version": "#1 SMP Wed Nov 23 01:01:46 UTC 2022",
    "python_full_version": "3.11.1",
    "python_version": "3.11",
    "sys_platform": "linux"
  }
}
8 Views
Powered by