https://pantsbuild.org/ logo
#general
Title
# general
w

white-address-70112

06/10/2021, 7:31 AM
Hi Experts, Actually i am very new to Pants.. I am facing issues running my pex file in another system . We are using the command (PEX_TOOLS=1 ./pants my_webapp:my_webapp) to package our python project to a pex file. We have developed our project using python 3.7 and i want to bundle and run it as a pex package in a system which has python 2.7 in it without any dependencies. But when i try to run this on the system i m getting the below error:
Copy code
[vagrant@localhost ~]$ ./my_webapp.pex 
/usr/bin/env: python3.9: No such file or directory
[vagrant@localhost ~]$ python --version
Python 2.7.5
So how do i ship venv with the project as a single PEX file, so that this can be executed in any system we want to without any python version requirement? Some expert opinion/advise is much appreciated!! TIA
e

enough-analyst-54434

06/10/2021, 2:57 PM
A few questions first: + What version of Pants are you using? + What is the intention of using
PEX_TOOLS=1
in
PEX_TOOLS=1 ./pants my_webapp:my_webapp
? + Can you provide your BUILD file target? You should be able to configure your BUILD file target to both package your PEX file for multiple target Python versions and use a neutral shebang. For modern Pants that would be by using the
interpreter_constraints
(or
platforms
) and
shebang
fields of the
pex_binary
target. So, part 1 - build the PEX file for multiple Pythons: 1.) You need to run Pants on a machine with all the Python versions you're targeting if the PEX will contain any non-universal wheels. 2.) Add
interpreter_constraints=[">=2.7,<4"]
to your
pex_binary
target: https://www.pantsbuild.org/docs/reference-pex_binary#codeinterpreter_constraintscode Part 2 - ensure the resulting PEX file shebang is
#!/usr/bin/env python
instead of `#!/usr/bin/python3.9`: 1.) Add
shebang="/usr/bin/env python"
to your
pex_binary
target. You can check the result of this step by running Pants to generate the PEX file and executing
head -1 <path to my PEX file>
. You should get
#!/usr/bin/env python
w

white-address-70112

06/14/2021, 10:34 AM
hi @enough-analyst-54434 We are able to get a single pex file but when we are trying to run that in a system that has python 2.7 , we are getting the below error:
Copy code
$./my_app_play.pex
Traceback (most recent call last):
  File "/home/sancbane/my_app_play.pex/.bootstrap/pex/pex.py", line 488, in execute
  File "/home/sancbane/my_app_play.pex/.bootstrap/pex/pex.py", line 144, in activate
  File "/home/sancbane/my_app_play.pex/.bootstrap/pex/pex.py", line 131, in _activate
  File "/home/sancbane/my_app_play.pex/.bootstrap/pex/environment.py", line 427, in activate
  File "/home/sancbane/my_app_play.pex/.bootstrap/pex/environment.py", line 775, in _activate
  File "/home/sancbane/my_app_play.pex/.bootstrap/pex/environment.py", line 601, in resolve
  File "/home/sancbane/my_app_play.pex/.bootstrap/pex/environment.py", line 686, in resolve_dists
pex.environment.ResolveError: Failed to resolve requirements from PEX environment @ /home/sancbane/my_app_play.pex.
Needed manylinux_2_17_x86_64-cp-27-cp27mu compatible dependencies for:
 1: psutil
    Required by:
      ansible-runner 1.4.7
    But this pex had no u'psutil' distributions.
 2: PyYAML
    Required by:
      ansible-runner 1.4.7
    But this pex had no u'PyYAML' distributions
Please find our pants.toml file below:
Copy code
[GLOBAL]
pants_version = "2.4.1"

backend_packages = [
  'pants.backend.python',
  'pants.backend.python.lint.black',
  'pants.backend.python.lint.isort',
  'pants.backend.python.lint.flake8',
  'pants.backend.python.typecheck.mypy',
]

[source]
root_patterns = [
  '/',
]

[python-setup]
# The default interpreter compatibility for code in this repo. Individual targets can override
#  this with the `interpreter_constraints` field. See
#  <https://www.pantsbuild.org/docs/python-interpreter-compatibility>.
interpreter_constraints = [">=2.7"]

# Use a constraints file. See <https://www.pantsbuild.org/docs/python-third-party-dependencies>.
requirement_constraints = "my_app_playbooks/python/constraints.txt"

# We search for interpreters on both on the $PATH and in the `$(pyenv root)/versions` folder.
#  If you're using macOS, you may want to leave off the <PATH> entry to avoid using the
#  problematic system Pythons. See
#  <https://www.pantsbuild.org/docs/python-interpreter-compatibility#changing-the-interpreter-search-path>.
interpreter_search_paths = ["<PATH>", "<PYENV>"]
Could you please advise us on the same
The Build file looks like this:
Copy code
python_library(
  dependencies = [
    'my_app_playbooks/configs',
    'my_app_playbooks/configs/casper_rba_config.py',
    'my_app_playbooks/telemetry',
    'my_app_playbooks/clients',
    'my_app_playbooks/playbooks',
    './ansible_runner',
    ':all',
    './configs:json_resources',
    'my_app_playbooks/python:requirements.txt',
  ],
)

pex_binary(
  #shebang="/usr/bin/env python",
  name='my_app_play',
  entry_point='my_app_play.py',
  zip_safe=False,
  #execution_mode="venv",
  interpreter_constraints=[">=2.7"],
)

archive(
    name="app_with_config",
    packages=[":my_app_play"],
    files=[":allfiles"],
    format="tar.gz",
)

resources(
  name='all',
  sources=['**'],
)

files(
  name="allfiles",
  sources=['hosts','ansible_runner/**','!*.py','!**/BUILD'],
)
e

enough-analyst-54434

06/14/2021, 1:13 PM
@white-address-70112 your interpreter constraints look good; however, Pants (really Pex) will only resolve using the interpreters it can find on your system when building the PEX file. I'm guessing you simply don't have a Python 2.7 interpreter on the machine where you are running Pants to create the PEX file. Is that right?
w

white-address-70112

06/14/2021, 2:25 PM
1. I have written code in Python3 and build the pex file on my mac system which has python 2.7 and 3.7 versions. 2. The PEX file is working fine on the mac system. 3. when i copy that pex file into a system which has linux and run python 2.7 on it , i get the mentioned error.
e

enough-analyst-54434

06/14/2021, 2:33 PM
Aha. Yeah, so for a multi-platform PEX you need to use
platforms
instead of interpreter constraints: https://www.pantsbuild.org/docs/reference-pex_binary#codeplatformscode Also, you'll need to have pre-built wheels available for all distributions. Neither Pants nor Pex has cross-building magic. This is confusing enough that I should provide an example. I'll prepare an example-python branch for you to look at.
w

white-address-70112

06/14/2021, 2:38 PM
I m currently using the below parameters
Copy code
platforms=[
    "manylinux_2_17_x86_64-cp-27-cp27mu",
    "linux_x86_64-cp-27-cp27mu",
    "macosx_10.12_x86_64-cp-36-cp36m",
  ],
but when i try to package it i m getting the below error:
Copy code
sanchayan@sanchayan-mac pex_build % ./pants package my_app_playbooks:my_app_play
19:27:42.68 [INFO] Completed: Resolving 2 requirements: ansible-runner, requests
19:27:42.68 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
  File "/Users/sanchayan/.cache/pants/setup/bootstrap-Darwin-x86_64/2.4.1_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 229, in _run_inner
    return self._perform_run(goals)
  File "/Users/sanchayan/.cache/pants/setup/bootstrap-Darwin-x86_64/2.4.1_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 168, in _perform_run
    return self._perform_run_body(goals, poll=False)
  File "/Users/sanchayan/.cache/pants/setup/bootstrap-Darwin-x86_64/2.4.1_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 190, in _perform_run_body
    poll_delay=(0.1 if poll else None),
  File "/Users/sanchayan/.cache/pants/setup/bootstrap-Darwin-x86_64/2.4.1_py37/lib/python3.7/site-packages/pants/init/engine_initializer.py", line 136, in run_goal_rules
    goal_product, params, poll=poll, poll_delay=poll_delay
  File "/Users/sanchayan/.cache/pants/setup/bootstrap-Darwin-x86_64/2.4.1_py37/lib/python3.7/site-packages/pants/engine/internals/scheduler.py", line 530, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/Users/sanchayan/.cache/pants/setup/bootstrap-Darwin-x86_64/2.4.1_py37/lib/python3.7/site-packages/pants/engine/internals/scheduler.py", line 500, in _raise_on_error
    wrapped_exceptions=tuple(t.exc for t in throws),

Exception message: 1 Exception encountered:

  ProcessExecutionFailure: Process 'Resolving 2 requirements: ansible-runner, requests' failed with exit code 1.
stdout:

stderr:
ERROR: Could not find a version that satisfies the requirement ansible-runner
ERROR: No matching distribution found for ansible-runner
ERROR: Could not find a version that satisfies the requirement ansible-runner
ERROR: No matching distribution found for ansible-runner
pid 84331 -> /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python /Users/sanchayan/.cache/pants/named_caches/pex_root/pip.pex/cf4106b4c7898b462f7c172dde686a9747103f1a --disable-pip-version-check --no-python-version-warning --exists-action a --use-feature 2020-resolver --isolated -q --cache-dir /Users/sanchayan/.cache/pants/named_caches/pex_root --log /private/var/folders/py/mw_r2yw93rv65p8zp60whqqw0000gn/T/process-executiontYLj2S/.tmp/tmpjXFBnp/pip.log download --dest /private/var/folders/py/mw_r2yw93rv65p8zp60whqqw0000gn/T/process-executiontYLj2S/.tmp/tmpZ4uxsO/linux_x86_64-cp-27-cp27mu --platform manylinux2014_x86_64 --platform linux_x86_64 --implementation cp --python-version 27 --abi cp27mu --only-binary :all: --constraint my_app_playbooks/python/constraints.txt ansible-runner requests --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with 1 and STDERR:
None



(Use --print-stacktrace to see more error details.)
e

enough-analyst-54434

06/14/2021, 2:41 PM
Yeah, that's just saying there is no Linux python 2.7 pre-built wheel available. That's the part where Pants / Pex is not magic. They cannot cross build a Linux wheel on your Mac. You'll need to pre-build any needed Linux wheels that aren't available on PyPI and host them at a location you can add to
[python-repos] repos
Does that make sense @white-address-70112?
👍 1
w

white-address-70112

06/15/2021, 6:13 PM
Yes @enough-analyst-54434 that makes sense.. But we are getting another issue 😞
we are packing with ansible but it failed to run the pex saying :
Copy code
/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: can't find '__main__' module in './dist//my_playbooks/play.pex'

⠂ 196.46s Resolving 5 requirements: ansible, ansible-runner, psutil, requests, sshtunnel
e

enough-analyst-54434

06/15/2021, 6:31 PM
Can you provide a listing of that PEX file? `unzip -l ./dist//my_playbooks/play.pex' should do it. Or better, can you provide the PEX file itself?
w

white-address-70112

06/15/2021, 6:42 PM
my_app_pex_unzip_lst.txt
e

enough-analyst-54434

06/15/2021, 7:10 PM
Hrm, that does have a
__main__.py
. The uncompressed size of the PEX zip is ~700MB which is not interesting. It has 120,950 files (entries) though and that is potentially interesting since storing more than ~2^16 files requires zip64 extensions. .... and - lo and behold: https://bugs.python.org/issue32959
So, you need to use python 3.9 for zipimport - which PEX relies on - to work with a zipfile containing more than ~2^16 entries.
f

full-intern-53951

06/15/2021, 7:11 PM
@enough-analyst-54434 FYI: if we put ansible in constraints it not packaging so we have imported the ansible in entrypoint pex_build % cat casper_sre_playbooks/python/constraints.txt requests sshtunnel psutil ansible-runner
e

enough-analyst-54434

06/15/2021, 7:12 PM
@full-intern-53951 one problem at a time please. I just want to make sure this 1st problem is understood. Does it make sense?
f

full-intern-53951

06/15/2021, 7:13 PM
@enough-analyst-54434 Got it, so do we need to try changing interpreter_constraints to 3.9 ?
Copy code
interpreter_constraints=[">=3.9.0","<=3.9.9"],
e

enough-analyst-54434

06/15/2021, 7:14 PM
I proved this bug is present in python 3.8 but not python 3.9 with this myself:
Copy code
mkdir test && for i in `seq 1 120950`; do touch test/example_$i.py; done
zip -q numerous_but_small.zip test/
$ PYTHONPATH=numerous_but_small.zip python3.8 -c 'import test.example_1; print(test.example_1.__file__)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'test.example_1'
$ PYTHONPATH=numerous_but_small.zip python3.9 -c 'import test.example_1; print(test.example_1.__file__)'
/home/jsirois/Downloads/test/example_1.py
@full-intern-53951 yes. Is that OK? In other words, are the places you want to build and run this PEX equipped with python 3.9 interpreters?
f

full-intern-53951

06/15/2021, 7:17 PM
Sounds good, our goal is build the executable and we can install python 3.9 in target system, I am currently trying to build with interpreter_constraints=[">=3.9.0","<=3.9.9"],
e

enough-analyst-54434

06/15/2021, 7:17 PM
To be clear here: Pex itself can handle zip64; it's only the bootstrap process that happens before Pex runtime code is executed, that cannot on python <= 3.8. That code is CPython c code that implements zip support. And that CPython c code doesn't handle zip64. The python zipfile module does though! That's implemented in python.
OK, great. To bullet-proof / simplify that interpreter constraints you could just say ">=3.9,<4"
Oh, yeah - actually, subtle but wrong too is the fact you have two entries in the list - you want just one list entry. Each seperate entry is ORed, so your constraints say >=3.9.0 OR <=3.9.9 which mean 2.7 works and that's probably not what you want. You need 1 string, exactly:
interpreter_constraints=[">=3.9,<4"],
The comma embedded in that string acts as AND
f

full-intern-53951

06/15/2021, 7:28 PM
@enough-analyst-54434 I used the same as you mentioned, but It seem pants using python3.8 for bootstrapping ./pants package my_app_playbooks:my_app_play --print-stacktrace -ldebug Bootstrapping Pants using /Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 Downloading the virtualenv zipapp
e

enough-analyst-54434

06/15/2021, 7:29 PM
That's fine. A confusing thing is Pants itself needs a Python to run Pants with. Pants can accept Python 3.7, 3.8 or 3.9. It will build your code though using the interpreter you asked for.
One way to verify is to run
head -1 my.pex
after you use Pants to create your PEX file. You should notice a python 3.9 shebang line.
f

full-intern-53951

06/15/2021, 7:32 PM
Copy code
pex_binary(
  shebang="/usr/bin/env python3.9",
  name='my_app_play',
  entry_point='my_app_play.py',
  zip_safe=False,
  execution_mode="unzip",
  interpreter_constraints=[">=3.9,<4"],

)
also, I see it not packaging ansible, please check the below 7.96s Resolving 4 requirements: ansible-runner, psutil, requests, sshtunnel
not sure why is it not picking up the ansible when I change interpreter_constraints to 3.9
% head -1 casper_sre_play.pex #!/usr/bin/env python3.9
e

enough-analyst-54434

06/15/2021, 7:34 PM
You shouldn't need that shebang setting, although it causes no harm - in other words, if you remove that field from your pex_binary - you should still get a PEX file with a python 3.9 shebang.
Thinking about the ansible problem ...
f

full-intern-53951

06/15/2021, 7:35 PM
after I built it 3.9, execution is failing with below error as ansible module did not packaged import ansible ModuleNotFoundError: No module named 'ansible'
e

enough-analyst-54434

06/15/2021, 7:36 PM
So, does any of your code actually
import ansible
? If not, Pants can't know ansible should be included.
f

full-intern-53951

06/15/2021, 7:36 PM
in entry point
Copy code
import ansible
w

white-address-70112

06/15/2021, 7:37 PM
Yes we are importing ansible inside "my_app_play.py"
e

enough-analyst-54434

06/15/2021, 7:39 PM
And you have a python_library() target along side that pex_binary target in the same BUILD file?
w

white-address-70112

06/15/2021, 7:44 PM
from python_library:
Copy code
python_library(
  dependencies = [
    'my_app_playbooks/configs',
    'my_app_playbooks/configs/my_rba_config.py',
    'my_app_playbooks/telemetry',
    'my_app_playbooks/clients',
    'my_app_playbooks/playbooks',
    './ansible_runner',
    ':all',
    './configs:json_resources',
    'my_app_playbooks/python:requirements.txt',
  ],
)
Copy code
cat  my_app_playbooks/python/requirements.txt
ansible
requests
sshtunnel
psutil
ansible-runner
e

enough-analyst-54434

06/15/2021, 7:46 PM
So one super suspicious thing is that your requirements.txt has no version constrainst for ansible and the latest ansible is 4.1.0 released 5 days agio, but your listing of the python3.8 zip has ansible 3.4.0.
w

white-address-70112

06/15/2021, 7:49 PM
we are adding ansible version in the requiremment.txt file and repackaging it
will let u know the outcome
e

enough-analyst-54434

06/15/2021, 7:50 PM
What version of Pants is this by the way? You may have answered in some other thread but I've forgotten if so.
w

white-address-70112

06/15/2021, 7:51 PM
2.4.1
@enough-analyst-54434 it didnt package ansible after we mentioned the version of ansible in the requirements.txt
Copy code
cat  my_app_playbooks/python/requirements.txt
ansible=4.1.0
requests
sshtunnel
psutil
ansible-runner
Copy code
3.45s Resolving 4 requirements: ansible-runner, psutil, requests, sshtunnel
e

enough-analyst-54434

06/15/2021, 7:55 PM
What does the following command say for your pex_binary target?:
./pants dependencies --transitive --type=source-and-3rdparty your/pex_binary:target
f

full-intern-53951

06/15/2021, 8:03 PM
Depens.txt
e

enough-analyst-54434

06/15/2021, 8:07 PM
In that listing I do not see my_app_play.py which is your pex_binary entrypoint. Am I just missing it?
Ok, there it is: my_app_playbooks/my_app_play.py
Um, looking way back up this thread - I really really don't like the look of your all target. That globs everything and so now the all target owns not just actual resource files, but your python files too. I think what's happening is both your python_library target and your all resources target "own" my_app_playbooks/my_app_play.py which defeats dependency inference. Can you trim down your all resources target to only grab resources?
f

full-intern-53951

06/15/2021, 8:20 PM
can we exclude python file like below, will that work
Copy code
resources(
  name='all',
  sources=['**', '!*.py'],
)
e

enough-analyst-54434

06/15/2021, 8:30 PM
That should work.
f

full-intern-53951

06/15/2021, 8:35 PM
This time it picked up, but we see below error WARNING: Discarding https://files.pythonhosted.org/packages/42/c6/e97fbab98d4d80f423cfdb3f82ed2d6461d9e608cd93422242b072d47b1a/ansible-4.1.0.tar.gz#sha256=f561cca7fbc4daa14d98e18cd0cb74bd8b173f1501b8fa11543f6ef002de3167 (from https://pypi.org/simple/ansible/) (requires-python:>=2.7,!=3.0.,!=3.1.,!=3.2.,!=3.3.,!=3.4.*). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output. ERROR: Could not find a version that satisfies the requirement ansible ERROR: No matching distribution found for ansible
3.1.0
this is what we should use it
let me try that
Copy code
gtripura@gtripura-mac pex_build % pip3.8 freeze | grep ansible
ansible==4.1.0
ansible-core==2.11.1
ansible-runner==1.4.7
gtripura@gtripura-mac pex_build % pip3.9 freeze | grep ansible
ansible==3.1.0
ansible-base==2.10.7
ansible-runner==1.4.7
e

enough-analyst-54434

06/15/2021, 8:42 PM
Just to back up a bit. There was a key lesson there. Pants dependency inference only works if files have unique target ownership. Keep that strongly in mind. There is a tension since ** seems more convenient. You'll get burned by that though as here.
With these lats edits are things looking better?
f

full-intern-53951

06/15/2021, 9:16 PM
@enough-analyst-54434 the PEX is working fine from my system, but see issue executing it from target system it is not working % ./my_app_play.pex -t IOS-33165 validating auth token.... Remediation starting for cschema-service-01301.node.ad1.1 heartbeat : casper_heartbeat.yml casper_heartbeat.yml ERROR! Unexpected Exception, this is probably a bug: No module named ‘psutil’ to see the full traceback, use -vvv
"ERROR! Unexpected Exception, this is probably a bug: No module named ‘psutil’"
e

enough-analyst-54434

06/15/2021, 9:17 PM
Sure, so tell me more about your system vs the other. How are they different?
f

full-intern-53951

06/15/2021, 9:17 PM
we see the psutil is packaged in the binary
e

enough-analyst-54434

06/15/2021, 9:18 PM
What prefix does psutil have in the PEX?
For example, is your system macOS and the other Linux?
a

acoustic-computer-61887

06/15/2021, 9:20 PM
f

full-intern-53951

06/15/2021, 9:21 PM
both are mac
when I extracted pex, I see .deps/psutil-5.8.0-cp39-cp39-macosx_10_9_x86_64.whl/
e

enough-analyst-54434

06/15/2021, 9:23 PM
Ok. And both macs do have a python3.9 interpreter on the PATH?
f

full-intern-53951

06/15/2021, 9:23 PM
currently I am using below parameters in pex_binary
Copy code
zip_safe=False,
execution_mode="unzip",
do I need to change anything there
e

enough-analyst-54434

06/15/2021, 9:23 PM
Maybe, but 1st, can you confirm both macs have python3.9 on the PATH?
f

full-intern-53951

06/15/2021, 9:25 PM
yes both are MAC with python3.9
other modules are working fine like requests and sshtunnel
e

enough-analyst-54434

06/15/2021, 9:25 PM
K. Try removing the zip_safe field and setting execution_mode="venv".
f

full-intern-53951

06/15/2021, 9:26 PM
okay
We are still seeing the same issue ERROR! Unexpected Exception, this is probably a bug: No module named ‘psutil’
e

enough-analyst-54434

06/15/2021, 9:56 PM
Hrm. One debugging approach is to try to build this same PEX on the other machine and then compare PEX files for differences.
f

full-intern-53951

06/15/2021, 10:03 PM
sure, we try tomorrow, actually we work in IST time zone (India) and its already Morning for us, it is a great help, we were able to fix some of the issues, Tomorrow we will try building it on other system and keep you posted
@enough-analyst-54434 Appreciate your Help!
e

enough-analyst-54434

06/15/2021, 10:06 PM
You're welcome. Hopefully you'll find differences in the PEX files tomorrow and that will make the remedy obvious.
🙏 1
w

white-address-70112

06/16/2021, 4:05 PM
hi @enough-analyst-54434 could you please help us with the procedure to build executable for linux from mac
f

full-intern-53951

06/16/2021, 4:18 PM
@enough-analyst-54434 Good Morning, It working fine after we installing the pip install Ansible but when I extract the pex I can see 2 ansible whl directories in .deps drwxr-xr-x 4 gtripura staff 128 Jan 1 1980 ansible-4.1.0-py3-none-any.whl drwxr-xr-x 4 gtripura staff 128 Jan 1 1980 ansible-3.1.0-py3-none-any.whl
Traceback (most recent call last): File "/usr/local/bin/ansible-playbook", line 34, in <module> from ansible import context ModuleNotFoundError: No module named 'ansible'
e

enough-analyst-54434

06/16/2021, 4:44 PM
Hey folks. I'm out of the office here for a few weeks. You'll need to cast a wider net for help. Before doing that though it may make sense to internally decide on one person to be the primary communicator and have that person come up with a good summary of the current problems such that whoever takes over helping you out can quickly understand the relevant details.