pantsbuild #pex

Channels

welcome

github-notifications

development

faint-holiday-3863

07/18/2016, 10:26 PM

conveniently, in bazel if you have it fetch sdists or eggs/wheels using http_file(), it results in a directory containing only the one file you’re looking for

rough-minister-58256

07/18/2016, 10:26 PM

ah nice

faint-holiday-3863

07/18/2016, 10:26 PM

so you can skip collecting the files in a new directory

faint-holiday-3863

07/18/2016, 10:27 PM

The one thing I still haven’t figured out is how to make this work without requiring the patch in https://github.com/pantsbuild/pex/pull/277

faint-holiday-3863

07/18/2016, 10:27 PM

which is really annoying

faint-holiday-3863

07/18/2016, 10:28 PM

I was hoping that building a pex.pex with setuptools and wheel embedded would work, but no luck so far

faint-holiday-3863

07/18/2016, 10:30 PM

but I think that’s because I’m doing something wrong while driving the API

faint-holiday-3863

07/18/2016, 10:31 PM

it’s ending with this mess:

Copy code

Traceback (most recent call last):
  File ".bootstrap/_pex/pex.py", line 328, in execute
  File ".bootstrap/_pex/pex.py", line 260, in _wrap_coverage
  File ".bootstrap/_pex/pex.py", line 292, in _wrap_profiling
  File ".bootstrap/_pex/pex.py", line 371, in _execute
  File ".bootstrap/_pex/pex.py", line 429, in execute_entry
  File ".bootstrap/_pex/pex.py", line 434, in execute_module
  File "/usr/lib/python2.7/runpy.py", line 180, in run_module
    fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/nonexistent/.pex/install/pex_wrapper-0.0.0-py2.7.egg.355ab1e4a2649222db04cd7b5a7f2a400079ce2a/pex_wrapper-0.0.0-py2.7.egg/pex_wrapper.py", line 256, in <module>
    sys.exit(main())
  File "/nonexistent/.pex/install/pex_wrapper-0.0.0-py2.7.egg.355ab1e4a2649222db04cd7b5a7f2a400079ce2a/pex_wrapper-0.0.0-py2.7.egg/pex_wrapper.py", line 253, in main
    pex_builder.build(output)
  File "/nonexistent/.pex/install/pex-1.1.13-py2.py3-none-any.whl.54e2486bd5c6fda99021a6500098c4976315d6fc/pex-1.1.13-py2.py3-none-any.whl/pex/pex_builder.py", line 424, in build
    self.freeze(bytecode_compile=bytecode_compile)
  File "/nonexistent/.pex/install/pex-1.1.13-py2.py3-none-any.whl.54e2486bd5c6fda99021a6500098c4976315d6fc/pex-1.1.13-py2.py3-none-any.whl/pex/pex_builder.py", line 411, in freeze
    self._precompile_source()
  File "/nonexistent/.pex/install/pex-1.1.13-py2.py3-none-any.whl.54e2486bd5c6fda99021a6500098c4976315d6fc/pex-1.1.13-py2.py3-none-any.whl/pex/pex_builder.py", line 335, in _precompile_source
    compiled_relpaths = compiler.compile(self._chroot.path(), source_relpaths)
  File "/nonexistent/.pex/install/pex-1.1.13-py2.py3-none-any.whl.54e2486bd5c6fda99021a6500098c4976315d6fc/pex-1.1.13-py2.py3-none-any.whl/pex/compiler.py", line 86, in compile
    out, _ = Executor.execute([self._interpreter.binary, fp.name])
  File "/nonexistent/.pex/install/pex-1.1.13-py2.py3-none-any.whl.54e2486bd5c6fda99021a6500098c4976315d6fc/pex-1.1.13-py2.py3-none-any.whl/pex/executor.py", line 83, in execute
    stdout_raw, stderr_raw = process.communicate(input=stdin_payload)
AttributeError: 'NoneType' object has no attribute 'communicate'

faint-holiday-3863

07/18/2016, 10:31 PM

ack, I wasn’t expecting slack to fill the whole screen with that, sorry

rough-minister-58256

07/18/2016, 10:32 PM

ah - think thats related to the new

Executor

stuff. mind filing a pex bug w/ the trace?

faint-holiday-3863

07/18/2016, 10:32 PM

pex_builder.interpreter.binary

appears to be set to a directory location, not an actual python binary. Is that expected?

faint-holiday-3863

07/18/2016, 10:33 PM

that’s after letting build_pex pick the interpreter

rough-minister-58256

07/18/2016, 10:33 PM

hm strange

faint-holiday-3863

07/18/2016, 10:34 PM

I’m not sure what to put in the bug description but I’m filing it

rough-minister-58256

07/18/2016, 10:36 PM

you might try 1.1.11

faint-holiday-3863

07/18/2016, 10:36 PM

this is 1.1.13

faint-holiday-3863

07/18/2016, 10:36 PM

oh wait it says that loud and clear

faint-holiday-3863

07/18/2016, 10:36 PM

okay

rough-minister-58256

07/18/2016, 10:37 PM

1.1.12 moves all subprocess execution under care of an

Executor

which provides better error handling etc; may be a couple uncovered bugs therein

faint-holiday-3863

07/18/2016, 10:37 PM

Heh, the error changes with 1.1.11 but:

faint-holiday-3863

07/18/2016, 10:37 PM

Copy code

Traceback (most recent call last):
  File ".bootstrap/_pex/pex.py", line 328, in execute
  File ".bootstrap/_pex/pex.py", line 260, in _wrap_coverage
  File ".bootstrap/_pex/pex.py", line 292, in _wrap_profiling
  File ".bootstrap/_pex/pex.py", line 371, in _execute
  File ".bootstrap/_pex/pex.py", line 429, in execute_entry
  File ".bootstrap/_pex/pex.py", line 434, in execute_module
  File "/usr/lib/python2.7/runpy.py", line 180, in run_module
    fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/nonexistent/.pex/install/pex_wrapper-0.0.0-py2.7.egg.355ab1e4a2649222db04cd7b5a7f2a400079ce2a/pex_wrapper-0.0.0-py2.7.egg/pex_wrapper.py", line 256, in <module>
    sys.exit(main())
  File "/nonexistent/.pex/install/pex_wrapper-0.0.0-py2.7.egg.355ab1e4a2649222db04cd7b5a7f2a400079ce2a/pex_wrapper-0.0.0-py2.7.egg/pex_wrapper.py", line 253, in main
    pex_builder.build(output)
  File "/nonexistent/.pex/install/pex-1.1.11-py2.py3-none-any.whl.abc7d0c18b74b47118dd27e85e69b2bc3dd9b36d/pex-1.1.11-py2.py3-none-any.whl/pex/pex_builder.py", line 424, in build
    self.freeze(bytecode_compile=bytecode_compile)
  File "/nonexistent/.pex/install/pex-1.1.11-py2.py3-none-any.whl.abc7d0c18b74b47118dd27e85e69b2bc3dd9b36d/pex-1.1.11-py2.py3-none-any.whl/pex/pex_builder.py", line 411, in freeze
    self._precompile_source()
  File "/nonexistent/.pex/install/pex-1.1.11-py2.py3-none-any.whl.abc7d0c18b74b47118dd27e85e69b2bc3dd9b36d/pex-1.1.11-py2.py3-none-any.whl/pex/pex_builder.py", line 335, in _precompile_source
    compiled_relpaths = compiler.compile(self._chroot.path(), source_relpaths)
  File "/nonexistent/.pex/install/pex-1.1.11-py2.py3-none-any.whl.abc7d0c18b74b47118dd27e85e69b2bc3dd9b36d/pex-1.1.11-py2.py3-none-any.whl/pex/compiler.py", line 86, in compile
    stderr=subprocess.PIPE)
  File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
    raise child_exception
OSError: [Errno 13] Permission denied

rough-minister-58256

07/18/2016, 10:38 PM

ah heh

faint-holiday-3863

07/18/2016, 10:38 PM

once again I see pex_builder.interpreter.binary is set to the root of a directory tree

faint-holiday-3863

07/18/2016, 10:38 PM

which seems weird

rough-minister-58256

07/18/2016, 10:38 PM

Copy code

>>> subprocess.check_call('/etc’)
…
OSError: [Errno 13] Permission denied

faint-holiday-3863

07/18/2016, 10:38 PM

exactly

rough-minister-58256

07/18/2016, 10:39 PM

yeah thats weird

faint-holiday-3863

07/18/2016, 10:41 PM

here’s the options hash it’s getting:

Copy code

{
  'use_wheel': True,
  'inherit_path': False,
  'script': None,
  'python': None,
  'repos': [PyPIFetcher('<https://pypi.python.org/simple/>')],
  'pex_name': None,
  'pypi': True,
  'requirement_files': [],
  'platform': 'linux-x86_64',
  'cache_dir’: '.pex/build',
  'entry_point': 'examples.foo',
  'verbosity': 0,
  'pex_root’: '.pex',
  'zip_safe': True,
  'interpreter_cache_dir': '.pex/interpreters',
  'ignore_errors': False,
  'cache_ttl': None,
  'python_shebang': None,
  'always_write_cache': False,
  'find_links': ''
}

faint-holiday-3863

07/18/2016, 10:43 PM

I’m really curious how it’s coming up with that directory as an interpreter

rough-minister-58256

07/18/2016, 11:02 PM

same