First the error feedback issue. I got this when running

./pants package

for a

pex_binary

that depends on a `python_distribution`:

10:18:41.98 [INFO] Completed: Extracting 1 requirement to build bin.pex from libs_lockfile.pex: y-cdi-server<7.0.0,>=6.18.10
10:18:41.99 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Extracting 1 requirement to build bin.pex from libs_lockfile.pex: y-cdi-server<7.0.0,>=6.18.10' failed with exit code 1.
stdout:

stderr:
Failed to resolve requirements from PEX environment @ /Users/x/.cache/pants/named_caches/pex_root/unzipped_pexes/6fba5552656884e2d57d5e870d4e77659011a422.
Needed cp39-cp39-macosx_12_0_x86_64 compatible dependencies for:
 1: thrift
    Required by:
      FingerprintedDistribution(distribution=y-logging 2.1.1 (/Users/x/.cache/pants/named_caches/pex_root/installed_wheels/1312836c8e4a7e2af38847a85c99385a4409622e6b172a979f55c799f0be6471/y_logging-2.1.1-py3-none-any.whl), fingerprint='979ed43ea397375058082a1e33ace9d24dbb60d8182a8721ce55f2e6e48086d8')
    But this pex had no 'thrift' distributions.

However, the statement that the pex doesn’t have a

thrift

distribution is untrue, as with verbose pex output, it says it has this distribution in the pex:

pex:  - thrift-0.15.0-cp38-cp38-macosx_11_0_x86_64.whl

so it doesn’t have any compatible distribution in the pex. Which is my error report feedback.

Which leads me to my question: why does it need the cp39 one ? Here’s the command line for that pex run:

/Users/x/.pyenv/versions/3.7.12/bin/python ./pex --tmpdir .tmp --jobs 1 --cert .cacert.pem --python-path $'/Users/x/.pyenv/versions/3.7.12/bin:/Users/x/.pyenv/versions/3.8.11/bin:/Users/x/.pyenv/versions/3.9.7/bin:/usr/local/bin:/root/pyenv/bin:/usr/bin' --output-file bin.pex --no-emit-warnings --manylinux manylinux2014 --include-tools --requirements-pex local_dists.pex --pex-repository libs_lockfile.pex --interpreter-constraint $'CPython>=3.8' --entry-point $'cdi.cli:main' $'--sources-directory=source_files' $'y-cdi-server<7.0.0,>=6.18.10' --layout zipapp

The

pex_binary

has a

interpreter_constraints=[">=3.8"],

and in my

pants.toml

I have

[python]
interpreter_constraints = [">=3.8"]

So I’d figure they ought to be compatible, but I got the feeling that pants picks the lowest possible version satisfying the constraints, while pex picks the highest.. is that possible? (will run some experiments wrgt this..)

Hmm.. of all the pythons, it seems it picked the system one, ugh!

pex: Constraints on interpreters: (Requirement.parse('CPython>=3.8'),), Matching Interpreter: /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/bin/python3.8

wait a minute, that’s a 3.8 one, so why does it then complain about?

Needed cp39-cp39-macosx_12_0_x86_64 compatible dependencies ...

(sorry for spamming here… using this area to think..)

OK, so pinning my interpreter constraints to

==3.8.*

does work. However I don’t think it was the interpreter selected to generate the lockfile that matters (noticed in the diff that I got rid of a lot of artifacts for various versions of python then..) so perhaps it is the one selected when creating the resolve pex that differs.

The springing point being that I’ve only got a thrift wheel for py38, as all the others are either py3-none-any or wheels for both py38 and py39..

Aha! The resolve lockfile.pex is built using

--interpreter-constraint $'CPython<3.9,>=3.8'

I wonder where that constraint came from..?

(starting to feel like it could be beneficial to include the selected python version and/or constraints for the various tasks in the debug log)

Hmm… so under what circumstances is this used?

11:47:33.76 [DEBUG] Completed: Find Python interpreter for constraints - Selected /Users/x/.pyenv/versions/3.8.11/bin/python3.8 to run PEXes with.

as most of the pex invocations I see starts with:

[DEBUG] spawned local process as Some(74821) for Process { argv: ["/Users/x/.pyenv/versions/3.7.12/bin/python", "./pex", ...

although the building of

build_backend.pex

did use the 3.8 one, but that was the only pex invocation that did, all the others used 3.7.12

Ok, I think I may have narrowed the issue down, somewhat, hopefully. Sorry for the lengthy issue, @enough-analyst-54434 https://github.com/pantsbuild/pex/issues/1688 Could use some expertise on rooting this out all the way.

Oh, btw I didn’t notice that

resolves_to_interpreter_constraints = { libs = [">=3.8,<3.10"] }

had any effect at all when applying ICs to install the resolve for the pex binary. @hundreds-father-404 is it a known limitation that this option doesn’t apply to

pex_binary

targets when using local dists? (if I’ve drawn the correct conclusions)

Conclusion I based on what ICs where used when invoking pex, where I had

interpreter_constraints = [">=3.8,<4"]

set, and I can see that IC a lot, but never

<3.10,>=3.8

…