if my python interpreter constraint is

interpreter_constraints: ['CPython>=3']

, how does pants pick a specific version of python?

On my machine it picks 3.7.0 and on my coworker's it picks 3.6.4

Ah @mysterious-farmer-45668 that Async issue is because Async is a reserved word in python 3.7. So, it’s an issue with that library itself and you won’t be able to use python 3.7 with it until they patch that

Re: how it resolves which Python to use, I don’t have a completely solid understanding of that but believe it looks on your PATH to see which are available, and which comply with your

interpreter_constraints

. What is the result of

which -a python3

on your computer vs your coworkers? Along with

python3 --version

?

yeah that is different. we have different root pythons. My coworker was under the impression that pants would use whatever python that is in the virtualenv that i invoke pants from, i guess that is not the case?

It picks the minimum compatible version found scanning PATH by default. You can control the scan though with

[python-setup] interpreter-search-paths

which is a list. If you have locked down machines, this can be a useful option to control exactly the python interpreter picked.

I've narrowed down my issue and I think pants is acting unexpectedly

this is my exception:

Exception message: Could not satisfy all requirements for flask-login==0.2.11:
    flask-login==0.2.11(from: apache-airflow==1.10.0; python_version < "3.7"), Flask-Login<0.5,>=0.3(from: apache-airflow==1.10.0; python_version < "3.7"->flask-appbuilder<2.0.0,>=1.11.1)

the former is specified here https://github.com/apache/incubator-airflow/blob/1.10.0/setup.py#L280

so on version

1.10.0

, it specifically denotes that

flask-login

should be

0.2.11

but for the latter, the version of

flask-login

it requires is different across versions that meet that

<2.0.0,>=1.11.1

requirement.

case in point:

https://github.com/dpgaspar/Flask-AppBuilder/blob/v1.11.1/setup.py#L41 vs https://github.com/dpgaspar/Flask-AppBuilder/blob/v1.12.1/setup.py#L42

1.11.1 and 1.12.1 both meet the requirement of

<2.0.0,>=1.11.1

but they require incompatible versions of

flask-login

so shouldn't pants be able to resolve to the earlier version and pick up flask-login

0.2.11

?

Probably. This is likely worthy of a bug against pex, but it's also really not ideal:

$ pex apache-airflow==1.10.0
**** Failed to install apache-airflow-1.10.0 (caused by: NonZeroExit("received exit code 1 during execution of `['/usr/bin/python3.7', '-', 'bdist_wheel', '--dist-dir=/tmp/tmp9oz6r7sr']` while trying to execute `['/usr/bin/python3.7', '-', 'bdist_wheel', '--dist-dir=/tmp/tmp9oz6r7sr']`")
):
stdout:

stderr:
Traceback (most recent call last):
  File "<stdin>", line 9, in <module>
  File "setup.py", line 124
    async = [
          ^
SyntaxError: invalid syntax


Traceback (most recent call last):
  File "/home/jsirois/.venv/pex/bin/pex", line 11, in <module>
    sys.exit(main())
  File "/home/jsirois/.venv/pex/lib/python3.7/site-packages/pex/bin/pex.py", line 735, in main
    pex_builder = build_pex(reqs, options, resolver_options_builder)
  File "/home/jsirois/.venv/pex/lib/python3.7/site-packages/pex/bin/pex.py", line 664, in build_pex
    for resolved_dist in resolveds:
  File "/home/jsirois/.venv/pex/lib/python3.7/site-packages/pex/resolver.py", line 566, in resolve_multi
    use_manylinux=use_manylinux):
  File "/home/jsirois/.venv/pex/lib/python3.7/site-packages/pex/resolver.py", line 502, in resolve
    return resolver.resolve(resolvables_from_iterable(requirements, builder))
  File "/home/jsirois/.venv/pex/lib/python3.7/site-packages/pex/resolver.py", line 300, in resolve
    dist = self.build(package, resolvable.options)
  File "/home/jsirois/.venv/pex/lib/python3.7/site-packages/pex/resolver.py", line 379, in build
    dist = super(CachingResolver, self).build(package, options)
  File "/home/jsirois/.venv/pex/lib/python3.7/site-packages/pex/resolver.py", line 258, in build
    raise Untranslateable('Package %s is not translateable by %s' % (package, translator))
pex.resolver.Untranslateable: Package SourcePackage('file:///home/jsirois/.pex/build/apache-airflow-1.10.0.tar.gz') is not translateable by ChainedTranslator(WheelTranslator, EggTranslator, SourceTranslator)

It would really help to have a working legit failure repro case.

You should be able to work around by declaring a direct dep on the correct solution for flask-appbuilder with appropriate comment pointing to the bug you file.

pypi says "Last released: Aug 27, 2018" - clearly no-one actually uses this version of airflow?

Or does everyone install from a git url with a fix or how even is this a thing?

that is why i had to put the constraint of <3.7 in my requirements. If you put that constraint i think you'd get a repro... will file an issue with pex.

Aha - I did not realize that was what the environment marker was for.

This thing is a nightmare!: "RuntimeError: By default one of Airflow's dependencies installs a GPL dependency (unidecode). To avoid this dependency set SLUGIFY_USES_TEXT_UNIDECODE=yes..."

CALA-AM02046714:de-monorepo seolh001$ pex apache-airflow==1.10.0 --interpreter-constraint='CPython<3.7'
Could not satisfy all requirements for flask-login==0.2.11:
    flask-login==0.2.11(from: apache-airflow==1.10.0), Flask-Login<0.5,>=0.3(from: apache-airflow==1.10.0->flask-appbuilder<2.0.0,>=1.11.1)

yup got a repro

I agree -.- i just got handed this ticket but seriously wtf how is nobody else googling for this problem..?

Repro - maybe - taking a long time to download the huge dep list

I got it with

pex apache-airflow==1.10.0 --interpreter-constraint='CPython<3.7'

Agreed:

$ SLUGIFY_USES_TEXT_UNIDECODE=yes pex --python=python2 'apache-airflow==1.10.0; python_version < "3.7"'
Could not satisfy all requirements for flask-login==0.2.11:
    flask-login==0.2.11(from: apache-airflow==1.10.0; python_version < "3.7"), Flask-Login<0.5,>=0.3(from: apache-airflow==1.10.0; python_version < "3.7"->flask-appbuilder<2.0.0,>=1.11.1)

That is a clean minimal repro

filed https://github.com/pantsbuild/pex/issues/614

Thank you! As soon as I confirm the workaround works I'll note that on the issue.