https://pantsbuild.org/ logo
Powered by
#pex
  • g

    glamorous-oil-7750

    07/27/2016, 1:42 AM
    Copy code
    # Create pex
docker run -i -t --rm -v /Users/mhite/tt/netvalidate:/data build-wheezy-airflow /bin/bash -c 'cd /data; mkdir -p /data/build/usr/local/bin; pex -v -v -v --no-pypi --find-links=/data/wheels "airflow[crypto, hive, ldap, mysql]" -c airflow -o /data/build/usr/local/bin/airflow'
pex: crawling link i=0 link=Link('file:///data/wheels') follow_links=False
Could not find compatible interpreterreter :: Setting up interpreter /usr/bin/python2.7 :: Interpreter cache resolving setuptools<20.11,>=2.2
make: *** [all] Error 102
  • g

    glamorous-oil-7750

    07/27/2016, 1:48 AM
    Welp, that appeared to be another setuptools issue
  • g

    glamorous-oil-7750

    07/27/2016, 1:48 AM
    setuptools... it's like draino for python developers
  • g

    glamorous-oil-7750

    07/27/2016, 1:48 AM
    if you drink it, it will kill ya
  • g

    glamorous-oil-7750

    07/27/2016, 1:49 AM
    Copy code
    # Create pex
docker run -i -t --rm -v /Users/mhite/tt/netvalidate:/data build-wheezy-airflow /bin/bash -c 'cd /data; mkdir -p /data/build/usr/local/bin; pex -v -v -v --no-pypi --find-links=/data/wheels "airflow[crypto, hive, ldap, mysql]" -c airflow -o /data/build/usr/local/bin/airflow'
pex: crawling link i=0 link=Link('file:///root/.pex/build') follow_links=False
pex: crawling link i=0 link=Link('file:///data/wheels') follow_links=False
pex: Building pex :: Resolving distributions :: Fetching file:///data/wheels/airflow-1.7.1.3-py2-none-any.whl :: Fetching file:///data/wheels/airflow-1.7.1.3-pypex: Building pex :: Resolving distributions :: Translating /tmp/tmpApFwOm/airflow-1.7.1.3-py2-none-any.whl into distribution                                   Could not satisfy all requirements for pandas<1.0.0,>=0.15.2:
    pandas<1.0.0,>=0.15.2(from: airflow[crypto,hive,ldap,mysql])
make: *** [all] Error 1
  • g

    glamorous-oil-7750

    07/27/2016, 1:51 AM
    do you hit the same thing if you try to build like this: 
    pex -v -v -v --no-pypi --find-links=/data/wheels "airflow[crypto, hive, ldap, mysql]" -c airflow -o /data/build/usr/local/bin/airflow
  • g

    glamorous-oil-7750

    07/27/2016, 1:58 AM
    (ie. airflow[crypto,hive,ldap,mysql] rather than just airflow)
  • g

    glamorous-oil-7750

    07/27/2016, 2:04 AM
    Not sure why it's getting tripped up on pandas 😞
  • g

    glamorous-oil-7750

    07/27/2016, 2:04 AM
    it's there -- wheels/pandas-0.18.1-cp27-cp27mu-manylinux1_x86_64.whl
  • r

    rough-minister-58256

    07/27/2016, 4:28 AM
    ah - pex doesn’t currently support manylinux, unfortunately.
  • r

    rough-minister-58256

    07/27/2016, 4:28 AM
    you could just rename that to linux_x86_64
  • r

    rough-minister-58256

    07/27/2016, 4:29 AM
  • r

    rough-minister-58256

    07/27/2016, 4:30 AM
  • r

    rough-minister-58256

    07/27/2016, 4:30 AM
    looks like there’s an odd way to tell pip not to match against manylinux
  • g

    glamorous-oil-7750

    07/27/2016, 6:24 PM
    Interesting. I tried 
    echo "manylinux1_compatible = False" > _manylinux.py
    in CWD and then running 
    pip wheel --no-cache-dir --wheel-dir=./wheels airflow
    but I can still it downloading the manylinux1 wheels.
  • g

    glamorous-oil-7750

    07/27/2016, 6:24 PM
    😦
  • g

    glamorous-oil-7750

    07/27/2016, 6:56 PM
    Erg, welp, for whatever reason CWD wasn't considered part of the module import path. Explicitly did an 
    export PYTHONPATH=/root
    and I can now see it downloading.
  • g

    glamorous-oil-7750

    07/27/2016, 6:56 PM
    Closer.
  • r

    rough-minister-58256

    07/27/2016, 6:58 PM
    yeah, bit odd of a user experience there to pass what might otherwise/should be a 
    --no-manylinux
    flag
  • f

    faint-holiday-3863

    07/27/2016, 9:05 PM
    tries to figure out how to get python to not include the system site-packages location in sys.path at startup
  • f

    faint-holiday-3863

    07/27/2016, 9:05 PM
    (like pex, but without pex, because it’s the build step that installs pex)
  • f

    faint-holiday-3863

    07/27/2016, 9:07 PM
    I see the stuff in pex/pex.py that does it
  • f

    faint-holiday-3863

    07/27/2016, 9:17 PM
    might just use virtualenv.
  • r

    rough-minister-58256

    07/27/2016, 9:21 PM
  • r

    rough-minister-58256

    07/27/2016, 9:21 PM
    python -S -s
    etc
  • g

    glamorous-oil-7750

    07/27/2016, 9:45 PM
    @rough-minister-58256 - initial tests of building the pex solely with wheels looks positive
  • g

    glamorous-oil-7750

    07/27/2016, 9:45 PM
    when built 100% with wheels, what exactly does this avoid on the host running the generated .pex file?
  • g

    glamorous-oil-7750

    07/27/2016, 9:45 PM
    i know from a high level it avoids eggs, but what does this get us?
  • g

    glamorous-oil-7750

    07/27/2016, 9:51 PM
    i'm guessing there is some weird dependency that then occurs with whatever setuptools is on the destination host? ¯\_(ツ)_/¯
  • f

    faint-holiday-3863

    07/27/2016, 10:19 PM
    oh holy crap, python -S actually does help
1...242526...74Latest