https://pantsbuild.org/ logo
#general
Title
# general
e

echoing-manchester-70122

01/21/2020, 6:50 PM
it really pollutes the logs
e

enough-analyst-54434

01/21/2020, 6:58 PM
Is this question in regards to hacking on pants in the pantsbuild/pants repo or using pantsbuild.pants==1.24.0.dev2 which was the only version to use Pex 2 which uses pip which has since been reverted?
e

echoing-manchester-70122

01/21/2020, 6:59 PM
we're still on 1.16.0
I just want to turn off these bars
Copy code
Collecting pycparser
  Downloading pycparser-2.19.tar.gz (158 kB)
[?25l
[K     |██                              | 10 kB 54.9 MB/s eta 0:00:01
[K     |████▏                           | 20 kB 60.4 MB/s eta 0:00:01
[K     |██████▏                         | 30 kB 73.6 MB/s eta 0:00:01
[K     |████████▎                       | 40 kB 84.6 MB/s eta 0:00:01
when running
pants binary
etc
e

enough-analyst-54434

01/21/2020, 7:00 PM
Um... I think the only place pip is used in old pants is in the setup script.
Is this on a machine where pants is being run for the 1st time?
e

echoing-manchester-70122

01/21/2020, 7:00 PM
no
it's being run by jenkins
e

enough-analyst-54434

01/21/2020, 7:01 PM
Hmm, not making sense. Pants doesn't use pip except to bootstrap itself. Can you provide full console output?
e

echoing-manchester-70122

01/21/2020, 7:01 PM
we do a pants clean-all between builds
I don't know whether it's pip or what, I just want to turn off those bars
I assumed it was pip, but whatever it is, is there a way to configure it to turn that off?
e

enough-analyst-54434

01/21/2020, 7:02 PM
That's definitely pip and that definitely only happens on pants bootstrap. The only way to turn it off is to only bootstrap once per pants version.
That would complexify your Jenkins setup though I imagine.
e

echoing-manchester-70122

01/21/2020, 7:03 PM
that's not really an option. So there is no way to pass an extra parameter to pip?
e

enough-analyst-54434

01/21/2020, 7:03 PM
There is no way.
Obviously you could hack your Pants bootstrap script, but I'm assuming you're looking for officially supported answers here. Pants in that version has no such option.
Here's the relevant code in the most modern pants bootstrap script: https://github.com/pantsbuild/setup/blob/gh-pages/pants#L170-L171 But also, here is the cache dir setting: https://github.com/pantsbuild/setup/blob/gh-pages/pants#L18 in case you want to arrange jenkins to re-use a bootstrapped Pants. IE: export a PANTS_HOME that points to a directory all jenkins runs can see.
But, reading pip docs I find https://pip.pypa.io/en/stable/user_guide/#environment-variables Have you tried that?
e

echoing-manchester-70122

01/21/2020, 7:14 PM
seems like adding the flag to the bootstrap script worked
thanks
this script is ancient though, is there any reason to upgrade if it's still working fine?
e

enough-analyst-54434

01/21/2020, 7:27 PM
Nope. I'd personally try the PIP_<env var> means of passing PIP flags just so I didn't have a hacked up script in case other issues arose later, but excellent.
e

echoing-manchester-70122

01/21/2020, 7:27 PM
troubleshooitng another issue now, any idea why this happens?
Copy code
pex: Activating PEX virtual environment from /mnt/workspace/.pants.d/test/pytest-prep/CPython-2.7.17/3c7ff9ac5a4760b2609534dd4afbdf0d09d99708 :: Resolving more-itertools<6.0.0; python_version < "3"
                     pex: Activating PEX virtual environment from /mnt/workspace/.pants.d/test/pytest-prep/CPython-2.7.17/3c7ff9ac5a4760b2609534dd4afbdf0d09d99708 :: Resolving pytest-cov==2.4.0
                     pex: Failed to resolve a requirement: The 'zipp>=0.5' distribution was not found and is required by importlib-metadata
                     pex: Activating PEX virtual environment from /mnt/workspace/.pants.d/test/pytest-prep/CPython-2.7.17/3c7ff9ac5a4760b2609534dd4afbdf0d09d99708 :: Resolving pytest-timeout==1.2.0
                     pex: Failed to resolve a requirement: The 'zipp>=0.5' distribution was not found and is required by importlib-metadata
                     pex: Activating PEX virtual environment from /mnt/workspace/.pants.d/test/pytest-prep/CPython-2.7.17/3c7ff9ac5a4760b2609534dd4afbdf0d09d99708 :: Resolving pytest==4.6.6
                     pex: Failed to resolve a requirement: The 'zipp>=0.5' distribution was not found and is required by importlib-metadata
                     pex: Activating PEX virtual environment from /mnt/workspace/.pants.d/test/pytest-prep/CPython-2.7.17/3c7ff9ac5a4760b2609534dd4afbdf0d09d99708 :: Resolving unittest2<=1.9.0,>=0.6.0
                     pex: Unresolved requirements:
                     pex:   - importlib-metadata
                     pex:   - zipp
                     pex: Distributions contained within this pex:
                     pex:   - more_itertools-5.0.0-py2-none-any.whl
                     pex:   - pathlib2-2.3.5-py2.py3-none-any.whl
                     pex:   - configparser-4.0.2-py2.py3-none-any.whl
                     pex:   - six-1.14.0-py2.py3-none-any.whl
                     pex:   - linecache2-1.0.0-py2.py3-none-any.whl
                     pex:   - py-1.8.1-py2.py3-none-any.whl
                     pex:   - pytest_timeout-1.2.0-py2.py3-none-any.whl
                     pex:   - scandir-1.10.0-cp27-cp27mu-linux_x86_64.whl
                     pex:   - zipp-0.0.0-py2-none-any.whl
                     pex:   - coverage-5.0.3-cp27-cp27mu-manylinux1_x86_64.whl
                     pex:   - traceback2-1.4.0-py2.py3-none-any.whl
                     pex:   - funcsigs-1.0.2-py2.py3-none-any.whl
                     pex:   - pytest_cov-2.4.0-py2.py3-none-any.whl
                     pex:   - atomicwrites-1.3.0-py2.py3-none-any.whl
                     pex:   - pluggy-0.13.1-py2.py3-none-any.whl
                     pex:   - contextlib2-0.6.0.post1-py2.py3-none-any.whl
                     pex:   - wcwidth-0.1.8-py2.py3-none-any.whl
                     pex:   - importlib_metadata-1.4.0-py2.py3-none-any.whl
                     pex:   - unittest2-1.1.0-py2.py3-none-any.whl
                     pex:   - argparse-1.4.0-py2.py3-none-any.whl
                     pex:   - pytest-4.6.6-py2.py3-none-any.whl
                     pex:   - attrs-19.3.0-py2.py3-none-any.whl
                     pex:   - packaging-20.0-py2.py3-none-any.whl
                     pex:   - pyparsing-2.4.6-py2.py3-none-any.whl
                     Failed to execute PEX file, missing linux_x86_64-cp-27-cp27mu compatible dependencies for:
                     importlib-metadata
                     zipp
e

enough-analyst-54434

01/21/2020, 7:41 PM
We were actually just bit by this this weekend - you'll need to pin zipp to version 1.0.0. I'll need a few minutes though to provide you info on how to do this in 1.16.0.
OK, the error above says "Resolving pytest==4.6.6" but pants 1.16.0 defaults to 'pytest>=3.0.7,<3.7' (https://github.com/pantsbuild/pants/blob/release_1.16.0/src/python/pants/backend/python/subsystems/pytest.py#L17-L18). This implies you have a
pants.ini
[pytest] requirements
entry. The fix would be to add the zipp pin to that entry, e.g.:
Copy code
[pytest]
requirements: ["pytest==4.6.6", "zipp==1.0.0"]
e

echoing-manchester-70122

01/21/2020, 8:20 PM
Copy code
timestamp: 2020-01-21T20:20:09.546109
Exception caught: (pkg_resources.RequirementParseError) (backtrace omitted)
Exception message: Invalid requirement, parse error at "'["pytest'"
I removed it, seems to be ok now
5 Views