https://pantsbuild.org/ logo
Join Slack
Powered by
Getting flaky installs in CI when building require...
# general
r
Getting flaky installs in CI when building requirements.pex. I tried the workaround of setting 
pip_version = "latest"
from the linked thread below but it didn't work. Anybody know how to work around this? Posting logs below. Using pants 2.20.1 and python 3.9. Relevant thread: https://pantsbuild.slack.com/archives/C046T6T9U/p1692644364840149
Copy code
stderr:
pid 1546 -> /home/andre/.cache/pants/named_caches/pex_root/venvs/39582e4dc3636270136e41c3bb04a860786f11e7/587d95bb4f21c70aedf0708bac1e6e728689715d/bin/python -sE /home/andre/.cache/pants/named_caches/pex_root/venvs/39582e4dc3636270136e41c3bb04a860786f11e7/587d95bb4f21c70aedf0708bac1e6e728689715d/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --use-deprecated legacy-resolver --isolated -q --cache-dir /home/andre/.cache/pants/named_caches/pex_root/pip/23.1.2/pip_cache wheel --no-deps --wheel-dir /home/andre/.cache/pants/named_caches/pex_root/built_wheels/sdists/autobahn-23.6.2.tar.gz/ec9421c52a2103364d1ef0468036e6019ee84f71721e86b36fe19ad6966c1181/cp39-cp39-manylinux_2_35_x86_64.2a9520c1423b4e0c89b7567fad231c9a.work /home/andre/.cache/pants/named_caches/pex_root/downloads/ec9421c52a2103364d1ef0468036e6019ee84f71721e86b36fe19ad6966c1181/autobahn-23.6.2.tar.gz --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with 2 and STDERR:
ERROR: Exception:
Traceback (most recent call last):
  File "/home/andre/.cache/pants/named_caches/pex_root/venvs/39582e4dc3636270136e41c3bb04a860786f11e7/587d95bb4f21c70aedf0708bac1e6e728689715d/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 169, in exc_logging_wrapper
    status = run_func(*args)
  File "/home/andre/.cache/pants/named_caches/pex_root/venvs/39582e4dc3636270136e41c3bb04a860786f11e7/587d95bb4f21c70aedf0708bac1e6e728689715d/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 248, in wrapper
    return func(self, options, args)
  File "/home/andre/.cache/pants/named_caches/pex_root/venvs/39582e4dc3636270136e41c3bb04a860786f11e7/587d95bb4f21c70aedf0708bac1e6e728689715d/lib/python3.9/site-packages/pip/_internal/commands/wheel.py", line 157, in run
    build_successes, build_failures = build(
  File "/home/andre/.cache/pants/named_caches/pex_root/venvs/39582e4dc3636270136e41c3bb04a860786f11e7/587d95bb4f21c70aedf0708bac1e6e728689715d/lib/python3.9/site-packages/pip/_internal/wheel_builder.py", line 334, in build
    wheel_cache.record_download_origin(cache_dir, req.download_info)
  File "/home/andre/.cache/pants/named_caches/pex_root/venvs/39582e4dc3636270136e41c3bb04a860786f11e7/587d95bb4f21c70aedf0708bac1e6e728689715d/lib/python3.9/site-packages/pip/_internal/cache.py", line 261, in record_download_origin
    origin = DirectUrl.from_json(origin_path.read_text())
  File "/home/andre/.cache/pants/named_caches/pex_root/venvs/39582e4dc3636270136e41c3bb04a860786f11e7/587d95bb4f21c70aedf0708bac1e6e728689715d/lib/python3.9/site-packages/pip/_internal/models/direct_url.py", line 231, in from_json
    return cls.from_dict(json.loads(s))
  File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
According to the above logs, it seems that 
latest
installs 
23.1.2
, which is privy to https://github.com/pypa/pip/issues/11985 since the latter was fixed in 
23.2
. Going to hardcode 
pip_version = "23.2"
in the hopes that this fixes it.
Same error just happened after hardcoding [python].pip_version to 23.2. The logs includes the same pip version shown in the original logs above. No caching is used. 🤔
b
Sorry for the trouble. I coincidentally just saw the same error now in my CI... I think the pip version is also encoded into lockfiles. There's a few built-in ones using 23.1.2 in Pants 20.1, and you may have some too. What do you see if you search for 
"pip_version": "23.1.2"
?
If you do find your lockfiles reference that pip version, maybe try 
pants generate-lockfiles ...
to regenerate them after changing the 
pants.toml
pip_version
value
r
Ooh great catch, completely overlooked the fact that lockfiles affected pip versioning (even though I just re-read the docs and lockfiles are mentioned in [python].pip_version 🤦‍♂️). Will be rerunning CI throughout the rest of the day to confirm that it's fixed for good. Thanks for the help!
👍 1
Been running CI pretty much all day and the issue hasn't appeared ever since regenerating lockfiles. Thanks again!
🎉 1
10 Views
Open in Slack
PreviousNext
Powered by