Hi, it seems lockfile generated by pex in 2.11 isn't storing correct URL for private pypi server This is the env variable I set when generating the lockfile. The

export PANTS_PYTHON_REPOS_INDEXES="+['<https://aws>:${CODEARTIFACT_AUTH_TOKEN}@org-11111111.d.codeartifact.eu-central-1.amazonaws.com/pypi/pyfleet/simple/']"

The

CODEARTIFACT_AUTH_TOKEN

is an env variable which gets exported. But in the generated lockfile in the

artifacts

section, I see this

"url": "<https://org-11111111.d.codeartifact.eu-central-1.amazonaws.com/pypi/pyfleet/simple/prophet/1.1/prophet-1.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl>"

Then later in our CI, I get 401 error when pants tries to install this package. Please see 🧵

You'll need 2.11.1rc1 or greater. 2.11.1rc2 is latest for 2.11.x

I am getting a new error now

14:28:31.53 [ERROR] 1 Exception encountered:


  ProcessExecutionFailure: Process 'Building 16 requirements for requirements.pex from the 3rdparty/pyfleet_vehicle_spec.lock resolve: PyYAML<6.0.0,>=5.4.1, SQLAlchemy<2.0.0,>=1.4.25, Unidecode<2.0.0,>=1.3.2, awswrangler<3.0.0,>=2.14.0; python_version >= "3.9" and python_version < "3.11", boto3<2.0.0,>=1.21.20, dynamic-yaml<2.0.0,>=1.3.0, fastapi<0.71.0,>=0.70.0, loguru<0.6.0,>=0.5.3, mangum<0.13.0,>=0.12.3, numpy>=1.20.0, pandas<2.0.0,>=1.3.3, prophet==1.1, pytest<7.0.0,>=6.0, python-jose[cryptography]<4.0.0,>=3.3.0, spacy<4.0.0,>=3.1.3, uvicorn<0.16.0,>=0.15.0' failed with exit code 1.

stdout:
stderr:
Traceback (most recent call last):
  File "/root/.cache/pants/named_caches/pex_root/unzipped_pexes/6059500ace72ed792367231dcc84ab6e7c3b99f0/.bootstrap/pex/pex.py", line 517, in execute
    exit_value = self._wrap_coverage(self._wrap_profiling, self._execute)
  File "/root/.cache/pants/named_caches/pex_root/unzipped_pexes/6059500ace72ed792367231dcc84ab6e7c3b99f0/.bootstrap/pex/pex.py", line 422, in _wrap_coverage
    return runner(*args)
 File "/root/.cache/pants/named_caches/pex_root/unzipped_pexes/6059500ace72ed792367231dcc84ab6e7c3b99f0/.bootstrap/pex/pex.py", line 453, in _wrap_profiling

   return runner(*args)

  File "/root/.cache/pants/named_caches/pex_root/unzipped_pexes/6059500ace72ed792367231dcc84ab6e7c3b99f0/.bootstrap/pex/pex.py", line 575, in _execute

    return self.execute_entry(

  File "/root/.cache/pants/named_caches/pex_root/unzipped_pexes/6059500ace72ed792367231dcc84ab6e7c3b99f0/.bootstrap/pex/pex.py", line 757, in execute_entry
    return self.execute_entry_point(entry_point)
  File "/root/.cache/pants/named_caches/pex_root/unzipped_pexes/6059500ace72ed792367231dcc84ab6e7c3b99f0/.bootstrap/pex/pex.py", line 788, in execute_entry_point

   return runner()

 File "/root/.cache/pants/named_caches/pex_root/installed_wheels/6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822/pex-2.1.90-py2.py3-none-any.whl/pex/bin/pex.py", line 762, in main

    do_main(

  File "/root/.cache/pants/named_caches/pex_root/installed_wheels/6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822/pex-2.1.90-py2.py3-none-any.whl/pex/bin/pex.py", line 783, in do_main

    pex_builder = build_pex(

  File "/root/.cache/pants/named_caches/pex_root/installed_wheels/6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822/pex-2.1.90-py2.py3-none-any.whl/pex/bin/pex.py", line 630, in build_pex
    resolve_from_lock(

  File "/root/.cache/pants/named_caches/pex_root/installed_wheels/6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822/pex-2.1.90-py2.py3-none-any.whl/pex/resolve/lock_resolver.py", line 310, in resolve_from_lock

   pool.map(

  File "/root/.pyenv/versions/3.9.5/lib/python3.9/multiprocessing/pool.py", line 364, in map

    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/root/.pyenv/versions/3.9.5/lib/python3.9/multiprocessing/pool.py", line 771, in get

    raise self._value
  File "/root/.pyenv/versions/3.9.5/lib/python3.9/multiprocessing/pool.py", line 125, in worker



    result = (True, func(*args, **kwds))



  File "/root/.pyenv/versions/3.9.5/lib/python3.9/multiprocessing/pool.py", line 48, in mapstar



    return list(map(*args))



  File "/root/.cache/pants/named_caches/pex_root/installed_wheels/6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822/pex-2.1.90-py2.py3-none-any.whl/pex/resolve/lock_resolver.py", line 166, in download_artifact



    return catch(



  File "/root/.cache/pants/named_caches/pex_root/installed_wheels/6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822/pex-2.1.90-py2.py3-none-any.whl/pex/result.py", line 103, in catch



    return func(*args, **kwargs)



  File "/root/.cache/pants/named_caches/pex_root/installed_wheels/6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822/pex-2.1.90-py2.py3-none-any.whl/pex/resolve/lockfile/download_manager.py", line 152, in store



    self.save(



  File "/root/.cache/pants/named_caches/pex_root/installed_wheels/6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822/pex-2.1.90-py2.py3-none-any.whl/pex/resolve/lock_resolver.py", line 66, in save



    with open(path, "wb") as fp, self._url_fetcher.get_body_stream(url) as stream:



  File "/root/.pyenv/versions/3.9.5/lib/python3.9/contextlib.py", line 117, in __enter__



    return next(self.gen)



  File "/root/.cache/pants/named_caches/pex_root/installed_wheels/6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822/pex-2.1.90-py2.py3-none-any.whl/pex/fetcher.py", line 103, in get_body_stream



    fp = cast(BinaryIO, opener.open(request, timeout=self._timeout))



  File "/root/.pyenv/versions/3.9.5/lib/python3.9/urllib/request.py", line 523, in open



    response = meth(req, response)



  File "/root/.pyenv/versions/3.9.5/lib/python3.9/urllib/request.py", line 632, in http_response



    response = self.parent.error(



  File "/root/.pyenv/versions/3.9.5/lib/python3.9/urllib/request.py", line 555, in error



    result = self._call_chain(*args)



  File "/root/.pyenv/versions/3.9.5/lib/python3.9/urllib/request.py", line 494, in _call_chain



    result = func(*args)



  File "/root/.pyenv/versions/3.9.5/lib/python3.9/urllib/request.py", line 1236, in http_error_401



    retry = self.http_error_auth_reqed('www-authenticate',



  File "/root/.pyenv/versions/3.9.5/lib/python3.9/urllib/request.py", line 1118, in http_error_auth_reqed



    raise ValueError("AbstractDigestAuthHandler does not support"



ValueError: AbstractDigestAuthHandler does not support the following scheme: 'Bearer'

It may be best to fork off an issue with these latest details + the fact this is AWS codeartifact at this point to track work. I'm on vacation through the end of June; so you may need to dig or else get someone else's attention who is willing to dig.

ah ok! No worries! I will create an issue in the meantime. https://github.com/pantsbuild/pants/issues/15766