Hi it seems lockfile generated by pex in 2 11 isn t storing

Question

Hi it seems lockfile generated by pex in 2 11 isn t storing correct URL for private pypi server This is the env variable I set when generating the lockfile The ```export PANTS PYTHON REPOS INDEXES= + $ CODEARTIFACT AUTH TOKEN org 11111111 d codeartifact eu central 1 amazonaws com pypi pyfleet simple ``` The `CODEARTIFACT AUTH TOKEN` is an env variable which gets exported But in the generated lockfile in the `artifacts` section I see this ``` url ``` Then later in our CI I get 401 error when pants tries to install this package Please see thread

refined-addition-53644 · Answer

This is the error generated inside CI ```11 56 48 94 ERROR 1 Exception encountered ProcessExecutionFailure Process Building 16 requirements for requirements pex from the 3rdparty pyfleet vehicle spec lock resolve PyYAML lt 6 0 0 gt =5 4 1 SQLAlchemy lt 2 0 0 gt =1 4 25 Unidecode lt 2 0 0 gt =1 3 2 awswrangler lt 3 0 0 gt =2 14 0 python version gt = 3 9 and python version lt 3 11 boto3 lt 2 0 0 gt =1 21 20 dynamic yaml lt 2 0 0 gt =1 3 0 fastapi lt 0 71 0 gt =0 70 0 loguru lt 0 6 0 gt =0 5 3 mangum lt 0 13 0 gt =0 12 3 numpy gt =1 20 0 pandas lt 2 0 0 gt =1 3 3 prophet==1 1 pytest lt 7 0 0 gt =6 0 python jose cryptography lt 4 0 0 gt =3 3 0 spacy lt 4 0 0 gt =3 1 3 uvicorn lt 0 16 0 gt =0 15 0 failed with exit code 1 stdout stderr Traceback most recent call last File root cache pants named caches pex root unzipped pexes 479026d892c9306a8907938e1e49f73e1514eabe bootstrap pex pex py line 504 in execute exit value = self wrap coverage self wrap profiling self execute File root cache pants named caches pex root unzipped pexes 479026d892c9306a8907938e1e49f73e1514eabe bootstrap pex pex py line 409 in wrap coverage return runner args File root cache pants named caches pex root unzipped pexes 479026d892c9306a8907938e1e49f73e1514eabe bootstrap pex pex py line 440 in wrap profiling return runner args File root cache pants named caches pex root unzipped pexes 479026d892c9306a8907938e1e49f73e1514eabe bootstrap pex pex py line 560 in execute return self execute entry self pex info entry point File root cache pants named caches pex root unzipped pexes 479026d892c9306a8907938e1e49f73e1514eabe bootstrap pex pex py line 696 in execute entry return self execute pkg resources entry point File root cache pants named caches pex root unzipped pexes 479026d892c9306a8907938e1e49f73e1514eabe bootstrap pex pex py line 728 in execute pkg resources return runner File root cache pants named caches pex root installed wheels 6a5463833452712ecf58a573b1b489c81d14aaed599e955c2af85cf793217a42 pex 2 1 84 py2 py3 none any whl pex bin pex py line 760 in main do main File root cache pants named caches pex root installed wheels 6a5463833452712ecf58a573b1b489c81d14aaed599e955c2af85cf793217a42 pex 2 1 84 py2 py3 none any whl pex bin pex py line 781 in do main pex builder = build pex File root cache pants named caches pex root installed wheels 6a5463833452712ecf58a573b1b489c81d14aaed599e955c2af85cf793217a42 pex 2 1 84 py2 py3 none any whl pex bin pex py line 629 in build pex result = try File root cache pants named caches pex root installed wheels 6a5463833452712ecf58a573b1b489c81d14aaed599e955c2af85cf793217a42 pex 2 1 84 py2 py3 none any whl pex result py line 82 in try raise ResultError error=result pex result ResultError There was 1 error downloading required artifacts 1 prophet 1 1 from <https org 11111111 d codeartifact eu central 1 amazonaws com pypi pyfleet simple prophet 1 1 prophet 1 1 cp39 cp39 manylinux 2 17 x86 64 manylinux2014 x86 64 whl> HTTP Error 401 Unauthorized```

refined-addition-53644 · Answer

I do export the exact env vars in our CI before running any test

refined-addition-53644 · Answer

Not sure if it s the URL in the lockfile or pants not being able to recognize the necessary authentication once it has generated this url inside lockfile

refined-addition-53644 · Answer

By the way this works with 2 10 which isn t using any custom resolve lockfile Pants is able to install from the private pypi server

enough-analyst-54434 · Answer

You ll need 2 11 1rc1 or greater 2 11 1rc2 is latest for 2 11 x

enough-analyst-54434 · Answer

The fixed code still does not store credentials in the lock file but it does use those same env vars when reading the lock file

refined-addition-53644 · Answer

I am getting a new error now ```14 28 31 53 ERROR 1 Exception encountered ProcessExecutionFailure Process Building 16 requirements for requirements pex from the 3rdparty pyfleet vehicle spec lock resolve PyYAML lt 6 0 0 gt =5 4 1 SQLAlchemy lt 2 0 0 gt =1 4 25 Unidecode lt 2 0 0 gt =1 3 2 awswrangler lt 3 0 0 gt =2 14 0 python version gt = 3 9 and python version lt 3 11 boto3 lt 2 0 0 gt =1 21 20 dynamic yaml lt 2 0 0 gt =1 3 0 fastapi lt 0 71 0 gt =0 70 0 loguru lt 0 6 0 gt =0 5 3 mangum lt 0 13 0 gt =0 12 3 numpy gt =1 20 0 pandas lt 2 0 0 gt =1 3 3 prophet==1 1 pytest lt 7 0 0 gt =6 0 python jose cryptography lt 4 0 0 gt =3 3 0 spacy lt 4 0 0 gt =3 1 3 uvicorn lt 0 16 0 gt =0 15 0 failed with exit code 1 stdout stderr Traceback most recent call last File root cache pants named caches pex root unzipped pexes 6059500ace72ed792367231dcc84ab6e7c3b99f0 bootstrap pex pex py line 517 in execute exit value = self wrap coverage self wrap profiling self execute File root cache pants named caches pex root unzipped pexes 6059500ace72ed792367231dcc84ab6e7c3b99f0 bootstrap pex pex py line 422 in wrap coverage return runner args File root cache pants named caches pex root unzipped pexes 6059500ace72ed792367231dcc84ab6e7c3b99f0 bootstrap pex pex py line 453 in wrap profiling return runner args File root cache pants named caches pex root unzipped pexes 6059500ace72ed792367231dcc84ab6e7c3b99f0 bootstrap pex pex py line 575 in execute return self execute entry File root cache pants named caches pex root unzipped pexes 6059500ace72ed792367231dcc84ab6e7c3b99f0 bootstrap pex pex py line 757 in execute entry return self execute entry point entry point File root cache pants named caches pex root unzipped pexes 6059500ace72ed792367231dcc84ab6e7c3b99f0 bootstrap pex pex py line 788 in execute entry point return runner File root cache pants named caches pex root installed wheels 6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822 pex 2 1 90 py2 py3 none any whl pex bin pex py line 762 in main do main File root cache pants named caches pex root installed wheels 6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822 pex 2 1 90 py2 py3 none any whl pex bin pex py line 783 in do main pex builder = build pex File root cache pants named caches pex root installed wheels 6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822 pex 2 1 90 py2 py3 none any whl pex bin pex py line 630 in build pex resolve from lock File root cache pants named caches pex root installed wheels 6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822 pex 2 1 90 py2 py3 none any whl pex resolve lock resolver py line 310 in resolve from lock pool map File root pyenv versions 3 9 5 lib python3 9 multiprocessing pool py line 364 in map return self map async func iterable mapstar chunksize get File root pyenv versions 3 9 5 lib python3 9 multiprocessing pool py line 771 in get raise self value File root pyenv versions 3 9 5 lib python3 9 multiprocessing pool py line 125 in worker result = True func args kwds File root pyenv versions 3 9 5 lib python3 9 multiprocessing pool py line 48 in mapstar return list map args File root cache pants named caches pex root installed wheels 6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822 pex 2 1 90 py2 py3 none any whl pex resolve lock resolver py line 166 in download artifact return catch File root cache pants named caches pex root installed wheels 6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822 pex 2 1 90 py2 py3 none any whl pex result py line 103 in catch return func args kwargs File root cache pants named caches pex root installed wheels 6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822 pex 2 1 90 py2 py3 none any whl pex resolve lockfile download manager py line 152 in store self save File root cache pants named caches pex root installed wheels 6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822 pex 2 1 90 py2 py3 none any whl pex resolve lock resolver py line 66 in save with open path wb as fp self url fetcher get body stream url as stream File root pyenv versions 3 9 5 lib python3 9 contextlib py line 117 in enter return next self gen File root cache pants named caches pex root installed wheels 6242b902db69f59e1092b406655c0fb1634486c47ce563f5fd27277cf4561822 pex 2 1 90 py2 py3 none any whl pex fetcher py line 103 in get body stream fp = cast BinaryIO opener open request timeout=self timeout File root pyenv versions 3 9 5 lib python3 9 urllib request py line 523 in open response = meth req response File root pyenv versions 3 9 5 lib python3 9 urllib request py line 632 in http response response = self parent error File root pyenv versions 3 9 5 lib python3 9 urllib request py line 555 in error result = self call chain args File root pyenv versions 3 9 5 lib python3 9 urllib request py line 494 in call chain result = func args File root pyenv versions 3 9 5 lib python3 9 urllib request py line 1236 in http error 401 retry = self http error auth reqed www authenticate File root pyenv versions 3 9 5 lib python3 9 urllib request py line 1118 in http error auth reqed raise ValueError AbstractDigestAuthHandler does not support ValueError AbstractDigestAuthHandler does not support the following scheme Bearer ```

enough-analyst-54434 · Answer

It may be best to fork off an issue with these latest details + the fact this is AWS codeartifact at this point to track work I m on vacation through the end of June so you may need to dig or else get someone else s attention who is willing to dig

refined-addition-53644 · Answer

ah ok No worries I will create an issue in the meantime <https github com pantsbuild pants issues 15766>