#1968 Parallell pex creation fails due to possible race conditions in package installation New issue created by tapetersen We're using pex with makefiles and parallell jobs in CI and are noticing spurious errors like the one below

pex --find-links out/wheels -o out/pexes/pex1.pex ./pex1 -e pex1:main
pex --find-links out/wheels -o out/pexes/pex2.pex -r pex2/requirements.txt --python-script pex2/entrypoint.py
Could not find the installation RECORD for zipp 3.10.0 under /root/.pex/installed_wheels/4fcb6f278987a6605757302a6e40e896257570d11c51628968ccb2a47e80c6c1/zipp-3.10.0-py3-none-any.whl.workdir

The other pex built fine and if I retry it will work. Problem is probably made worse by having all wheels cached so no time is spent downloading them which makes it more likely to run into races when installing. This would probably need some locking around the installed_wheels folder or more fine grained locking/atomik operations for the real installation folder. It's not urgent for us as I can avoid making those in parallell or ensure separate pexdirs but it was somewhat of a nasty issue to debug and would perhaps save someone the time and frustration in the future. pantsbuild/pex

#1968 Parallell pex creation fails due to possible race conditions in package installation Issue closed as completed by jsirois pantsbuild/pex