Hey! A colleague of mine has been having a miserable time trying to use Pants in the one production repository we switched to Pants so far. She can’t get Pants to work at all, since it fails to bootstrap itself. This is for Pants 2.8.0 on a MacOS x86 laptop running 10.14.6. I tried to diagnose the issue with her and we kept on running into the exact same problem, independent of whether the available Python was provided by pyenv or homebrew and after diligently nuking both

~/.pex

and

~/.cache/pants

. A typical error trace is attached and any help would be much appreciated. This is a blocker for her picking up issues that touch on the repository (which coincidentally contains a new tool to validate package requirements across all our repositories that is about to go CI for all). Also, to exclude any peculiarities due to our local Python package mirror or some such, I nuked the Pex and Pants caches on my work laptop and tried to bootstrap from scratch. The results are mixed: The good news is that I was able to bootstrap and Pants does run mypy, tests, etc. The bad news is that I cannot format or lint the code anymore since Pants just can’t find the lockfile for Black. I tried regenerating the main lockfiles etc to no avail. Part of me is ecstatic about never having to deal with Pylint again, but I fear that CI nor the CTO won’t let me get away with that. Again, any help would be much appreciated.

14:24:15.68 [ERROR] 1 Exception encountered:

  FileNotFoundError: [Errno 2] No such file or directory: '/Users/grr/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.DZStIS/install/lib/python3.9/site-packages/pants/backend/python/lint/black/lockfile.txt'

Addressing your colleague's issue and not yours to start, the interesting log lines are:

ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==2.8.0 (from versions: 0.0.17, 0.0.18, 0.0.20, 0.0.21, 0.0.22, 0.0.23, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.39, 0.0.40, 0.0.41, 0.0.42, 0.0.43, 0.0.44, 0.0.45, 0.0.46, 0.0.47, 0.0.48, 0.0.49, 0.0.50, 0.0.51, 0.0.52, 0.0.53, 0.0.54, 0.0.55, 0.0.56, 0.0.57, 0.0.58, 0.0.59, 0.0.60, 0.0.61, 0.0.62, 0.0.63, 0.0.64, 0.0.65, 0.0.66, 0.0.67, 0.0.68, 0.0.69, 0.0.70, 0.0.71, 0.0.72, 0.0.73, 0.0.74, 0.0.75, 0.0.76, 0.0.77, 0.0.79, 0.0.80, 0.0.81, 0.0.82, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.15.0, 1.16.0, 1.17.0, 1.18.0, 1.19.0, 1.20.0, 1.21.0, 1.22.0, 1.23.0, 1.24.0, 1.25.0, 1.26.0, 1.27.0, 1.28.0)

You'll notice that version list doesn't contain any 2.x versions at all (its also missing 1.29.x -> 1.30.x). That implies an sdist is being searched for since we do not (cannot actually) publish an sdist any longer due to the engine being in Rust and not having invested in getting an sdist build working. So, they only way the wheels for all those versions fail to present themselves as satisfactory to Pex (Pip) is when the interpreter running Pip does not support any of the wheel tags. You can see the tags Pip thinks your interpreter supports by running:

/Users/sandra/.cache/pants/setup/bootstrap-Darwin-x86_64/2.8.0_py39/bin/pip debug -v

Could you do that and report back? It will be a very large list of tags.

Thank you for the explanation of where you are heading with this. I’ll find out what tags pip supports and report back.

Often this happens when using older versions of macOS. See https://www.pantsbuild.org/docs/prerequisites

Ah, that might be an explanation. She is running 10.14.6 after all. Meanwhile, she also did run

pip debug -v

on the two cached bootstrap directories still present on her machine. The one in the above log didn’t exist anymore. I’ve attached the output for both. As to my issue with the lockfile for Black: I tried debugging that last night and, after adding

-ldebug

to

./pants

, Pants decided to rebuild the build tool PEXens (PEXes?) and everything is back to working order. Since that means Pylint is nagging me again, I’m still deciding whether that’s progress or a regression. 🙄 Pylint aside, it’s a bit surprising that a debug flag triggers a rebuild. Do you have any insight as to what triggers that? Thank you!

Hmm it definitely should not rebuild because of a debug flag. I've not seen that before. Are you able to reproduce it?

yea, it will.

So I just tried to reproduce the black lockfile issue, retracing my steps in two variations, to no avail. That seems to have been a Heisenbug. But the bit about

-ldebug

restarting the scheduler is useful knowledge for situations like this. I’ve had the impression over the last few weeks that Pants wouldn’t pick up changes at times, so this seems an easy way to prod Pants. In fact, I’m tempted to add that to the troubleshooting section of the docs. Should I?

If you're seeing what you think are cache issues, see https://www.pantsbuild.org/docs/troubleshooting#cache-or-pantsd-invalidation-issues

I'm leery of adding that to the docs because TBH I'm not sure it should restart the scheduler. Possibly we want a more overt way to do that.

That’s fair — though what makes this appealing to me is that

-ldebug

is generally useful (it certainly has helped me before) and it also triggering a scheduler restart is a bit like a hidden superpower.