Hi everyone, sorry if I should have been able to find this already. Sometimes

pants run

gets stuck in the "building requirements" point (seemingly for ever). It seems like

pants generate-lockfiles

clears it up, but it feels like the wrong thing to be doing. Does anyone have an idea about what could be happening? Unfortunately I have no way to reproduce this. Is there another command that could be run that's more appropriate vs regenerating the lock file?

On the surface, your report doesn't seem to make much sense. It should be the case that if lockfiles need to be updated, Pants errors fast and tells you to run

pants generate-lockfiles ...

. What version of Pants is this and do you have

[python] enable_resolves = true

configured in

pants.toml

?

So it doesn't seem to be the new lock file which actually makes any difference. If I revert the lock file it continues to run just fun. It seems almost as if generating the lockfile has some other side effect that unblocks, particularly because the hanging build is repeatable when it occurs (rarely). Pants version is 2.16.0 and enable_resolves is true. This is Python, yes.

I guess we only say "requirements" for Python though,

We're using some git/ssh requirements that need to be compiled. I've no understanding on what goes on behind the scenes, but generate lockfiles is slow enough that it seems like it's compiling the wheels. It seems to be, for whatever reason, that on rare occasions

pants run

hangs up when doing that but

generate-lockfiles

doesn't, and does enough to unblock

pants run

the next time. Right now our workflow is, if pants is hanging then just run

generate-lockfiles

. Are there any logs I can collect next time this happens?

pants generate-lockfiles

should not be compiling wheels for VCS requirements. It should be using PEP-517 https://peps.python.org/pep-0517/#get-requires-for-build-wheel + https://peps.python.org/pep-0517/#prepare-metadata-for-build-wheel in that sequence to generate metadata for the VCS project (gather its Requires-Python + Requires-Dist entries). It only falls back to building a wheel when the optional

prepare-metadata-for-build-wheel

API is not implemented by the project's build backend. To speed up locks, you might try using

[python] pip_version = "23.0.1"

in `pants.toml`: https://www.pantsbuild.org/v2.16/docs/reference-python#pip_version I still am stuck back though on

pants run

working at all when the lock file is out of date.

pants run

should fail fast and prompt you to re-gen the lock. If it does not, I think that represents a Pants bug.

I'm not sure if the lock file is out of date in this case - none of the requirement versions have changed and we use exact == versions Is there anything I can run for you to help determine if there is a bug or not?

Ok, so you have a lock file and it is up to date; therefore Pants does not error when you try to run

pants run ...

, it just hangs sometimes. You say running

pants generate-lockfiles ...

, even though not required, does help a subsequent

pants run ...

not hang. It took me a while to understand this, but it looks like this is exactly what you said above. It's just very strange. The way to start debugging this would be to: 1. Get back in the hang state and ctrl-C. 2. Re-run the hanging

pants run ...

with

pants run --keep-sandboxes=always ...

as outlined here: https://www.pantsbuild.org/docs/rules-api-tips#debugging-look-inside-the-chroot 3. With the hanging sandbox preserved and found, look at its

__run.sh

script, find the Pex command line and add args

--preserve-pip-download-log -vvvvvvvvv

to it. That will both flood the console with logs as well as print out a tmp path where the pip download log is preserved. The Pex verbose log and / or the Pip download log from step 3 may provide useful information.

Thanks @enough-analyst-54434, I'll do this next time it happens