Hi Y'all since this morning, we are facing issues ...
# general
f
Hi Y'all since this morning, we are facing issues with lockfile generation as the newly released setuptools 72 seems to be incompatible with some of the 3rd party packages we have dependency on (vcrpy for instance, but there seem to be others we have not managed to track down yet). When calling
generate-lockfiles
, I'm getting a semi-helpful
Copy code
ProcessExecutionFailure: Process 'Generate lockfile for python-default' failed with exit code 1.

stderr: <lengthy pex call>

and STDERR:
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
I have tried down-pinning setuptools, both in the offending resolve(s) themselves, as well as globally by creating a resolve for setuptools - though the former seems to be ignored and the latter only be relevant for python_distribution targets - not building resolves. When in pip, we apparently need to disable build isolation or pip will happily use the latest version of setuptools unless the package itself has requirements that disable this. I haven't been able to figure out how to set the same in pants (or at least limit the highest version of setuptools). Would appreciate any pointers. Thanks
f
Same here, also trying to figure out something
w
f
yes
b
yes, which can be worked around on other builds with Poetry or on Pip parameters but not on Pants as far as I know
w
And from what I understood, I believe you tried some of the customization detailed here? https://www.pantsbuild.org/2.21/docs/python/overview/building-distributions Are you able to use the contraints tweak with resolves? https://www.pantsbuild.org/2.21/reference/subsystems/python#resolves_to_constraints_file The last option, which is something I was looking at myself was how to provide CLI arguments to pex, as pex has a
no-build-isolation
flag... But Pants doesn't seem to have exposed it
f
I've both tried to pin setuptools within the offending resolves directly, as well as create it's own resolve, e.g. https://www.pantsbuild.org/2.20/reference/subsystems/setuptools the latter just gives me an empty lockfile as setuptools seems to be ignored as a requirement, possibly as the lockfile states
Copy code
"build_isolation": true,
which suggest that setuptools is used on the fly anyway (from what I understood). I also haven't found a way to disable build isolation.
w
Yeah, pex allows that - give me a sec to try something, I just created a repro repo
Interesting, using
--no-build-isolation
in pip doesn't appear to work
Copy code
pants --python-resolves-to-constraints-file="{'python-default': 'constraints.txt'}" generate-lockfiles ::
Does this work for anyone? With constraints.txt being:
Copy code
setuptools<72
Or better yet, if anyone has an example dep that causes this problem, as the one I'm using seems to only sometimes not work 🤦‍♂️
f
Copy code
python_requirement(
    name="pytest",
    resolve="pytest",
    requirements=["pytest", "pytest-cov", "ipdb", "pytest-vcr", "pytest-asyncio"],
)
seems to be reliably broken for us
issue being pytest-vcr or vcrpy in this case
w
Alright, so THAT worked when using
--no-build-isolation
in the pex cli, and then broke when removing it. That's a good thing
Now, the question is, is that exposable right now... Are you able to build pants from source to test on your repo?
f
never tried that, but the documentation seems straight forward enough so let me give it a try
w
👍 Maybe someone more familiar with some of this can figure out how to pass the constraints file in - doesnt' seem to be working for me. I recall that we used to have a constraints option that went away, so this is weird
What version of pants are you using?
Maybe try using a more recent pex cli?
Copy code
[pex-cli]
version = "v2.11.0"
known_versions = [
  "v2.11.0|macos_arm64|554a3ad5d1662d93d06a7e65bf5444589d1956752df39e2665a4d54e1e05c949|4181790",
  "v2.11.0|macos_x86_64|554a3ad5d1662d93d06a7e65bf5444589d1956752df39e2665a4d54e1e05c949|4181790",
  "v2.11.0|linux_x86_64|554a3ad5d1662d93d06a7e65bf5444589d1956752df39e2665a4d54e1e05c949|4181790",
  "v2.11.0|linux_arm64|554a3ad5d1662d93d06a7e65bf5444589d1956752df39e2665a4d54e1e05c949|4181790"
]
f
we have been on 2.17 for far too long, this just put upgrading to the top of my to do list for this week though
w
👍 You can upgrade a bunch of stuff in isolation, like pex, but yeah - I need to look into this pex-cli args. It should be trivial to passthrough args to pex, but I don't know if we exposed that ability 😕
👍 1
f
the yank has put us back in business anyway. Now I have time to upgrade things 😃 thank you for your help either way!
👍 1