https://pantsbuild.org/ logo
#general
Title
# general
l

lemon-oxygen-72498

12/07/2022, 9:25 AM
I'm generalizing my client's CI to demonstrate testing his libraries with two python versions, so I have
interpreter_constraints = ["==3.8.*", "==3.9.*"]
in the top-level
pants.toml
file and in one library, I have
interpreter_constraints=parametrize(py38=["==3.8.*"], py39=["==3.9.*"])
to generate targets for the two interpreters. My problem is that
pants
doesn't want to run
black
anymore,
./pants lint --only=black ::
fails with:
Copy code
InvalidLockfileError: You are using the lockfile at pants-utils/3rdparty/python/black_lockfile.lock to install the tool `black`, but it is not compatible with your configuration:

- You have set interpreter constraints (`CPython<4,>=3.7`) that are not compatible with those used to generate the lockfile (`CPython==3.8.* OR CPython==3.9.*`).You can fix this by not setting `[black].interpreter_constraints`, or by using a new custom lockfile.
But
./pants lint --only=isort ::
works 🤯 Running
./pants generate-lockfiles --resolve=black
doesn't solve the issue. What is the root explanation and what can I do?
1
w

wide-midnight-78598

12/07/2022, 1:04 PM
After resolving Black, what does the generated lock file say about its interpreter constraints?
l

lemon-oxygen-72498

12/07/2022, 2:47 PM
Copy code
→ git grep -h -C 4 interpreter_constraints pants-utils/3rdparty/python/black_lockfile.lock
//
// --- BEGIN PANTS LOCKFILE METADATA: DO NOT EDIT OR REMOVE ---
// {
//   "version": 3,
//   "valid_for_interpreter_constraints": [
//     "CPython==3.8.*",
//     "CPython==3.9.*"
//   ],
//   "generated_with_requirements": [
It looks correct 😕
h

happy-kitchen-89482

12/07/2022, 5:07 PM
Where is that
CPython<4,>=3.7
coming from? Were those the old interpreter constraints from pants.toml? Or the default if you weren't setting those?
e

enough-analyst-54434

12/07/2022, 5:09 PM
Unfortunately? Those are from us:
Copy code
$ git grep "CPython>=3.7,<4"
src/python/pants/backend/cc/lint/clangformat/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/codegen/protobuf/python/python_protobuf_subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/docker/subsystems/dockerfile_parser.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/goals/coverage_py.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/lint/add_trailing_comma/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/lint/autoflake/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/lint/black/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/lint/docformatter/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/lint/isort/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/lint/pyupgrade/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/lint/yapf/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/subsystems/setup.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/subsystems/setuptools_scm.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/subsystems/twine.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/typecheck/mypy/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/typecheck/pyright/subsystem.py:        default=["CPython>=3.7,<4"],
src/python/pants/backend/terraform/dependency_inference.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
g

gentle-gigabyte-52115

12/07/2022, 5:13 PM
yeah it seems to pick the constraints from that lockfile ( pants/src/python/pants/backend/python/lint/black/black.lock ) even though we use a custom lockfile. We seem to be able to set interpreter_constraints in the toml file to resolve this, but it seems somewhat unexpected that it would use the other lockfile for this. (I work with @lemon-oxygen-72498)
w

wide-midnight-78598

12/07/2022, 5:29 PM
We seem to be able to set interpreter_constraints in the toml file to resolve this
Which
interpreter_constraints
did you have to set to solve this? And which one didn't solve it?
g

gentle-gigabyte-52115

12/07/2022, 6:24 PM
Copy code
[black] # <https://www.pantsbuild.org/docs/reference-black>
interpreter_constraints = [">=3.8"]
version = "black==22.3.0"
lockfile = "pants-utils/3rdparty/python/black_lockfile.lock"
e

enough-analyst-54434

12/07/2022, 6:42 PM
So, no clue if this helps, but that IC is dangerous. I think black is non-platform specific, so you get lucky, but if it was platform specific, certainly there are no Python 3.12 wheels out there today and who knows if you could build the sdist in that case. So, you may want to restrict the ICs there to what you need. Maybe just
>=3.8,<3.10
👍 2
Pants is setting a bit of a bad example actually with the
>=3.7,<4
default. That permits 3.12 alphas. We've had no issues reported as a result though that I'm aware of.
l

lemon-oxygen-72498

12/07/2022, 7:42 PM
Thanks everyone, I didn't know that one can attach
interpreter_constraints
declarations to tools. @gentle-gigabyte-52115's suggestion indeed works 👍
h

happy-kitchen-89482

12/07/2022, 9:06 PM
Yeah, I think our initial pants install script should force you to pick an IC, and strongly recommend that it be for a single minor version if possible
That default is... bad
4 Views