https://pantsbuild.org/ logo
Join Slack
Powered by
I'm generalizing my client's CI to demonstrate tes...
# general
l
I'm generalizing my client's CI to demonstrate testing his libraries with two python versions, so I have 
interpreter_constraints = ["==3.8.*", "==3.9.*"]
in the top-level 
pants.toml
file and in one library, I have 
interpreter_constraints=parametrize(py38=["==3.8.*"], py39=["==3.9.*"])
to generate targets for the two interpreters. My problem is that 
pants
doesn't want to run 
black
anymore, 
./pants lint --only=black ::
fails with:
Copy code
InvalidLockfileError: You are using the lockfile at pants-utils/3rdparty/python/black_lockfile.lock to install the tool `black`, but it is not compatible with your configuration:

- You have set interpreter constraints (`CPython<4,>=3.7`) that are not compatible with those used to generate the lockfile (`CPython==3.8.* OR CPython==3.9.*`).You can fix this by not setting `[black].interpreter_constraints`, or by using a new custom lockfile.
But 
./pants lint --only=isort ::
works 🤯 Running 
./pants generate-lockfiles --resolve=black
doesn't solve the issue. What is the root explanation and what can I do?
✅ 1
w
After resolving Black, what does the generated lock file say about its interpreter constraints?
l
Copy code
→ git grep -h -C 4 interpreter_constraints pants-utils/3rdparty/python/black_lockfile.lock
//
// --- BEGIN PANTS LOCKFILE METADATA: DO NOT EDIT OR REMOVE ---
// {
//   "version": 3,
//   "valid_for_interpreter_constraints": [
//     "CPython==3.8.*",
//     "CPython==3.9.*"
//   ],
//   "generated_with_requirements": [
It looks correct 😕
h
Where is that 
CPython<4,>=3.7
coming from? Were those the old interpreter constraints from pants.toml? Or the default if you weren't setting those?
e
Unfortunately? Those are from us:
Copy code
$ git grep "CPython>=3.7,<4"
src/python/pants/backend/cc/lint/clangformat/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/codegen/protobuf/python/python_protobuf_subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/docker/subsystems/dockerfile_parser.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/goals/coverage_py.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/lint/add_trailing_comma/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/lint/autoflake/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/lint/black/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/lint/docformatter/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/lint/isort/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/lint/pyupgrade/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/lint/yapf/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/subsystems/setup.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/subsystems/setuptools_scm.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/subsystems/twine.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/typecheck/mypy/subsystem.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
src/python/pants/backend/python/typecheck/pyright/subsystem.py:        default=["CPython>=3.7,<4"],
src/python/pants/backend/terraform/dependency_inference.py:    default_interpreter_constraints = ["CPython>=3.7,<4"]
g
yeah it seems to pick the constraints from that lockfile ( pants/src/python/pants/backend/python/lint/black/black.lock ) even though we use a custom lockfile. We seem to be able to set interpreter_constraints in the toml file to resolve this, but it seems somewhat unexpected that it would use the other lockfile for this. (I work with @lemon-oxygen-72498)
w
We seem to be able to set interpreter_constraints in the toml file to resolve this
Which 
interpreter_constraints
did you have to set to solve this? And which one didn't solve it?
g
Copy code
[black] # <https://www.pantsbuild.org/docs/reference-black>
interpreter_constraints = [">=3.8"]
version = "black==22.3.0"
lockfile = "pants-utils/3rdparty/python/black_lockfile.lock"
e
So, no clue if this helps, but that IC is dangerous. I think black is non-platform specific, so you get lucky, but if it was platform specific, certainly there are no Python 3.12 wheels out there today and who knows if you could build the sdist in that case. So, you may want to restrict the ICs there to what you need. Maybe just 
>=3.8,<3.10
👍 2
Pants is setting a bit of a bad example actually with the 
>=3.7,<4
default. That permits 3.12 alphas. We've had no issues reported as a result though that I'm aware of.
l
Thanks everyone, I didn't know that one can attach 
interpreter_constraints
declarations to tools. @gentle-gigabyte-52115's suggestion indeed works 👍
h
Yeah, I think our initial pants install script should force you to pick an IC, and strongly recommend that it be for a single minor version if possible
That default is... bad
4 Views
Open in Slack
PreviousNext
Powered by