Hello, I’ve a 3rd-party lockfile generation issue ...
# general
b
Hello, I’ve a 3rd-party lockfile generation issue 🧵
Pants is correctly giving me the below error when I’m trying to change my interpreter constraints from 
>=3.6.6,<=3.8.5
to
>=3.8.5,<=3.10.0
Copy code
22:16 $ ./pants test ::
22:17:17.55 [ERROR] 1 Exception encountered:

  ValueError: You are using the lockfile at 3rdparty/pytest_lockfile.txt to install the tool `pytest`, but it is not compatible with your configuration:

- You have set interpreter constraints (`CPython==3.10.0`) that are not compatible with those used to generate the lockfile (`CPython<=3.8.5,==3.6.6,>=3.6.6 OR CPython<=3.8.5,==3.8.5,>=3.6.6 OR CPython==3.6.6 OR CPython==3.8.5`). `pytest` determines its interpreter constraints based on your code's own constraints. To fix this error, you can either change your code's constraints (see <https://www.pantsbuild.org/v2.8/docs/python-interpreter-compatibility>) or by generating a new custom lockfile.

To regenerate your lockfile based on your current configuration, run `./pants generate-lockfiles --resolve=pytest`.
But when I try to re-generate the lockfile with the new constraints, I get the below poetry error -
Copy code
22:17 $ ./pants generate-lockfiles --resolve=pytest
22:21:18.00 [WARN] The option `[python-repos].indexes` is configured, but it does not currently work with lockfile generation. Lockfile generation will fail if the relevant requirements cannot be located on PyPI.

ally generate a lockfile, such as by using pip-compile or `pip freeze`. Set the `[tool].lockfile` option to the path you manually generated. When manually maintaining lockfiles, set `[python].invalid_lockfile_behavior = 'ignore'.
22:21:37.74 [INFO] Completed: Generate lockfile for pytest
22:21:37.74 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Generate lockfile for pytest' failed with exit code 1.
stdout:
The currently activated Python version 3.6.6 is not supported by the project (<=3.10.0,==3.10.0,>=3.8.5 || <=3.10.0,==3.8.5,>=3.8.5 || ==3.10.0 || ==3.8.5).
Trying to find and use a compatible version.

  NoCompatiblePythonVersionFound

  Poetry was unable to find a compatible version. If you have one, you can explicitly use it via the "env use" command.

  at ~/.cache/pants/named_caches/pex_root/venvs/s/78ebfe9c/venv/lib/python3.6/site-packages/poetry/utils/env.py:737 in create_venv
       733│                     break
       734│
       735│             if not executable:
       736│                 raise NoCompatiblePythonVersionFound(
    →  737│                     self._poetry.package.python_versions
       738│                 )
       739│
       740│         if root_venv:
       741│             venv = venv_path

stderr:



Use --no-process-execution-local-cleanup to preserve process chroots for inspection.
I’ve a 3.10.0 venv activated on the repo using pyenv
I’m on Pants
2.8.0
Copy code
[GLOBAL]
pants_version = "2.8.0"
pythonpath = ["%(buildroot)s/pants-plugins"]
backend_packages = [
  "pants.backend.python",
  "pants.backend.python.mixed_interpreter_constraints",
  "pants.backend.python.lint.black",
  "pants.backend.python.lint.flake8",
  "pants.backend.python.lint.isort",
  "pants.backend.python.typecheck.mypy",
  "internal_plugins.packaging"
]
ignore_warnings = [
  "$regex$Unmatched glob*"
]
build_file_prelude_globs = [
  "pants-plugins/macros.py"
]

[source]
root_patterns = [
  "src/python/gh-*",
  "pants-plugins"
]

[black]
config = "pyproject.toml"

[flake8]
config = ".flake8"

[isort]
version = "isort[pyproject,colors]>=5.5.1,<5.6"
config = ["pyproject.toml"]
lockfile = "3rdparty/isort_lockfile.txt"

[mypy]
config = "mypy.ini"

[pytest]
args = ["-vv"]
extra_requirements.add = [
  "pytest-icdiff",
  "pygments",
  "coverage",
  "pytest-cov"
]
lockfile = "3rdparty/pytest_lockfile.txt"

[python]
interpreter_constraints = [">=3.8.5,<=3.10.0"]
interpreter_search_paths = ["<PYENV_LOCAL>", "<PYENV>", "<PATH>"]

[python-infer]
inits = true

[python-repos]
indexes.add = ["<https://www.acme.com/artifactory/api/pypi/pypi-repos/simple>"]

[setup-py-generation]
first_party_dependency_version_scheme = "compatible"

[anonymous-telemetry]
enabled = false
h
If you temporarily comment out the custom index in
[python-repos]
does it work?
This is about to become significantly more robust in 2.10.x
b
Getting the same error when I remove the custom index
So I’ve a macro that runs tests for multiple python versions. I changed the interpreter constraints there from
==3.10.0
and
==3.8.5
to
==3.10.*
and
==3.8.*
👍 1
and now everything works. interesting
h
This is about to become significantly more robust in 2.10.x
Nack, we still use Poetry to generate lockfiles in 2.10. We are hoping to have Pex generate lockfiles in 2.11, but I'm not certain that would make this UX much better
Okay glad it's working now...double checking btw if you've seen the tip about the
py-constraints
goal: https://www.pantsbuild.org/docs/python-interpreter-compatibility#tip-activate-pantsbackendpythonmixed_interpreter_constraints I wonder if it'd be possible to use fewer interpreter constraints than what you have. It might make things easier to reason about
h
Oh yeah, reading the error message more carefully, that index thing was just a warning, and a red herring. Glad it's working now.
b
@hundreds-father-404 I did use the py-constraints goal and did not see 3.6.6 in constraints (even transitive). So not sure what made Pants pick 3.6.6
h
Oh hm, it looks like Poetry's interpreter constraints are set by
[poetry].interpreter_constraints
, rather than based on your own code. It defaults to 3.6+ because Poetry generally works with 3.6+ Indeed, that confusing experience would be solved in 2.11 when we switch to Pex for lockfile generation (which uses pip under-the-hood)