https://pantsbuild.org/ logo
Join Slack
Powered by
Hello! I am running ```pants lint --lint-only=pyli...
# general
s
Hello! I am running
Copy code
pants lint --lint-only=pylint ::
and getting this error:
Copy code
pip: ERROR: No matching distribution found for pantsbuild.pants==2.21.0
I did encounter a ticket about something similar but the solution there was to not run pants with 
./pants
, which I am not. Any ideas?
Copy code
stderr:
pid 410038 -> /home/user/.cache/pants/named_caches/pex_root/venvs/0/9013e023ea1ba5759869ffac8bf45ba27c7fad99/5985ed09b49a653d6596b0e14d134c5456cf1a9f/bin/python /home/user/.cache/pants/named_caches/pex_root/venvs/0/9013e023ea1ba5759869ffac8bf45ba27c7fad99/5985ed09b49a653d6596b0e14d134c5456cf1a9f/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --isolated --log /tmp/pants-sandbox-GxHt0d/.tmp/pex-pip-log.5fsmrn42/pip.log -q --cache-dir /home/user/.cache/pants/named_caches/pex_root/pip/1/24.2/pip_cache download --dest /home/user/.cache/pants/named_caches/pex_root/downloads/0/resolver_download.2fmrlxav/usr.bin.python3.10 PyQt5-stubs==5.15.6.0 PyQt5==5.15.6 pandas~=2.0.2 pantsbuild.pants==2.21.0 pyserial~=3.5 roboticstoolbox-python~=1.0.2 spatialgeometry==1.0.1 spatialmath-python==1.0.5 --index-url <https://pypi.org/simple/> --extra-index-url https://<hidden>:<hidden>@<hidden>/artifactory/api/pypi/python/simple --extra-index-url https://<hidden>:<hidden>@<hidden>/artifactory/api/pypi/release/simple --extra-index-url https://<hidden>:<hidden>@<hidden>/artifactory/api/pypi/simple --find-links <https://wheels.pantsbuild.org/simple> --retries 5 --timeout 15 exited with 1 and STDERR:
pip: Ignoring the following environment variables in Pex venv mode:
pip: _PEX_CACHE_ACCESS_LOCK=0|5|/home/user/.cache/pants/named_caches/pex_root/access.lck
pip: ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==2.21.0 (from versions: 0.0.17, 0.0.18, 0.0.20, 0.0.21, 0.0.22, 0.0.23, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.39, 0.0.40, 0.0.41, 0.0.42, 0.0.43, 0.0.44, 0.0.45, 0.0.46, 0.0.47, 0.0.48, 0.0.49, 0.0.50, 0.0.51, 0.0.52, 0.0.53, 0.0.54, 0.0.55, 0.0.56, 0.0.57, 0.0.58, 0.0.59, 0.0.60, 0.0.61, 0.0.62, 0.0.63, 0.0.64, 0.0.65, 0.0.66, 0.0.67, 0.0.68, 0.0.69, 0.0.70, 0.0.71, 0.0.72, 0.0.73, 0.0.74, 0.0.75, 0.0.76, 0.0.77, 0.0.79, 0.0.80, 0.0.81, 0.0.82, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.15.0, 1.16.0, 1.17.0, 1.18.0, 1.19.0, 1.20.0, 1.21.0, 1.22.0, 1.23.0, 1.24.0, 1.25.0, 1.26.0, 1.27.0, 1.28.0)
This issue only happens with pylint. I do not get this issue with isort or black.
b
That’s strange! Sorry for the trouble Can you share your pants.toml?
s
No worries at all. Here it is:
Copy code
[GLOBAL]
pants_version = "2.21.0"
backend_packages.add = [
  "pants.backend.build_files.fmt.black",
  "pants.backend.experimental.python",
  "pants.backend.python",
  "pants.backend.python.lint.pylint",
  "pants.backend.python.lint.black",
  "pants.backend.python.lint.isort",
  "pants.backend.plugin_development",
  "pants.backend.python.mixed_interpreter_constraints",

  # Custom Plugins
  "versioning",
]
pants_ignore.add =[
  # Other common directories to ignore
  "__pycache__",
  "!.git",
  "!dist/",
]
pythonpath = ["%(buildroot)s/pants-plugins"]
print_stacktrace = true

[anonymous-telemetry]
enabled = false

[source]
marker_filenames = [
  "BUILD"
]

[python-bootstrap]
search_path = [
    "<PYENV>",
    "<PATH>",
    "%(env._PANTS_SERVER_EXE)s/..",
]

[python-infer]
unowned_dependency_behavior = "error"

[python-repos]
indexes.add = [
    "https://%(env.ARTIFACTORY_USR)s:%(env.ARTIFACTOR_PSW)s@<hidden>/artifactory/api/python/simple",
]

[twine]
config = "devops/artifactory.pypirc"

[isort]
config = ["devops/.isort.cfg"]

[stats]
log = true

[python]
enable_resolves = true

[python.resolves]
python-default = "devops/pantsbuild/lockfiles/python-default.lock"
pants-plugins = "devops/pantsbuild/lockfiles/pants-plugins.lock"

[python]
interpreter_constraints = ["~=3.10.6"]

[python.resolves_to_interpreter_constraints]
pants-plugins = ["==3.9.*"]
Some more info here, this happens at the end of the command, the command itself does executes and spits out pylinty stuff.
This also only happens when I run pylint with 
::
. Individual files work fine.
b
ah, okay. You've got some plugins 🤔 Can you share any more information about those plugins? Of particular interest: 1. how do you define the dependencies in the 
pants-plugins
resolve? 2. what sort of plugins are they, e.g. are they defining new targets or new goals, or hooking into existing ones?
s
I actually don't have any explicit 3rd party dependencies in my plugin. I just have this (but no requirements.txt):
Copy code
__defaults__(all=dict(resolve="pants-plugins"))

pants_requirements(
    name="pants",
    resolve="pants-plugins",
)
In my pants-plugins.lock, I have the following that has pylint:
Copy code
"project_name": "pantsbuild-pants",
          "requires_dists": [
            "PyYAML<7.0,>=6.0",
            "ansicolors==1.1.8",
            "chevron==0.14.0",
            "fasteners==0.16.3",
            "ijson==3.2.3",
            "node-semver==0.9.0",
            "packaging==21.3",
            "pex==2.3.1",
            "psutil==5.9.8",
            "python-lsp-jsonrpc==1.0.0",
            "setproctitle==1.3.2",
            "setuptools<64.0,>=63.1.0",
            "toml==0.10.2",
            "types-PyYAML==6.0.3",
            "types-setuptools==62.6.1",
            "types-toml==0.10.8",
            "typing-extensions==4.3.0"
          ],
          "requires_python": "==3.9.*",
          "version": "2.21.0"
and then
Copy code
"project_name": "python-lsp-jsonrpc",
          "requires_dists": [
            "coverage; extra == \"test\"",
            "pycodestyle; extra == \"test\"",
            "pyflakes; extra == \"test\"",
            "pylint; extra == \"test\"",
            "pytest-cov; extra == \"test\"",
            "pytest; extra == \"test\"",
            "ujson>=3.0.0"
          ],
I have one custom plugin that uploads the pexes that pants creates, using the curl bash command. It is an independent goal.
I can see that this might be an issue? The first code snippet says requires_python == 3.9.*, but my project's interpreter constraints are ~=3.10.12. I do use mixed_interpreter_constraints and I also have pylint defined in my project's resolve.
Let me also add that I added to my pants.toml:
Copy code
[pylint]
install_from_resolve = "python-default"
Ah, on https://wheels.pantsbuild.org/simple/, I do not see pants2.21.0 for python3.10.12. Is there an alternative to get this working? I see no wheels for python3.10 whatsoever. I think this is the root cause.
b
Yeah, that version of Pants only runs on 3.9.
Copy code
[python.resolves_to_interpreter_constraints]
pants-plugins = ["==3.9.*"]
should be handling that.
It's definitely confusing. There's ~three levels of Python code/versions: • your "main" code <= 3.10 • any external tools written in Python, like pylint <= potentially "arbitrary" versions, depending on what the tool supports (e.g. an tool might require being executed using Python 3.12, but can analyse a Python 3.10 input file) • the "orchestration" Pants code, that applies the tools to your main code <= 3.9
Looking at the failing command output, it looks like it's attempting to install these dependencies:
Copy code
PyQt5-stubs==5.15.6.0 PyQt5==5.15.6 pandas~=2.0.2 pantsbuild.pants==2.21.0 pyserial~=3.5 roboticstoolbox-python~=1.0.2 spatialgeometry==1.0.1 spatialmath-python==1.0.5
Those mostly sound like "main" dependencies, so it's unexpected to me that pants itself is being pulled in there. A few follow-up questions, to try to work out why it's being pulled in: 1. is there more logging around the time of the failure you can share? e.g. potentially stuff like "Building 7 requirements for ..." 2. if you run with 
pants --print-stacktrace ...
, does that show a stacktrace? Can you share that? 3. does 
pantsbuild
appear in the 
python-default.lock
file?
s
Stack trace:
Copy code
PANTS_CONFIG_FILES=pants.release.ci.toml pants --print-stacktrace lint --lint-only=pylint ::
17:06:05.48 [ERROR] 1 Exception encountered:

Engine traceback:
  in root
    ..
  in pants.core.goals.lint.lint
    `lint` goal

Traceback (most recent call last):
  File "/home/user/.cache/nce/ab1acf935c4cc43338c604ae7d0f6aa2419f2415d94eb9cae381601dbba70a61/bindings/venvs/2.21.0/lib/python3.9/site-packages/pants/core/goals/lint.py", line 469, in lint
    all_batch_results = await MultiGet(
  File "/home/user/.cache/nce/ab1acf935c4cc43338c604ae7d0f6aa2419f2415d94eb9cae381601dbba70a61/bindings/venvs/2.21.0/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 376, in MultiGet
    return await _MultiGet(tuple(__arg0))
  File "/home/user/.cache/nce/ab1acf935c4cc43338c604ae7d0f6aa2419f2415d94eb9cae381601dbba70a61/bindings/venvs/2.21.0/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 174, in __await__
    result = yield self.gets
  File "/home/user/.cache/nce/ab1acf935c4cc43338c604ae7d0f6aa2419f2415d94eb9cae381601dbba70a61/bindings/venvs/2.21.0/lib/python3.9/site-packages/pants/backend/python/lint/pylint/rules.py", line 168, in run_pylint
    ) = await MultiGet(
  File "/home/user/.cache/nce/ab1acf935c4cc43338c604ae7d0f6aa2419f2415d94eb9cae381601dbba70a61/bindings/venvs/2.21.0/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 436, in MultiGet
    return await _MultiGet((__arg0, __arg1, __arg2, __arg3))
  File "/home/user/.cache/nce/ab1acf935c4cc43338c604ae7d0f6aa2419f2415d94eb9cae381601dbba70a61/bindings/venvs/2.21.0/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 174, in __await__
    result = yield self.gets
  File "/home/user/.cache/nce/ab1acf935c4cc43338c604ae7d0f6aa2419f2415d94eb9cae381601dbba70a61/bindings/venvs/2.21.0/lib/python3.9/site-packages/pants/backend/python/util_rules/pex.py", line 826, in create_pex
    result = await Get(BuildPexResult, PexRequest, request)
  File "/home/user/.cache/nce/ab1acf935c4cc43338c604ae7d0f6aa2419f2415d94eb9cae381601dbba70a61/bindings/venvs/2.21.0/lib/python3.9/site-packages/pants/engine/internals/selectors.py", line 124, in __await__
    result = yield self
  File "/home/user/.cache/nce/ab1acf935c4cc43338c604ae7d0f6aa2419f2415d94eb9cae381601dbba70a61/bindings/venvs/2.21.0/lib/python3.9/site-packages/pants/backend/python/util_rules/pex.py", line 720, in build_pex
    requirements_setup = await _setup_pex_requirements(request, python_setup)
  File "/home/user/.cache/nce/ab1acf935c4cc43338c604ae7d0f6aa2419f2415d94eb9cae381601dbba70a61/bindings/venvs/2.21.0/lib/python3.9/site-packages/pants/backend/python/util_rules/pex.py", line 601, in _setup_pex_requirements
    validate_metadata(
  File "/home/user/.cache/nce/ab1acf935c4cc43338c604ae7d0f6aa2419f2415d94eb9cae381601dbba70a61/bindings/venvs/2.21.0/lib/python3.9/site-packages/pants/backend/python/util_rules/pex_requirements.py", line 525, in validate_metadata
    raise InvalidLockfileError(msg)
pants.core.util_rules.lockfile_metadata.InvalidLockfileError: You are consuming `pantsbuild.pants==2.21.0` from the `pants-plugins` lockfile at devops/pantsbuild/lockfiles/pants-plugins.lock with incompatible inputs.



- The inputs use interpreter constraints (`CPython~=3.10.6`) that are not a subset of those used to generate the lockfile (`CPython==3.9.*`).

- The input interpreter constraints are specified by your code, using the `[python].interpreter_constraints` option and the `interpreter_constraints` target field.

- To create a lockfile with new interpreter constraints, update the option `[python].resolves_to_interpreter_constraints`, and then generate the lockfile (see below).

See <https://www.pantsbuild.org/2.21/docs/python/overview/interpreter-compatibility> for details.

To regenerate your lockfile, run `pants generate-lockfiles --resolve=pants-plugins`.

See <https://www.pantsbuild.org/2.21/docs/python/overview/third-party-dependencies> for details.
pantsbuild-pants does not appear in the python-default lock. I have some more info to share. This issue only appears when I try to include my pants-plugins directory in the subset of files I want to lint. I think this is because I am specifying that pylint be installed from the python-default resolve which uses python3.10.12. If I specify a set of directories that all use 3.10.12, I do not see this error. pants-plugins is the only resolve that uses 3.9.*.
b
ah okay, that's helpful! yeah, I think that's a limitation that's hard to avoid at the moment, with the pylint style of linter.
s
Thanks a bunch for helping me debug this. 🙂
16 Views
Open in Slack
PreviousNext
Powered by