ripe-analyst-17330
04/18/2022, 1:19 PMpython
in the PATH
) or otherwise use my system python from MacOS. however, it seems to behave in a completely unexpected way. In the output (see thread), you can see that I'm running in a 3.10
venv, pants uses a seemingly random 3.9
binary from my pyenv shims, and then when the goal is run, it uses a 3.7
binary (maybe the original system binary?).% python --version
Python 3.10.3
% pyenv version
global-3.10.3 (set by PYENV_VERSION environment variable)
% ./pants generate-lockfiles
Bootstrapping Pants using /Users/mike/.pyenv/shims/python3.9
Creating the virtualenv PEX.
Downloading the Pex PEX.
SHA256 fingerprint of <https://github.com/pantsbuild/pex/releases/download/v2.1.62/pex> verified.
Installing pantsbuild.pants==2.11.0rc3 into a virtual environment at /Users/mike/.cache/pants/setup/bootstrap-Darwin-x86_64/2.11.0rc3_py39
New virtual environment successfully created at /Users/mike/.cache/pants/setup/bootstrap-Darwin-x86_64/2.11.0rc3_py39.
09:21:04.25 [INFO] Completed: Generate lockfile for python-default
09:21:04.26 [ERROR] 1 Exception encountered:
ProcessExecutionFailure: Process 'Generate lockfile for python-default' failed with exit code 1.
stdout:
The currently activated Python version 3.7.12 is not supported by the project (>=3.10).
Trying to find and use a compatible version.
NoCompatiblePythonVersionFound
pants.toml
file making it even more confusing:
[python]
interpreter_constraints = ["CPython>=3.10"]
enough-analyst-54434
04/18/2022, 1:44 PMinterpreter_constraints
you configured when trying to generate a lock for the python-default
resolve.
As to the Python 3.9 - Pants itself needs Python to run. This is true even if you only use Pants to work with Go or the JVM. As such it has its own interpreter constraints of >=3.7,<3.10
; i.e. it runs on Python 3.{7,8,9}. So you see it picking a Python 3.9 interpreter to run on from your machine. That should generally be of no concern to you; except here it caused understandable confusion when downstream errors made you pause to read the logs more fully.
After picking the interpreter it needs, Pants then (generally!) picks the interpreter your code needs if you're using Pants to work with Python code like you are.[python-bootstrap] search_path
?: https://www.pantsbuild.org/docs/reference-python-bootstrap#section-search-pathripe-analyst-17330
04/18/2022, 1:51 PM3.9
, thanks. It does seem like pants eventually runs a different venv.
I've tried both with and without the following with the same result:
[python-bootstrap]
search_path = ["<PYENV>"]
enough-analyst-54434
04/18/2022, 2:11 PM[python] lockfile_generator
? I'm on a little firmer footing talking about the Pex locker. This may be using the Poetry locker though under the covers.ripe-analyst-17330
04/18/2022, 2:57 PMlockfile_generator = 'pex'
[poetry]
version = "poetry==1.1.13"
interpreter_constraints = ["CPython>=3.10,<4"]
Is there a per-project poetry config that can be set for this as it seems it would break a monorepo with different version constraints per project? We have this issue in our repo I just don't know if its enough to cause problems here (most are 3.9+ compatible)enough-analyst-54434
04/18/2022, 3:24 PMreq
ICs: https://github.com/pantsbuild/pants/blob/2d2e287193e5d2ee68058c0b016a184d4f02489d/src/python/pants/backend/python/goals/lockfile.py#L208-L214
Just above though, using the Pex locker, the right thing is done:
https://github.com/pantsbuild/pants/blob/2d2e287193e5d2ee68058c0b016a184d4f02489d/src/python/pants/backend/python/goals/lockfile.py#L159-L178
So, it looks like you've found a bug for locking using the Poetry locker. I'll check to see if that's filed / known. You might try switching to the Pex locker if you need per-project support. We're headed to that being the default anyhow and likely will strip support for Poetry being the locker since it simply can't handle some permutations of the inputs - like custom find links repos.[python] enable-resolves
configured to true
? (It may be useful to just dump the `[python]`section of your pants.toml
to prevent a bit of back-forth): https://www.pantsbuild.org/v2.11/docs/reference-python#section-enable-resolvespyproject.toml
here: https://github.com/pantsbuild/pants/blob/2d2e287193e5d2ee68058c0b016a184d4f02489d/src/python/pants/backend/python/subsystems/poetry.py#L68-L83
I think the problem is when The Poetry tool (PEX) is installed using different ICs than the ICs fed to it in pyproject.toml
. IOW we use 1 global ~venv for Poetry and we then try to feed it many different generated pyproject.toml
, each with custom ICs. If the ICs conflict - boom.pants.toml
config. That was very helpful. I'm pretty sure my analysis is correct so I went ahead and filed: https://github.com/pantsbuild/pants/issues/15171
Hopefully you can use the workaround you found or else switch to the "pex" lockfile generator in the meantime.ripe-analyst-17330
04/18/2022, 3:49 PMHEAD
but with 3rd party deps, we sometimes use different versions in different projects. But it sounds like this breaks the multiple resolves as well, which I hadn't yet figured outenough-analyst-54434
04/18/2022, 3:54 PM(~/dev/repo) $ ./pants foo poetry_project1/...
having different results than (~/dev/repo/poetry_project1) $ poetry foo ...
.ripe-analyst-17330
04/18/2022, 3:58 PM[tool.poetry.scripts]
- but I'm guessing we would just need per-project poetry build
commands. Sorry I'm just starting to look at pants, so I'm not too familiar with the build steps yet. I was building CI matrixes with scripts when this was suggested.hundreds-father-404
05/12/2022, 3:12 PM