Hi all sorry for some questions on using pants I just starte

Question

Hi all sorry for some questions on using pants I just started trying to use it for a monorepo with existing projects While I ve hit a few issues I m currently trying to understand this least astonishment issue for pants with python I suspected that pants would use either my currently activated virtual environment default to `python` in the `PATH` or otherwise use my system python from MacOS however it seems to behave in a completely unexpected way In the output see thread you can see that I m running in a `3 10` venv pants uses a seemingly random `3 9` binary from my pyenv shims and then when the goal is run it uses a `3 7` binary maybe the original system binary

ripe-analyst-17330 · Answer

``` python version Python 3 10 3 pyenv version global 3 10 3 set by PYENV VERSION environment variable pants generate lockfiles Bootstrapping Pants using Users mike pyenv shims python3 9 Creating the virtualenv PEX Downloading the Pex PEX SHA256 fingerprint of <https github com pantsbuild pex releases download v2 1 62 pex> verified Installing pantsbuild pants==2 11 0rc3 into a virtual environment at Users mike cache pants setup bootstrap Darwin x86 64 2 11 0rc3 py39 New virtual environment successfully created at Users mike cache pants setup bootstrap Darwin x86 64 2 11 0rc3 py39 09 21 04 25 INFO Completed Generate lockfile for python default 09 21 04 26 ERROR 1 Exception encountered ProcessExecutionFailure Process Generate lockfile for python default failed with exit code 1 stdout The currently activated Python version 3 7 12 is not supported by the project gt =3 10 Trying to find and use a compatible version NoCompatiblePythonVersionFound```

ripe-analyst-17330 · Answer

I also have this set in my `pants toml` file making it even more confusing ``` python interpreter constraints = CPython gt =3 10 ```

enough-analyst-54434 · Answer

I can t directly speak to the final problem Pants should be respecting the `interpreter constraints` you configured when trying to generate a lock for the `python default` resolve As to the Python 3 9 Pants itself needs Python to run This is true even if you only use Pants to work with Go or the JVM As such it has its own interpreter constraints of ` gt =3 7 lt 3 10` i e it runs on Python 3 7 8 9 So you see it picking a Python 3 9 interpreter to run on from your machine That should generally be of no concern to you except here it caused understandable confusion when downstream errors made you pause to read the logs more fully After picking the interpreter it needs Pants then generally picks the interpreter your code needs if you re using Pants to work with Python code like you are

enough-analyst-54434 · Answer

< ripe analyst 17330> do you have custom configuration for ` python bootstrap search path`

ripe-analyst-17330 · Answer

that makes sense for pants itself to use `3 9` thanks It does seem like pants eventually runs a different venv I ve tried both with and without the following with the same result ``` python bootstrap search path = lt PYENV gt ```

enough-analyst-54434 · Answer

What do you have configured for ` python lockfile generator` I m on a little firmer footing talking about the Pex locker This may be using the Poetry locker though under the covers

ripe-analyst-17330 · Answer

yes I m using poetry Currently all our projects have their own folders and individual poetry lock files I was just trying to get to the point I could use those and ran across this issue

ripe-analyst-17330 · Answer

it does seem to work if I explicitly set `lockfile generator = pex `

ripe-analyst-17330 · Answer

after looking at the code I determined this was the cause <https github com pantsbuild pants blob 2d2e287193e5d2ee68058c0b016a184d4f02489d src python pants backend python subsystems poetry py L30> Setting this explicitly in the poetry config got me past it ``` poetry version = poetry==1 1 13 interpreter constraints = CPython gt =3 10 lt 4 ``` Is there a per project poetry config that can be set for this as it seems it would break a monorepo with different version constraints per project We have this issue in our repo I just don t know if its enough to cause problems here most are 3 9+ compatible

enough-analyst-54434 · Answer

Ah yeah and that s used here with no regard to the `req` ICs Just above though using the Pex locker the right thing is done So it looks like you ve found a bug for locking using the Poetry locker I ll check to see if that s filed known You might try switching to the Pex locker if you need per project support We re headed to that being the default anyhow and likely will strip support for Poetry being the locker since it simply can t handle some permutations of the inputs like custom find links repos

enough-analyst-54434 · Answer

Hrm I could be wrong Do you have ` python enable resolves` configured to `true` It may be useful to just dump the ` python `section of your `pants toml` to prevent a bit of back forth <https www pantsbuild org v2 11 docs reference python section enable resolves>

enough-analyst-54434 · Answer

< hundreds father 404> will certainly better be able to guide you through all this more efficiently I ve been working on the Pex support for generating and consuming locks and have a lot less familiarity with the Pants side of this

enough-analyst-54434 · Answer

Aha the custom ICs do get threaded through for Poetry into a generated `pyproject toml` here <https github com pantsbuild pants blob 2d2e287193e5d2ee68058c0b016a184d4f02489d src python pants backend python subsystems poetry py L68 L83> I think the problem is when The Poetry tool PEX is installed using different ICs than the ICs fed to it in `pyproject toml` IOW we use 1 global ~venv for Poetry and we then try to feed it many different generated `pyproject toml` each with custom ICs If the ICs conflict boom

enough-analyst-54434 · Answer

< hundreds father 404> if you can sanity check my guess immediately above that seems like an un filed bug

enough-analyst-54434 · Answer

Ok thanks < ripe analyst 17330> for digging into the code and fiddling with your `pants toml` config That was very helpful I m pretty sure my analysis is correct so I went ahead and filed Hopefully you can use the workaround you found or else switch to the pex lockfile generator in the meantime

ripe-analyst-17330 · Answer

I m glad you called that out my next step is how to do multiple resolves as we currently have poetry lock files on a per project basis internal code will basically all use `HEAD` but with 3rd party deps we sometimes use different versions in different projects But it sounds like this breaks the multiple resolves as well which I hadn t yet figured out

enough-analyst-54434 · Answer

Although you can mix Poetry projects with the Pex locker that is probably not a great idea since they could produce different locks That would then lead to ` ~ dev repo $ pants foo poetry project1 ` having different results than ` ~ dev repo poetry project1 $ poetry foo `

ripe-analyst-17330 · Answer

I could probably make a global lock work I would just need to figure out if there is any other work that needs to be done around the poetry config in ` tool poetry scripts ` but I m guessing we would just need per project `poetry build` commands Sorry I m just starting to look at pants so I m not too familiar with the build steps yet I was building CI matrixes with scripts when this was suggested

hundreds-father-404 · Answer

Hey < ripe analyst 17330> my apologies for not getting back to you earlier I was moving from AZ to Mexico City when this happened Where are you at now with things Anything we can help with