https://pantsbuild.org/ logo
#general
Title
# general
c

cold-vr-15232

04/04/2023, 1:34 PM
Hi! Regarding the new python.providers.experimental.pyenv backend, when I enable it, it installs 3.7 despite having
interpreter_constraints=[">=3.8,<3.9"]
. There is no docs for this, so maybe there is some configuration that needs to be done?
1
s

sparse-lifeguard-95737

04/04/2023, 1:39 PM
what version of pants are you running?
I’m also seeing odd behavior around interpreter selection (https://github.com/pantsbuild/pants/issues/18662) - wondering if it’s the same root issue
b

bitter-ability-32190

04/04/2023, 2:13 PM
3.7 would be unexpected. It should use the lowest compatible version. Mind filing an issue? Small reproduction repo would be great
@sparse-lifeguard-95737 that's an unrelated issue. There were several changes at the same time, but your issue doesn't relate to the experimental pyenv change
👍 1
Also @cold-vr-15232, glad you're trying out the new feature!
c

cold-vr-15232

04/04/2023, 2:14 PM
I am using 2.16.0rc0. I will try to reproduce in a sample repo.
❤️ 1
Ok, I think I got the culprits. I can reproduce in any repo, when using the config below.
Copy code
[GLOBAL]
pants_version = "2.16.0rc0"
backend_packages = [
  # Any of the following three backends install python 3.7
  "pants.backend.build_files.fmt.black",
  "pants.backend.python.lint.docformatter",
  "pants.backend.python.lint.isort",
  #
  "pants.backend.python",
  "pants.backend.experimental.python",
  "pants.backend.python.providers.experimental.pyenv",
]

[python]
interpreter_constraints=[">=3.8,<3.9"]
Also, "pants.backend.python.typecheck.mypy" will use 3.8 but fails because it can't find module _sqlite3.
Copy code
File "/home/vdumitre/.cache/pants/named_caches/pyenv/versions/3.8.16/lib/python3.8/sqlite3/dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ModuleNotFoundError: No module named '_sqlite3'
b

bitter-ability-32190

04/05/2023, 2:58 PM
Oh, I'm guessing that's because the ICs you set are for your code. But they do t necessarily need to match the ICs for the tools being run
c

cold-vr-15232

04/05/2023, 4:14 PM
I don't think it is reasonable to let the user add resolves_to_interpreter_constraints for each tool, if they want to use this backend. In a normal setup, one wouldn't have python 3.7 (for example) installed, so everything works fine. But the pyenv backend will install whatever, and some tools probably want to use the same version as the code. Maybe we could have a config like global_interpreter_constraints_for_everything? 🙂
b

bitter-ability-32190

04/05/2023, 4:16 PM
Oh yeah sorry, I should've mentioned I don't think this is ideal Was just musing as to why the current code might behave this way
👍🏽 1
b

broad-processor-92400

04/05/2023, 7:56 PM
This might be https://github.com/pantsbuild/pants/issues/15358#issuecomment-1491262069 which others have encountered too
b

bitter-ability-32190

04/05/2023, 8:55 PM
Oh good find
b

brash-family-72057

04/06/2023, 11:41 PM
Just in case you need more info and an extra repro, I just hit what seems to be a similar issue, though, in my case, it seems to select the correct python version during packaging. My toml file:
Copy code
[GLOBAL]
pants_version = "2.16.0rc0"
backend_packages = [
  ...
  "pants.backend.python.providers.experimental.pyenv",
]
...

[python]
interpreter_constraints = ["CPython>=3.10,<3.11"]
When running the package command, I see that it installs python 3.10:
Copy code
15:58:10.20 [INFO] Completed: Choose specific version for Python 3.10
16:00:14.27 [INFO] Completed: Install Python 3.10
I didn’t have python 3.10 installed on my machine (which is why I chose it for this test):
Copy code
❯ pyenv versions
  system
  3.9.10
* 3.11.2 (set by /Users/vzakharov/mdevxp/.python-version)
When I run the resulting pex, I get this error:
Copy code
❯ ./dist/src.python.mdevxp.health_score/health_score_processor.pex --help
pyenv: python3.9: command not found

The `python3.9' command exists in these Python versions:
  3.9.10

Note: See 'pyenv help global' for tips on allowing both
      python2 and python3 to be found.
looking inside the pex, I see that
__main__.py
starts with
Copy code
#!/usr/bin/env python3.9
Let me know if you’d like more info on this.
b

bitter-ability-32190

04/07/2023, 12:09 AM
Yeah that checks out with my theory
b

brash-family-72057

04/07/2023, 5:30 AM
@bitter-ability-32190 Since several issues were discussed in this thread, please clarify what your theory is. Thank you!
Also, I don't think using the pyenv provider will work seamlessly with trying to run a packaged PEX binary. The pyenv provider is for pants-run stuff. Once you try to run code yourself, pants can't help you. That being said, you can adjust the shebang in the PEX to be something compatible with the host system. That's a field of the target
b

brash-family-72057

04/07/2023, 7:45 PM
I see. I must have misunderstood the feature. I was under the impression that it was able to package the python interpreter required for running the pex into the pex itself, so you could ship a fully portable binary. Is there a plan for supporting that? (Or maybe already something out there).
b

bitter-ability-32190

04/07/2023, 7:49 PM
Yeah existing solutions are pyoxidizer which there's experimental support for. And scie which support is coming
👍 2
🙏 1
4 Views