https://pantsbuild.org/ logo
Join Slack
Powered by
Hello, I am new to pants. I have been having tro...
# general
r
Hello, I am new to pants. I have been having trouble understanding how pants determines the python version for different goals. For context I am running locally on a Mac with M2 chip and using pyenv. Any thoughts on possible configuration issues would be appreciated. Thanks. I can successfully run 
pants generate-lockfiles
which generates lockfiles using the expected python version When I try to run 
pants lint ::
I get the error
Copy code
11:09:35.84 [ERROR] 1 Exception encountered:

Engine traceback:
  in `lint` goal

ValueError: Couldn't determine a compatible Interpreter Constraint from ('3.11', '3.12')
Finally when I try to generate a pex file for one of the targets - the issue seems to be that the packages exist for python 3.11 but not 3.9 ( which I "think" is the version that is being used to build the pex file )
Copy code
pex: Building pex :: Resolving distributions for requirements: apache-flink<2.0.0,>=1.20.0 setuptools<75.0.0,>=74.1.2 :: Resolving requirements from lock file 3rdparty/python/flink-wordcnt.lock :: Categorizing 84 downloaded artifacts
There were 4 errors downloading required artifacts:
1. apache-flink 1.20 from <https://files.pythonhosted.org/packages/37/85/830c518b8a80095c371d3af5143c90d51a21958f1d0c4095ce799445bcfe/apache-flink-1.20.0.tar.gz>
    note: This error originates from a subprocess, and is likely not a problem with pip.
2. numpy 1.24.4 from <https://files.pythonhosted.org/packages/a4/9b/027bec52c633f6556dba6b722d9a0befb40498b9ceddd29cbe67a45a127c/numpy-1.24.4.tar.gz>
    note: This error originates from a subprocess, and is likely not a problem with pip.
3. pandas 2.1 from <https://files.pythonhosted.org/packages/6f/31/a4a8e7367856d9584d0332793edfe631182a9cca885f12dbe2dd77c10c4a/pandas-2.1.0.tar.gz>
    hint: See above for details.
4. pemja 0.4.1 from <https://files.pythonhosted.org/packages/90/2b/2ca14e70edd1c8bb38fcef6c6902d08702fa9a3a1e60da81b1a9cb22a200/pemja-0.4.1.tar.gz>
    note: This error originates from a subprocess, and is likely not a problem with pip.
More information my 
pants.toml
Copy code
[GLOBAL]
pants_version = '2.21.0'

level = 'debug'

backend_packages = [
  'pants.backend.build_files.fmt.black',
  'pants.backend.python',
  'pants.backend.python.lint.black',
  'pants.backend.python.lint.flake8',
  'pants.backend.python.lint.isort',
  'pants.backend.python.typecheck.mypy',
  'pants.backend.docker',
  'pants.backend.shell',
  'pants.backend.experimental.javascript',
  "pants.backend.project_info",
  "pants.backend.awslambda.python",
  "pants.backend.python.mixed_interpreter_constraints",
  "pants.backend.python.providers.experimental.pyenv",
]

[python-bootstrap]
search_path = [
    "<PYENV>",
    "/Users/brooks-jolly/.pyenv/versions",
]

[tailor]
ignore_paths = [
    "python/src/awslambdas/deploy/**",
]

[python]
interpreter_constraints = ['>=3.11,<3.13']
interpreter_versions_universe = ["3.11", "3.12"]
enable_resolves = true
default_resolve = "python-default"

[generate-lockfiles]
diff = true

[source]
root_patterns = [
  "python/src/awslambdas",
  "python/src/common",
  "python/src/services",
  "python/src/common",
  "python/src/acloudguru",
]

[test]
output = "all"

[docker]
build_verbose = false
env_vars = ["DOCKER_CONFIG=${HOME}/.docker"]
tools = [
  "docker-credential-desktop"
]

[python.resolves]
flink-wordcnt = "3rdparty/python/flink-wordcnt.lock"
flink-movie = "3rdparty/python/flink-movie.lock"
python-default = "3rdparty/python/default.lock"
My BUILD target for 
pex
Copy code
python_sources(
    name="flinkwordcount",
    interpreter_constraints=[">=3.11,<3.13"],
    resolve="flink-wordcnt",
)

python_tests(
    name="tests",
    sources=["tests/test_*.py", "tests/*_test.py", "tests/tests.py"],
    resolve="flink-wordcnt",
    interpreter_constraints=[">=3.11,<3.13"],
)

poetry_requirements(
    name="flink-wordcnt",
    source="pyproject.toml",
    resolve="flink-wordcnt",
)

pex_binary(
    name="bin",
    entry_point="flow-stream-processing-poc.flink-wordcnt-example.datastream_wordcount",
    execution_mode="venv",
    dependencies=[":flinkwordcount", ":flink-wordcnt", "//:zscaler"],
    interpreter_constraints=[">=3.11,<3.13"],
    resolve="flink-wordcnt",
)
w
Pants itself using a bundled python 3.9, but try removing the interpreter_universe, and using only the global (pants.toml) interpreter_constraints (removing the others on a per-target basis) That way, everything should use the same constraints, and it might be easier to debug
r
Thanks. I was able to "solve" the pex build by removing all versions of python in pyenv ( 3.10 and 3.12 ) so I think the pex build was trying to use 3.10 for some reason. The lint still fails but now lists a larger number of possible versions.
Copy code
Engine traceback:
  in `lint` goal

ValueError: Couldn't determine a compatible Interpreter Constraint from ('2.7', '3.5', '3.6', '3.7', '3.8', '3.9', '3.10', '3.11', '3.12')
Interestingly if I specify the lint target it works so
Copy code
pants lint python/src/services/flow-stream-processing-poc/flink-wordcnt-example/:
for example executes successfully.
w
Generally that means there is -some- target or dependency that falls over, so you'd eventually run into it. Maybe try 
pants peek ::
as sometime that surfaces errors
I haven't used the pyenv provider, so I'm not the best to speak on that - but generally it feels like a dependency might not be satisfied by the interpreter constraints you're offering - or, not all the targets are using the same interpreter_constraints for some reason
r
That sounds like the correct tract. I was attempting to have different projects use different python versions and likely misconfigured somewhere. Is it valid to have nested BUILD files? I created a BUILD file in the root directory to define some file and other static resources. I see that it is based on the example project.
@wide-midnight-78598 thank you for your suggestions - using peek I was able to clean up some of my targets fix the linting issues I had.
👍 1
3 Views
Open in Slack
PreviousNext
Powered by