Hi there! I'm having some trouble getting multiple...
# general
m
Hi there! I'm having some trouble getting multiple lockfiles tied to different interpreters working:
Copy code
āÆ pants package --docker-tools=docker-credential-osxkeychain src/apps/running_man:_binary-deps
09:30:04.98 [INFO] Initializing Nailgun pool for 24 processes...
09:30:07.12 [INFO] Initializing Nailgun pool for 24 processes...
ā ‚ 0.42s Find interpreter for constraints: CPython<3.11,>=3.10.16
09:30:11.21 [INFO] Completed: Building 31 requirements for src.apps.running_man/_binary-deps.pex from the 3rdparty/python/python-torch.lock resolve: *** (580 characters truncated)
09:30:11.21 [ERROR] 1 Exception encountered:

Engine traceback:
  in `package` goal

ProcessExecutionFailure: Process 'Building 31 requirements for src.apps.running_man/_binary-deps.pex from the 3rdparty/python/python-torch.lock resolve: ***' failed with exit code 1.
stdout:

stderr:
Failed to resolve compatible artifacts from lock 3rdparty/python/python-torch.lock for 2 targets:
1. cp312-cp312-manylinux_2_36_aarch64:
    Failed to resolve all requirements for complete platform cp312-cp312-manylinux_2_36_aarch64 from 3rdparty/python/python-torch.lock:
In my pants.toml I added:
Copy code
[python]
interpreter_constraints = [">=3.12.7,<3.13"]
enable_resolves = true

[python.resolves]
python-default = "3rdparty/python/default.lock"
python-torch = "3rdparty/python/python-torch.lock"

[python.resolves_to_constraints_file]
python-default = "3rdparty/python/constraints.txt"

[python.resolves_to_interpreter_constraints]
python-default = [">=3.12.7,<3.13"]
python-torch = [">=3.10.16,<3.11"]
In my target I've added the following:
Copy code
python_sources(
    name="test_py",
    resolve="python-torch",
    interpreter_constraints=[">=3.10.16,<3.11"]
)

dependencies = [
    "3rdparty/python:reqs-torch#python-multipart",
    ":test_py"
]

pex_binary(
    name="dev",
    resolve="python-torch",
    restartable=True,
    entry_point="main.py",
    execution_mode="venv",
    layout="packed",
    dependencies=dependencies,
    interpreter_constraints=[">=3.10.16,<3.11"]
)

pex_binary(
    name="_binary-deps",
    resolve="python-torch",
    complete_platforms=[
        "3rdparty/platforms:linux_x86_64_py312",
        "3rdparty/platforms:linux_aarch64_py312",
    ],
    entry_point="main.py",
    layout="packed",
    include_sources=False,
    include_tools=True,
    dependencies=dependencies,
    interpreter_constraints=[">=3.10.16,<3.11"]
)

# similar logic for building a docker container
But it still seems like it's trying to package with python 3.12 which is the default resolve/resolve constraint. Also is it really the case that for each
BUILD
I need to add something like:
Copy code
python_sources(
    resolve="python-default",
    interpreter_constraints=[">=3.12.7,<3.13"]
)

python_sources(
    name="src_torch",
    resolve="python-torch",
    interpreter_constraints=[">=3.10.16,<3.11"]
)
r
The story looks similar to mine, but in my case I just build the pex binaries for the default resolve...
m
I was reading through your thread šŸ™‚ I have it most of the way but its a different error message
r
in my eyes, your configuration has all the required information, like the explicit
resolve
field in the
pex_binary
target. šŸ¤”
maybe.. just regenerate the lockfiles if you've changed your config?
m
I'll try šŸ™‚ I think they're fresh though
I figured it out
it was the
complete_platforms
Does it really take so much boiler plate to have pants work with two different python interpreters depending on the lockfile?
c
Regrading the boilerplate, what I have done is a combination of defaults and macros. This looks something like "all py sources use both CPython versions" and then a shorthand
pex_binary_311
macro.
m
@curved-manchester-66006 how would that work? Do you have an example?
c
Something like:
Copy code
-def foo_bookworm_py310_pex(**kwargs):
-    _foo_complete_platform_pex(
-        shebang="/usr/bin/env python3.10",
-        complete_platforms=("3rdparty/platforms/docker/debian/bookworm:python_310",),
-        interpreter_constraints=["CPython==3.10.*"],
-        **kwargs,
-    )
-