Hi everyone! :) I ran into an issue with lockfile generation and wanted to check if there are any better ways to handle it. I'd love to hear your thoughts! Case:

requirements_pyarrow.txt

with a single entry

awswrangler==3.12.1

results in the following error when running `pants generate-lockfiles --resolve=python-pyarrow`:

pip: ERROR: Cannot install awswrangler==3.12.1 because these package versions have conflicting dependencies.
pip: ERROR: ResolutionImpossible: for help visit <https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts>
pip:  
pip:  The conflict is caused by:
pip:      awswrangler 3.12.1 depends on pyarrow<18.0.0 and >=8.0.0; sys_platform == "darwin" and platform_machine == "x86_64"
pip:      awswrangler 3.12.1 depends on pyarrow<21.0.0 and >=18.0.0; sys_platform != "darwin" or platform_machine != "x86_64"

@great-leather-68110 with https://github.com/pex-tool/pex/pull/2940 you can do:

python -mpex.cli lock create \
    --pip-version latest-compatible \
    --style universal \
    --target-system linux \
    --target-system mac \
    --interpreter-constraint "==3.11.*" \
    awswrangler==3.12.1 \
    'pyarrow<18,>=8; sys_platform == "darwin" and platform_machine == "x86_64"' \
    'pyarrow<21,>=18; sys_platform != "darwin" or platform_machine != "x86_64"' \
    --indent 2 -o lock.json \
    --pip-log pip.log \
    --elide-unused-requires-dist

In other words, the universal lock splitting referenced in https://pantsbuild.slack.com/archives/C046T6T9U/p1760360455443789 will also work for any top-level requirements that split the lock (that currently works today except for the

platform_machine

marker needed in your case). Here I elevated the conflict error you received to top-level requirements to let Pex know to split the universal lock resolve. I assume you could do the same in Pants by declaring those two dependencies at top-level with a comment indicating why you're doing this even though you have no direct dependency on pyarrow (presumably). I expect to release Pex with that fix for universal lock splitting by end of week. All that said, Pants supporting non-universal locks would be great. I still think universal locks are generally a bad idea despite the ~wide ecosystem support in poetry, pdm, uv, ... etc.

@brief-scientist-13682 Thank you for looking into that! I’d also prefer having the option to directly specify the platforms in Pants. We’ll go with the approach from my first message then, as it’s a more straightforward way to define the expected platforms compared to using

target_system

or, say,

platform_machine

. So your fix isn’t blocking, and kudos for developing Pex! I’ll explore the options for passing platforms during Pants lockfile generation.

Sounds good. That said, the Pex support for universal resolve splitting is now available here: https://github.com/pex-tool/pex/releases/tag/v2.62.0

Thank you!