Looking for feedback before I open a PR Hello Our team uses Pants #general

Looking for feedback before I open a PR: Hello! O...

rapid-journalist-97001

10/13/2025, 1:00 PM

Looking for feedback before I open a PR: Hello! Our team uses pytorch and we’ve been having issues related to [#18965]. As of pex 2.56.0 it is possible to create a universal lock with two locked resolves (one for macos, one for linux) by adding an appropriately scoped index. Example:

Copy code

pex3 lock create \
    --style universal \
    --target-system linux \
    --target-system mac \
    --elide-unused-requires-dist \
    --interpreter-constraint "CPython==3.13.*" \
    --index pytorch=<https://download.pytorch.org/whl/cpu> \
    --source "pytorch=torch; sys_platform != 'darwin'" \
    --indent 2 \
    -o lock.json \
    torch

It looked like a simple change so I forked pants and experimented with adding the options to specify

index

and

source

when running

generate-lockfiles

. It seems to work smoothly. Is there anything you think I should know before I have a go at adding tests and opening a PR?

happy-kitchen-89482

10/14/2025, 1:35 AM

Thanks for stepping up on this! A PR would be great! So in your experiment are these options on

generate-lockfiles

subsystem, so they then apply to all invocations of that goal? We'd probably want them to be properties of individual resolves, which we currently implement rather clunkily as

_resolves_to_*

`DictOption`s in src/python/pants/backend/python/subsystems/setup.py

👍 1

happy-kitchen-89482

10/14/2025, 1:35 AM

Which is not that much more complicated

happy-kitchen-89482

10/14/2025, 1:35 AM

Let us know here if that gets difficult

average-breakfast-91545

10/14/2025, 8:25 AM

Have you any advice on how best to test this? The current lockfile/export tests are using a single well-known dependency, which I imagine is small and a leaf node. Are we able to easily test the behaviour of this without actually downloading a bunch of torch variants? I guess we could run a rule and then look at the requests it generates.

rapid-journalist-97001

10/14/2025, 11:55 AM

Thank you Benjy! Yes that makes sense and it was a straightforward change. I actually realised we technically only really need to add a

_resolves_to_sources

to get this to work, as we can already set indexes (for all resolves) through

PythonRepos

. It might be nice to also have the option of specifying resolve-specific indexes (and it's hardly any more work), though it is redundant, what do you think? And yes - any insight on Bob's question would be handy

❤️ 1

rapid-journalist-97001

10/15/2025, 10:58 AM

There may be a workaround regarding the tests...

cowsay

is published in both PyPI and TestPyPI, so we could use the latter as a second index for testing. (ansicolors
, the package used in other lockfile tests, is not on TestPyPI this was a half lie, there is an

ansiColor

but it has a different maintainer)

👌 1

average-breakfast-91545

10/15/2025, 10:59 AM

it'd be ideal to find something cross-platform, though. Almost tempting to publish a python package just for this purpose.

rapid-journalist-97001

10/18/2025, 5:34 PM

PR: https://github.com/pantsbuild/pants/pull/22760 Had some issues testing (described in comments on the PR) so keen for feedback 🙂

👀 2

rapid-journalist-97001

10/21/2025, 6:07 PM

Hey @happy-kitchen-89482! Just saw your comment on the PR: release notes added and formatting fixed, but fyi I can only run the

lint check fmt fix

goals on certain targets (like

src/python/pants/core/goals::

and

src/python/pants/backend/python::

), not on

::

. If I try the latter I get

Copy code

18:54:27.38 [INFO] Completed: Scheduling: Ensure download of JDK --jvm=temurin:1.11.
18:54:27.38 [ERROR] 1 Exception encountered:
...
  File "/Users/chiara/dev/pants/src/python/pants/jvm/jdk_rules.py", line 250, in prepare_jdk_environment
    raise ValueError(
ValueError: Failed to locate Java for JDK `temurin:1.11`:

Which is weird cause I do have temurin 1.11 installed:

Copy code

$ cs java --installed 
adoptium:1.11.0.28 installed at /Users/chiara/Library/Caches/Coursier/arc/https/github.com/adoptium/temurin11-binaries/releases/download/jdk-11.0.28%252B6/OpenJDK11U-jdk_aarch64_mac_hotspot_11.0.28_6.tar.gz/jdk-11.0.28+6/Contents/Home
adoptium:1.17.0.16 installed at /Users/chiara/Library/Caches/Coursier/arc/https/github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.16%252B8/OpenJDK17U-jdk_aarch64_mac_hotspot_17.0.16_8.tar.gz/jdk-17.0.16+8/Contents/Home
temurin:1.11.0.28 installed at /Users/chiara/Library/Caches/Coursier/arc/https/github.com/adoptium/temurin11-binaries/releases/download/jdk-11.0.28%252B6/OpenJDK11U-jdk_aarch64_mac_hotspot_11.0.28_6.tar.gz/jdk-11.0.28+6/Contents/Home
temurin:1.17.0.16 installed at /Users/chiara/Library/Caches/Coursier/arc/https/github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.16%252B8/OpenJDK17U-jdk_aarch64_mac_hotspot_17.0.16_8.tar.gz/jdk-17.0.16+8/Contents/Home

I don't think it's a problem for this particular PR as I can run the goals for the relevant targets, but thought you should know as I saw there had been a somewhat similar issue that was later resolved.

happy-kitchen-89482

10/22/2025, 4:51 PM

Ah yes, that will run linters on the entire repo, including JVM code. But you can do

pants --changed-since=main fmt fix lint check

to act just on files that have changed compared to

main

7 Views

Open in Slack

Previous Next