wave i m having a very weird issues with parametrization no Pants #general

:wave: - i'm having a very weird issues with param...

some-restaurant-1855

07/16/2024, 12:36 AM

👋 - i'm having a very weird issues with parametrization not being respected in only a

pex_binary

target. at the highest level, i'm exploring solutions for dealing with

torch

for multiple platforms + gpu or non-gpu setups (yet again) . i've defined a few grouped parametrization targets for the three resolves/lockfiles i want to respect

python_linux, python_linux_cuda, python_darwin

in my repo defaults for python sources /tests, have an explicit torch

python_requirement

(

:torch_manual

) with the same parametrization, and an override on

torch

from my main

3rdparty/python/BUILD

python_requirements

generator (which is also parametrized the same way). basically, the

pex_binary

target i use in the project, which requires torch in the underlying

python_sources

, identifies the

3rdparty:torch@parametrize=python_linux_cuda

requirement as a dep even though it shouldn't be there and my

python_sources

target doesn't include it as one (via

peek

and

dependencies/dependents

querying. i have a [gist](https://gist.github.com/binaryaaron/cf5638fcdde79b2537a6716ebd1e529f) with the pertinent files - the 3rdparty build file, project build, pants.toml, and root build file, and outputs showing the issues. anyone familiar with

parametrize

able to look?

curved-television-6568

07/16/2024, 4:07 PM

I don't recall if pants does any adjustments to explicitly provided dependencies, which would explain why you need to provide them per parametrization as you have in the "BUILD file with explicitly parametrized deps".

some-restaurant-1855

07/16/2024, 5:34 PM

interesting. more that in the case of a

pex_binary

- how would you let pants infer the required deps without specifying them explicitly?

some-restaurant-1855

07/16/2024, 5:35 PM

feels like a weird case for a macro / plugin? sorta standardizing the

parametrize

groups / deps in cases like this

some-restaurant-1855

07/16/2024, 5:49 PM

i saw a few convos related to past work on `parametrize`; sorta thinking out loud here that i'd expect that a

parametrize=X

would 'match' other

parametrize=X

targets. in my head, i sorta assumed the group name would "match" correctly - is the

pex_binary

requirements target something that could support parametrization like i'm sorta assuming it could? as a bit of an aside, would it be nuts to think about specifying the parametrize groups more like the following? user defines the 'common' parametrize groups somewhere, e.g. a build file like:

Copy code

parametrize_group(name="A", <options> )
parametrize_group(name="B", <options> )
parametrize_group(name="C", <options> )

parametrize_groups(dependencies = [":A", ":B", ":C"]

and when you add a group to a target, any group that matches one of the

A, B, C

groups will use the 'global' group name more directly with args that overwrite what you might have defined in the hypothetical

parametrize_group

target? e.g., still use

Copy code

pex_binary(..., 
**parametrize("A", **kargs), **parametrize("B", **kwargs)
)

and if the arg

matches one of the above groups it'll ... just work? 😅 i imagine that'd be a reasonable amount of work to do, just curious

4 Views

Open in Slack

Previous Next