https://pantsbuild.org/ logo
#general
Title
# general
c

curved-manchester-66006

01/24/2023, 8:30 PM
Am I reading https://github.com/pantsbuild/pants/issues/16602 right that today if I want some of my code to depend on the 3rd party python package
foo[bar]
, other code to depend on
foo[bar,bax]
and yet other code to depend on
foo[qux]
, then those dependencies need to be in separate resolvers?
e

enough-analyst-54434

01/24/2023, 8:36 PM
I don't think you are. If the requirement you write down contains all extras needed by your repo - the union - then everything should just work,
Did you try?
So, write down
foo[bar,bax,qux]>=1.0
in your requirements.txt for example - no clue where you write your reqs.
If a new internal lib needs the
baz
extra, add
baz
to the extras list there, etc. The resolve / lock sub-setting Pants already does will ensure everyone gets only what they need and not the full union.
Josh's case fundamentally involved a dependency that should expose extras but does not. You are talking about the
foo
dependency which does expose extras.
c

curved-manchester-66006

01/24/2023, 8:48 PM
ah. In my case I was trying to have a
pex_binary
depend on
'3rdparty/py:reqs#foo[bar]
which gives "ValueError: The explicit dependency ... does not provide enough address parameters to identify which parametrization of the dependency target should be used." But in the more typical case where it is 1st party code doing the depending I can have `foo[bar,bax,qux]>=1.0`` and inferred dependencies on
foo
will only use the needed extras and not the full union?
e

enough-analyst-54434

01/24/2023, 8:56 PM
Hmm, I have no clue what that error message is about. Do you currently use 1 resolve or many?
c

curved-manchester-66006

01/24/2023, 9:00 PM
Several. With less made up example reqs of
feast[aws,snowflake]==0.25.0
and this
Copy code
pex_binary(
    name="feast-cli",
    entry_point="feast.cli:cli",
    dependencies=['3rdparty/py:reqs#feast[aws]'
                  ],
    resolve="feast_demo"
)
gives
Copy code
15:59:12.84 [ERROR] 1 Exception encountered:

Engine traceback:
  in `run` goal

ValueError: The explicit dependency `3rdparty/py:reqs#feast[aws]` of the target at `src/docker/feastui:feast-cli` does not provide enough address parameters to identify which parametrization of the dependency target should be used.
Target `3rdparty/py:reqs` can be addressed as:
  * 3rdparty/py:reqs
  * 3rdparty/py:reqs#feast
  * 3rdparty/py/requirements.txt:reqs
My interpretation of which is that the
[]
syntax is not understood by
dependencies=
e

enough-analyst-54434

01/24/2023, 9:00 PM
What version of Pants is in play here?
c

curved-manchester-66006

01/24/2023, 9:00 PM
"2.16.0.dev5"
e

enough-analyst-54434

01/24/2023, 9:01 PM
Ok, and feast_demo has the feast[x,y,z] requirement, right?
c

curved-manchester-66006

01/24/2023, 9:03 PM
Correct
Copy code
$ cat 3rdparty/py/requirements.txt 
# TODO: Clarify targets per extra?
feast[aws,snowflake]==0.25.0
e

enough-analyst-54434

01/24/2023, 9:04 PM
I think this is almost certainly a bug in Pants. It probably handles extras naively / not at all and neglects to expand such a requirement to its powerset before doing further calculations. That was a bug of omission in Pex itself a long time ago.
If you don't mind filing / have a smallish example you can provide / point at, that would be great.
c

curved-television-6568

01/24/2023, 9:34 PM
+1 for John’s conclusion, I don’t think Pants treats extras beyond the requirements.txt and lockfile. (i.e. you can’t name the extras you need in your dependency)
c

curved-manchester-66006

01/25/2023, 9:30 PM
👍 1
9 Views