Scenario: I'm building a lambda function that depe...
# general
Scenario: I'm building a lambda function that depends on aws-lambda-powertools and boto3. Accordingly, I need to add an override to my python_dependencies to pull in jmespath and pydantic, which are undeclared transitive deps. When I deploy my lambda function, I want to elide those deps because they'll ship as part of a lambda layer. My lambda func is a pex_binary:
Copy code
pex_binary (
I'd have expected that jmespath and pydantic would be excluded, because they're transitive dependencies of two excluded deps, but by default they're pulled into the pex, so I have to exclude those as well. Is that intentional behaviour?
I think it's expected behavior, but only if you know Pants is 3rd party dumb. It only tracks 1st party sources transitively and its knowledge ends at direct 3rd party dependencies. It offloads handling of those (3rd party resolution, locking, subsetting) to tools. The only sign you might see of this if not in the know, besides the sign you ran into here, is running
pants dependencies --transitive <target>
I think this is definitely buggy though. Pants really ought to be able to handle transitivity 1st class including in 3rd party deps. At the least, doc about what
can and can't do should be improved.
That's surprising to me. What does
./pants paths --from=path/to/pex:address --to=path/to/reqs:jmespath#resolve
say (you'll need to fill in your own values)
Copy code
$ pants paths --paths-from=src/dz/recsys/anomaly_flagger:main --paths-to=3rdparty/python:jmespath
12:11:12.01 [ERROR] 1 Exception encountered:

Engine traceback:
  in `paths` goal

KeyError: Address(3rdparty/python#aws-lambda-powertools)
Copy code
❯ pants peek 3rdparty/python#aws-lambda-powertools
    "address": "3rdparty/python#aws-lambda-powertools",
    "target_type": "python_requirement",
    "dependencies": [
    "dependencies_raw": [
    "description": null,
    "modules": null,
    "requirements": [
    "resolve": null,
    "tags": null,
    "type_stub_modules": null
Does your code depends on AWS lambda power tools? (Also file a bug if you can about the key error, you can use --print-traceback to see the calling code) (Also you don't have to use --path prefix after the
goal, after a goal the prefix is implied)
The code does, yes, but I want to exclude it from the package because it'll be present in a lambda layer.
Oh yeah forgot that part of OP 🙃
I'll see if I can repro the bug. The stacktrace isn't terribly informative
Copy code
13:07:19.35 [ERROR] 1 Exception encountered:

Engine traceback:
  in select
  in pants.backend.project_info.paths.paths
    `paths` goal

Traceback (most recent call last):
  File "/home/bob/.cache/nce/d036c571b74670fb7cccdb2ee973b51625c6419b29dd5e2e5390d6267b873904/bindings/venvs/2.15.0/lib/python3.9/site-packages/pants/engine/internals/", line 593, in native_engine_generator_send
    res = func.send(arg)
  File "/home/bob/.cache/nce/d036c571b74670fb7cccdb2ee973b51625c6419b29dd5e2e5390d6267b873904/bindings/venvs/2.15.0/lib/python3.9/site-packages/pants/backend/project_info/", line 133, in paths
    for path in find_paths_breadth_first(adjacency_lists, root.address, destination.address):
  File "/home/bob/.cache/nce/d036c571b74670fb7cccdb2ee973b51625c6419b29dd5e2e5390d6267b873904/bindings/venvs/2.15.0/lib/python3.9/site-packages/pants/backend/project_info/", line 73, in find_paths_breadth_first
    for dep in adjacency_lists[target]:
KeyError: Address(3rdparty/python#aws-lambda-powertools)
Ok yeah it's either getting the dep from somewhere else (but we can't see that because of a pants bug) Or the transitive exclude doesnt exclude its deps. I'm not sure which way of handling this would be more intuitive 🫤
Ahh I think that's enough info for the bug. I bet it isn't handling the exclude correctly 😅
Okies, I'll open a bug 🙂
Maybe we maybe we need another exclusion, lol
I genuinely almost tried that, so it's at least intuitive
Hahahaha excellent
Soon enough I think we're trending towards "tagged dependencies" so we won't be stuck with string mucking for modifiers