Can some help me find the error in the following m...
# general
r
Can some help me find the error in the following macro (see threaded message). The
pex
binary DOES NOT include
langgraph.json
.
๐Ÿงต 1
Copy code
def langgraph_app(name, **kwargs):
  deps = kwargs.pop("dependencies", [])
  graphs = kwargs.pop("graphs")
  assert graphs is not None, "graphs is a required parameter"

  langgraph_json_config = f"""
{{
  "graphs": {graphs},
  "python_version": "3.12",
  "dependencies": ["."]
}}
"""
  
  # Create langgraph.json file.
  shell_command(
    name="echo_langgraph_json",
    command=f"echo {langgraph_json_config} > langgraph.json",
    tools=["echo"],
    output_files=["langgraph.json"],
  )
  
  # Bundle it as a resource for pex_binary.
  resources(
    name="langgraph_json",
    sources=["langgraph.json"],
    dependencies=[":echo_langgraph_json"],
  )

  pex_binary(
    name=name,
    entry_point="registry.py",
    dependencies=[":langgraph_json"] + deps,
    **kwargs,
  )
h
As an experiment - does this work if you have a hardcoded langgraph.json on disk?
So that the
resources
target points to a real file, I mean
r
Yes, it works if I have the langgraph in disk.
I solved it. I had to use
experimentas_wrapped_as_resource
. To be honest, that's rather surprising, and not intuitive. Do you know who I can talk to, maybe I can see if I can help make it part of
resources
directly.
g
So
experimental_wrap_as_resources
instead of
resources
? Or did you need both?
r
Just
experimental_wrap_as_resources
. I think a better API should not use it, but rather directly
resources
.
g
Ack! I have a few ideas for what goes wrong... There's a few different concepts of Pants that get in the way of using resources directly. I'd have expected at least a warning that your
resources
did not match any files.
Copy code
17:28:45.40 [WARN] Unmatched glob from //:langgraph_json's `sources` field: "langgraph.json"

Do the file(s) exist? If so, check if the file(s) are in your `.gitignore` or the global `pants_ignore` option, which may result in Pants not being able to see the file(s) even though they exist on disk. Refer to <https://www.pantsbuild.org/v2.18/docs/troubleshooting#pants-cannot-find-a-file-in-your-project>.
Optimally we'd not need any intermediate target but that gives
Copy code
17:36:02.30 [WARN] The target //:foobar (`pex_binary`) transitively depends on the below `files` targets, but Pants will not include them in the built package. Filesystem APIs like `open()` may be not able to load files within the binary itself; instead, they read from the current working directory.
I'm not sure why we detect but don't include file targets in pex's... it does seem actively unhelpful. @rough-engineer-58925 can you file an issue on GitHub?
r
Yes of course. Thanks a lot for looking into it as well.
๐Ÿ™‡ 1
@gorgeous-winter-99296
I'm not sure why we detect but don't include file targets in pex's
I think the answer is written here: https://www.pantsbuild.org/stable/docs/using-pants/assets-and-archives
However, i am would want to avoid the use of
experimental_wrap_as_resources
. Is there someone who I can talk to so that I can help solve this issue?
g
That someone is any of us maintainers. The issue is the first step of that. To me it's not clear how this should be solved, so at least from my POV a discussion has to happen to understand what can/should change. As a brief summary, sources in Pants are "typed", so a
shell_command
outputs a
file
source. The
experimental_wrap_as_*
helpers convert these sources to other types. These types are hard-coded inside Pants (i.e., a specific target has a specific source type), and different rules in Pants look for these specific types of files to do their work. So changing the types of sources at runtime has to happen at the target level (
experimental...
), and changing the hardcoded source type of an existing target type might introduce issues for those expecting it to be the other type.
If you want to look at contributing right away, see https://github.com/pantsbuild/pants/pull/17877 and https://github.com/pantsbuild/pants/issues/17926 for some relevant discussions!
r
Thanks a lot for the pointers ๐Ÿ™ I also understand now why
resources
don't work out of the box.