https://pantsbuild.org/ logo
Join Slack
Powered by
If I wanted to include all source files within a m...
# general
g
If I wanted to include all source files within a module in a pex binary, is there a better way other than just using 
pants list path/to/module::
and adding them as explicit deps to the pex_binary target? I don't know what people do here and I'm curious.
b
Assuming that module has a BUILD file (which I’m assuming it does based on working with 
pants list
) - you should be able to add the 
python_sources
target in that file as a dependency for the 
pex_binary
target?
Ex: here’s what this looks like for me
Copy code
pex_binary(
    name="binary-srcs",
    entry_point="main.py",
    dependencies=[
        ":lib",
        "my/nested/dir/scripts:lib",
    ],
    ...
)
g
Yeah, I just have a crap ton
b
Can you lean on dependency inference to automatically pull in other files you need? For me, the two directories I posted above are the only files I add dependencies for explicitly, and that’s because those are the primary entry points that I might need - everything else from the repo is automatically pulled in via dependency inference
h
The 
pex_binary
should only need an explicit dep on the entry point, everything else should come along via inferred deps
g
in theory that is true, but in reality that doesn't always work. In this context whoever authored this code did some weird unusual things.
c
I'd lean towards adding explict dependencies for those weird files, keep the explicitness close to where it is needed, so if/when that is fixed, it's easy to remove and let dep inference do its work. But if this is in a gazillion places, I can see it's not a very viable option..
h
Probably the way to do this is at the source file level. That is, if 
foo.py
depends on 
bar.py
through some "weird unusual thing" that isn't amenable to dep inference, then make that dep explicit via an override on the relevant 
python_sources()
g
@happy-kitchen-89482 can you elaborate? I looked the docs linked and I'm not sure how I would use overrides to add a dependency.
h
Something like
Copy code
python_sources(
    overrides={
        "foo.py": {"dependencies": ["path/to/bar.py"]},
    }
)
That says "add this dep to just foo.py, not the entire python_sources()"
Pants needs an accurate view of your deps. Usually dep inference gets it there. Sometimes you need to manually tweak the deps, as in your case. So this is the most precise way of doing so. Rather than trying to glob a bunch of dangling deps into the pex_binary itself, let Pants follow deps, as it likes to do, but just augment those deps as needed. Does that make sense?
g
ah, thank you! This is very helpful.
h
What is the weirdness that prevents dep inference from working? Some sort of runtime classloading by name?
If this is ubiquitous you can also write a custom dep inference plugin to adapt this
g
It's a one off in our monorepo. I honestly don't know how it was working before pants 🤣 It is using uvicorn and the module has a custom a entrypoint.py file that loads a Settings class that has an entry point in a string (e.g. 
app: str = "app:app"
) The weird thing is should actually be 
app: str = "<http://module_name.app:app|module_name.app:app>"
I have no idea how it was working before I changed it. As I was typing this out I realized that I didn't try removing explicit deps after I fixed the above string and enabled 
string_imports
and 
string_imports_min_dots = 2
Copy code
[python-infer]
string_imports = true
string_imports_min_dots = 2
I bet it will actually work properly now with inference.
4 Views
Open in Slack
PreviousNext
Powered by