https://pantsbuild.org/ logo
Join Slack
Powered by
Hi all, I have a question regarding the dependency...
# general
p
Hi all, I have a question regarding the dependency inference for 
pex_binary
. I have a directory structure that looks like so:
Copy code
app
  BUILD
  __init__.py
  script.py
__init__.py
contains the routes for a fastapi server. Now whenever I try to run a pex_binary with 
script.py
as an entrypoint (but no dependencies on the 
__init__.py
module) it also runs the code from the init file. Excluding the 
python_sources
that reference the init module does not change anything. So my question would be if there is some way to limit the source files for a pex binary.
r
How does your BUILD file look like? I mean the content of the file.
p
Copy code
python_sources(
    name = "lib",
)

pex_binary(
    name = "bin",
    args = [
        "notification:app",
        "--host",
        "0.0.0.0",
        "--port",
        "8080",
    ],
    dependencies = [
        "//:requirements#uvicorn",
        "//:requirements#aiohttp",
        "//:requirements#python-multipart",
        ":lib",
    ],
    platforms = [
        "linux_x86_64-cp-39-cp39",
        #  "macosx-11.0-arm64-cp-39-cp39",
    ],
    script = "uvicorn",
)

pex_binary(
    name = "script",
    dependencies = [
        "!:lib",
    ],
    entry_point = "script.py",
)
r
I assume you are running the 
script
pex? What does 
pants dependencies
for 
script
shows? It feels bit weird that you are ignoring the 
:lib
target which owns the 
script.py
but later you provide it as an entry point. 
python_sources
globs all the 
.py
files in current directory except I think 
test_
or conftest
p
Ah yes, that was just my effort to exclude the init file. Removing it does not make a difference. But even when I exclude it 
pants dependencies
shows 
app/__init__.py:lib
as a dependency.
r
Can you ignore the 
__init__.py
directly inside 
python_sources
and try?
Copy code
python_sources(name="lib", dependencies=["script.py"])
I am not sure if when fetching a single script as entry point, pants packages the whole target which owns it.
Sorry it should be 
sources
not 
dependencies
Copy code
python_sources(name="lib", sources=["script.py"])
🙌 1
p
Hm no it still attempts to run the init file. However now it fails to infer the third party dependencies used in the init file. I will set up a reproduction repo later to make this simpler. Thanks for your time sofar! ❤️
👍 1
@refined-addition-53644 there you go https://github.com/chrismatix/pants-pex-repro
e
@refined-addition-53644 and @plain-quill-25213 I think Pants just doesn't support this currently. Pex does, you'd say 
--exe script.py
and that would slurp in 
script.py
and nothing else.
👍 2
p
Thanks @enough-analyst-54434. So it seems that there is no way around having a dedicated 
scripts
directory for a given app.
e
I think so if you're using Pants right now.
👍 1
r
Ah ok yeah I checked pex docs and saw the example of 
--exe
e
By design, Pants is in the position of being a middleman unfortunately; so you often have to teach it about the underlying tool in the form of more middleman code.
h
I think this is due to this option, which by default infers deps on "structural" 
__init__.py
files. Try setting that to 
never
to see if that helps?
That may not be what you want in general though, but at least it'll be instructive
p
@happy-kitchen-89482 good call, but unfortunately the option does not change anything. I even tried moving scripts related to the app to their own directory and now it executes both init files.
Copy code
.
└── apps/
    ├── BUILD
    ├── __init__.py # outer init
    └── scripts/
        ├── BUILD
        ├── __init__.py # inner init
        └── script.py
But this seems to be expected:
Copy code
Even if this is set to never or content_only, Pants will still always include any ancestor __init__.py files in the sandbox. Only, they will not be "proper" dependencies, e.g. they will not show up in pants dependencies and their own dependencies will not be used.
Of course I can work around this, but it’s very surprising behavior. Anyway, thank you for Pants and all your work!
h
Hmmm, yes, I remember this now. Python, and many tools in its ecosystem, make structural assumptions about the presence of, and evaluation of, 
__init__.py
so we had to be aggressive about that
I think the unwanted behavior is in some sense "correct" from a Python perspective, alas, if the script file is being imported. The key bit here is that you're running it as a script, not importing it. But Pants isn't using that information. Probably it should.
Can you open a ticket? This seems like wrong behavior.
p
4 Views
Open in Slack
PreviousNext
Powered by