https://pantsbuild.org/ logo
Join Slack
Powered by
anyone know why `pkgutil.iter_modules` doesn't wor...
# general
e
anyone know why 
pkgutil.iter_modules
doesn't work as expected when an application is packaged as a PEX? Gives an empty array, but it works when run in a different environment. I verified that the modules in question are present in the PEX.
✅ 1
specifically, if I do 
pants package <target>
and then just
Copy code
cd dist/<target>/bin.pex
python __main__.py
everything works as expected. but, doing 
pants run <target>
does not.
I've tried a lot of options, and am using 
layout="loose"
to get the PEX in the dir format I can 
cd
into
ahhh it has something to do with Pants maybe. If I add an explicit import to the module in question, 
pkgutil.iter_modules
finds it. So seems like Pants/PEX thinks the module isn't needed because it's never explicitly imported, and so the PEX doesn't include it in the right way somehow. the whole point of the 
pkgutil.iter_modules
is to do dynamic imports. I guess I need to do some manual dependency specification somewhere...
f
Is there a way Pants could have inferred the dependency?
e
I don't think so.
f
You can add explicit dependencies to the targets in the relevant 
BUILD
files.
(via the 
dependencies
field on a target)
That is the general way in Pants to add the dependencies if dependency inference would not find it.
e
yeah, my confusion is that I thought I was doing manual specification:
Copy code
python_sources(name="thing", sources=["app/**/*.py"]) # contains the modules that are dynamically imported

pex_binary(
    name="bin",
    dependencies=[":thing"],
    execution_mode="venv",
    layout="loose",
    script="bentoml",
    args=["serve", "app.service:svc"]
)
I've tried variations of this, adding the specific .py which does the dynamic imports as a 
python_source
and adding the imported modules as dependencies of that
and when I do 
pants dependencies thing:bin
the dynamically imported modules show up in the list...
I figured it out. It wasn't a Pants/PEX issue, and actually the dependencies were correct all along. The issue was that 
pkgutil.iter_modules
accepts a path, not a module, and the path was different inside the PEX. Doing this makes it work everywhere: https://stackoverflow.com/a/1310912
thanks for the replies!
6 Views
Open in Slack
PreviousNext
Powered by