I’m trying to debug an issue I’m having with

pants run

. If I try to run the

pex

using

execution_mode="venv"

I’m getting an import error for

google-auth

EDIT: I’m running

pants==2.19.1

Pants is detecting

google-auth

as a transitive dependency

> pants dependencies --transitive src/python/apps/ava:ava_app | grep 'google-auth'
3rdparty/python:google-auth

I’ve declared

google.oauth2

as a module of

google-auth

> cat 3rdparty/python/BUILD | grep 'google-auth'
python_requirement(name="google-auth", requirements=["google-auth"], modules=["google.auth", "google.oauth2"], dependencies=[":setuptools"])

I’ve validated that it’s included in the lockfile and checked that that version does indeed have the

google.oauth2

module

> cat 3rdparty/python/python-default.lock | grep 'google-auth=='
//     "google-auth==2.16.2",
    "google-auth==2.16.2",

Here’s the build file

> cat src/python/apps/ava/BUILD
python_sources(name="run_sanic", sources=["run_sanic.py"])

pex_binary(
  name="ava_app",
  dependencies=[":run_sanic"],
  entry_point="run_sanic.py:main",
  execution_mode="venv",
)

And here’s what happens when I run it:

> pants run src/python/apps/ava:ava_app
Traceback (most recent call last):
  File "/Users/josha/.pex/venvs/f5ad7fc6f9308a9164792203bf58aaf22a09c919/779eb2cc0ca9e2fdd204774cbc41848e4e7c5055/pex", line 282, in <module>
    module = importlib.import_module(module_name)
  File "/Users/josha/.pyenv/versions/3.9.16/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/Users/josha/.pex/venvs/f5ad7fc6f9308a9164792203bf58aaf22a09c919/779eb2cc0ca9e2fdd204774cbc41848e4e7c5055/lib/python3.9/site-packages/run_sanic.py", line 8, in <module>
    from ava.server import app
  File "/Users/josha/.pex/venvs/f5ad7fc6f9308a9164792203bf58aaf22a09c919/779eb2cc0ca9e2fdd204774cbc41848e4e7c5055/lib/python3.9/site-packages/ava/server.py", line 21, in <module>
    from ava.worker.bootstrap import create_app  # noqa: E402
  File "/Users/josha/.pex/venvs/f5ad7fc6f9308a9164792203bf58aaf22a09c919/779eb2cc0ca9e2fdd204774cbc41848e4e7c5055/lib/python3.9/site-packages/ava/worker/bootstrap.py", line 9, in <module>
    from ava.worker.audit import init_auditing
  File "/Users/josha/.pex/venvs/f5ad7fc6f9308a9164792203bf58aaf22a09c919/779eb2cc0ca9e2fdd204774cbc41848e4e7c5055/lib/python3.9/site-packages/ava/worker/audit.py", line 1, in <module>
    from pf.sanic.extras.audit.setup import setup_audit
  File "/Users/josha/.pex/venvs/f5ad7fc6f9308a9164792203bf58aaf22a09c919/779eb2cc0ca9e2fdd204774cbc41848e4e7c5055/lib/python3.9/site-packages/pf/sanic/extras/audit/setup.py", line 6, in <module>
    from google.oauth2.service_account import Credentials
ModuleNotFoundError: No module named 'google.oauth2'

It does this even if I add

“3rdparty/python:google-auth”

as an explicit dependency to the

pex_binary

def. Given I’ve done

pants export --py-resolve-format=symlinked_immutable_virtualenv --resolve=python-default

when I try running the

run_sanic.py

directly it works fine

/Users/josha/pf/c3/dist/export/python/virtualenvs/python-default/3.9.16/bin/python src/python/apps/ava/run_sanic.py
<expected output>

But if I remove

execution_mode="venv"

I get a different error:

pants run src/python/apps/ava:ava_app
/Users/josha/.pyenv/versions/3.9.16/bin/python3.9: Error while finding module specification for '__main__' (ValueError: __main__.__spec__ is None)

And for clarity at the bottom of

run_sanic.py

is just

if __name__ == "__main__":
    main()

I’m clearly doing something wrong, and any ideas would be greatly appreciated!

I've haven't tried things like

python_requirement(name="google-auth", requirements=["google-auth"], modules=["google.auth", "google.oauth2"], dependencies=[":setuptools"])

before but I've had luck with mapping modules to packages via

python_requirements(
    ...
    module_mapping={
        "google-auth": ["google.oauth2"],
    }
)

Yeah my read from the docs is that

python_requirements

is just a convenience function to generate

python_requirement

nodes from a

requirement.txt

file. I could be wrong though

Have you tried adding (excuse the formatting)

overrides={
    "...": {
        "the sanic audit package name": [
           "//3rdparty/python:reqs#google-auth",
        ]
     },
  }

Hmm good idea! Let me try that

Have you tried

--keep-sandboxes=always

?

Good idea. Let me try that

I'm not sure what the answer to that is. If you include

include_tools=True

in your pex you can run something like

PEX_TOOLS=1 python item-rank/cli.pex/ venv --compile ./foo

You can then run

./foo/bin/python3

and try to import

google.oauth2

in the interpreter.*

Oh nice one. I’ll try that

Does

lib/python3.10/site-packages/

look like its supposed to?

Hmm actually, the version of google-api-core we’re running does use the old style also… https://github.com/googleapis/python-api-core/tree/v1.34.0/google

fwiw

>>> import pkgutil
>>> import google
>>> def list_modules(package):
...     return [modname for _, modname, _ in pkgutil.walk_packages(path=package.__path__)]
... 
>>> modules = list_modules(google)
>>> for module in modules:
...     print(module)
... 
auth
oauth2

Any chance you can try with this set?

python_requirement(name="google-api-core", requirements=["google-api-core==1.34.0"], modules=["google.api_core"], dependencies=[":setuptools"])
python_requirement(name="google-api-python-client", requirements=["google-api-python-client==1.12.8"], modules=["googleapiclient"], dependencies=[":setuptools"])
python_requirement(name="google-auth", requirements=["google-auth"], modules=["google.auth", "google.oauth2"], dependencies=[":setuptools"])
python_requirement(name="google-cloud-compute", requirements=["google-cloud-compute==1.10.0"], modules=["google.cloud.compute", "google.cloud.compute_v1"], dependencies=[":setuptools"])
python_requirement(name="google-cloud-error-reporting", requirements=["google-cloud-error-reporting<2.0.0,>=1.8.2"], modules=["google.cloud.error_reporting"], dependencies=[":setuptools"])
python_requirement(name="google-cloud-profiler", requirements=["google-cloud-profiler<5.0.0,>=4.0.0"], modules=["googlecloudprofiler"], dependencies=[":setuptools"])
python_requirement(name="google-cloud-secret-manager", requirements=["google-cloud-secret-manager==2.12.6"], modules=["google.cloud.secretmanager"], dependencies=[":setuptools"])
python_requirement(name="google-cloud-sqlcommenter", requirements=["google-cloud-sqlcommenter<3.0.0,>=2.0.0"], modules=["google.cloud.sqlcommenter"], dependencies=[":setuptools"])
python_requirement(name="google-cloud-trace", requirements=["google-cloud-trace<2.0.0,>=1.11.0"], modules=["google.cloud.trace", "google.cloud.trace_v1", "google.cloud.trace_v2"], dependencies=[":setuptools"])

How are you mapping that the package that installs pf.sanic.extras.audit.setup depends on google-auth?

pf.sanic.extras

is an internal lib in the same monorepo. So it’s just getting mapped by pants

I take it that's returning nothing?

🙂

In my experience Pex has been pretty solid

Yeah the traceback gets us there

File "/Users/josha/.pex/venvs/b8e2aec83f24690d9d16e74d79fc645c385ad07c/779eb2cc0ca9e2fdd204774cbc41848e4e7c5055/lib/python3.9/site-packages/run_sanic.py", line 8, in <module>
    from ava.server import app
  File "/Users/josha/.pex/venvs/b8e2aec83f24690d9d16e74d79fc645c385ad07c/779eb2cc0ca9e2fdd204774cbc41848e4e7c5055/lib/python3.9/site-packages/ava/server.py", line 21, in <module>
    from ava.worker.bootstrap import create_app  # noqa: E402
  File "/Users/josha/.pex/venvs/b8e2aec83f24690d9d16e74d79fc645c385ad07c/779eb2cc0ca9e2fdd204774cbc41848e4e7c5055/lib/python3.9/site-packages/ava/worker/bootstrap.py", line 9, in <module>
    from ava.worker.audit import init_auditing
  File "/Users/josha/.pex/venvs/b8e2aec83f24690d9d16e74d79fc645c385ad07c/779eb2cc0ca9e2fdd204774cbc41848e4e7c5055/lib/python3.9/site-packages/ava/worker/audit.py", line 1, in <module>
    from pf.sanic.extras.audit.setup_audit import setup_audit
  File "/Users/josha/.pex/venvs/b8e2aec83f24690d9d16e74d79fc645c385ad07c/779eb2cc0ca9e2fdd204774cbc41848e4e7c5055/lib/python3.9/site-packages/pf/sanic/extras/audit/setup_audit.py", line 6, in <module>
    from google.oauth2.service_account import Credentials
ModuleNotFoundError: No module named 'google.oauth2'

so:

- run_sanic.py
  - ava/server.py
    - ava/worker/bootstrap.py
      - ava/worker/audit.py
        - pf/sanic/extras/audit/setup_audit.py

And just as a sanity check you can

pants run

each of those without getting import errors (well I guess run_sanic.py gives an error)?

👀

dang. It might be worth trying to make a minimal reproduction of the code base.

Yeah I think you’re right. I really appreciate your time/help with this

Maybe a dumb question. But why do you have

dependencies=[":run_sanic"],

in the pex?

I’ll go and see if I can build a minimal reproduction

got it