Hello, I experienced a module that cannot be found only during

pants test

but it can be found using

pants run

ImportError: cannot import name 'module_something' from 'src.common' (unknown location)

There was one time that the test was successful but I changed the code with the same import structure and the above error shows up. Does anyone know how could this happen? Small reproducible repository: https://github.com/tomgun132/pants_test_proto Thank you beforehand!

Is pants printing any warnings?

Yes, a lot of warning regarding cannot infer owners.

Fix those and the imports should work. Pants will only include files as a dependency when it knows they're required, so if an import isn't being inferred, it's likely Pants doesn't know that they're required

Yeah, that's a bit difficult as

src/common

contains protobuf generated codes...

How is the code being generated? Can you create a reduced example that demonstrates what's going on?

it's by

python -m grpc_tools.protoc -I. --python_out=src/common --pyi_out=src/common --grpc_python_out=src/common module_something.proto

But the strange thing is that, this only happens in

pants test

. When I run

pants run

to the code that has the same import, it works without any problem even though it has the same warning.

Ah, so they're "normal" .py files on disk, just not written by hand?

yes

Can you create a reduced example that demonstrates what's going on?

@broad-processor-92400 it's this https://github.com/tomgun132/pants_test_proto

Ah, cool, that's helpful. This import is attempting to import a top-level package called

entities_pb2

, but the adjacent file is "meant" to be accessible as

import common.entities_pb2

(or

from . import entities_pb2

) according to how the source roots are configured.

import entities_pb2 as entities__pb2

e.g. if I manually change

-import entities_pb2 as entities__pb2
+from . import entities_pb2 as entities__pb2

then

pants test ::

gets further. Some options are: 1. change the pants source roots so that it understands that each protobuf file is meant to be top-level packages, maybe something like

src/common

2. change the protobuf code generation command to generate code appropriate for submodule (I think this means the result you're looking for is an

__init__.py

file, and it using relative imports), no idea how to do this, you'll have to investigate 3. investigate pants' builtin support for protobuf and see if that helps (it might help with other things) https://www.pantsbuild.org/docs/protobuf-python

@broad-processor-92400 Thank you for the explanation! I changed the repository's protobuf structure to be similar with my real project. The import error changed to

home/tomgun/.pyenv/versions/3.10.11/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
app/tests/test_app.py:3: in <module>
    from <http://app.app|app.app> import foo
E   ModuleNotFoundError: No module named 'app'

You'll get more useful help if you make a reduced example that actually reproduces the real problem. (everyone in this slack is a volunteer, with limited time) If your real project is still getting the warnings about dependencies not being able to be inferred, that'll indicate why pants isn't including the files in the sandbox. Getting rid of those by working out which imports are 'bad' and whether you need to update your source roots or imports or whatever, will be the first step

the error is something like this:

src/service/common_pb2.py:14: in <module>
    from entities import entities_bar_pb2 as entities_bar__pb2
E   ImportError: cannot import name 'entities_bar_pb2' from 'entities' (unknown location)