Our company is using pants, Python and protobufs. One thing I didn't see written up anywhere, but appears to be true, is that if you do all these things, you either have to use namespace packages (in other words, no

__init__.py

), or put all your protos in a root directory, not underneath any directory that's a module. The issue is that protoc doesn't generate

__init__.py

files, so any package in

dist/codegen

under a module package (with the init file), cannot be found. Has anyone hit upon any reasonable way to solve the problem? Forcing us into a package structure just to satisfy python module rules seems weird. Using namespace packages is also weird, and causes its own problems (frequently it means that

pants test

doesn't work for something but

pants run

does). It's hard to figure out a reasonable solution here.

To clarify a point -

dist/codegen

is just where Pants exports generated code when you ask it to. It doesn’t use that for

pants test

,

pants run

and so on.

FWIW we use Python + protobuffs and require our first party python code to not be namespace packages. (All the namespace interactions still confuse me greatly and I am sure there are plenty of corner cases.)

@happy-kitchen-89482 right - there's no breakage in Pants here, but developers often find it nice to run without Pants for speed. Because

dist/codegent

doesn't have any

__init__.py

, anything there that's namespaced within a package that does have

__init_.py_

is not usable. One solution might be for Pants to create

__init__.py

, which I think is technically possible, but it would break anyone using namespace packages, so it would have to be an option. For example, if our regular code has package

foo

, and we have a

foo/protos

, then

foo/protos

, then trying to

import foo.protos

would only work in Pants. To be clear, this is not a Pants problem, but it could be that Pants could make this better. Or maybe just people have some standard way to deal with this. @curved-manchester-66006 do you do this nesting of your proto packages?

do you do this nesting of your proto packages?

The proto names are pretty "flat", so I dont' think you have a case of src / foo / some proto / bar / some python

I can see how that might be the case. In our case, at least initially, the protos were living side-by-side with code, so tended not to have a flat namespace.

I guess the true solution would be for the proto compiler to add

__init__.py

files, but I guess Pants could also do so

We ended up just moving our protos to their own directory, but yeah, Pants could in theory solve this issue (I think)