Hey All, I am getting a weird error while trying to package a script containing some proto code as a wheel file. Some of the proto files are being packaged inside the wheels just fine, whereas some of them are being skipped even after being explicitly specified in the BUILD file. Any ideas/thoughts what might be happening here ??

Are all your proto files accounted for when running

pants dependencies --transitive path/to/your/distribution:target

?

when you say proto files, are the

.proto

files expected to be in the wheel, or the

_pb.py

files generated from them?

Andreas - Yes, I can see all the files being accounted for. Benjy - I meant the _pb.py files being generated from the proto.

Are the missing _pb.py files generated to a path that is under a source root (https://www.pantsbuild.org/docs/source-roots)?

Are the missing _pb.py files generated to a path that is under a source root

Yes

Do you see them when you run

./pants export-codegen

?

oh I wasn’t aware of this Thanks. I tried this & am running into some errors related to golang -

Please specify either:
        • a "go_package" option in the .proto source file, or
        • a "M" argument on the command line.

See <https://developers.google.com/protocol-buffers/docs/reference/go-generated#package> for more information.

--go_out: protoc-gen-go: Plugin failed with status code 1.

I will try to figure a way around this and share result. In the meantime, if you have any suggestions pls let me know. (EDIT: Check below added notes)

Can you think of any differences between the ones that are in the wheel and the ones that aren’t?

No, not really. Both are defined in the BUILD file, and in some other

pex_binary

target they are being imported as expected. One thing is - the missing files (let’s call it depA.proto) is being imported in other parent proto file (let’s call is parent.proto) & the parent.proto is being imported as expected (but missing the dependency).

And I assume you’ve checked (with

unzip -t

, say) that the files are actually not in the wheel file?

Yes that’ right. I have unzipped the tar.gz and checked EDIT: The

codegen

was resolved and it was able to generate all the protos.

if you want to keep the project structure, you may exclude files that should go into the other distrubution by adding

!!src/backend/abc/libs/service/quitter.py

to the dependencies of conflicting distribution. https://www.pantsbuild.org/docs/targets#dependencies-field

Thanks Andreas, I tried that but there are some common files that needs to be shared across both the distributions so I would probably need to find some other way here

Ah yes, when Pants generates setup.py etc. for you from a

python_distribution

, it has to have some way of assigning each python module to a wheel it builds. The algorithm it uses for this is described here: https://www.pantsbuild.org/docs/python-distributions#mapping-source-files-to-distributions

Yep, I am currently going that route only. 😄