https://pantsbuild.org/ logo
Join Slack
Powered by
looks like there’s no codegen for protobuf + go, c...
# general
h
looks like there’s no codegen for protobuf + go, can you all point me in the right direction? might contribute a plugin for this. seems simple enough, especially since you already have a protobuf + python implementation
f
However, that PR uncovered a fundamental limitation with the design of the existing Pants codegen rules: Those rules do not provide a way for the backends for compiled languages (e.g., Go, Scala, Java) to wrap the generated code in a target for use in the build graph to enable compilation of that code. The existing rules are geared toward Python where the mere existence of generated files is sufficient for them to be usable by a Python interpreter.
I was discussing this issue with @hundreds-father-404 earlier this week. I’ll be opening an issue within a day or so with a proposal on how to fix using the “target generation” feature in Pants. But it is going to require work in core codegen rules to be able to support Go protobuf (and for Scala/Java as well).
Once I’ve written up the issue, neither Eric nor I particularly needs to be the one to actually fix the core codegen rules. Would you be interested? We can support you in that endeavor.
h
sure - I can give it a shot
🙌 3
@fast-nail-55400: if I am understanding this correctly, once 14258 is completed, then https://github.com/pantsbuild/pants/pull/13985 can be merged (which will add go + protobuf codegen support)
h
Yes. Tom figured out the parts already like "How do we invoke Protoc to generate Go code?", and now it's only adjusting Pants's codegen setup so that we can then compile those generaetd files
👍 1
h
@hundreds-father-404 will look into it tonight, thanks
i am trying to decipher this:
Copy code
Define a _scala_protobuf_sources target type with a dependency on its protobuf_sources target. (This would be used internally to wrap generated Scala sources, and would not be exposed to the user.)
where in the codebase would be this defined?
is it 
target_types.py
?
h
I suspect that implementing this particular change is going to be pretty involved, unfortunately, so it might not be the best starter ticket. I think it still requires some design work & discussion. But @fast-nail-55400 maybe we can make more progress on the design this week, and then translate that into some steps on how to implement? (thank you for offering to dive in! I really appreciate that)
h
indeed - i thought it would be only a few lines to it up and running
f
no it is actually a pretty involved change. and I left some open questions in the sketch, since it will require some research as well into what the right approach should be. I did not mean to imply that it was an easy change.
following up here. Pants v2.11.0rc0 has protobuf/go codegen support.
🙌 1
h
thanks @fast-nail-55400
h
I think we might have not activated the backend properly until 2.11.0rc1, which will be released shortly
h
awesome
glad to know this project is very active 🙂
@hundreds-father-404 will ./pants tailor pick it up automatically after i enable the backend in rc1?
h
For sure! we often have too much to fit into our release blogs in fact 🙈 in case you missed it, 2.10 blog: https://blog.pantsbuild.org/introducing-pants-2-10/ talks about improvements to Go backend
and yes, activating 
pants.backend.experimental.codegen.protobuf.go
will have 
tailor
add 
protobuf_sources
options for you We still need to add docs for this new feature, but I think other than activating the backend & running 
tailor
, you need to make sure each relevant file sets 
option go_package
, and then I think your 
go.mod
and 
go.sum
include the relevant modules like 
<http://github.com/golang/protobuf|github.com/golang/protobuf>
🙌 1
f
note: if any necessary support libraries are missing from go.mod, the generated code will fail to compile and you should see the import paths needed in the error messages
👍 2
h
i look forward to testing it out when rc1 is released
@hundreds-father-404 @fast-nail-55400: I see 2.11.0rc1 is available. I tried enabling the 
pants.backend.experimental.codegen.protobuf.go
backend but reports a “ModuleNotFoundError”
Copy code
[GLOBAL]
pants_version = "2.11.0rc1"
backend_packages = [
    "pants.backend.experimental.go",
    "pants.backend.experimental.codegen.protobuf.go"
]

[golang]
expected_version = "1.18"
Copy code
Traceback (most recent call last):
  File "/Users/cest-la-vie/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.rHDk1N/install/lib/python3.9/site-packages/pants/init/extension_loader.py", line 130, in load_backend
    module = importlib.import_module(backend_module)
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'pants.backend.experimental.codegen.protobuf.go'
08:52:53.67 [ERROR] Failed to load the pants.backend.experimental.codegen.protobuf.go.register backend: ModuleNotFoundError("No module named 'pants.backend.experimental.codegen.protobuf.go'")
ignore this, I see tdyas is already working on fixing it as per another conversation in #development channel
f
Open in Slack
PreviousNext
Powered by