https://pantsbuild.org/ logo
#general
Title
# general
c

clean-night-52582

08/03/2021, 12:21 AM
Is there any way standard way in pants to add some additional validation to proto?
proto3
is very relaxed for the sake of compatibility, but am looking for a way not to have to write validation into the endpoints. I'm looking for something that after we deserialize the proto we can check for what we consider a valid message that is a subset of what proto would allow, string not being empty, int's not being 0 etc. Something like a
jsonschema
validator layer for JSON. I was looking at
protoc-gen-validate
but it seems like I would have to swap out the whole
proto
compiler and give it access to some
go
resources. https://github.com/envoyproxy/protoc-gen-validate wondering if anyone else has found a good solution.
f

fast-nail-55400

08/03/2021, 12:36 AM
the
--descriptor_set_out=FILE
option to
protoc
will write metadata about the protobufs to FILE (in protobuf format of course)
Writes a FileDescriptorSet (a protocol buffer,
defined in descriptor.proto) containing all of the input files to FILE.
do you want to this to be a pants plugin?
h

hundreds-father-404

08/03/2021, 12:41 AM
the --descriptor_set_out=FILE option to protoc will write metadata about the protobufs to FILE
Like the
./pants export-codegen
goal? Not sure how automated you mean by "deserialize"
f

fast-nail-55400

08/03/2021, 12:48 AM
so the following is one possible solution: 1. add `fmt`/`lint` support for protobuf files 2. write a “protobuf metadata” lint plugin that obtains the descriptor set and then asks the engine to resolve a
ProtobufMetadataLintRequest
using a union rule where particular metadata linters can register. 3. in a particular repo, add a pants plugin to hook
ProtobufMetadataLintRequest
via
UnionRule
and process any of its`ProtobufMetadataLintRequest` subclass that come to it,
the plugin in step 2 will take care of using protoc to obtain the metadata. the `ProtobufMetadataLintRequest`plugins would just process the metadata and impose whatever policy they want.
c

clean-night-52582

08/03/2021, 12:53 AM
I'm not really worried about the current output of the code generation, well might want to modify it at some point. Basically was asking if there is a way to add annotations to
proto
files such that I don't have to write a bunch of code to check fields are not the default. Trying to get grpc off the ground but switching from json with a schema validator to proto if we have to have to hand write the validation might be a tough sell.
f

fast-nail-55400

08/03/2021, 12:56 AM
Basically was asking if there is a way to add annotations to 
proto
 files such that I don’t have to write a bunch of code to check fields are not the default.
not that I know of. this is a general problem with any user of protobuf.
you could switch to using the
google.protobuf.*
wrapper types
some languages will turn that into
Option<T>
instead of just
T
for whatever T
The
prost
crate that we use in the Rust parts of Pants does that.
your question is not a Pants problem per se but rather one of how protoc generates code and how your code uses that generated code