mirror of
https://github.com/protocolbuffers/protobuf-go.git
synced 2024-12-28 00:19:55 +00:00
Go support for Google's protocol buffers
8012b444ee
This CL takes a significantly different approach to generating support for protobuf reflection. The previous approach involved generating a large number of Go literals to represent the reflection information. While that approach was correct, it resulted in too much binary bloat. The approach taken here initializes the reflection information from the raw descriptor proto, which is a relatively dense representation of the protobuf reflection information. In order to keep initialization cost low, several measures were taken: * At program init, the bare minimum is parsed in order to initialize naming information for enums, messages, extensions, and services declared in the file. This is done because those top-level declarations are often relevant for registration. * Only upon first are most of the other data structures for protobuf reflection actually initialized. * Instead of using proto.Unmarshal, a hand-written unmarshaler is used. This allows us to avoid a dependendency on the descriptor proto and also because the API for the descriptor proto is fundamentally non-performant since it requires an allocation for every primitive field. At a high-level, the new implementation lives in internal/fileinit. Several changes were made to other parts of the repository: * cmd/protoc-gen-go: * Stop compressing the raw descriptors. While compression does reduce the size of the descriptors by approximately 2x, it is a pre-mature optimization since the descriptors themselves are around 1% of the total binary bloat that is due to generated protobufs. * Seeding protobuf reflection from the raw descriptor significantly simplifies the generator implementation since it is no longer responsible for constructing a tree of Go literals to represent the same information. * We remove the generation of the shadow types and instead call protoimpl.MessageType.MessageOf. Unfortunately, this incurs an allocation for every call to ProtoReflect since we need to allocate a tuple that wraps a pointer to the message value, and a pointer to message type. * internal/impl: * We add a MessageType.GoType field and make it required that it is set prior to first use. This is done so that we can avoid calling MessageType.init except for when it is actually needed. The allows code to call (*FooMessage)(nil).ProtoReflect().Type() without fearing that the init code will run, possibly triggering a recursive deadlock (where the init code depends on getting the Type of some dependency which may be declared within the same file). * internal/cmd/generate-types: * The code to generate reflect/prototype/protofile_list_gen.go was copied and altered to generated internal/fileinit.desc_list_gen.go. At a high-level this CL adds significant technical complexity. However, this is offset by several possible future changes: * The prototype package can be drastically simplified. We can probably reimplement internal/legacy to use internal/fileinit instead, allowing us to drop another dependency on the prototype package. As a result, we can probably delete most of the constructor types in that package. * With the prototype package significantly pruned, and the fact that generated code no longer depend on depends on that package, we can consider merging what's left of prototype into protodesc. Change-Id: I6090f023f2e1b6afaf62bd3ae883566242e30715 Reviewed-on: https://go-review.googlesource.com/c/158539 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com> |
||
|---|---|---|
| cmd | ||
| encoding/textpb | ||
| internal | ||
| proto | ||
| protogen | ||
| reflect | ||
| runtime | ||
| types | ||
| .gitignore | ||
| .travis.yml | ||
| AUTHORS | ||
| CONTRIBUTING.md | ||
| CONTRIBUTORS | ||
| go.mod | ||
| go.sum | ||
| LICENSE | ||
| PATENTS | ||
| README.md | ||
| regenerate.bash | ||
| test.bash | ||
Next Generation Go Protocol Buffers
WARNING: This repository is in active development. There are no guarantees about API stability. Breaking changes will occur until a stable release is made and announced.
This repository is for the development of the next major Go implementation of protocol buffers. This library makes breaking API changes relative to the existing Go protobuf library. Of particular note, this API aims to make protobuf reflection a first-class feature of the API and implements the protobuf ecosystem in terms of reflection.
Design Documents
List of relevant design documents:
Contributing
We appreciate community contributions. See CONTRIBUTING.md.
Reporting Issues
Issues regarding the new API can be filed at
github.com/golang/protobuf.
Please use a APIv2: prefix in the title to make it clear that
the issue is regarding the new API work.