For a file "foo/bar.proto", put the FileDescriptor in "ProtoFile_foo_bar"
rather than "Bar_fileDescriptor".
Avoid name clashes when a package contains "a/foo.proto" and "b/foo.proto".
Don't camelcase the filename: These vars weren't fully camelcased to begin
with, and leaving the filename relatively unchanged is clearer and more
predictable.
Move "ProtoFile" from the end of the var name to the start, so that vars
will sort better in packages with multiple descriptors.
These changes do add a chance of name collision when the input filename
begins with an uppercase letter: Foo.proto becomes "ProtoFile_Foo", which
could be the result of camelcasing "proto_file.foo". The readability
benefits seem worth it.
Change-Id: If27d3a0d7b5bf3535aa1607a8579eb057c74d2dc
Reviewed-on: https://go-review.googlesource.com/c/163199
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Herbie Ong <herbie@google.com>
This test examines the result of converting math.NaN() to a fixed byte
string. Change it to use a specific NaN value instead, since the value
returned by math.NaN is specified only as being a NaN, not a specific
one.
Use specific float32 and float64 NaN values, since the result of
converting a float64 NaN to a float32 can and does vary.
Fixes test failure on ARM.
Change-Id: Ia1517fdba768cdd88e5ee5f5af37f0b481e651b4
Reviewed-on: https://go-review.googlesource.com/c/162117
Reviewed-by: Herbie Ong <herbie@google.com>
These methods are no longer used. Drop them.
Change-Id: Ie5595c1a56dd76183782019384d16ea97d5f7e9e
Reviewed-on: https://go-review.googlesource.com/c/160357
Reviewed-by: Herbie Ong <herbie@google.com>
Test the fileinit and protodesc packages by verifying that passing a
FileDescriptorProto through a round trip of these packages leaves it
unchanged.
Change-Id: I6bfb894d95f1736f9908adee1ab63e9653b3f1be
Reviewed-on: https://go-review.googlesource.com/c/159762
Reviewed-by: Herbie Ong <herbie@google.com>
This CL takes a significantly different approach to generating support
for protobuf reflection. The previous approach involved generating a
large number of Go literals to represent the reflection information.
While that approach was correct, it resulted in too much binary bloat.
The approach taken here initializes the reflection information from
the raw descriptor proto, which is a relatively dense representation
of the protobuf reflection information. In order to keep initialization
cost low, several measures were taken:
* At program init, the bare minimum is parsed in order to initialize
naming information for enums, messages, extensions, and services declared
in the file. This is done because those top-level declarations are often
relevant for registration.
* Only upon first are most of the other data structures for protobuf
reflection actually initialized.
* Instead of using proto.Unmarshal, a hand-written unmarshaler is used.
This allows us to avoid a dependendency on the descriptor proto and also
because the API for the descriptor proto is fundamentally non-performant
since it requires an allocation for every primitive field.
At a high-level, the new implementation lives in internal/fileinit.
Several changes were made to other parts of the repository:
* cmd/protoc-gen-go:
* Stop compressing the raw descriptors. While compression does reduce
the size of the descriptors by approximately 2x, it is a pre-mature
optimization since the descriptors themselves are around 1% of the total
binary bloat that is due to generated protobufs.
* Seeding protobuf reflection from the raw descriptor significantly
simplifies the generator implementation since it is no longer responsible
for constructing a tree of Go literals to represent the same information.
* We remove the generation of the shadow types and instead call
protoimpl.MessageType.MessageOf. Unfortunately, this incurs an allocation
for every call to ProtoReflect since we need to allocate a tuple that wraps
a pointer to the message value, and a pointer to message type.
* internal/impl:
* We add a MessageType.GoType field and make it required that it is
set prior to first use. This is done so that we can avoid calling
MessageType.init except for when it is actually needed. The allows code
to call (*FooMessage)(nil).ProtoReflect().Type() without fearing that the
init code will run, possibly triggering a recursive deadlock (where the
init code depends on getting the Type of some dependency which may be
declared within the same file).
* internal/cmd/generate-types:
* The code to generate reflect/prototype/protofile_list_gen.go was copied
and altered to generated internal/fileinit.desc_list_gen.go.
At a high-level this CL adds significant technical complexity.
However, this is offset by several possible future changes:
* The prototype package can be drastically simplified. We can probably
reimplement internal/legacy to use internal/fileinit instead, allowing us
to drop another dependency on the prototype package. As a result, we can
probably delete most of the constructor types in that package.
* With the prototype package significantly pruned, and the fact that generated
code no longer depend on depends on that package, we can consider merging
what's left of prototype into protodesc.
Change-Id: I6090f023f2e1b6afaf62bd3ae883566242e30715
Reviewed-on: https://go-review.googlesource.com/c/158539
Reviewed-by: Herbie Ong <herbie@google.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
When encoding/textpb marshals out float32 values, it was previously
formatting it as float64 bitsize since both float types are stored as
float64 and internal/encoding/text only has one Float type. A
consequence of this is that the output may display a different value
than expected, e.g. 1.02 becomes 1.0199999809265137.
This CL splits Float type into Float32 and Float64 to keep track of
which bitsize to use when formatting. Values of both types are still
stored as float64 to keep the logic simple.
Decoding will always use Float64, but users can ask for a float32 value
from it.
Change-Id: Iea5b14b283fec2236a0c3946fac34d4d79b95274
Reviewed-on: https://go-review.googlesource.com/c/158497
Reviewed-by: Damien Neil <dneil@google.com>
A "_ProtoFile" suffix can potentially conflict with a sub-message named
"ProtoFile" nested within a message that matches the camel-cased
form of the basename of the .proto source file.
Avoid unlikely conflicts and rename this to use a "_protoFile" suffix,
which can never conflict except with an enum value that is also named
"protoFile" (which is a violation of the style guide).
Change-Id: Ie9d22f9f741a63021b8f76906b20c6c2f599885b
Reviewed-on: https://go-review.googlesource.com/c/157218
Reviewed-by: Damien Neil <dneil@google.com>
This change preserves consistency with CL/157077,
where New returns a value closer to the reflective type.
Change-Id: I85bfdae24e1ce1a10c3c7b939420fa1043bff743
Reviewed-on: https://go-review.googlesource.com/c/157078
Reviewed-by: Damien Neil <dneil@google.com>
Most usages of New actually prefer to interact with the reflective view
rather than the native Go type. Thus, change New to return that instead.
This parallels reflect.New, which returns the reflective view
(i.e., reflect.Value) instead of native type (i.e., interface{}).
We make the equivalent change to KnownFields.NewMessage, List.NewMessage,
and Map.NewMessage for consistency.
Since this is a subtle change where the type system will not always
catch the changed type, this change was made by both changing the type
and renaming the function to NewXXX and manually looking at every usage
of the the function to ensure that the usage correctly operates
on either the native Go type or the reflective view of the type.
After the entire codebase was cleaned up, a rename was performed to convert
NewXXX back to New.
Change-Id: I153fef627b4bf0a427e4039ce0aaec52e20c7950
Reviewed-on: https://go-review.googlesource.com/c/157077
Reviewed-by: Damien Neil <dneil@google.com>
Drop the protoreflect.ProtoEnum type (containing a single method
returning a protoreflect.Enum) and make generated enum types
directly implement protoreflect.Enum instead.
Messages have a two-level type split (ProtoMessage and Message) to
minimize conflicts between reflection methods and field names. Enums
need no such split, since enums do not have fields and therefore have
no source of conflicts.
Change-Id: I2b6222e9404253e6bfef2217859e1b760ffcd29b
Reviewed-on: https://go-review.googlesource.com/c/156902
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Change the go_package option to match the actual import path of this package.
Change-Id: Ie8630878ce75e34ca76d97c6e2922254cf964801
Reviewed-on: https://go-review.googlesource.com/c/156344
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The purego tag (see https://golang.org/issue/23172) is a community agreed
upon signal that a given build environment does not support unsafe.
The appengine environment is supposed to respect this tag, but does not
properly do so. Add this tag back in until they fix their environment.
Change-Id: I9a70062be4339c2e1a93cac31d387698c561b8aa
Reviewed-on: https://go-review.googlesource.com/c/154743
Reviewed-by: Damien Neil <dneil@google.com>
The v2 decoder isn't 100% complete, but it's good enough.
Delete the vendored copy of the v1 Unmarshal implementation.
Change-Id: Ibeabbb2e9109a1ec3df57e71f98b7aa4a583fc5b
Reviewed-on: https://go-review.googlesource.com/c/154577
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Add proto.Unmarshal.
Test cases all produce identical results to the v1 unmarshaller.
Change-Id: I42259266018a14e88a650c5d83a043cb17a3a15d
Reviewed-on: https://go-review.googlesource.com/c/153918
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
PBValueOf returns a protoreflect.Message, not a
protoreflect.ProtoMessage.
Change-Id: I88ed55f52bada6fc2b29ffd63e30de09e1febe8c
Reviewed-on: https://go-review.googlesource.com/c/153917
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Record annotations on methods in interface types (e.g., gRPC
server/client methods).
Generate annotations in golden tests, even when not checking the
content, to catch annotations without a corresponding symbol.
Change-Id: I44ae6caf66f709dc7f4686e931be04b8b6fa843d
Reviewed-on: https://go-review.googlesource.com/c/153877
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Was using a pref.Message where we want a pref.ProtoMessage.
Change-Id: I61d986a43eaf8f945a1378a7a10120474aa89d6f
Reviewed-on: https://go-review.googlesource.com/c/153697
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Add fields to the Message and Field builder structs which hold the value
of MessageOptions.map_entry, FieldOptions.packed, and FieldOptions.weak
options. Remove all access to the contents of options messages from the
prototype package.
Change IsPacked to always return false for unpackable field types,
which is consistent with the equivalent C++ API.
This change helps avoid dependency cycles between prototype and the
options messages. (Previously this was resolved by accessing options
with reflection, but just breaking the dependency from prototype to the
options message is cleaner and simpler.)
Change-Id: I756aefe2e04cfa8fea31eaaaa0b5a99d4ac9e851
Reviewed-on: https://go-review.googlesource.com/c/153517
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Remove the Mutable methods from KnownFields, List, and Map, replacing
them with methods which return a new, empty message value without adding
that value to the collection.
The new API is simpler, since it clearly applies only to message values,
and more orthogonal, since it provides a way to create a value without
mutating the collection. This latter point is particularly useful in
map deserialization, where the key may be unknown at the time the value
is deserialized.
Drop the Mutable interface, since it is no longer necessary.
Change-Id: Ic5f3d06a2aa331a5d5cd2b4e670a3dba4a74f77c
Reviewed-on: https://go-review.googlesource.com/c/153278
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Logic for serializing the default value in textual form exists in
multiple places in trivially similar forms. Centralize that logic.
Change-Id: I4408ddfeef2c0dfa5c7468e01a4d4df5654ae57f
Reviewed-on: https://go-review.googlesource.com/c/153022
Reviewed-by: Herbie Ong <herbie@google.com>
These properties of descriptors are currently missing and makes it impossible
to convert a FileDescriptorProto into one of the structured Go representations
and convert it back to a proto message without loss of information.
Furthermore, ReservedRanges and ReservedNames has semantic importance
to text serialization.
Change-Id: Ic33c30020ad51912b143156b95f47a4fb8da3503
Reviewed-on: https://go-review.googlesource.com/c/153019
Reviewed-by: Damien Neil <dneil@google.com>
Considerable thought was given to whether a seperate ExtensionRanges interface
should be made that encapsulates FieldNumbers with an Options method.
However, I decided against this design for the following reasons:
* Extension ranges share syntax with reserved numbers and fields.
Why is extension ranges so special that it can have options, while the other
two currently do not? How do we know that those other two won't grow options
in the future? If they do, then those APIs can be expanded in the same way as
how extension range options is being expanded here today.
* Extension range options stand out like a sore thumb compared to the other
eight options. The other options correspond with a named declaration and have
a full protobuf name that they are associated with. Extension range options
is the only options that is not correlated with a full name.
* Extension range options are an extremely rarely used feature and
it seems unfortunate complicating the common case with additional structure.
Change-Id: Ib284a0b798c57dc264febe304692eee5b9c8e91b
Reviewed-on: https://go-review.googlesource.com/c/153018
Reviewed-by: Damien Neil <dneil@google.com>
In CL/152020, we checked in pre-generated versions of descriptor and plugin.
This CL makes it such that they are generated by protoc-gen-go.
We modify protoc-gen-go to avoid reflection support by default
since the binary size increase is still an issue to investigate.
Reflection support is temporarily enabled by setting a special
PROTOC_GEN_GO_ENABLE_REFLECT environment variable.
Reflection support is always enabled for descriptor and plugin.
Furthermore, we change descriptor to depend on the protoapi package
instead of the proto package. The reason we do not switch to protoapi
for all generated protos is because we still depend on v1 proto
for the table-driven InternalMessageInfo type. Dropping it from descriptor
is semantically correct, but does incur slight performance cost.
It does not seem appropriate to drop it for all generated messages.
We could move InternalMessageInfo to protoapi, but the logic behind that
is significant.
Change-Id: I5c3fff7f6eab1a5a2399049d42fa6bf42d4c93f9
Reviewed-on: https://go-review.googlesource.com/c/152547
Reviewed-by: Damien Neil <dneil@google.com>
Custom descriptor types would want to benefit from descriptor formatting.
As such, move the logic out from prototype into an internal package for
the benefit of usages outside the prototype package.
Change-Id: I4bb2144221e656aa36909d33a77189fe084f700b
Reviewed-on: https://go-review.googlesource.com/c/152777
Reviewed-by: Herbie Ong <herbie@google.com>
In order to transition more of v1 proto package to use the v2 API,
we need the v2 API to stop depending on v1 proto. The legacy package currently
depends on v1 proto because it needs to unmarshal the descriptor protos.
Ideally, we would switch this to use the v2 implementation of wire unmarshaling.
However, that is not available yet. So, instead, we vendor a minified version
of the v1 proto package that only supports unmarshaling.
The only changes to the vendored v1 code are:
* Delete code not needed to implement proto.Unmarshal
* Drop support for message sets
* Drop support for reporting the full field name for required not set errors
The unused tool was used to delete unrelated code:
https://github.com/dominikh/go-tools/tree/master/cmd/unused
To verify that the dependency was dropped:
$ cd internal/legacy
$ go list -f "{{join .Deps \"\n\"}}" | sort | uniq | grep protobuf
github.com/golang/protobuf/protoapi
github.com/golang/protobuf/v2/internal/detrand
github.com/golang/protobuf/v2/internal/encoding/tag
github.com/golang/protobuf/v2/internal/encoding/text
github.com/golang/protobuf/v2/internal/encoding/wire
github.com/golang/protobuf/v2/internal/errors
github.com/golang/protobuf/v2/internal/flags
github.com/golang/protobuf/v2/internal/impl
github.com/golang/protobuf/v2/internal/legacy/protoV1
github.com/golang/protobuf/v2/internal/pragma
github.com/golang/protobuf/v2/internal/scalar
github.com/golang/protobuf/v2/internal/set
github.com/golang/protobuf/v2/internal/value
github.com/golang/protobuf/v2/reflect/protoreflect
github.com/golang/protobuf/v2/reflect/prototype
github.com/golang/protobuf/v2/runtime/protoimpl
github.com/golang/protobuf/v2/types/descriptor
Change-Id: I470865f1a987203574339fefc7d83843a12af966
Reviewed-on: https://go-review.googlesource.com/c/152545
Reviewed-by: Damien Neil <dneil@google.com>
In order to generate descriptor.proto, the generated code would want to depend
on the prototype package to construct the reflection data structures.
However, this is a problem since descriptor itself is one of the dependencies
for prototype. To break this dependency, we do the following:
* Avoid using concrete *descriptorpb.XOptions messages in the public API, and
instead just use protoreflect.ProtoMessage. We do lose some type safety here
as a result.
* Use protobuf reflection to interpret the Options message.
* Split out NewFileFromDescriptorProto into a separate protodesc package since
constructing protobuf reflection from the descriptor proto obviously depends
on the descriptor protos themselves.
As part of this CL, we check in a pre-generated version of descriptor and plugin
that supports protobuf reflection natively and switchover all usages of those
protos to the new definitions. These files were generated by protoc-gen-go
from CL/150074, but hand-modified to remove dependencies on the v1 proto runtime.
Change-Id: I81e03c42eeab480b03764e2fcbe1aae0e058fc57
Reviewed-on: https://go-review.googlesource.com/c/152020
Reviewed-by: Damien Neil <dneil@google.com>
The legacy prefix made sense when this functionality was part of impl.
Now that it is in its own package called legacy, the legacy prefix is silly.
Change-Id: I9e6ddb6185ce1f701e02768b505e6a05f3986f77
Reviewed-on: https://go-review.googlesource.com/c/152543
Reviewed-by: Herbie Ong <herbie@google.com>
Converting to/from v1/v2 extension descriptor types is a common operation
for v1 and v2 interoperability. Optimize these operations with a cache.
Change-Id: I5feca810f60376847c791654982acd3b6a37a5db
Reviewed-on: https://go-review.googlesource.com/c/152542
Reviewed-by: Herbie Ong <herbie@google.com>
Similar to how generated messages allow you to call Get methods on a
nil pointer, we permit similar functionality when protobuf reflection
is used on a nil pointer.
Change-Id: Ie2f596d39105c191073b42d7d689525c3b715240
Reviewed-on: https://go-review.googlesource.com/c/152021
Reviewed-by: Damien Neil <dneil@google.com>
The impl package currently supports wrapping legacy v1 enums and messages
so that they implement the v2 reflective APIs. This functionality is necessary
for v1 and v2 to interoperate. However, the existence of this functionality
presents several problems:
* A significant portion of the complexity in impl is for legacy wrapping.
* This complexity is linked into a Go binary even if all the other messages
in the binary natively support v2 reflection.
* It presents a cyclic dependency when trying to generate descriptor proto.
Suppose you are generating descriptor.proto. The generated code would want to
depend on the impl package because impl is the runtime implementation for
protobuf messages. However, impl currently depends depends on descriptor in
order to wrap legacy enum and messages since it needs the ability to dynamically
create new protobuf descriptor types. In the case of descriptor.proto, it would
presumably be generated with native reflection support, so the legacy wrapping
logic is unneccessary.
To break the dependency of impl on descriptor, we move the legacy support logic
to a different package and instead add hooks in impl so that legacy support could
be dynamically registered at runtime. This is dependency injection.
Change-Id: I01a582908ed5629993f6699e9bf2f4bee93857a4
Reviewed-on: https://go-review.googlesource.com/c/151877
Reviewed-by: Herbie Ong <herbie@google.com>
Add a Proto prefix before the Unwrap method to reduce the probability that
it would ever conflict with a method of the same name that a
custom implementation of Enum, Message, List, or Map may have.
Change-Id: I628bf8335583f2747ab4589f3e6ff82e4501ce98
Reviewed-on: https://go-review.googlesource.com/c/151817
Reviewed-by: Herbie Ong <herbie@google.com>
The new v1 protoapi package enables:
* Referencing types in the protoapi package instead of protoV1, which further
reduces the number of situations where we need to depend on protoV1.
This is for the goal of eventually breaking all cases where the v2 implementation
relies on v1, so that in the near future, proto v1 can rely on proto v2 instead.
* Removes the need for legacy_extension_hack.go since that functionality has now
been exported into the protoapi package.
Change-Id: If71002d9ec711bfabfe494636829df9abf19e23e
Reviewed-on: https://go-review.googlesource.com/c/151403
Reviewed-by: Herbie Ong <herbie@google.com>
The XXX_OneofWrappers method is a simplified way to obtain the wrapper structs
compared the previous XXX_OneofFuncs method which returned far more information
that was strictly necessary.
Change-Id: I2670506a2a8f7e8e724846b8c4083e7995371007
Reviewed-on: https://go-review.googlesource.com/c/151679
Reviewed-by: Herbie Ong <herbie@google.com>
Since detrand is an internal package, we can safely provide a function
that can be called to disable its functionality for testing purposes.
Change-Id: I26383e12a5832eb5af01952898a4c73f627d7aa5
Reviewed-on: https://go-review.googlesource.com/c/151678
Reviewed-by: Herbie Ong <herbie@google.com>
Add the scalar package to reduce dependencies on the v1 proto runtime package.
It may very well be the case that these functions should be exposed in the
public API of v2, but that is not a decision we need to make now.
Change-Id: Ifbc6d15311ba5837909ac72af47c630a80a142ef
Reviewed-on: https://go-review.googlesource.com/c/151402
Reviewed-by: Herbie Ong <herbie@google.com>
The marshaler, unmarshaler, and sizer functions are unused ever since
the underlying implementation was switched to be table-driven.
Change the function to only return the wrapper structs.
This change:
* enables generated protos to drop dependencies on certain proto types
* reduces the size of generated protos
* simplifies the implementation of oneofs in protoc-gen-go
Updates #708
Change-Id: I845c9009bc0236d1b51d34b014dc3e184303c0f2
Reviewed-on: https://go-review.googlesource.com/c/151357
Reviewed-by: Damien Neil <dneil@google.com>
Clearly specify that Get on an unpopulated field:
* returns the default value for scalars
* returns a mutable (but empty) List for repeated fields
* returns a mutable (but empty) Map for map fields
* returns an invalid value for message fields
The difference in semantics between List+Maps and Messages is because
protobuf semantics provide no distinction between an unpopulated and empty list
or map. On the other hand, there is a semantic difference between an unpopulated
message and an empty message.
Default values for scalars is trivial to implement with FieldDescriptor.Default.
A mutable, but empty List and Map is easy to implement for known fields since
known fields are generated as a slice or map field in a struct.
Since struct fields are addressable, the implementation can just return a
reference to the slice or map.
Repeated, extension fields are a little more tricky since extension fields
are implemented under the hood as a map[FieldNumber]Extension.
Rather than allocating an empty list in KnownFields.Get upon first retrieval
(which presents a race), delegate the work to ExtensionFieldTypes.Register,
which must occur before any Get operation. Register is not a concurrent-safe
operation, so that is an excellent time to initilize empty lists.
The implementation of extensions will need to be careful that Clear on a repeated
field simply truncates it zero instead of deleting the object.
For unpopulated messages, we return an invalid value, instead of the prior
behavior of returning a typed nil-pointer to the Go type for the message.
The approach is problematic because it assumes that
1) all messages are always implemented on a pointer reciever
2) a typed nil-pointer is an appropriate "read-only, but empty" message
These assumptions are not true of all message types (e.g., dynamic messages).
Change-Id: Ie96e6744c890308d9de738b6cf01d3b19e7e7c6a
Reviewed-on: https://go-review.googlesource.com/c/150319
Reviewed-by: Damien Neil <dneil@google.com>
The use of math/rand in serialization is to provide some form of instability
to the output to provide a clear signal to the user that the should not
depend on the the property of stability. However, it is reasonable that users
expect the output for these to be deterministic.
As such, add a detrand package that provides deterministic, yet unstable
randomization functionality.
Since this package hashes the binary, it does impose a small initialization cost:
Benchmark 100000 20712 ns/op 480 B/op 6 allocs/op
Change-Id: I232d0fea1789a4278079837a67ee2f63474a4364
Reviewed-on: https://go-review.googlesource.com/c/151340
Reviewed-by: Herbie Ong <herbie@google.com>
Make output deliberately unstable so users don't rely on exactness.
For multi-line output, add another extra random space after <key>: for
at most one field per message.
-- example --
key1: field1
key2: {
foo: bar
}
For single-line output, add another extra random space after a field per
message.
-- example --
key1:field1 key2:{foo:bar}
Change-Id: I3ab25d4d970fdebb88bbd9dd8fa6d73af84338ea
Reviewed-on: https://go-review.googlesource.com/c/150977
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The definition of Has for proto3 scalars is whether the value is non-zero.
Change-Id: I6aee92dd518d63a66515ad35da84b2be7aa22527
Reviewed-on: https://go-review.googlesource.com/c/150320
Reviewed-by: Herbie Ong <herbie@google.com>
Reviewed-by: Joe Tsai <joetsai@google.com>
Add more extensive tests to ensure that the reflective API works for both
enums and messages. We tests the situation where a v2 message has dependencies
on v1 messages and vice versa.
Change-Id: Ib85d465711728ae13743bea700b678d9dda5e85c
Reviewed-on: https://go-review.googlesource.com/c/149758
Reviewed-by: Herbie Ong <herbie@google.com>
The terminology Vector does not occur in protobuf documentation at all,
so we should rename the Go use of the term to something more recognizable.
As such, all instances that match the regexp "[Vv]ect(or)?" were replaced.
The C++ documentation uses the term "Repeated", which is a reasonable name.
However, the term became overloaded in 2014, when maps were added as a feature
and implementated under the hood as repeated fields. This is confusing as it
means "repeated" could either refer to repeated fields proper (i.e., explicitly
marked with the "repeated" label in the proto file) or map fields. In the case
of the C++ reflective API, this is not a problem since repeated fields proper
and map fields are interacted with through the same RepeatedField type.
In Go, we do not use a single type to handle both types of repeated fields:
1) We are coming up with the Go protobuf reflection API for the first time
and so do not need to piggy-back on the repeated fields API to remain backwards
compatible since no former usages of Go protobuf reflection exists.
2) Map fields are commonly represented in Go as the Go map type, which do not
preserve ordering information. As such it is fundamentally impossible to present
an unordered map as a consistently ordered list. Thus, Go needs two different
interfaces for lists and maps.
Given the above situation, "Repeated" is not a great term to use since it
refers to two different things (when we only want one of the meanings).
To distinguish between the two, we'll use the terms "List" and "Map" instead.
There is some precedence for the term "List" in the protobuf codebase
(e.g., "getRepeatedInt32List").
Change-Id: Iddcdb6b78e1e60c14fa4ca213c15f45e214b967b
Reviewed-on: https://go-review.googlesource.com/c/149657
Reviewed-by: Damien Neil <dneil@google.com>
Implement support for extension fields for messages that use the v1
data structures for extensions. The legacyExtensionFields type wraps a
v1 map to implement the v2 protoreflect.KnownFields interface.
Working on this change revealed a bug in the dynamic construction of
message types for protobuf messages that had cyclic dependencies (e.g.,
message Foo has a sub-field of message Bar, and Bar has a sub-field of Foo).
In such a situation, a deadlock occurs because initialization code depends on
the very initialization code that is currently running. To break these cycles,
we make some systematic changes listed in the following paragraphs.
Generally speaking, we separate the logic for construction and wrapping,
where constuction does not recursively rely on dependencies,
while wrapping may recursively inspect dependencies.
Promote the MessageType.MessageOf method as a standalone MessageOf function
that dynamically finds the proper *MessageType to use. We make it such that
MessageType only supports two forms of messages types:
* Those that fully implement the v2 API.
* Those that do not implement the v2 API at all.
This removes support for the hybrid form that was exploited by message_test.go
In impl/message_test.go, switch each message to look more like how future
generated messages will look like. This is done in reaction to the fact that
MessageType.MessageOf no longer exists.
In value/{map,vector}.go, fix Unwrap to return a pointer since the underlying
reflect.Value is addressable reference value, not a pointer value.
In value/convert.go, split the logic apart so that obtaining a v2 type and
wrapping a type as v2 are distinct operations. Wrapping requires further
initialization than simply creating the initial message type, and calling it
during initial construction would lead to a deadlock.
In protoreflect/go_type.go, we switch back to a lazy initialization of GoType
to avoid a deadlock since the user-provided fn may rely on the fact that
prototype.GoMessage returned.
Change-Id: I5dea00e36fe1a9899bd2ac0aed2c8e51d5d87420
Reviewed-on: https://go-review.googlesource.com/c/148826
Reviewed-by: Herbie Ong <herbie@google.com>
Add a corpus of generated protobuf messages generated at specific versions
of protoc-gen-go to ensure that we continue to support for generated messages
that have may never be updated.
Change-Id: I04a1b74306f471d7c99f5daf52399a5bd9adcbbc
Reviewed-on: https://go-review.googlesource.com/c/148831
Reviewed-by: Herbie Ong <herbie@google.com>
Rather than having the Converter carry a NewMessage method, have the struct
simply expose the MessageType or EnumType since they carry more information
and are retrieved anyways as part of the functionality of NewConverter.
While changing Converter, export the fields and remove all the methods.
Also, add an IsLegacy boolean, which is useful for the later implementation
of the extension fields.
Add a wrapLegacyEnum function which is used to wrap v1 enums as v2 enums.
We use this functionality in NewLegacyConverter to detrive the EnumType.
Additionally, modify wrapLegacyMessage to return a protoreflect.ProtoMessage
to be consistent with wrapLegacyEnum which must return a protoreflect.ProtoEnum.
Change-Id: Idc8989d07e4895d30de4ebc22c9ffa7357815cad
Reviewed-on: https://go-review.googlesource.com/c/148827
Reviewed-by: Herbie Ong <herbie@google.com>
The Go type descriptors protoreflect.{Enum,Message,Extension}Type are simple
wrappers over protoreflect.{Enum,Message,Extension}Descriptor with a small
number of additional methods. It is very unlikely that more will be added in
the near future.
For this reason, construct the types directly using arguments to the constructor
function, as opposed to taking in another struct (which was originally done
to provide flexibility in-case we needed more fields).
Furthmore, rename GoNew and New.
Change-Id: Ic7fb5bc250cdb2761ae03b388b5147ff50f37d15
Reviewed-on: https://go-review.googlesource.com/c/148822
Reviewed-by: Herbie Ong <herbie@google.com>
Follow the precedence of Go maps where deletion on a key without an entry in
the map is a noop. Similarly, document that the following methods are safe
to call with entries that do not exist:
* Map.Clear
* KnownFields.Clear
* ExtensionFieldTypes.Remove
Change the implementation for each of these to match the documented behavior.
Change-Id: Ifccff9b7b03baaeffdc366d05f6286ba60e14934
Reviewed-on: https://go-review.googlesource.com/c/148317
Reviewed-by: Herbie Ong <herbie@google.com>
