Ensure that EnumOf, EnumDescriptorOf, EnumTypeOf, ProtoMessageV1Of,
ProtoMessageV2Of, MessageOf, MessageDescriptorOf, and MessageTypeOf
all return nil if passed a nil interface.
This parallels the behavior of reflect.TypeOf or reflect.ValueOf,
which return nil or an invalid value rather than panicking.
Change-Id: I461f15542f16cb0922d627bca6fcad5fc27d87e2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213239
Reviewed-by: Damien Neil <dneil@google.com>
The JSONName constructor returns a struct value which shallow copies
a sync.Once within it; this is a dubious pattern.
Instead, add a jsonName.Init method to initialize the value.
Change-Id: I190a7239b1b62a8041ee7e4e09c0fe37b64ff623
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213237
Reviewed-by: Damien Neil <dneil@google.com>
The size calculation for packed repeated extension fields was
considering a zero-length list as encoding to a zero-length
wire.BytesType field, rather than being omitted entirely.
Change-Id: I7d4424a21ca8afd4fa81391caede49cadb4e2505
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/212297
Reviewed-by: Joe Tsai <joetsai@google.com>
All the fuzzers have the same test, which runs the fuzzer against every
entry in the corpus. Move the test logic into a separate package.
Change-Id: I3a7e2ca75d20a5ff6d51ed9e6151629e6667684b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/212258
Reviewed-by: Joe Tsai <joetsai@google.com>
Add a new Fuzz message containing all the message types we want to make
available to fuzzers. Previously, testing (for example) required fields
would require modifying the fuzzer; now, it's just a matter of adding a
message with required fields as a field of the top-level Fuzz message.
Add internal/cmd/generate-corpus to codify where the fuzz seed corpus
comes from. This will simplify adding text and json fuzzers.
Rename internal/fuzz/wire to internal/fuzz/wirefuzz to minimize package
name ambiguity. Also, the addition of the Fuzz container message
invalidates the existing corpus, so using a new name seems like a good
idea.
Change-Id: I94f8f64ba93596c8e8cecb4d42bcc5b98c17d838
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/212218
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Fix a bug in handling the case where the encoding for a map item places
the value field (2) before the key field (1).
Change-Id: I2e6ad9af729a199e960e566ed7ef96bba3726990
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/211804
Reviewed-by: Joe Tsai <joetsai@google.com>
Move the test inputs for the wire marshaler and unmarshaler out of
decode_test.go and into a new file. Consolidate some tests for invalid
messages (UTF-8 validation failures, field numbers out of range) into
a single list of invalid messages. Break out the no-enforce-utf8 test
into a separate file, since it is both complicated and conditional on
legacy support.
Change-Id: Ide80fa9d3aec2b6d42a57e6f9265358aa5e661a7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/211557
Reviewed-by: Joe Tsai <joetsai@google.com>
In very old messages predating the existence of the size cache or the
proto3 unknown fields, it is possible that the generated struct lacks both
XXX_ fields and ones tagged with "protobuf". This can happen with a message
that only contains oneofs. As such, check for the "protobuf_oneof" tag as well.
Change-Id: I1981cd7dde68aece1a013356b6bc91cc5529f951
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/210747
Reviewed-by: Damien Neil <dneil@google.com>
These are not necessary now that weak fields are unexported.
Change-Id: Ida18b984abedfdf52fd3d5f3cb2f4ca580659a5c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/210745
Reviewed-by: Damien Neil <dneil@google.com>
When loading a *MessageInfo for a legacy message type, check to see if
the Go type contains at least one field which looks like a message
field. Specifically, look for at least one field with a `protobuf:` tag,
or an XXX_unrecognized field.
If a message has no recognizable fields, assume that it's something we
don't know how to interpret and treat it as an aberrant message.
Change-Id: If5c09087f1a0187271c98539d761395a2ee70a9e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/210617
Reviewed-by: Joe Tsai <joetsai@google.com>
The conditional was accidentally inverted.
This function provides dubious support for encoding/json.
Change-Id: Ib4131a229afa14d9aef1ad31fec51f4dac417a3b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/210638
Reviewed-by: Damien Neil <dneil@google.com>
Both the generator and the runtime need to agree upon the names of
specialized Go struct fields. Centralize that information in an
internal genname package.
In the mean time, also change the XXX_weak field name to match
the name used internally at Google.
Change-Id: I026bf354418c363482e5902f21aa5e0cacae24b0
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/207080
Reviewed-by: Damien Neil <dneil@google.com>
Change size, marshal, and isinit operations on oneofs to look up the
currently-set oneof type in a map rather than testing for each possible
oneof field in turn.
Significantly improves oneof encoding speed for oneofs with a
substantial number of fields:
go test ./proto -bench=./oneof.*string.*test.TestAll -benchmem -count=8 -cpu=1
name old time/op new time/op delta
Encode/oneof_(string)_(*test.TestAllTypes) 911ns ± 1% 397ns ± 3% -56.45% (p=0.000 n=8+7)
Decode/oneof_(string)_(*test.TestAllTypes) 899ns ± 1% 922ns ± 1% +2.49% (p=0.001 n=7+7)
Fixesgolang/protobuf#950
Change-Id: I9393a87975ce09011d885a8af4a63a639ea8452f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/210281
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Stash fast-path information for extensions on the ExtensionInfo. In
the usual case where an ExtensionType's underlying implementation is
an *ExtensionInfo, fetching the fast-path information becomes a type
assertion rather than a mutex-guarded map access.
Maintain a global sync.Map for the case where an ExtensionType isn't an
*ExtensionInfo.
Substantially improves performance for fast-path operations on
extensions:
Encode/MessageSet_type_id_before_message_content-12 267ns ± 1% 185ns ± 1% -30.44% (p=0.001 n=7+7)
Encode/basic_scalar_types_(*test.TestAllExtensions)-12 1.94µs ± 1% 0.40µs ± 1% -79.32% (p=0.000 n=8+7)
Change-Id: If048b521deb3665a090ea3d0a178c61691d4201e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/210540
Reviewed-by: Joe Tsai <joetsai@google.com>
The deprecated messageset format permits extension fields with numbers
greater than the usual maximum (1<<29-1). To support this, the
internal/encoding/wire package has disabled field number validation when
legacy support is enabled.
We shouldn't skip validating all field numbers for validity just because
we support larger ones in messagesets.
This change drops range validation from the wire package (other than
checking that numbers fit in an int32) and adds it to the wire
unmarshalers instead. This gives us validation where we care
about it (when unmarshaling a wire-format message) and allows for
best-effort handling of out-of-range numbers everywhere else.
Fixesgolang/protobuf#996
Change-Id: I4e11b8a8aa177dd60e89723570af074a317c2451
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/210290
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The fast-path decoder decodes zero-length repeated bytes values as
non-nil, zero-length []bytes. Do the same in the reflection decoder.
This isn't really a correctness issue, since there's no ambiguity about what a
nil entry in a [][]byte means. Still a good idea for consistency, and
retains v1 behavior.
Change-Id: Icd2cb726d14ff1f2b9f142e65756777a359971f3
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/210257
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The protoreflect documentation states that Get on an empty repeated or
map field returns an invalid (empty, read-only) List or Map. We weren't
doing this; fix it.
The documentation also states that an invalid List or Map may not be
assigned via Set. We were permitting Set with an invalid map; fix this.
Add tests for this behavior in prototest.
Change-Id: I4678af532e192210af0bde7c96a1439a4cd26efa
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/209019
Reviewed-by: Joe Tsai <joetsai@google.com>
Various protoreflect methods can return an "empty, read-only" message,
list, or map value. Provide a method to test if a value is one of these.
Fixesgolang/protobuf#966
Change-Id: I793d8426d6e2201755983c06f024412a7e09bc4c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/209018
Reviewed-by: Joe Tsai <joetsai@google.com>
This CL adds the following helper options:
func IgnoreEnums(...protoreflect.Enum) cmp.Option
func IgnoreMessages(...proto.Message) cmp.Option
func IgnoreFields(proto.Message, ...protoreflect.Name) cmp.Option
func IgnoreOneofs(proto.Message, ...protoreflect.Name) cmp.Option
func IgnoreDescriptors(...protoreflect.Descriptor) cmp.Option
func IgnoreDefaultScalars() cmp.Option
func IgnoreEmptyMessages() cmp.Option
func IgnoreUnknown() cmp.Option
It also augments transformMessage to unmarshal and expand Any messages
with the value of the underlying message. At this moment in time
we do not provide an API to provide a custom type resolver.
Change-Id: I51e1d9ff0d56d71161e510f366a7dcc32236d760
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/204577
Reviewed-by: Damien Neil <dneil@google.com>
Accessing Number as a method, rather than a field paves the
way to have Enum potentially implement protoreflect.Enum
in the future.
Change-Id: Iebe9c0ec12e067decf2121d12fe2fb1549477b32
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/207077
Reviewed-by: Damien Neil <dneil@google.com>
Simplify the implementation of makeStructInfo by checking the
names of the internal fields in a switch statement.
Also, add "weakFields" as the unexported name for weak fields
in preparation for actually unexporting it.
Change-Id: Ide970a39e9caa5a24bc288ba3e3a0d223a6bfcb6
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/207057
Reviewed-by: Damien Neil <dneil@google.com>
The v1 wire marshaler sorts fields as follows:
- All extensions, sorted by field number.
- All non-oneof fields, sorted by field number.
- All oneof fields, in indeterminate order.
We already make some steps toward supporting this ordering: The
fast path encoder places extensions in sorted order at the start
of the message.
This commit moves oneof fields to the end of the message, makes the
reflection-based encoder use this ordering when deterministic marshaling
is enabled, and adds a test to catch unintentional changes to the
ordering.
Users SHOULD NOT depend on stability of the marshal output. It is
subject to change over time. Without deterministic marshaling enabled,
it is subject to change over calls to Marshal.
Change-Id: I6cfd89090d790a3bb50785f32b94d2781d7d08db
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/206800
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
When creating a MessageDescriptor for a legacy message with a
Descriptor method, we call that method on the type's zero value to
get the message's DescriptorProto. Some existing dynamic message
types have a Descriptor method which panics in this case.
Catch the panic and continue as if the Descriptor method wasn't present.
Change-Id: I98d4625d6917cc1ec25737e5670a443f5d02a404
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/206637
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
In the v1 implementation, unknown MessageSet items are stored in a
message's unknown fields section in non-MessageSet format. For example,
consider a MessageSet containing an item with type_id T and value V.
If the type_id is not resolvable, the item will be placed in the unknown
fields as a bytes-valued field with number T and contents V. This
conversion is then reversed when marshaling a MessageSet containing
unknown fields.
Preserve this behavior in v2.
One consequence of this change is that actual unknown fields in a
MessageSet (any field other than 1) are now discarded. This matches
the previous behavior.
Change-Id: I3d913613f84e0ae82481078dbc91cb25628651cc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/205697
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Remove previously deprecated types, functions, and methods.
Update github.com/golang/protobuf module version to one which does not
depend on any deprecated APIs.
Fixexs golang/protobuf#963
Change-Id: Ida451ef5ef3f34830808f737cc0d1c98f32ce76a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/206017
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Avoid a deadlock when registering a legacy ExtensionType, caused by
initialization of the "internal/impl".ExtensionInfo calling IsMessageSet
on the MessageDescriptor of the type being extended.
We can avoid this deadlock either by initializing the ExtensionType
outside of the GlobalTypes mutex, or by moving IsMessageSet to L1
initialization of the MessageDescriptor so that it doesn't trigger lazy
init.
CL 204804 takes the former approach; this CL takes the latter.
Change-Id: Idfc1ed36a23a139839290ea32492142a17f68cf5
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/205957
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The v1 implementation calls Marshal methods when deterministic
serialization is requested, even though it has no way to verify that the
method supports determinism. Preserve this behavior.
Change-Id: I383f2ec4bd4d5b996d96d604e92dfa43cb6f1bdc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/205719
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The resolverOnly type embeds a *protoregistry.Files and overrides one of
of its methods. When that method was renamed (Register -> RegisterFile),
the resolverOnly type continued to compile but no longer overrode the
desired method.
Update the method name, and change this type to explicitly forward
methods to the underlying *protoregistry.Files to catch errors of this
nature in the future.
Change-Id: I09a529034b6e2f310977bc635288fa15ce14cc7e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/205242
Reviewed-by: Joe Tsai <joetsai@google.com>
Remove repeated extension fields from the set of nullable fields,
so that Has reports false and Range does not visit a a zero-length
repeated extension field.
This corrects a fuzzer-detected case where unmarshaling and remarshaling
a wire-format message could result in a semantic change. For a repeated
extension field in non-packed encoding, unmarshaling a packed
representation of the field would result in a message which Has the
extension. Remarshaling it would discard the the field.
Fixesgolang.org/protobuf#975
Change-Id: Ie836559c93d218db5b5201742a3b8ebbaacf54ed
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/204897
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Joe Tsai <joetsai@google.com>
Add type-safe methods to register message, enum, and extension types.
Deprecate the NewTypes function and the (*Types).Register method.
Add (*File).RegisterFile and deprecate the NewFiles function and
the (*File).Register method.
Updates golang/protobuf#963
Change-Id: Ie89e77526e0874539e9bd929ca0ba8d758e65a6e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/199898
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Remove compiler flags that originally existed to assist the internal
divergences of protoc-gen-go. However, these ended up not being used
since the internal "patch" ended up being an entirely different fork.
Change-Id: I936d1db96bc7d737a2f2e5b90d2bd09fcc8c7b88
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/201738
Reviewed-by: Herbie Ong <herbie@google.com>
High-level API:
func Transform() cmp.Option
type Enum struct{ ... }
type Message map[string]interface{}
The Transform function transform messages into a Message type that
cmp.Equal and cmp.Diff then knows how to traverse and compare.
Change-Id: I445f3b5c69f054b6984f28c205cda69e44af3b89
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/164680
Reviewed-by: Damien Neil <dneil@google.com>
Test the fuzzer with a minimal seed corpus. (Currently one file
containing a valid TestAllTypes messge with most fields set.)
Change-Id: I8dcec4e26f1e8374993cc3a4bef0496f36cccd41
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/201639
Reviewed-by: Herbie Ong <herbie@google.com>
The MessageInfo cache, once set, must not be cleared, otherwise
there exists a *messageState value where the MessageInfo value is nil.
Fix the generation of the Reset method to avoid clearing this value.
Change-Id: Ic84ca8b2640a43e967c36993da1ccd3f2b7096c4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/201478
Reviewed-by: Damien Neil <dneil@google.com>
The Message.String method is intended for debugging,
so outputting <nil> for typed nil messages actually has use.
This matches the current behavior of Message.String
for messages generated by the v1 protoc-gen-go.
Change-Id: I6e31183961c83d7bce6338eb99aa7758cfda1ff4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/199397
Reviewed-by: Damien Neil <dneil@google.com>
Change tests which use private types registries to use the global one.
Except in cases where we want to explicitly test that the private
registry is used, it's simpler to use the global registry.
Change-Id: I998fb463b6beef91c7f5ce2ca2083251ae24d1db
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/199897
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Herbie Ong <herbie@google.com>
These methods are difficult or impossible to use correctly; types
created by the dynamicpb package, for example, all have the same GoType.
Fixesgolang/protobuf#938
Change-Id: I33d4ef381579ff18569b11df501d0ba7f38a6b5c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/199060
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The global registry is initialized via generated code.
The Go language guarantees that these are serialized (non concurrently).
The main concern is when a concurrent read operation occurs while
registration is still ongoing. In such a case, we do need a lock to
serialize the read with regard to the writes (i.e. registrations).
Change-Id: Ied35d6f8d2620f448cb281c3ec46d8de893b5671
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/199217
Reviewed-by: Damien Neil <dneil@google.com>
The protolegacy package is a minimal version of the v1 proto package.
This allows us to use this stub version as the dependency for
internal/testprotos/legacy packages and avoid a dependency
on the real v1 proto package.
The implementation of most v1 functionality will panic if called.
This way, we know if we the v2 code depends on one of those
unimplemented functions.
Updates golang/protobuf#962
Change-Id: I20b4091706fd456e4b01ae0931cce30a872639b0
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/199297
Reviewed-by: Damien Neil <dneil@google.com>
