The v1 implementation calls Marshal methods when deterministic
serialization is requested, even though it has no way to verify that the
method supports determinism. Preserve this behavior.
Change-Id: I383f2ec4bd4d5b996d96d604e92dfa43cb6f1bdc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/205719
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The resolverOnly type embeds a *protoregistry.Files and overrides one of
of its methods. When that method was renamed (Register -> RegisterFile),
the resolverOnly type continued to compile but no longer overrode the
desired method.
Update the method name, and change this type to explicitly forward
methods to the underlying *protoregistry.Files to catch errors of this
nature in the future.
Change-Id: I09a529034b6e2f310977bc635288fa15ce14cc7e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/205242
Reviewed-by: Joe Tsai <joetsai@google.com>
Remove repeated extension fields from the set of nullable fields,
so that Has reports false and Range does not visit a a zero-length
repeated extension field.
This corrects a fuzzer-detected case where unmarshaling and remarshaling
a wire-format message could result in a semantic change. For a repeated
extension field in non-packed encoding, unmarshaling a packed
representation of the field would result in a message which Has the
extension. Remarshaling it would discard the the field.
Fixesgolang.org/protobuf#975
Change-Id: Ie836559c93d218db5b5201742a3b8ebbaacf54ed
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/204897
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Joe Tsai <joetsai@google.com>
Add type-safe methods to register message, enum, and extension types.
Deprecate the NewTypes function and the (*Types).Register method.
Add (*File).RegisterFile and deprecate the NewFiles function and
the (*File).Register method.
Updates golang/protobuf#963
Change-Id: Ie89e77526e0874539e9bd929ca0ba8d758e65a6e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/199898
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Remove compiler flags that originally existed to assist the internal
divergences of protoc-gen-go. However, these ended up not being used
since the internal "patch" ended up being an entirely different fork.
Change-Id: I936d1db96bc7d737a2f2e5b90d2bd09fcc8c7b88
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/201738
Reviewed-by: Herbie Ong <herbie@google.com>
High-level API:
func Transform() cmp.Option
type Enum struct{ ... }
type Message map[string]interface{}
The Transform function transform messages into a Message type that
cmp.Equal and cmp.Diff then knows how to traverse and compare.
Change-Id: I445f3b5c69f054b6984f28c205cda69e44af3b89
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/164680
Reviewed-by: Damien Neil <dneil@google.com>
Test the fuzzer with a minimal seed corpus. (Currently one file
containing a valid TestAllTypes messge with most fields set.)
Change-Id: I8dcec4e26f1e8374993cc3a4bef0496f36cccd41
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/201639
Reviewed-by: Herbie Ong <herbie@google.com>
The MessageInfo cache, once set, must not be cleared, otherwise
there exists a *messageState value where the MessageInfo value is nil.
Fix the generation of the Reset method to avoid clearing this value.
Change-Id: Ic84ca8b2640a43e967c36993da1ccd3f2b7096c4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/201478
Reviewed-by: Damien Neil <dneil@google.com>
The Message.String method is intended for debugging,
so outputting <nil> for typed nil messages actually has use.
This matches the current behavior of Message.String
for messages generated by the v1 protoc-gen-go.
Change-Id: I6e31183961c83d7bce6338eb99aa7758cfda1ff4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/199397
Reviewed-by: Damien Neil <dneil@google.com>
Change tests which use private types registries to use the global one.
Except in cases where we want to explicitly test that the private
registry is used, it's simpler to use the global registry.
Change-Id: I998fb463b6beef91c7f5ce2ca2083251ae24d1db
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/199897
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Herbie Ong <herbie@google.com>
These methods are difficult or impossible to use correctly; types
created by the dynamicpb package, for example, all have the same GoType.
Fixesgolang/protobuf#938
Change-Id: I33d4ef381579ff18569b11df501d0ba7f38a6b5c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/199060
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The global registry is initialized via generated code.
The Go language guarantees that these are serialized (non concurrently).
The main concern is when a concurrent read operation occurs while
registration is still ongoing. In such a case, we do need a lock to
serialize the read with regard to the writes (i.e. registrations).
Change-Id: Ied35d6f8d2620f448cb281c3ec46d8de893b5671
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/199217
Reviewed-by: Damien Neil <dneil@google.com>
The protolegacy package is a minimal version of the v1 proto package.
This allows us to use this stub version as the dependency for
internal/testprotos/legacy packages and avoid a dependency
on the real v1 proto package.
The implementation of most v1 functionality will panic if called.
This way, we know if we the v2 code depends on one of those
unimplemented functions.
Updates golang/protobuf#962
Change-Id: I20b4091706fd456e4b01ae0931cce30a872639b0
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/199297
Reviewed-by: Damien Neil <dneil@google.com>
Fix a reversed error check in impl.Export{}.WeakNil.
Check to see if we have a type for the weak field on marshal/size.
Treat a typed nil valued in XXX_Weak as not indicating presence for
the field.
Change-Id: Id667ac7eb4f53236be9e181017082bd8cd21d115
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/198717
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Support, to some limited degree, types which implement protoV1.Message
but which are not struct pointers. Our ability to work with these types
is largely limited to calling Marshal or Unmarshal methods, when
present.
Change-Id: Ie1b851d9e753e2b2cb189b17ffeefebe2d8b3a8f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/198237
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
When allocating more space for the destination message in MarshalAppend,
use the same slice growth algorithm as the Go runtime's append rather
than allocating precisely the desired space.
Change-Id: If6033f6f7abdca473bc5188c4d3938ce57d3bdd2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/197758
Reviewed-by: Joe Tsai <joetsai@google.com>
Call the Marshal or Unmarshal method on legacy messages implementing
protoV1.Marshaler or protoV2.Unmarshaler.
We do this in the impl package by creating an appropriate function in
the protoiface.Methods struct for legacy messages.
In proto.MarshalAppend, return the bytes provided by the fast-path
marshal function even when the returned error is non-nil.
Fixesgolang/protobuf#955
Change-Id: I36924af9ff959a946c43f2295ef3202216e81b32
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/197357
Reviewed-by: Joe Tsai <joetsai@google.com>
Modify Equal to treat nil messages as equal iff both are nil.
Of special note, a typed nil pointer to T is equal to a new(T)
since they are indistinguishable from a protobuf reflection.
Change-Id: Ibf90b43a982e7376e07b4159be198f06230ec194
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/196618
Reviewed-by: Damien Neil <dneil@google.com>
While odd, it is possible to merge a message into itself.
In such a situation, the material impact is that repeated
and unknown fields are duplicated. The previous logic would
inifinite loop since the list iteration logic uses the current
length, but since the current length is ever growing, this loop
will never terminate. Instead, record the list length once
and iterate exactly that many times.
Change-Id: Ief98afa1b20bd950a9c2422d4462b170dbe6fa11
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/196857
Reviewed-by: Damien Neil <dneil@google.com>
Some existing targets (whether correctly or not) rely on it Merge
being safe to call concurrently so long as the set of fields being
merged are disjoint.
Change-Id: I4db9e64efccc7a2d44a5f9b52261b611cce461b0
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/196737
Reviewed-by: Damien Neil <dneil@google.com>
Add support for an ExtensionDesc with only Field populated as returned by
protoV1.ExtensionDescs. Rather than panicking when TypeDescriptor is called,
return a placeholder that preserves the name and/or number.
Change-Id: I60352a7aec8ccd8a0c1fb08db5891043a441695f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/193725
Reviewed-by: Damien Neil <dneil@google.com>
Suppose a oneof has N fields, the previous marshaling logic
would traverse every field checking for presence. This is O(N).
Using the protoreflect.Message.WhichOneof method, we can reduce
this to O(1). This optimization is exceptionally useful for oneofs
with a large number of fields.
Change-Id: I5f4aa8b1a899930f5c95e9cf1d68bac4b0b7884d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/196121
Reviewed-by: Herbie Ong <herbie@google.com>
This change performs two optimizations:
* It uses a pre-constructed rangeInfos slice to iterate over
all the fields. This is more performant since iterating over a slice
is faster than iterating over a map. Furthermore, this slice
does not contain fields that are part of a oneof. If a oneof has
N fields, the time to check presence on the oneof is now O(1)
instead of O(N).
* It uses a dense field info slice that is optmized for the common
case where the field number is relatively low and close in value
to the index itself.
We also fix a minor bug in the construction of oneofInfo where
it wasn't treating a typed nil pointer to a wrapper struct as if
it were unset. This ensures WhichOneof and Has always agree.
name old time/op new time/op delta
Reflect/Has-4 7.81µs ± 3% 6.74µs ± 3% -13.61% (p=0.000 n=9+9)
Reflect/Get-4 12.7µs ± 1% 11.3µs ± 4% -10.85% (p=0.000 n=8+10)
Reflect/Set-4 19.5µs ± 5% 17.8µs ± 2% -8.99% (p=0.000 n=10+10)
Reflect/Clear-4 12.0µs ± 4% 10.2µs ± 3% -14.86% (p=0.000 n=9+10)
Reflect/Range-4 6.58µs ± 1% 4.17µs ± 2% -36.65% (p=0.000 n=8+9)
Change-Id: I2c48b4d3fb6103ab238924950529ded0d37f8c8a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/196358
Reviewed-by: Damien Neil <dneil@google.com>
Ironically, the "real" protobuf name of a group is not the name
of the field descriptor, but the message descriptor.
Change-Id: I26ab546a94e934766fa6af6252cacd294442a221
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/195780
Reviewed-by: Herbie Ong <herbie@google.com>
We change Unmarshal to reset a message by default.
* We add a Merge option to UnmarshalOptions for explicit merging.
* We speed up Reset by checking for the Reset method.
* Remove TODOs in prototext and protojson about reset behavior.
Fixesgolang/protobuf#890
Change-Id: Ibd8963c741053f564acf061fbdb846699942109c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/195457
Reviewed-by: Damien Neil <dneil@google.com>
If an enum dependency is unresolved, there will be no enum values.
In such a case, just fall back to using number 0.
No tests because this situation is hard to replicate.
Change-Id: I41f5e1a5c6c8afd1ab6173d2dcfa33e611fa9560
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/195982
Reviewed-by: Damien Neil <dneil@google.com>
This CL adds support for discarding unknown fields from the input.
We add support for parsing and resolving field numbers, so that
the DiscardUnknown option can ignore all unresolvable fields.
We continue to reject known fields identified by field number
since there are a number of edge cases that a difficult to resolve.
Change-Id: I5c88b7bae8656ce20e85e4b5c92d8564a5ff8bb6
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/195779
Reviewed-by: Herbie Ong <herbie@google.com>
This changes text marshaling to avoid unknown fields by default
and instead adds an option so that unknown fields be emitted.
This ensures that the default marshal/unknown can round-trip.
Change-Id: I85c84ba6ab7916d538ec6bfd4e9d399a8fcba14e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/195778
Reviewed-by: Herbie Ong <herbie@google.com>
This CL:
* Make the meaning of impl/ExtensionInfo.goType consistent. Before,
it was sometimes a T and other times a []T depending on the current
state of initialization. Change it so that it is the constructor's
responsibility to pass in a []T if it is repeated.
* Make internal/filetype responsible for constructing a []T for
repeated extension fields.
* Makes filedesc/Extension.Cardinality one of the eagerly initialized
pieces of information since it is useful to internal/filetype.
* Unify ExtensionInfo.desc and ExtensionInfo.tdesc.ExtensionField,
which held the same information.
* Remove the internal implementation for impl.X.ExtensionDescFromType
since we are dropping support for this from v1.
Change-Id: Ie95c4de66cd674c1d886da4f63b133b7d763c7ef
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/195777
Reviewed-by: Damien Neil <dneil@google.com>
This is more consistent with the indent documentation:
If indent is a non-empty string, it causes every entry in a List or Message
to be preceded by the indent and trailed by a newline.
Since an empty message has no entries, there should be no newlines.
Change-Id: I5d57165aaf94ca6b184bb35bf05d5d68f5ee9dd5
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/194877
Reviewed-by: Herbie Ong <herbie@google.com>
The xi.init flag should not be set until after we have truly
initialized everything.
Change-Id: I43dcb300917145ce19a7199fa871acd0df325c6c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/194439
Reviewed-by: Damien Neil <dneil@google.com>
Call XXX_MessageName with best-effort. If it panics, oh-well.
Change-Id: I605ea074470b0c90b0bea8b36fa7d4a69368692d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/194598
Reviewed-by: Damien Neil <dneil@google.com>
There are currently at least 5 ways to derive Go package information:
* From the 'M' command-line flag.
* From the 'import_path' command-line flag.
* From the 'go_package' proto file option.
* From the proto package name in proto source file.
* From the path of the proto source file.
Technically, there are more than 5 ways since information can be derived
from a convoluted combination of all the methods.
We should move towards a sensible and consistent world where each
Go package information for each proto source file is:
* only derived from the command-line OR
* only derived from the proto source file itself.
It should never be derived from a mixture of methods.
In the future, all .proto source files will be required to have a
"go_package" option specified, unless the "M" command-line argument is
specified. If the "M" flag is given it takes precedence over 'go_package'.
This CL prints warnings if the user is generating proto files without
a "M" flag or "go_package" option specified. It suggests to the user
a "go_package" option that preserves the current semantics.
Change-Id: I5cf8d40a245146bb145b3b610d42f1bcd140b449
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/194158
Reviewed-by: Damien Neil <dneil@google.com>
Since the internal body calls ExtensionType.ValueOf, it seems that the
intent is for lazyExtensionValue.value to store a:
func() protoreflect.Value
instead of a:
func() interface{}
This seems more apparent given that GetValue returns a pref.Value.
Change-Id: I1679fe56088c20d5c8d36360e75dd773850da4c2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/193757
Reviewed-by: Damien Neil <dneil@google.com>
ProtoMessageV1(nil) now returns nil.
ProtoMessageV2(nil) now returns nil.
Note that the following continue to panic:
MessageOf(nil)
MessageDescriptorOf(nil)
MessageTypeOf(nil)
It may be reasonable for them to also return nil in the future.
Change-Id: Icc14857252d844eb6d4dbfe0c248cef22023e930
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/193758
Reviewed-by: Damien Neil <dneil@google.com>
Use a non-breaking space instead of two spaces to vary the output.
This keeps the mutated version aesthetically similar to the normal one.
Change-Id: Ib4ade2795004fe5b30e454e7e533e5a0e3a9ffa2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/194157
Reviewed-by: Herbie Ong <herbie@google.com>
This is meant to deter users from doing byte for byte comparison.
Change-Id: If005d2dc1eba45eaa4254171d2f247820db109e4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/194037
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Simply move logic into similar code block. Maintains the same logic.
Change-Id: I7b5a3f3d57f6102c7919cdc03dd105f08d21aca3
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/194039
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Use internal/detrand in the construction of our error messages.
This alters whether there is one or two spaces following the "proto:" prefix.
While it is easy for users to still work around this mutation,
sit at least forces them to write test infrastructure to more fuzzily
match on error strings.
Change-Id: I4ddca717526ee3fc4dbb1e0b36cfca8c6e0df36d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/194038
Reviewed-by: Herbie Ong <herbie@google.com>
UseProtoNames=true uses proto field name in JSON field names.
Change-Id: I23249dc1787d9735bef780b1ef8d294a9c55c043
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/193998
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Add option to marshal out all fields including unset ones except for
unset oneof fields and extension fields.
This is to make V2 compatible with V1's EmitDefaults option.
Change-Id: Ifa7bae48e82740b623c74f936bcbe9e66b11344a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/193759
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
