Correctly report the type we were looking for when panicking.
Previously would say: "invalid type: got *T, want *T".
Change-Id: I90ea0dd1fc64f3aec37a5a5828c1832fb0ab2887
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/176258
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Minor refactoring of impl.MessageType initialization: Pull the
information gathered about a message struct by makeKnownFieldsFunc out
into a struct that we can pass around.
At the moment, makeKnownFieldsFunc is the only user of this struct, but
this will simplify the table (un)marshaler.
Drop the 'specialByName' map indexing XXX_ fields because it currently
isn't used anywhere.
Change-Id: I992c9f490982a05f3919d7d4e08052e2ab54d44d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/176220
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The protobuf data model makes no distinction between unknown fields
that are within the extension field ranges or not. Now that we eagerly
unmarshal extensions, there is even less need for storing unknown
fields in the extension map. Instead, use the XXX_unrecognized field
exclusively for this purpose.
Change-Id: I673a7d6259fe9fdbdc295bcfa8252ef4da415343
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/175579
Reviewed-by: Damien Neil <dneil@google.com>
CL/172399 switches the v1 code to eagerly unmarshal extensions.
This CL does the equivalent for v2.
For the test, we simply switch from protoV1.Equal to protoV2.Equal,
since the v2 equal does not magically unmarshal raw extensions.
Change-Id: I6f64455b0a75bbc9a9a82108558641a29bd2b982
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/175838
Reviewed-by: Damien Neil <dneil@google.com>
Weak fields are an obsolete proto1 feature. They have been superseded
by extensions. However, some vestigial support for weak fields does
remain, mostly as Google-internal patches. (They aren't exciting;
extensions really do everything weak fields do in a cleaner and
more portable fashion.)
At the moment, the only visible impact of marking a field [weak=true]
is to exclude it from "internal/fileinit".FileBuilder.DependencyIndexes.
We want to preserve that behavior just in case we ever do add full weak
field support here.
Extend fileinit to look up message descriptors for weak fields in the
global registry. If the descriptor cannot be found, use a placeholder
instead.
Remove special-case handling of weak fields in the impl package. The
code generator doesn't do anything special for them, so they can be
treated as any other field.
Change-Id: Ifa2ee3d30d63680a0eeb59c66ebc9521f38fd660
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/175997
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Weak imports are added to Imports as placeholders even if they can be
found in the Files registry, so we have to look at the name rather than
the actual FileDescriptor.
Change-Id: I28f62e945f233119014e5e8cb1bcbde7dca831a7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/175897
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This covers most of the TODO around validation. I left open the ones
that we didn't have clear consensus on yet.
Change-Id: I336c53173ee8d7447558b1e3a0c1ef945e986cd5
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/175140
Reviewed-by: Joe Tsai <joetsai@google.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The protobuf documentation explicitly specifies (1<<29)-1 as the maximum
field number, but the C++ implementation itself has a special-case where it
allows field numbers up to MaxInt32 for MessageSet fields, but continues
to apply the former limit in all non-MessageSet cases.
To avoid complicated branching logic, we use the larger limit for all cases
if MessageSet is supported, otherwise, we impose the documented limit.
Change-Id: I710a2a21aa3beba161c3e6ca2f2ed9a266920a5a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/175817
Reviewed-by: Damien Neil <dneil@google.com>
Change the rewritten Go package name for the conformance protos to match
the basename of the directory (i.e., "conformance").
This is a trivial change, but avoids confusion when the package name and
import path don't match.
Change-Id: I9b091c78ce4a85f7051c55ee1a48ef6dbba68db6
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/174944
Reviewed-by: Herbie Ong <herbie@google.com>
The prototest package takes a message and exercises its implementation
of the protoreflect.Message interface in a variety of ways.
Change-Id: I4c150d1f1a6c41048f7e7f0d77efa4440ae8db0d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/174577
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Add support for basic equality comparison of messages.
Messages are equal if they have the same type and marshal to the
same bytes with deterministic serialization, with some exceptions:
- Messages with different registered extensions are unequal.
- NaN is not equal to itself.
Unlike the v1 Equal, a nil message is equal to an empty message of
the same type.
Change-Id: Ibabdadd8c767b801051b8241aeae1ba077e58121
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/174277
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Change use of regexp for matching literals true,false,null to simple
bytes comparison. Small gain from doing this.
Remove computing for position in Value as that is only needed in error
messages. In order to preserve ability to compute for position later,
store the original input in Value instead of just the slice containing
the value, however, need to also store the start index and size of the
parsed value.
Using benchmark in encoding/bench_test.go now shows faster time and less
memory usage than V1.
name old time/op new time/op delta
JSONEncode-4 30.3ms ± 1% 10.3ms ± 1% -66.02% (p=0.000 n=9+8)
JSONDecode-4 54.4ms ± 3% 18.9ms ± 2% -65.33% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
JSONEncode-4 10.3MB ± 0% 3.9MB ± 0% -61.74% (p=0.000 n=10+9)
JSONDecode-4 19.0MB ± 0% 3.6MB ± 0% -81.29% (p=0.000 n=10+9)
name old allocs/op new allocs/op delta
JSONEncode-4 465k ± 0% 64k ± 0% -86.30% (p=0.000 n=10+8)
JSONDecode-4 289k ± 0% 163k ± 0% -43.69% (p=0.000 n=10+9)
Change-Id: I0a3108d675d6442674facb065aaebd14051f6c5d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172662
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This feature seems to be used quite a bit, and the conformance tests
treat this as required, perhaps as a "required option" since the
developer guide states:
"Proto3 JSON parser should reject unknown fields by default but may
provide an option to ignore unknown fields in parsing."
Also, all invalid UTF-8 errors in skipped values are also returned as it
is similar to a parse error, except it is a non-fatal one.
Change-Id: Ia26e9a355daecdbf99af23f3061353fffa32d47d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/174017
Reviewed-by: Damien Neil <dneil@google.com>
According to IEEE-754, equality on floats reports true for both +0 and -0.
However, this definition causes us to lose information. Instead, we want
a definition that checks for exact equality with +0.
Otherwise, we would lose the ability to serialize -0.
Change-Id: I36450c24258fc4f856bfd4bc4c53a90199b216b9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172970
Reviewed-by: Herbie Ong <herbie@google.com>
CL/172238 added equivalent methods without the Type suffix,
while keeping the old methods.
CL/172582 updates the v1 codebase to use the new methods.
This CL removes the methods with the Type suffix.
Change-Id: Iaaaa4fff11cac1694735657db2e5fd7cadc90afe
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/173138
Reviewed-by: Herbie Ong <herbie@google.com>
The protobuf type system uses the word "descriptor" instead of "type".
We should avoid the "type" verbage when we aren't talking about Go types.
The old names are temporarily kept around for compatibility reasons.
Change-Id: Icc99c913528ead011f7a74aa8399d9c5ec6dc56e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172238
Reviewed-by: Herbie Ong <herbie@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Previous calls to indexNeedEscape with a type conversion from []byte
to string incurs allocation.
Make 2 different calls instead, one for string and one for bytes.
Type converting string to []byte does not incur extra allocation,
however, the benchmark results still show it to be slower by ~3% for
textpb and 6+% for jsonpb, hence decided to go with 2 separate calls
instead.
Results over current head:
name old time/op new time/op delta
TextEncode-4 18.1ms ± 2% 18.3ms ± 2% ~ (p=0.065 n=10+9)
TextDecode-4 233ms ± 3% 102ms ± 1% -56.34% (p=0.000 n=9+10)
JSONEncode-4 10.4ms ± 2% 10.5ms ± 0% +0.56% (p=0.019 n=9+9)
JSONDecode-4 870ms ± 2% 354ms ± 4% -59.33% (p=0.000 n=9+10)
name old alloc/op new alloc/op delta
TextEncode-4 28.9MB ± 0% 28.9MB ± 0% +0.00% (p=0.000 n=10+9)
TextDecode-4 1.16GB ± 0% 0.03GB ± 0% -97.44% (p=0.000 n=9+10)
JSONEncode-4 3.94MB ± 0% 3.94MB ± 0% +0.00% (p=0.000 n=10+10)
JSONDecode-4 3.35GB ± 0% 0.01GB ± 0% -99.83% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
TextEncode-4 73.5k ± 0% 73.5k ± 0% ~ (all equal)
TextDecode-4 278k ± 0% 255k ± 0% -8.26% (p=0.000 n=9+10)
JSONEncode-4 63.8k ± 0% 63.8k ± 0% ~ (all equal)
JSONDecode-4 247k ± 0% 210k ± 0% -14.92% (p=0.000 n=10+10)
Change-Id: Ibc64e9a7827ec1fffa213eb79f60497950203700
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172239
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
These deprecated features add a non-trivial amount of binary bloat.
Protect these under a constant to make it easy to patch them out.
Change-Id: I12cc33613c19ef60ceec4c4449d0cf3692835170
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172407
Reviewed-by: Damien Neil <dneil@google.com>
Using a named fields gives us the flexibility to change the underlying
representation of special fields without needing to regenerate user code.
We add a named type for ExtensionFields, UnknownFields, and SizeCache.
Change-Id: I107cf82899850ea76665310ce79def60f0f7ab97
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172402
Reviewed-by: Damien Neil <dneil@google.com>
The v2 implementation will eagerly unmarshal extension fields,
which means that the logic to synchronize the extensions fields
is no longer necessary. Using type aliases, we can ensure
impl.ExtensionsV1 is identical to the extensions map prior to
a4ab9ec5 from May 2016. This makes it such that we only have one
legacy implementation that we have to deal with.
Change-Id: I41b977e912fbb899c5bc4f055890d474b419429c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172240
Reviewed-by: Damien Neil <dneil@google.com>
Lower-casing the FileDescriptor variable provides for shorter and
more readable global variable names. This has a very minor benefit
that the read-only data section of a binary is slightly smaller
since function names do end up in the final binary.
Change-Id: I077364905e5c9adea69873b3ea580fddc68b9eb8
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172119
Reviewed-by: Herbie Ong <herbie@google.com>
This should be redundant with "go test ./...", and on pre-Go1.11 versions it
fails on directories that contain no non-test files.
Change-Id: I2c16cf3b26e67f3baa7732f0f7744aa547972e1a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172237
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Drop the protoreflect.FileDescriptor.DescriptorByName method.
Descriptor lookup will always happen through a protoregistry.Files, which
is more generally useful (it's rare that you want to find a descriptor in a
specific file, as opposed to a package which may be composed of multiple files).
Split protoregistry.Files descriptor lookup into individual per-type functions
(enum, message, extension, service), matching the preg.Types API.
Drop the ability to look up enum values, message fields, and service methods
for now. This can be easily added later if needed, and is trivial to implement
in user code. (e.g., look up the service and then consult sd.Methods.ByName().)
Change-Id: I2b3d8ef888921a8464ba1434eddab20c7d3a458e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172118
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Changes:
* Remove protoreflect.OptionsMessage and use protoreflect.ProtoMessage
now that the generated options natively implement reflection.
* Make registration of concrete option types the responsibility of
internal/fileinit since the init logic of descriptor.pb.go uses that.
* Remove equivalent logic in protoc-gen-go to special-case descriptor.
Change-Id: Id814651fafa4d888ff4532d59c6a4f9b68145157
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171465
Reviewed-by: Damien Neil <dneil@google.com>
Keeping this consistent with textpb - golang.org/cl/168439.
Change-Id: Ie4c57615f4ac5eaf6b3836edaf7ae2082ebaae64
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171240
Reviewed-by: Damien Neil <dneil@google.com>
Go1.12 on GoLLVM with LLVM 9.0.0 has a bug where the simpler but
equivalent expression results in memory corruption.
Revert to the more complex expression to avoid breaking GoLLVM and
it is not worth waiting for a GoLLVM fix.
Change-Id: I8b42965ca9bd333deb8693021e7b22f966e13521
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171463
Reviewed-by: Damien Neil <dneil@google.com>
Stop adding options to legacy MessageDescriptors and FieldDescriptors.
We would synthesize options messages containing semantic options
(FieldOptions.is_packed, FieldOptions.is_weak, MessageOptions.map_entry).
This information is already contained in the protoreflect descriptors,
so there is no real need to define a correct options message.
If we do want to include options in legacy descriptors, then we should
get the original value out of the FileDescriptorProto, since it may
include additional extensions or other information.
This change completely removes the dependency from internal/legacy to
descriptor.proto.
Change-Id: Ib6bbe4ca6e0fe7ae501f3e9b11d5fa0222808410
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171458
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
We jump through many hoops to avoid generated protos depending on
internal/legacy. Break the cycle in the other direction: Remove
the dependency on descriptor.proto from internal/legacy by
using a hand-written parser for the few descriptor fields we need.
Still to do: Remove the descriptor.proto dependency from
internal/encoding/tag.
Change-Id: I5fd99a2170470ba8530eb2679b6dde899821bf3e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171457
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Add a method that provides efficiently querying for which member field
in a oneof is actually set. This is useful when dealing with oneofs
with many member fields.
Change-Id: I918b566c432f8bdd24dcecbb5501d231ffefef29
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170580
Reviewed-by: Damien Neil <dneil@google.com>
Seperate out the generation of the oneof wrapper types from the
code block that is about getter methods.
Change-Id: Ief44ef953d0b5ad8c998a8542c830ca70468a3bf
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171029
Reviewed-by: Damien Neil <dneil@google.com>
Run the encode/decode tests as benchmarks. These are trivial microbenchmarks
and not representative of real-world usage, but serve as a useful overview
of the relative cost of various operations and are handy for profiling
specific narrow scenarios.
When run with the -v1 flag, benchmark the v1 implementation for comparison.
Change-Id: I469dd6759bef50e2bd039327095f82d29d70b8fc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171120
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Using an option instead of a separate method has several useful properties:
It makes it explicit whether the fast-path AppendMarshal is expected to use
cached sizes or not.
It properly plumbs the decision to use cached sizes through the call stack.
Consider the case where message A includes B includes C: If A and C support
cached sizes but B does not, we would like to use the size cache in all
messages which support it. Placing this decision in the options allows this
to work properly with no additional effort.
Placing this option in MarshalOptions permits users to request use of
existing cached sizes. This is a two-edged sword: There are places where
this ability can permit substantial efficiencies, but this is also an
exceedingly sharp-edged API. I believe that on balance the benefits
outweigh the risks, especially since the prerequisites for using
cached sizes are intuitively obvious. (You must have called Size, and
you must not have changed the message.)
This CL adds a Size method to MarshalOptions, rather than adding a SizeOptions
type. Future additions to MarshalOptions may affect the size of the encoded
output (e.g., an option to skip encoding unknown fields) and using the same
options for both Marshal and Size makes it easier to use a consistent
configuration for each.
Change-Id: I6adbb55b717dd03d39f067e1d0b7381945000976
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171119
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Was overwriting the output buffer rather than appending to it.
Change-Id: I6ffb72a440f464f4259cfebc42c1dc75b73fb5ae
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171117
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Remove the Reflection field from MarshalOptions and UnmarshalOptions.
Disable the fast path and use the reflection-based implementation when
the 'protoreflect' build tag is set.
Change-Id: Ic674e3af67501de27fb03ec2712fbed40eae7fef
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170896
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Move all checks for required fields into a proto.IsInitialized function.
Initial testing makes me confident that we can provide a fast-path
implementation of IsInitialized which will perform more than
acceptably. (In the degenerate-but-common case where a message
transitively contains no required fields, this check can be nearly
zero cost.)
Unifying checks into a single function provides consistent behavior
between the wire, text, and json codecs.
Performing the check after decoding eliminates the wire decoder bug
where a split message is incorrectly seen as missing required fields.
Performing the check after decoding also provides consistent and
arguably more correct behavior when the target message was partially
prepopulated.
Change-Id: I9478b7bebb263af00c0d9f66a1f26e31ff553522
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170787
Reviewed-by: Herbie Ong <herbie@google.com>
