Keeping this consistent with textpb - golang.org/cl/168439.
Change-Id: Ie4c57615f4ac5eaf6b3836edaf7ae2082ebaae64
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171240
Reviewed-by: Damien Neil <dneil@google.com>
Go1.12 on GoLLVM with LLVM 9.0.0 has a bug where the simpler but
equivalent expression results in memory corruption.
Revert to the more complex expression to avoid breaking GoLLVM and
it is not worth waiting for a GoLLVM fix.
Change-Id: I8b42965ca9bd333deb8693021e7b22f966e13521
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171463
Reviewed-by: Damien Neil <dneil@google.com>
Stop adding options to legacy MessageDescriptors and FieldDescriptors.
We would synthesize options messages containing semantic options
(FieldOptions.is_packed, FieldOptions.is_weak, MessageOptions.map_entry).
This information is already contained in the protoreflect descriptors,
so there is no real need to define a correct options message.
If we do want to include options in legacy descriptors, then we should
get the original value out of the FileDescriptorProto, since it may
include additional extensions or other information.
This change completely removes the dependency from internal/legacy to
descriptor.proto.
Change-Id: Ib6bbe4ca6e0fe7ae501f3e9b11d5fa0222808410
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171458
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
We jump through many hoops to avoid generated protos depending on
internal/legacy. Break the cycle in the other direction: Remove
the dependency on descriptor.proto from internal/legacy by
using a hand-written parser for the few descriptor fields we need.
Still to do: Remove the descriptor.proto dependency from
internal/encoding/tag.
Change-Id: I5fd99a2170470ba8530eb2679b6dde899821bf3e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171457
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Add a method that provides efficiently querying for which member field
in a oneof is actually set. This is useful when dealing with oneofs
with many member fields.
Change-Id: I918b566c432f8bdd24dcecbb5501d231ffefef29
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170580
Reviewed-by: Damien Neil <dneil@google.com>
Seperate out the generation of the oneof wrapper types from the
code block that is about getter methods.
Change-Id: Ief44ef953d0b5ad8c998a8542c830ca70468a3bf
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171029
Reviewed-by: Damien Neil <dneil@google.com>
Run the encode/decode tests as benchmarks. These are trivial microbenchmarks
and not representative of real-world usage, but serve as a useful overview
of the relative cost of various operations and are handy for profiling
specific narrow scenarios.
When run with the -v1 flag, benchmark the v1 implementation for comparison.
Change-Id: I469dd6759bef50e2bd039327095f82d29d70b8fc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171120
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Using an option instead of a separate method has several useful properties:
It makes it explicit whether the fast-path AppendMarshal is expected to use
cached sizes or not.
It properly plumbs the decision to use cached sizes through the call stack.
Consider the case where message A includes B includes C: If A and C support
cached sizes but B does not, we would like to use the size cache in all
messages which support it. Placing this decision in the options allows this
to work properly with no additional effort.
Placing this option in MarshalOptions permits users to request use of
existing cached sizes. This is a two-edged sword: There are places where
this ability can permit substantial efficiencies, but this is also an
exceedingly sharp-edged API. I believe that on balance the benefits
outweigh the risks, especially since the prerequisites for using
cached sizes are intuitively obvious. (You must have called Size, and
you must not have changed the message.)
This CL adds a Size method to MarshalOptions, rather than adding a SizeOptions
type. Future additions to MarshalOptions may affect the size of the encoded
output (e.g., an option to skip encoding unknown fields) and using the same
options for both Marshal and Size makes it easier to use a consistent
configuration for each.
Change-Id: I6adbb55b717dd03d39f067e1d0b7381945000976
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171119
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Was overwriting the output buffer rather than appending to it.
Change-Id: I6ffb72a440f464f4259cfebc42c1dc75b73fb5ae
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171117
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Remove the Reflection field from MarshalOptions and UnmarshalOptions.
Disable the fast path and use the reflection-based implementation when
the 'protoreflect' build tag is set.
Change-Id: Ic674e3af67501de27fb03ec2712fbed40eae7fef
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170896
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Move all checks for required fields into a proto.IsInitialized function.
Initial testing makes me confident that we can provide a fast-path
implementation of IsInitialized which will perform more than
acceptably. (In the degenerate-but-common case where a message
transitively contains no required fields, this check can be nearly
zero cost.)
Unifying checks into a single function provides consistent behavior
between the wire, text, and json codecs.
Performing the check after decoding eliminates the wire decoder bug
where a split message is incorrectly seen as missing required fields.
Performing the check after decoding also provides consistent and
arguably more correct behavior when the target message was partially
prepopulated.
Change-Id: I9478b7bebb263af00c0d9f66a1f26e31ff553522
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170787
Reviewed-by: Herbie Ong <herbie@google.com>
In marshalMap, replace Map.Range and sortMap with
internal/mapsort.Range. Also, make sure to capture fatal error properly
for return.
In appendExtensions, capture fatal error properly for return. Added a
testcase that shows this was a bug.
In unmarshalAny, remove unnecessary checks and use internal/fieldnum for
Any's field numbers.
Change-Id: Id8574d5b4eb820ad961f6ad5e886f8ae2e9b90f0
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170627
Reviewed-by: Damien Neil <dneil@google.com>
So far only Ints is being used and it does not seem like the
other set types will ever be used. Remove them.
We can always add them back if we need them again.
Change-Id: I9a9e8ce76bd231d1fe5b726af7da690dc4019bb8
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170625
Reviewed-by: Herbie Ong <herbie@google.com>
Unmarshaling should fail if multiple fields in the same oneof exists in
the input.
Change-Id: I76efd88681a50c18f3eaf770c9eb48727efb412b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170517
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Allow message implementations to provide optimized versions of standard
operations. Generated messages now include a ProtoReflectMethods method,
returning a protoiface.Methods struct containing pointers to assorted
optional functions.
The Methods struct also includes a Flags field indicating support for
optional features such as deterministic marshaling.
Implementation of the fast paths (and tests) will come in later CLs.
Change-Id: Idd1beed0ecf43ec5e5e7b8da2ee1e08d3ce32213
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170340
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Change use of regular expression to manually parsing the value.
Allow value with + symbol in front, e.g. "+3s". Previous regex missed
this.
Do not allow values without numbers, e.g. "-s". Previous regex missed
this as well.
name old time/op new time/op delta
Unmarshal_Duration-4 1.96µs ± 0% 1.24µs ± 0% ~ (p=1.000 n=1+1)
name old alloc/op new alloc/op delta
Unmarshal_Duration-4 703B ± 0% 512B ± 0% ~ (p=1.000 n=1+1)
name old allocs/op new allocs/op delta
Unmarshal_Duration-4 20.0 ± 0% 17.0 ± 0% ~ (p=1.000 n=1+1)
Change-Id: I4db58d70f55607213631c49d698ee6a048b5e094
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170012
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
A JSON "null" field should set the NullValue enum field because
NullValue has the custom encoding format of "null".
Change-Id: I2bfa0900de64d7e2874f7c6db04b1cbc0b61b904
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170107
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Also added json.Decoder.Clone API for unmarshaling Any to look
ahead remaining bytes for @type field.
Change-Id: I2f803743534dfb64f9092d716805b115faa5975a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170102
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Minor changes:
* Use x as the receiver since "e" and "m" are meaningless in the presence
of user-defined enum and message names.
* Consistently keep enum methods together, rather awkwardly split apart
by the value maps.
Change-Id: I68e5666efb56ac7a4d062fb223b9f826dc72aba9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170357
Reviewed-by: Herbie Ong <herbie@google.com>
This reduces the init-time cost slightly since the GZIP'd
raw descriptor is constructed lazily on demand.
Change-Id: I482c6a2201b8786e425d7dee5612fdfd60ab1500
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169917
Reviewed-by: Herbie Ong <herbie@google.com>
The "custom" JSON representation for Empty is {}. And hence, when
embedded in Any, it needs to be represented as a field value in the
"value" field.
Discussion at https://github.com/protocolbuffers/protobuf/issues/5390.
As of this CL, code currently handles marshaling of Empty embedded
in Any, but not unmarshaling of Any yet. I'd like to get this corrected
first in order to proceed with unmarshaling of Any.
Change-Id: Ic0c321be188a979909b99d6b467aabc57c48e95c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169702
Reviewed-by: Damien Neil <dneil@google.com>
Added tests related to required fields and AllowPartial. I somehow
missed this before.
Change-Id: I3eb17347b1f3a99be3d65af06c4549abcc87ae39
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169701
Reviewed-by: Damien Neil <dneil@google.com>
I had defined these types at some point in order to reuse another
instance of these within, but it's not needed anymore and hence removing
them.
Change-Id: I8aa127326a5926c6a8688d83cce7ca83c160b39b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169700
Reviewed-by: Damien Neil <dneil@google.com>
Provide AllowPartial option to accept messages with missing required
field during marshaling and unmarshaling.
Change-Id: Ia23783870a8125633f8ddc0b686984b4c5ca15ba
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169500
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Rearranged well_known_types.go such that marshaling and unmarshaling
funcs for a particular type are closer, and added description for its
JSON representation.
Change-Id: Id9f5e4060158f66392a351debaffe1d7260919e4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169497
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Also, fixed unmarshaling of map messages where non-fatal errors were not
propagated up.
Change-Id: I06415b4a4ccd12135f0fdfaa38ccda54866139e7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168997
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This removes yet another set of dependencies of v2 on v1.
The only remaining dependency are in the _test.go files,
primarily for proto.Equal.
Changes made:
* cmd/protoc-gen-go no longer generates any functionality that depends
on the v1 package, and instead only depends on v2.
* internal/fileinit.FileBuilder.MessageOutputTypes is switched from
protoreflect.MessageType to protoimpl.MessageType since the
implementation must be fully inialized before registration occurs.
* The test for internal/legacy/file_test.go is switched to a legacy_test
package to avoid a cyclic dependency.
This requires Load{Enum,Message,File}Desc to be exported.
Change-Id: I43e2fe64cff4eea204258ce11e791aca5eb6e569
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169298
Reviewed-by: Damien Neil <dneil@google.com>
Description of message Value states:
`Value` represents a dynamically typed value which can be either null, a
number, a string, a boolean, a recursive struct value, or a list of values. A
producer of value is expected to set one of that variants, absence of any
variant indicates an error.
https://github.com/protocolbuffers/protobuf/blob/3.7.x/src/google/protobuf/struct.proto#L57-L60
Previous implementation was following C++ lib behavior.
Change-Id: Id51792e2fc8cc465a05a978e63410d3b6802b522
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168901
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Per Joe's suggestion, remove producing numberParts when parsing a JSON
number to produce corresponding Value. This saves having to store it
inside Value as well. Only produce numberParts for calls to
Value.{Int,Uint} call.
numberParts is only used for producing integers and removing the logic to
produce numberParts improves overall decoding speed for floats, and shows no
change for integers.
name old time/op new time/op delta
Float-4 559ns ± 0% 288ns ± 0% ~ (p=1.000 n=1+1)
Int-4 471ns ± 0% 466ns ± 0% ~ (p=1.000 n=1+1)
Change-Id: I21bf304ca67dda8d41a4ea0022dcbefd51058c1c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168781
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
As a goal, v2 should not depend on v1. As another step towards that end,
we move all the types that used to be in the v1 protoapi package over to v2.
For now, we place MessageV1, ExtensionRangeV1, and ExtensionDescV1
in runtime/protoiface since these are types that generated messages will
probably have to reference forever. An alternative location could be
reflect/protoreflect, but it seems unfortunate to have to dirty the
namespace of that package with these types.
We move ExtensionFieldV1, ExtensionFieldsV1, and ExtensionFieldsOf
to internal/impl, since these are related to the implementation of a
generated message.
Since moving these types from v1 to v2 implies that the v1 protoapi
package is useless, we update all usages of v1 protoapi in the v2
repository to point to the relevant v2 type or functionality.
CL/168538 is the corresponding change to alter v1.
There will be a temporary build failure as it is not possible
to submit CL/168519 and CL/168538 atomically.
Change-Id: Ide4025c1b6af5b7f0696f4b65b988b4d10a50f0b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168519
Reviewed-by: Herbie Ong <herbie@google.com>
The protobuf type system hacks the representation of map entries into that
of a pseudo-message descriptor.
Previously, we made all message descriptors implement MessageType
where type descriptors had a GoType method that simply returned nil.
Unfortunately, this violates a nice property in the Go type system
where being able to assert to a MessageType guarantees that Go type
information is truly associated with that descriptor.
This CL makes it such that message descriptors for map entries
do not implement MessageType.
Change-Id: I23873cb71fe0ab3c0befd8052830ea6e53c97ca9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168399
Reviewed-by: Damien Neil <dneil@google.com>
Remove unnecessary v1 wrapping. Change most tests to operate on message
Any directly.
Change-Id: I19bbca6c1af72894f6a292aab8ebd09e0301a260
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168517
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Generate field numbers for the well-known types,
so that encoding/jsonpb can benefit from them as well.
This CL fixes internal/cmd/generate-protos, which was silently failing
because the modulePath was not properly initialized. We fix this by
moving it to the start of the init function.
Change-Id: I87637176f29218cffa512b4baa49f39dae924061
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168497
Reviewed-by: Herbie Ong <herbie@google.com>
Also, changed MarshalOptions.Compact to Indent for consistency with v1
and to make compact as the default.
Change-Id: Id08aaa5ca5656f18e7925d2eabc0b6b055b1cebb
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168352
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This makes it consistent with jsonpb.MarshalOptions. This does change
the default to be in compact form.
Change-Id: I1b07f06f282c019b30f3f1cbb43f6c8cba18f385
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168405
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Changes:
* Modify protoc-gen-go to use a helper from internal/impl
to print the message as text.
* Add a helper function to internal/impl that calls v2 textpb.
* Modify encoding/textpb to avoid depending on descriptor proto,
which would cause an import cycle.
* Modify internal/fileinit to populate a pseudo-internal
method on MessageDescriptor to check whether the message should
use the message set wire format. We avoid adding this to the
main MessageDescriptor interface since message sets are a
deprecated proto1 feature that we should avoid promoting.
Change-Id: Ibaf79a563af695756f11ddc4db69b38e25a8f1a7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168439
Reviewed-by: Herbie Ong <herbie@google.com>
The v1 registration leaks the message types out to the proto package.
When doing that, it must ensure that the reflection data structures
for those types are properly initialized first. We achieve that by
doing v1 registration at the end of the reflection init function.
Change-Id: If6df18df59d05bad50ff39c2eff6beb19e7466cc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168348
Reviewed-by: Damien Neil <dneil@google.com>
