Temporarily remove go.mod, since we can't generate an accurate one until
the corresponding v1 change is submitted.
Change-Id: I1e1ad97f2b455e33f61ffaeb8676289795e47e72
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/177000
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Added API:
FieldDescriptor.IsExtension
FieldDescriptor.IsList
FieldDescriptor.MapKey
FieldDescriptor.MapValue
FieldDescriptor.ContainingOneof
FieldDescriptor.ContainingMessage
Deprecated API (to be removed in subsequent CL):
FieldDescriptor.Oneof
FieldDescriptor.Extendee
These methods help cleanup several common usage patterns.
Change-Id: I9a3ffabc2edb2173c536509b22f330f98bba7cf3
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/176977
Reviewed-by: Damien Neil <dneil@google.com>
Added methods:
Enum.Descriptor
Message.Descriptor
EnumType.Descriptor
MessageType.Descriptor
ExtensionType.Descriptor
Message.New
All functionality is switched over to use those methods instead of
implicitly relying on the fact that {Enum,Message}Type implicitly
implement the associated descriptor interface.
This CL does not yet remove {Enum,Message}.Type or prevent
{Enum,Message,Extension}Type from implementating a descriptor.
That is a subsequent CL.
The Message.New method is also added to replace functionality
that will be lost when the Type methods are removed.
Change-Id: I7fefde1673bbd40bfdac489aca05cec9a6c98eb1
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/174918
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Herbie Ong <herbie@google.com>
This feature seems to be used quite a bit, and the conformance tests
treat this as required, perhaps as a "required option" since the
developer guide states:
"Proto3 JSON parser should reject unknown fields by default but may
provide an option to ignore unknown fields in parsing."
Also, all invalid UTF-8 errors in skipped values are also returned as it
is similar to a parse error, except it is a non-fatal one.
Change-Id: Ia26e9a355daecdbf99af23f3061353fffa32d47d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/174017
Reviewed-by: Damien Neil <dneil@google.com>
The protobuf type system uses the word "descriptor" instead of "type".
We should avoid the "type" verbage when we aren't talking about Go types.
The old names are temporarily kept around for compatibility reasons.
Change-Id: Icc99c913528ead011f7a74aa8399d9c5ec6dc56e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172238
Reviewed-by: Herbie Ong <herbie@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Using a named fields gives us the flexibility to change the underlying
representation of special fields without needing to regenerate user code.
We add a named type for ExtensionFields, UnknownFields, and SizeCache.
Change-Id: I107cf82899850ea76665310ce79def60f0f7ab97
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172402
Reviewed-by: Damien Neil <dneil@google.com>
Lower-casing the FileDescriptor variable provides for shorter and
more readable global variable names. This has a very minor benefit
that the read-only data section of a binary is slightly smaller
since function names do end up in the final binary.
Change-Id: I077364905e5c9adea69873b3ea580fddc68b9eb8
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172119
Reviewed-by: Herbie Ong <herbie@google.com>
Keeping this consistent with textpb - golang.org/cl/168439.
Change-Id: Ie4c57615f4ac5eaf6b3836edaf7ae2082ebaae64
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171240
Reviewed-by: Damien Neil <dneil@google.com>
Go1.12 on GoLLVM with LLVM 9.0.0 has a bug where the simpler but
equivalent expression results in memory corruption.
Revert to the more complex expression to avoid breaking GoLLVM and
it is not worth waiting for a GoLLVM fix.
Change-Id: I8b42965ca9bd333deb8693021e7b22f966e13521
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171463
Reviewed-by: Damien Neil <dneil@google.com>
Seperate out the generation of the oneof wrapper types from the
code block that is about getter methods.
Change-Id: Ief44ef953d0b5ad8c998a8542c830ca70468a3bf
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171029
Reviewed-by: Damien Neil <dneil@google.com>
Move all checks for required fields into a proto.IsInitialized function.
Initial testing makes me confident that we can provide a fast-path
implementation of IsInitialized which will perform more than
acceptably. (In the degenerate-but-common case where a message
transitively contains no required fields, this check can be nearly
zero cost.)
Unifying checks into a single function provides consistent behavior
between the wire, text, and json codecs.
Performing the check after decoding eliminates the wire decoder bug
where a split message is incorrectly seen as missing required fields.
Performing the check after decoding also provides consistent and
arguably more correct behavior when the target message was partially
prepopulated.
Change-Id: I9478b7bebb263af00c0d9f66a1f26e31ff553522
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170787
Reviewed-by: Herbie Ong <herbie@google.com>
In marshalMap, replace Map.Range and sortMap with
internal/mapsort.Range. Also, make sure to capture fatal error properly
for return.
In appendExtensions, capture fatal error properly for return. Added a
testcase that shows this was a bug.
In unmarshalAny, remove unnecessary checks and use internal/fieldnum for
Any's field numbers.
Change-Id: Id8574d5b4eb820ad961f6ad5e886f8ae2e9b90f0
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170627
Reviewed-by: Damien Neil <dneil@google.com>
Unmarshaling should fail if multiple fields in the same oneof exists in
the input.
Change-Id: I76efd88681a50c18f3eaf770c9eb48727efb412b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170517
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Allow message implementations to provide optimized versions of standard
operations. Generated messages now include a ProtoReflectMethods method,
returning a protoiface.Methods struct containing pointers to assorted
optional functions.
The Methods struct also includes a Flags field indicating support for
optional features such as deterministic marshaling.
Implementation of the fast paths (and tests) will come in later CLs.
Change-Id: Idd1beed0ecf43ec5e5e7b8da2ee1e08d3ce32213
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170340
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Change use of regular expression to manually parsing the value.
Allow value with + symbol in front, e.g. "+3s". Previous regex missed
this.
Do not allow values without numbers, e.g. "-s". Previous regex missed
this as well.
name old time/op new time/op delta
Unmarshal_Duration-4 1.96µs ± 0% 1.24µs ± 0% ~ (p=1.000 n=1+1)
name old alloc/op new alloc/op delta
Unmarshal_Duration-4 703B ± 0% 512B ± 0% ~ (p=1.000 n=1+1)
name old allocs/op new allocs/op delta
Unmarshal_Duration-4 20.0 ± 0% 17.0 ± 0% ~ (p=1.000 n=1+1)
Change-Id: I4db58d70f55607213631c49d698ee6a048b5e094
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170012
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
A JSON "null" field should set the NullValue enum field because
NullValue has the custom encoding format of "null".
Change-Id: I2bfa0900de64d7e2874f7c6db04b1cbc0b61b904
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170107
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Also added json.Decoder.Clone API for unmarshaling Any to look
ahead remaining bytes for @type field.
Change-Id: I2f803743534dfb64f9092d716805b115faa5975a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170102
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Minor changes:
* Use x as the receiver since "e" and "m" are meaningless in the presence
of user-defined enum and message names.
* Consistently keep enum methods together, rather awkwardly split apart
by the value maps.
Change-Id: I68e5666efb56ac7a4d062fb223b9f826dc72aba9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170357
Reviewed-by: Herbie Ong <herbie@google.com>
This reduces the init-time cost slightly since the GZIP'd
raw descriptor is constructed lazily on demand.
Change-Id: I482c6a2201b8786e425d7dee5612fdfd60ab1500
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169917
Reviewed-by: Herbie Ong <herbie@google.com>
The "custom" JSON representation for Empty is {}. And hence, when
embedded in Any, it needs to be represented as a field value in the
"value" field.
Discussion at https://github.com/protocolbuffers/protobuf/issues/5390.
As of this CL, code currently handles marshaling of Empty embedded
in Any, but not unmarshaling of Any yet. I'd like to get this corrected
first in order to proceed with unmarshaling of Any.
Change-Id: Ic0c321be188a979909b99d6b467aabc57c48e95c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169702
Reviewed-by: Damien Neil <dneil@google.com>
Added tests related to required fields and AllowPartial. I somehow
missed this before.
Change-Id: I3eb17347b1f3a99be3d65af06c4549abcc87ae39
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169701
Reviewed-by: Damien Neil <dneil@google.com>
I had defined these types at some point in order to reuse another
instance of these within, but it's not needed anymore and hence removing
them.
Change-Id: I8aa127326a5926c6a8688d83cce7ca83c160b39b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169700
Reviewed-by: Damien Neil <dneil@google.com>
Provide AllowPartial option to accept messages with missing required
field during marshaling and unmarshaling.
Change-Id: Ia23783870a8125633f8ddc0b686984b4c5ca15ba
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169500
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Rearranged well_known_types.go such that marshaling and unmarshaling
funcs for a particular type are closer, and added description for its
JSON representation.
Change-Id: Id9f5e4060158f66392a351debaffe1d7260919e4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169497
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Also, fixed unmarshaling of map messages where non-fatal errors were not
propagated up.
Change-Id: I06415b4a4ccd12135f0fdfaa38ccda54866139e7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168997
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This removes yet another set of dependencies of v2 on v1.
The only remaining dependency are in the _test.go files,
primarily for proto.Equal.
Changes made:
* cmd/protoc-gen-go no longer generates any functionality that depends
on the v1 package, and instead only depends on v2.
* internal/fileinit.FileBuilder.MessageOutputTypes is switched from
protoreflect.MessageType to protoimpl.MessageType since the
implementation must be fully inialized before registration occurs.
* The test for internal/legacy/file_test.go is switched to a legacy_test
package to avoid a cyclic dependency.
This requires Load{Enum,Message,File}Desc to be exported.
Change-Id: I43e2fe64cff4eea204258ce11e791aca5eb6e569
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169298
Reviewed-by: Damien Neil <dneil@google.com>
Description of message Value states:
`Value` represents a dynamically typed value which can be either null, a
number, a string, a boolean, a recursive struct value, or a list of values. A
producer of value is expected to set one of that variants, absence of any
variant indicates an error.
https://github.com/protocolbuffers/protobuf/blob/3.7.x/src/google/protobuf/struct.proto#L57-L60
Previous implementation was following C++ lib behavior.
Change-Id: Id51792e2fc8cc465a05a978e63410d3b6802b522
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168901
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
As a goal, v2 should not depend on v1. As another step towards that end,
we move all the types that used to be in the v1 protoapi package over to v2.
For now, we place MessageV1, ExtensionRangeV1, and ExtensionDescV1
in runtime/protoiface since these are types that generated messages will
probably have to reference forever. An alternative location could be
reflect/protoreflect, but it seems unfortunate to have to dirty the
namespace of that package with these types.
We move ExtensionFieldV1, ExtensionFieldsV1, and ExtensionFieldsOf
to internal/impl, since these are related to the implementation of a
generated message.
Since moving these types from v1 to v2 implies that the v1 protoapi
package is useless, we update all usages of v1 protoapi in the v2
repository to point to the relevant v2 type or functionality.
CL/168538 is the corresponding change to alter v1.
There will be a temporary build failure as it is not possible
to submit CL/168519 and CL/168538 atomically.
Change-Id: Ide4025c1b6af5b7f0696f4b65b988b4d10a50f0b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168519
Reviewed-by: Herbie Ong <herbie@google.com>
Remove unnecessary v1 wrapping. Change most tests to operate on message
Any directly.
Change-Id: I19bbca6c1af72894f6a292aab8ebd09e0301a260
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168517
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Generate field numbers for the well-known types,
so that encoding/jsonpb can benefit from them as well.
This CL fixes internal/cmd/generate-protos, which was silently failing
because the modulePath was not properly initialized. We fix this by
moving it to the start of the init function.
Change-Id: I87637176f29218cffa512b4baa49f39dae924061
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168497
Reviewed-by: Herbie Ong <herbie@google.com>
Also, changed MarshalOptions.Compact to Indent for consistency with v1
and to make compact as the default.
Change-Id: Id08aaa5ca5656f18e7925d2eabc0b6b055b1cebb
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168352
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This makes it consistent with jsonpb.MarshalOptions. This does change
the default to be in compact form.
Change-Id: I1b07f06f282c019b30f3f1cbb43f6c8cba18f385
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168405
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Changes:
* Modify protoc-gen-go to use a helper from internal/impl
to print the message as text.
* Add a helper function to internal/impl that calls v2 textpb.
* Modify encoding/textpb to avoid depending on descriptor proto,
which would cause an import cycle.
* Modify internal/fileinit to populate a pseudo-internal
method on MessageDescriptor to check whether the message should
use the message set wire format. We avoid adding this to the
main MessageDescriptor interface since message sets are a
deprecated proto1 feature that we should avoid promoting.
Change-Id: Ibaf79a563af695756f11ddc4db69b38e25a8f1a7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168439
Reviewed-by: Herbie Ong <herbie@google.com>
The v1 registration leaks the message types out to the proto package.
When doing that, it must ensure that the reflection data structures
for those types are properly initialized first. We achieve that by
doing v1 registration at the end of the reflection init function.
Change-Id: If6df18df59d05bad50ff39c2eff6beb19e7466cc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168348
Reviewed-by: Damien Neil <dneil@google.com>
The EnumName, UnmarshalJSONEnum, and CompressGZIP helpers currently live
in v1 protoapi, which would cause all generated messages to depend on v1.
In an effort to break the dependency of v2 on v1, we move these helper
functions to v2 (and re-written to take advantage of protobuf reflection).
These helpers are unfortunate, but we cannot eliminate the functionality
that they implement since they are exposed in the publicly generated API.
Since EnumName does not rely on the enum maps, it removes another dependency
on those variables. Eventually, we can get to the point where these variables
(though declared) are not linked into the binary if the user does not use them.
Also, we rely on the v1 proto package for registration instead of v1 protoapi.
This may re-introduce a cyclic dependency on descriptor proto again in the
future, but the better approach is to just start registering with v2.
Change-Id: Id755585a7a1df14e4a6a2dfa650df221a3c153fb
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/167921
Reviewed-by: Damien Neil <dneil@google.com>
Added temporary work-around for now by always using deterninistic
marshaling so that protoV1.Equals knows how to do comparison.
Change-Id: Idec412647269d59f86ed3e485a1893c070355b3d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/167917
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Substitute protoV1.Marshal calls to use proto v2.
A few tests still need to use v1 due to missing required field checks in
v2.
Change-Id: I97ee208b81afc85c81142e70c1d7bdc47db76ce4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/167774
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Unmarshaling of scalar, messages, repeated, and maps.
Need to further improve on error messages for consistency, some error
messages contain the position info while some currently do not. There
are cases where position info is wrong as well when a value is decoded
in another pass, e.g. numbers in string value, or map keys.
Change-Id: I6f9e903c499b5e87fb258dbdada7434389fc7522
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/166338
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
