19 Commits

Author SHA1 Message Date
Herbie Ong
300b9fe4da encoding/jsonpb: fix unmarshaling of NullValue field
A JSON "null" field should set the NullValue enum field because
NullValue has the custom encoding format of "null".

Change-Id: I2bfa0900de64d7e2874f7c6db04b1cbc0b61b904
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170107
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-04-02 04:50:03 +00:00
Herbie Ong
8ac9dd257b encoding/jsonpb: add support for unmarshaling Any
Also added json.Decoder.Clone API for unmarshaling Any to look
ahead remaining bytes for @type field.

Change-Id: I2f803743534dfb64f9092d716805b115faa5975a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170102
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-04-02 04:49:03 +00:00
Herbie Ong
82014a51c5 encoding/jsonpb: update handling of Empty message type
The "custom" JSON representation for Empty is {}. And hence, when
embedded in Any, it needs to be represented as a field value in the
"value" field.

Discussion at https://github.com/protocolbuffers/protobuf/issues/5390.

As of this CL, code currently handles marshaling of Empty embedded
in Any, but not unmarshaling of Any yet. I'd like to get this corrected
first in order to proceed with unmarshaling of Any.

Change-Id: Ic0c321be188a979909b99d6b467aabc57c48e95c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169702
Reviewed-by: Damien Neil <dneil@google.com>
2019-03-28 18:04:43 +00:00
Herbie Ong
329be5b5fc encoding/jsonpb: add AllowPartial option to MarshalOptions and UnmarshalOptions
Added tests related to required fields and AllowPartial. I somehow
missed this before.

Change-Id: I3eb17347b1f3a99be3d65af06c4549abcc87ae39
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169701
Reviewed-by: Damien Neil <dneil@google.com>
2019-03-27 22:37:31 +00:00
Herbie Ong
822de2d88c encoding/jsonpb: remove encoder and decoder types
I had defined these types at some point in order to reuse another
instance of these within, but it's not needed anymore and hence removing
them.

Change-Id: I8aa127326a5926c6a8688d83cce7ca83c160b39b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169700
Reviewed-by: Damien Neil <dneil@google.com>
2019-03-27 22:34:27 +00:00
Herbie Ong
c4450377bd encoding/jsonpb: add support for unmarshaling Duration and Timestamp
Change-Id: Ia8319ed82d1d031e344ad7b095df2018286dcd43
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169698
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-03-27 18:09:54 +00:00
Herbie Ong
307a7930c8 encoding/jsonpb: rearrange and add descriptions in well_known_types.go
Rearranged well_known_types.go such that marshaling and unmarshaling
funcs for a particular type are closer, and added description for its
JSON representation.

Change-Id: Id9f5e4060158f66392a351debaffe1d7260919e4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169497
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-03-27 17:06:06 +00:00
Herbie Ong
e63c4c4b81 encoding/jsonpb: add support for unmarshaling wrapper and struct types
Also, fixed unmarshaling of map messages where non-fatal errors were not
propagated up.

Change-Id: I06415b4a4ccd12135f0fdfaa38ccda54866139e7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168997
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-03-26 20:15:50 +00:00
Herbie Ong
1c7462c1ba encoding/jsonpb: fix encoding of empty google.protobuf.Value
Description of message Value states:

`Value` represents a dynamically typed value which can be either null, a
number, a string, a boolean, a recursive struct value, or a list of values. A
producer of value is expected to set one of that variants, absence of any
variant indicates an error.

https://github.com/protocolbuffers/protobuf/blob/3.7.x/src/google/protobuf/struct.proto#L57-L60

Previous implementation was following C++ lib behavior.

Change-Id: Id51792e2fc8cc465a05a978e63410d3b6802b522
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168901
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-03-25 23:01:49 +00:00
Joe Tsai
4fddebafc0 all: move v1 types over to the v2 repository
As a goal, v2 should not depend on v1. As another step towards that end,
we move all the types that used to be in the v1 protoapi package over to v2.

For now, we place MessageV1, ExtensionRangeV1, and ExtensionDescV1
in runtime/protoiface since these are types that generated messages will
probably have to reference forever. An alternative location could be
reflect/protoreflect, but it seems unfortunate to have to dirty the
namespace of that package with these types.

We move ExtensionFieldV1, ExtensionFieldsV1, and ExtensionFieldsOf
to internal/impl, since these are related to the implementation of a
generated message.

Since moving these types from v1 to v2 implies that the v1 protoapi
package is useless, we update all usages of v1 protoapi in the v2
repository to point to the relevant v2 type or functionality.

CL/168538 is the corresponding change to alter v1.
There will be a temporary build failure as it is not possible
to submit CL/168519 and CL/168538 atomically.

Change-Id: Ide4025c1b6af5b7f0696f4b65b988b4d10a50f0b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168519
Reviewed-by: Herbie Ong <herbie@google.com>
2019-03-22 20:01:07 +00:00
Joe Tsai
ca46d8c924 internal/fieldnum: generate field numbers for the google.protobuf package
Generate field numbers for the well-known types,
so that encoding/jsonpb can benefit from them as well.

This CL fixes internal/cmd/generate-protos, which was silently failing
because the modulePath was not properly initialized. We fix this by
moving it to the start of the init function.

Change-Id: I87637176f29218cffa512b4baa49f39dae924061
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168497
Reviewed-by: Herbie Ong <herbie@google.com>
2019-03-21 00:57:40 +00:00
Herbie Ong
0b0f4036db encoding/jsonpb: add support for marshaling well-known types
Also, changed MarshalOptions.Compact to Indent for consistency with v1
and to make compact as the default.

Change-Id: Id08aaa5ca5656f18e7925d2eabc0b6b055b1cebb
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168352
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-03-20 23:30:00 +00:00
Herbie Ong
e52379a5c2 encoding/jsonpb: add support for unmarshaling extensions and messagesets
Change-Id: I7f3e0091c4f46924d2e8a08c614c7ab64917014c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/167773
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-03-16 07:23:11 +00:00
Herbie Ong
c96a79da29 encoding/jsonpb: add support for basic unmarshaling
Unmarshaling of scalar, messages, repeated, and maps.

Need to further improve on error messages for consistency, some error
messages contain the position info while some currently do not.  There
are cases where position info is wrong as well when a value is decoded
in another pass, e.g. numbers in string value, or map keys.

Change-Id: I6f9e903c499b5e87fb258dbdada7434389fc7522
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/166338
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-03-15 18:53:18 +00:00
Herbie Ong
f83d5bb6f0 encoding/jsonpb: add support for marshaling of extensions and messagesets
Change-Id: I839660146760a66c5cbf25d24f81f0ba5096d9e1
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/167395
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-03-14 20:53:14 +00:00
Herbie Ong
87608a75eb encoding/jsonpb: switch MarshalOptions to use new JSON encoder
Delete temporary copy of old JSON encoder/decoder internal/encoding/jsonx.

Change-Id: I8b23d7907370d069d0930c360979a2d8b62adc93
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/165778
Reviewed-by: Damien Neil <dneil@google.com>
2019-03-11 21:54:53 +00:00
Herbie Ong
1e1e78b71b internal/encoding/jsonx: copy internal/encoding/json
We're rewriting internal/encoding/json. So, make a copy of it first in
order not to break encoding/jsonpb package.

Change-Id: I8b63c468d3f432102d2af4db22a7549998ce3876
Reviewed-on: https://go-review.googlesource.com/c/164642
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-03-02 00:23:28 +00:00
Joe Tsai
19058431cd internal/cmd/generate-protos: initial commit
Create a single binary for handling generation of protos.
This replaces previous logic spread throughout the repo in:
* regenerate.bash
* cmd/protoc-gen-go/golden_test.go
* cmd/protoc-gen-go-grpc/golden_test.go
* (indirectly) internal/protogen/goldentest

One of the problems with the former approaches is that they relied on
a version of protoc that was specific to a developer's workstation.
This meant that the result of generation was not hermetic.
To address this, we rely on the hard-coded version of protobuf specified
in the test.bash script.

A summary of changes in this CL are:
* The internal_gengo.GenerateFile and internal_gengogrpc.GenerateFile
functions are unified to have consistent signatures. It seems that the
former accepted a *protogen.GeneratedFile to support v1 where gRPC code
was generated into the same file as the base .pb.go file. However, the
same functionality can be achieved by having the function return
the generated file object.
* The test.bash script patches the protobuf toolchain to have properly
specified go_package options in each proto source file.
* The test.bash script accepts a "-regenerate" argument.
* Add generation for the well-known types. Contrary to how these were
laid out in the v1 repo, all the well-known types are placed in the
same Go package.
* Add generation for the conformance proto.
* Remove regenerate.bash
* Remove internal/protogen
* Remove cmd/protoc-gen-go/golden_test.go
* Remove cmd/protoc-gen-go-grpc/golden_test.go
* Add cmd/protoc-gen-go/annotation_test.go

Change-Id: I4a1a97ae6f66e2fabcf4e4d292c95ab2a2db0248
Reviewed-on: https://go-review.googlesource.com/c/164477
Reviewed-by: Damien Neil <dneil@google.com>
2019-03-01 20:47:52 +00:00
Herbie Ong
7b828bca66 encoding/jsonpb: basic JSON marshaling
This does not handle extensions, messagesets, and well-known types yet.

Change-Id: I2786c429f490fe8c57f3f85cd25058d936b58bf7
Reviewed-on: https://go-review.googlesource.com/c/162637
Reviewed-by: Damien Neil <dneil@google.com>
2019-02-15 22:00:28 +00:00