286 Commits

Author SHA1 Message Date
Joe Tsai
7390346c05 cmd/protoc-gen-go: invert conditional check for proto syntax
To avoid accidentally emitting the UnmarshalMethod in the unlikely
(but possible) event that a proto4 syntax was added, swich the conditional
to only trigger on the proto2 syntax.

Change-Id: I0c201eace56e9ecc92cd791e45ca5d3b99e2ba86
Reviewed-on: https://go-review.googlesource.com/c/154317
Reviewed-by: Damien Neil <dneil@google.com>
2018-12-17 22:19:10 +00:00
Damien Neil
d068d302ef proto: use extension field type in decoding when available
If a field has an ExtensionType defined, use it in decoding.

Change-Id: I85f3da0f52a11578500cf28e4611fa4eb31f0623
Reviewed-on: https://go-review.googlesource.com/c/154581
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-12-17 22:15:23 +00:00
Damien Neil
4be2fb4ada internal/legacy: use v2 proto.Unmarshal
The v2 decoder isn't 100% complete, but it's good enough.
Delete the vendored copy of the v1 Unmarshal implementation.

Change-Id: Ibeabbb2e9109a1ec3df57e71f98b7aa4a583fc5b
Reviewed-on: https://go-review.googlesource.com/c/154577
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-12-17 22:15:00 +00:00
Damien Neil
ba23aa55b2 proto: wire decoding support
Add proto.Unmarshal.

Test cases all produce identical results to the v1 unmarshaller.

Change-Id: I42259266018a14e88a650c5d83a043cb17a3a15d
Reviewed-on: https://go-review.googlesource.com/c/153918
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-12-17 18:48:00 +00:00
Joe Tsai
2c6f09887d protogen: simplify and fix naming derivation logic
We want to codify the naming logic to be understandable in a specification.
Clean up the camelCase function to be a little more readable.

Cleanup and fix the cleanGoName function when mustExport is true.
It is okay to change this behavior since this is new logic in v2 that has not
yet been exposed to the universe.

In the mustExport code-path, we do not need to check for conflicts with keywords
since Go keywords are never uppercase, so the uppercasing of the first letter
is sufficient to break a conflict. Also, we fix the logic for uppercasing
the first character since not every lowercase character has an uppercase form.

Change-Id: If14422d773bb89ed7038d874135e3dcd12683101
Reviewed-on: https://go-review.googlesource.com/c/154180
Reviewed-by: Damien Neil <dneil@google.com>
2018-12-14 18:30:52 +00:00
Herbie Ong
7c624e2501 encoding/textpb: ignore reserved field names during unmarshal
Do not treat reserved field names as unknown. Skip over these instead
of returning error for unknown field.

Change-Id: Iba9371fd901a0690e5b7329bccf261570352b55d
Reviewed-on: https://go-review.googlesource.com/c/154178
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-12-14 00:27:35 +00:00
Herbie Ong
70651959ff encoding/textpb: switch tests to use V2 generated messages
Change-Id: I817568aec5fbf053c3566b311e92b79aff1caf7e
Reviewed-on: https://go-review.googlesource.com/c/154177
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-12-14 00:26:06 +00:00
Herbie Ong
20a1d31685 encoding/textpb: add marshaling of unknown fields
Change-Id: Ifa2a86b3edd51d6c53d9cf7226b6f4f8d2f88a44
Reviewed-on: https://go-review.googlesource.com/c/153757
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-12-13 03:19:01 +00:00
Herbie Ong
800c990248 encoding/textpb: initial textproto unmarshaling
This initial textproto unmarshaling implementation covers messages
without the use of extensions, Any expansion, nor weak.

Updated encoding tests. Split some testcases to keep each simpler.

Added TestRoundTrip for example messages like the well-known types.

Change-Id: Icffab02834aa004fa8409a9da70624f687f604fb
Reviewed-on: https://go-review.googlesource.com/c/153020
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-12-13 03:14:40 +00:00
Damien Neil
9ecf75b72e internal/value: bugfix for v2 message conversion
PBValueOf returns a protoreflect.Message, not a
protoreflect.ProtoMessage.

Change-Id: I88ed55f52bada6fc2b29ffd63e30de09e1febe8c
Reviewed-on: https://go-review.googlesource.com/c/153917
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-12-13 01:11:07 +00:00
Damien Neil
ae2a561335 protogen: annotate interface methods
Record annotations on methods in interface types (e.g., gRPC
server/client methods).

Generate annotations in golden tests, even when not checking the
content, to catch annotations without a corresponding symbol.

Change-Id: I44ae6caf66f709dc7f4686e931be04b8b6fa843d
Reviewed-on: https://go-review.googlesource.com/c/153877
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-12-12 19:34:33 +00:00
Damien Neil
887028d7ae internal/value: fix GoValueOf for v2 message types
Was using a pref.Message where we want a pref.ProtoMessage.

Change-Id: I61d986a43eaf8f945a1378a7a10120474aa89d6f
Reviewed-on: https://go-review.googlesource.com/c/153697
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-12-11 22:30:29 +00:00
Damien Neil
232ea15589 reflect/prototype: hoist semantic options into builders
Add fields to the Message and Field builder structs which hold the value
of MessageOptions.map_entry, FieldOptions.packed, and FieldOptions.weak
options. Remove all access to the contents of options messages from the
prototype package.

Change IsPacked to always return false for unpackable field types,
which is consistent with the equivalent C++ API.

This change helps avoid dependency cycles between prototype and the
options messages. (Previously this was resolved by accessing options
with reflection, but just breaking the dependency from prototype to the
options message is cleaner and simpler.)

Change-Id: I756aefe2e04cfa8fea31eaaaa0b5a99d4ac9e851
Reviewed-on: https://go-review.googlesource.com/c/153517
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-12-11 20:25:45 +00:00
Damien Neil
692690ff4e cmd/protoc-gen-go-grpc: fix camel-casing of names in generated code
Fix a few points of divergence between the v1 and v2 generators around
when to apply camel-casing to service and method names.

Change-Id: I862f89c0995c540e4862013316d7af772e1ab0d8
Reviewed-on: https://go-review.googlesource.com/c/153658
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-12-11 19:20:28 +00:00
Damien Neil
08c3cdf59f cmd/protoc-gen-go-grpc: change test files to exercise name rewriting
Change the service and method names in grpc.proto to not use camel-case,
to make it explicit where rewriting occurs in the generated code.

Change-Id: I9e4a851097b0ee14817a589f5f959adcc5a14fe3
Reviewed-on: https://go-review.googlesource.com/c/153657
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-12-11 19:11:24 +00:00
Joe Tsai
2653420232 reflect/protoreflect: remove DescriptorProto method
The DescriptorProto method was originally intended to return a raw descriptor.
However, that functionality will be responsibility of the protodesc package
in the near future.

Also document that any callers of Options must import the descriptor package.
This is reasonable since any usage of that method will almost always be of
the form:
	fd.Options().(*descriptorpb.FieldOptions)

Change-Id: I1085f02305faef8dd724e0cb3ad45ac12bd5df0a
Reviewed-on: https://go-review.googlesource.com/c/153497
Reviewed-by: Damien Neil <dneil@google.com>
2018-12-10 21:53:06 +00:00
Damien Neil
97e7f57dbb reflect/protoreflect: replace Mutable with NewMessage
Remove the Mutable methods from KnownFields, List, and Map, replacing
them with methods which return a new, empty message value without adding
that value to the collection.

The new API is simpler, since it clearly applies only to message values,
and more orthogonal, since it provides a way to create a value without
mutating the collection. This latter point is particularly useful in
map deserialization, where the key may be unknown at the time the value
is deserialized.

Drop the Mutable interface, since it is no longer necessary.

Change-Id: Ic5f3d06a2aa331a5d5cd2b4e670a3dba4a74f77c
Reviewed-on: https://go-review.googlesource.com/c/153278
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-12-10 21:17:16 +00:00
Joe Tsai
c9899da4d5 internal/encoding/defval: unify logic for handling default values
Logic for serializing the default value in textual form exists in
multiple places in trivially similar forms. Centralize that logic.

Change-Id: I4408ddfeef2c0dfa5c7468e01a4d4df5654ae57f
Reviewed-on: https://go-review.googlesource.com/c/153022
Reviewed-by: Herbie Ong <herbie@google.com>
2018-12-07 22:40:20 +00:00
Joe Tsai
bce82b8e0d reflect/protoreflect: add HasJSONName, ReservedRanges, and ReservedNames
These properties of descriptors are currently missing and makes it impossible
to convert a FileDescriptorProto into one of the structured Go representations
and convert it back to a proto message without loss of information.

Furthermore, ReservedRanges and ReservedNames has semantic importance
to text serialization.

Change-Id: Ic33c30020ad51912b143156b95f47a4fb8da3503
Reviewed-on: https://go-review.googlesource.com/c/153019
Reviewed-by: Damien Neil <dneil@google.com>
2018-12-07 20:10:15 +00:00
Joe Tsai
381f04c102 reflect/protoreflect: add MessageDescriptor.ExtensionRangeOptions
Considerable thought was given to whether a seperate ExtensionRanges interface
should be made that encapsulates FieldNumbers with an Options method.
However, I decided against this design for the following reasons:
* Extension ranges share syntax with reserved numbers and fields.
Why is extension ranges so special that it can have options, while the other
two currently do not? How do we know that those other two won't grow options
in the future? If they do, then those APIs can be expanded in the same way as
how extension range options is being expanded here today.
* Extension range options stand out like a sore thumb compared to the other
eight options. The other options correspond with a named declaration and have
a full protobuf name that they are associated with. Extension range options
is the only options that is not correlated with a full name.
* Extension range options are an extremely rarely used feature and
it seems unfortunate complicating the common case with additional structure.

Change-Id: Ib284a0b798c57dc264febe304692eee5b9c8e91b
Reviewed-on: https://go-review.googlesource.com/c/153018
Reviewed-by: Damien Neil <dneil@google.com>
2018-12-07 19:42:29 +00:00
Joe Tsai
a4cbffe4bc reflect/prototype: add registration hooks for options
Add pseudo-hidden functions to register the concrete Go type used for
the optional types. Also, augment protoc-gen-go to specially generate the
descriptor proto to register these types.

This change does not add validation logic yet to ensure that the correct option
types are passed to the API.

Change-Id: I5decc897e14b4bf570a61cf17b57a066a2a0f9d7
Reviewed-on: https://go-review.googlesource.com/c/153017
Reviewed-by: Damien Neil <dneil@google.com>
2018-12-07 19:38:57 +00:00
Joe Tsai
9667c4816d cmd/protoc-gen-go: reduce technical debt
The following TODOs were addressed:
* Consistently collect all enums, messages, and extensions in a breadth-first order.
The practical affect of this is that the declaration order in a Go file may change.
This simplifies reflection generation, which relies on consistent ordering.
* Removal of placeholder declarations (e.g., "var _ = proto.Marshal") since
protogen is intelligent about including imports as necessary.
* Always generate a default variable or constant for explicit empty strings.
The practical effect of this is the addition of new declarations in some cases.
However, it simplifies our logic such that it matches the protobuf data model.
* Generate the registration statements in a consistent order.

Change-Id: I627bb72589432bb65d53b50965ea88e5f7983977
Reviewed-on: https://go-review.googlesource.com/c/152778
Reviewed-by: Damien Neil <dneil@google.com>
2018-12-07 03:13:48 +00:00
Joe Tsai
05d60cda2c reflect/prototype: disable detrand for test
Change-Id: I62cadc7f02afb9a6ee84c5ae2d7857be4b405360
Reviewed-on: https://go-review.googlesource.com/c/153021
Reviewed-by: Herbie Ong <herbie@google.com>
2018-12-07 00:53:04 +00:00
Joe Tsai
24ceb2b095 cmd/protoc-gen-go: generate descriptor and plugin with reflection
In CL/152020, we checked in pre-generated versions of descriptor and plugin.
This CL makes it such that they are generated by protoc-gen-go.

We modify protoc-gen-go to avoid reflection support by default
since the binary size increase is still an issue to investigate.
Reflection support is temporarily enabled by setting a special
PROTOC_GEN_GO_ENABLE_REFLECT environment variable.

Reflection support is always enabled for descriptor and plugin.
Furthermore, we change descriptor to depend on the protoapi package
instead of the proto package. The reason we do not switch to protoapi
for all generated protos is because we still depend on v1 proto
for the table-driven InternalMessageInfo type. Dropping it from descriptor
is semantically correct, but does incur slight performance cost.
It does not seem appropriate to drop it for all generated messages.

We could move InternalMessageInfo to protoapi, but the logic behind that
is significant.

Change-Id: I5c3fff7f6eab1a5a2399049d42fa6bf42d4c93f9
Reviewed-on: https://go-review.googlesource.com/c/152547
Reviewed-by: Damien Neil <dneil@google.com>
2018-12-06 17:47:27 +00:00
Joe Tsai
b3960a7de0 internal/typefmt: move descriptor formatting logic to typefmt
Custom descriptor types would want to benefit from descriptor formatting.
As such, move the logic out from prototype into an internal package for
the benefit of usages outside the prototype package.

Change-Id: I4bb2144221e656aa36909d33a77189fe084f700b
Reviewed-on: https://go-review.googlesource.com/c/152777
Reviewed-by: Herbie Ong <herbie@google.com>
2018-12-05 22:14:52 +00:00
Joe Tsai
b6405bd782 cmd/protoc-gen-go: add support for protobuf reflection
Implement support in protoc-gen-go for generating messages and enums
that satisfy the v2 protobuf reflection interfaces. Specifically, the following
are added:
* top-level variable representing the file descriptor
* ProtoReflect method on enums (to implement protoV2.Enum)
* ProtoReflect method on messages (to implement protoV2.Message)

The following are not supported yet:
* resolving transitive dependencies for file imports
* Extension descriptors
* Service descriptors

The implementation approach creates a single array for all the message and enum
declarations and references sections of that array to complete dependencies.
Since protobuf declarations can form a graph (a message may depend on itself),
it is difficult to construct a graph as a single literal. One way is to use
placeholder descriptors, but that is not efficient as it requires encoding
the full name of each dependent enum and message and then later resolving it;
thus, both expanding the binary size and also increasing initialization cost.
Instead, we add a prototype.{Enum,Message}.Reference method to obtain a
descriptor reference for the purposes for satisfying dependencies.
As such, nested declarations and dependencies are populated in an init function.

Other changes to support the implementation:
* Added a protoimpl package to expose the MessageType type and also the
MessageTypeOf and EnumTypeOf helper functions.
* Added a protogen.File.GoIdent field to provide a suggested variable name
for the file descriptor.
* Added prototype.{Enum,Message}.Reference that provides a descriptor reference
for the purposes for satisfying cyclic dependencies.
* Added protoreflect.{Syntax,Cardinality,Kind}.GoString to obtain a Go source
identifier that represents the given constant.

Change-Id: I9455764882dee6ad10f251901e7d419091e8bf1d
Reviewed-on: https://go-review.googlesource.com/c/150074
Reviewed-by: Damien Neil <dneil@google.com>
2018-12-05 19:32:12 +00:00
Joe Tsai
eeca8bb946 internal/legacy: drop dependency on protoV1 via vendoring
In order to transition more of v1 proto package to use the v2 API,
we need the v2 API to stop depending on v1 proto. The legacy package currently
depends on v1 proto because it needs to unmarshal the descriptor protos.

Ideally, we would switch this to use the v2 implementation of wire unmarshaling.
However, that is not available yet. So, instead, we vendor a minified version
of the v1 proto package that only supports unmarshaling.

The only changes to the vendored v1 code are:
* Delete code not needed to implement proto.Unmarshal
* Drop support for message sets
* Drop support for reporting the full field name for required not set errors

The unused tool was used to delete unrelated code:
	https://github.com/dominikh/go-tools/tree/master/cmd/unused

To verify that the dependency was dropped:
$ cd internal/legacy
$ go list -f "{{join .Deps \"\n\"}}" | sort | uniq | grep protobuf
github.com/golang/protobuf/protoapi
github.com/golang/protobuf/v2/internal/detrand
github.com/golang/protobuf/v2/internal/encoding/tag
github.com/golang/protobuf/v2/internal/encoding/text
github.com/golang/protobuf/v2/internal/encoding/wire
github.com/golang/protobuf/v2/internal/errors
github.com/golang/protobuf/v2/internal/flags
github.com/golang/protobuf/v2/internal/impl
github.com/golang/protobuf/v2/internal/legacy/protoV1
github.com/golang/protobuf/v2/internal/pragma
github.com/golang/protobuf/v2/internal/scalar
github.com/golang/protobuf/v2/internal/set
github.com/golang/protobuf/v2/internal/value
github.com/golang/protobuf/v2/reflect/protoreflect
github.com/golang/protobuf/v2/reflect/prototype
github.com/golang/protobuf/v2/runtime/protoimpl
github.com/golang/protobuf/v2/types/descriptor

Change-Id: I470865f1a987203574339fefc7d83843a12af966
Reviewed-on: https://go-review.googlesource.com/c/152545
Reviewed-by: Damien Neil <dneil@google.com>
2018-12-05 19:14:18 +00:00
Joe Tsai
e1f8d50e17 reflect/protodesc: split descriptor related functionality from prototype
In order to generate descriptor.proto, the generated code would want to depend
on the prototype package to construct the reflection data structures.
However, this is a problem since descriptor itself is one of the dependencies
for prototype. To break this dependency, we do the following:
* Avoid using concrete *descriptorpb.XOptions messages in the public API, and
instead just use protoreflect.ProtoMessage. We do lose some type safety here
as a result.
* Use protobuf reflection to interpret the Options message.
* Split out NewFileFromDescriptorProto into a separate protodesc package since
constructing protobuf reflection from the descriptor proto obviously depends
on the descriptor protos themselves.

As part of this CL, we check in a pre-generated version of descriptor and plugin
that supports protobuf reflection natively and switchover all usages of those
protos to the new definitions. These files were generated by protoc-gen-go
from CL/150074, but hand-modified to remove dependencies on the v1 proto runtime.

Change-Id: I81e03c42eeab480b03764e2fcbe1aae0e058fc57
Reviewed-on: https://go-review.googlesource.com/c/152020
Reviewed-by: Damien Neil <dneil@google.com>
2018-12-05 00:38:30 +00:00
Joe Tsai
6dbffb765a internal/legacy: drop legacy prefix
The legacy prefix made sense when this functionality was part of impl.
Now that it is in its own package called legacy, the legacy prefix is silly.

Change-Id: I9e6ddb6185ce1f701e02768b505e6a05f3986f77
Reviewed-on: https://go-review.googlesource.com/c/152543
Reviewed-by: Herbie Ong <herbie@google.com>
2018-12-05 00:24:04 +00:00
Joe Tsai
62517cc551 internal/legacy: improve performance of extension descriptor conversions
Converting to/from v1/v2 extension descriptor types is a common operation
for v1 and v2 interoperability. Optimize these operations with a cache.

Change-Id: I5feca810f60376847c791654982acd3b6a37a5db
Reviewed-on: https://go-review.googlesource.com/c/152542
Reviewed-by: Herbie Ong <herbie@google.com>
2018-12-05 00:14:16 +00:00
Damien Neil
4c057caa84 cmd/protoc-gen-go-grpc: switch to the stdlib "context" package
Also replace the ident function with the handy GoImportPath.Ident method.

Change-Id: Ie4e820556fb83e659ab7e7af98f27fc24cdcd760
Reviewed-on: https://go-review.googlesource.com/c/152177
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-12-03 20:00:03 +00:00
Joe Tsai
6cf80c4f76 internal/impl: allow reflection on typed nil pointers
Similar to how generated messages allow you to call Get methods on a
nil pointer, we permit similar functionality when protobuf reflection
is used on a nil pointer.

Change-Id: Ie2f596d39105c191073b42d7d689525c3b715240
Reviewed-on: https://go-review.googlesource.com/c/152021
Reviewed-by: Damien Neil <dneil@google.com>
2018-12-03 19:36:32 +00:00
Joe Tsai
f9123a3cd0 runtime: expose protoimpl and protolegacy
The internal/impl and internal/legacy packages are only visible from packages
within the v2 repository. However, selected parts of these packages need to
be visible from the v1 repository or from generated code.

Change-Id: I1d87227dacc38f5b1ac464e9b9dd43bd34559cda
Reviewed-on: https://go-review.googlesource.com/c/151858
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-30 23:16:22 +00:00
Joe Tsai
08e0030032 internal/legacy: extract legacy support out from the impl package
The impl package currently supports wrapping legacy v1 enums and messages
so that they implement the v2 reflective APIs. This functionality is necessary
for v1 and v2 to interoperate. However, the existence of this functionality
presents several problems:
	* A significant portion of the complexity in impl is for legacy wrapping.
	* This complexity is linked into a Go binary even if all the other messages
	in the binary natively support v2 reflection.
	* It presents a cyclic dependency when trying to generate descriptor proto.

Suppose you are generating descriptor.proto. The generated code would want to
depend on the impl package because impl is the runtime implementation for
protobuf messages. However, impl currently depends depends on descriptor in
order to wrap legacy enum and messages since it needs the ability to dynamically
create new protobuf descriptor types. In the case of descriptor.proto, it would
presumably be generated with native reflection support, so the legacy wrapping
logic is unneccessary.

To break the dependency of impl on descriptor, we move the legacy support logic
to a different package and instead add hooks in impl so that legacy support could
be dynamically registered at runtime. This is dependency injection.

Change-Id: I01a582908ed5629993f6699e9bf2f4bee93857a4
Reviewed-on: https://go-review.googlesource.com/c/151877
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-30 23:16:16 +00:00
Joe Tsai
ba0ef9a571 internal/value: rename the Unwrap method as ProtoUnwrap
Add a Proto prefix before the Unwrap method to reduce the probability that
it would ever conflict with a method of the same name that a
custom implementation of Enum, Message, List, or Map may have.

Change-Id: I628bf8335583f2747ab4589f3e6ff82e4501ce98
Reviewed-on: https://go-review.googlesource.com/c/151817
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-30 01:23:40 +00:00
Damien Neil
7e5c64778d cmd/protoc-gen-go: fix forwarding decls for inf and NaN defaults
Floating-point defaults of -inf, +inf, and NaN must be generated as
vars rather than consts. Fix the forwarding declaration for public
imports of these vars. (Was "const", is now "var".)

Change-Id: Ic6dc90ab7f88378ba477bff39e047de5f5193c35
Reviewed-on: https://go-review.googlesource.com/c/151757
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-11-29 23:43:35 +00:00
Herbie Ong
cddf8195e1 encoding/textpb: initial implementation of textproto marshaling
This initial implementation covers marshaling Message without use
of extensions, Any expansion, weak yet.

Change-Id: Ic787939c1d2a4e70e40c3a1654c6e7073052b7d3
Reviewed-on: https://go-review.googlesource.com/c/151677
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-11-29 23:06:35 +00:00
Joe Tsai
f18ab539ab all: make use of the protoapi package in v1
The new v1 protoapi package enables:
* Referencing types in the protoapi package instead of protoV1, which further
reduces the number of situations where we need to depend on protoV1.
This is for the goal of eventually breaking all cases where the v2 implementation
relies on v1, so that in the near future, proto v1 can rely on proto v2 instead.
* Removes the need for legacy_extension_hack.go since that functionality has now
been exported into the protoapi package.

Change-Id: If71002d9ec711bfabfe494636829df9abf19e23e
Reviewed-on: https://go-review.googlesource.com/c/151403
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-29 22:46:29 +00:00
Joe Tsai
25cc69d405 internal/impl: fix legacy logic to know about the new XXX_OneofWrappers method
The XXX_OneofWrappers method is a simplified way to obtain the wrapper structs
compared the previous XXX_OneofFuncs method which returned far more information
that was strictly necessary.

Change-Id: I2670506a2a8f7e8e724846b8c4083e7995371007
Reviewed-on: https://go-review.googlesource.com/c/151679
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-29 08:04:36 +00:00
Joe Tsai
71acbc7b7d internal/detrand: support disabling detrand
Since detrand is an internal package, we can safely provide a function
that can be called to disable its functionality for testing purposes.

Change-Id: I26383e12a5832eb5af01952898a4c73f627d7aa5
Reviewed-on: https://go-review.googlesource.com/c/151678
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-29 07:49:45 +00:00
Joe Tsai
009e067ed8 internal/scalar: add scalar package for primitive wrappers
Add the scalar package to reduce dependencies on the v1 proto runtime package.
It may very well be the case that these functions should be exposed in the
public API of v2, but that is not a decision we need to make now.

Change-Id: Ifbc6d15311ba5837909ac72af47c630a80a142ef
Reviewed-on: https://go-review.googlesource.com/c/151402
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-28 07:06:11 +00:00
Joe Tsai
f757390b52 cmd/protoc-gen-go: remove generation of RegisterMessageSetType
PR#741 in the v1 repository deprecated this behavior.

Change-Id: Ife48f1d586f178d875b9b3002a88b3336a2cd3b4
Reviewed-on: https://go-review.googlesource.com/c/151401
Reviewed-by: Damien Neil <dneil@google.com>
2018-11-28 00:40:08 +00:00
Joe Tsai
b6dbf23033 cmd/protoc-gen-go: remove generation of {Unmarshal,Marshal}JSON for MessageSets
PR#741 in the v1 repository deprecated this behavior.

Change-Id: Idffa3884d43f9cc5528ce8d9f303676e0e501b67
Reviewed-on: https://go-review.googlesource.com/c/151400
Reviewed-by: Damien Neil <dneil@google.com>
2018-11-27 23:47:23 +00:00
Joe Tsai
411f33952f protogen: clarify that GoIdent.GoName is a single idenfitier
While we're at it, also add the license header for the file.

Change-Id: If33f1764111aff96efcdb5a4e88c2a70ae28c09c
Reviewed-on: https://go-review.googlesource.com/c/150060
Reviewed-by: Damien Neil <dneil@google.com>
2018-11-27 19:56:42 +00:00
Joe Tsai
d7e97bc71b cmd/protoc-gen-go: generate XXX_OneofWrappers instead of XXX_OneofFuncs
The marshaler, unmarshaler, and sizer functions are unused ever since
the underlying implementation was switched to be table-driven.
Change the function to only return the wrapper structs.

This change:
* enables generated protos to drop dependencies on certain proto types
* reduces the size of generated protos
* simplifies the implementation of oneofs in protoc-gen-go

Updates #708

Change-Id: I845c9009bc0236d1b51d34b014dc3e184303c0f2
Reviewed-on: https://go-review.googlesource.com/c/151357
Reviewed-by: Damien Neil <dneil@google.com>
2018-11-27 19:36:27 +00:00
Joe Tsai
f6d4a4215f reflect/protoreflect: clarify Get semantics on unpopulated fields
Clearly specify that Get on an unpopulated field:
* returns the default value for scalars
* returns a mutable (but empty) List for repeated fields
* returns a mutable (but empty) Map for map fields
* returns an invalid value for message fields

The difference in semantics between List+Maps and Messages is because
protobuf semantics provide no distinction between an unpopulated and empty list
or map. On the other hand, there is a semantic difference between an unpopulated
message and an empty message.

Default values for scalars is trivial to implement with FieldDescriptor.Default.

A mutable, but empty List and Map is easy to implement for known fields since
known fields are generated as a slice or map field in a struct.
Since struct fields are addressable, the implementation can just return a
reference to the slice or map.

Repeated, extension fields are a little more tricky since extension fields
are implemented under the hood as a map[FieldNumber]Extension.
Rather than allocating an empty list in KnownFields.Get upon first retrieval
(which presents a race), delegate the work to ExtensionFieldTypes.Register,
which must occur before any Get operation. Register is not a concurrent-safe
operation, so that is an excellent time to initilize empty lists.
The implementation of extensions will need to be careful that Clear on a repeated
field simply truncates it zero instead of deleting the object.

For unpopulated messages, we return an invalid value, instead of the prior
behavior of returning a typed nil-pointer to the Go type for the message.
The approach is problematic because it assumes that
1) all messages are always implemented on a pointer reciever
2) a typed nil-pointer is an appropriate "read-only, but empty" message
These assumptions are not true of all message types (e.g., dynamic messages).

Change-Id: Ie96e6744c890308d9de738b6cf01d3b19e7e7c6a
Reviewed-on: https://go-review.googlesource.com/c/150319
Reviewed-by: Damien Neil <dneil@google.com>
2018-11-27 19:13:59 +00:00
Joe Tsai
492a476312 internal/detrand: new package for deterministically random functionality
The use of math/rand in serialization is to provide some form of instability
to the output to provide a clear signal to the user that the should not
depend on the the property of stability. However, it is reasonable that users
expect the output for these to be deterministic.

As such, add a detrand package that provides deterministic, yet unstable
randomization functionality.

Since this package hashes the binary, it does impose a small initialization cost:
	Benchmark    100000    20712 ns/op    480 B/op    6 allocs/op

Change-Id: I232d0fea1789a4278079837a67ee2f63474a4364
Reviewed-on: https://go-review.googlesource.com/c/151340
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-27 02:14:04 +00:00
Herbie Ong
c3f4d48629 internal/encoding/text: add extra random space to make output unstable.
Make output deliberately unstable so users don't rely on exactness.

For multi-line output, add another extra random space after <key>: for
at most one field per message.

-- example --
key1: field1
key2:  {
    foo:  bar
}

For single-line output, add another extra random space after a field per
message.

-- example --
key1:field1  key2:{foo:bar}

Change-Id: I3ab25d4d970fdebb88bbd9dd8fa6d73af84338ea
Reviewed-on: https://go-review.googlesource.com/c/150977
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-11-27 00:46:16 +00:00
Joe Tsai
ccf645912d reflect/prototype: print {Enum,Message,Extension}Type.GoType
Add GoType to the list of accessors that can be pretty-printed.
Useful for debugging.

Change-Id: Ib2a240249ca5bf0176cb77fa5edc639d44ae0561
Reviewed-on: https://go-review.googlesource.com/c/150537
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-20 20:02:59 +00:00
Joe Tsai
44e389ca6c internal/impl: fix Has for proto3 scalars
The definition of Has for proto3 scalars is whether the value is non-zero.

Change-Id: I6aee92dd518d63a66515ad35da84b2be7aa22527
Reviewed-on: https://go-review.googlesource.com/c/150320
Reviewed-by: Herbie Ong <herbie@google.com>
Reviewed-by: Joe Tsai <joetsai@google.com>
2018-11-19 23:34:00 +00:00