102 Commits

Author SHA1 Message Date
Koichi Shiraishi
784c482554 all: remove shorthand import aliases
Change-Id: I62ec79419354a2c65ac69e4bcc1cc864a6a40a1a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/407934
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2022-05-24 20:05:50 +00:00
Kir Kolyshkin
bf9455640d all: fix typos
Brought to you by codespell v2.1.0, using the command

	codespell -S .cache,vendor -L ot,ba,fo,unparseable -w

Note that the misspelled "unparseable" comes from the
github.com/protocolbuffers/protobuf, where it is explicitly ignored
(see [1] and some explanation at [2]), so we ignore it here, too.

[1] https://github.com/protocolbuffers/protobuf/pull/7752
[2] https://github.com/protocolbuffers/protobuf/pull/7751#discussion_r460170422

Change-Id: Ie1ca705db4f11df8ec8b22fdc22b6a6ee667ae5b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/406845
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
2022-05-19 09:32:38 +00:00
Joe Tsai
acaef6adb7 cmd/protoc-gen-go: avoid referencing remote enum values by name
The Go generator has historically always prefixed an enum value
with the name of the enum type, when it was unnecessary to do so.

For example:
	enum Status {
		STATUS_FAILED = 0;
		STATUS_PASSED = 1;
	}
would be generated as:
	type Status int32
	const (
		Status_STATUS_FAILED Status = 0
		Status_STATUS_PASSED Status = 1
	)

It is common for the enum values to be manually prefixed by the
enum type since protobuf enums use C++ namespace rules where
enum types and enum values are in the same namespace scope.
Thus, having the Go generator add a prefix is redundant.
See https://github.com/golang/protobuf/issues/513.

Some custom Go generators like protoc-gen-gogo allow removing
the prefix with the gogoproto.goproto_enum_prefix feature.
However, this leads to interoperability issues between
protoc-gen-go and protoc-gen-gogo, where the enum value names
cannot be accurately inferred.

Avoid this problem by just hard-coding the enum value number
for values declared in other packages. This provides benefits
in interoperability at the small cost of enum values possibly
being stale if their value were ever changed in a remote package.
However, this would only occur with use of proto2 enums and
default values, which seems to be an exceptionally rare situation.

Before:
	Default_MyMessage_MyField = remotepb.FooEnum_FOO_ENUM
After:
	Default_MyMessage_MyField = remotepb.FooEnum(4) // remotepb.FooEnum_FOO_ENUM

Before:
	func (x *MyMessage) GetField() remotepb.FooEnum {
		...
		return remotepb.FooEnum_FOO_ZERO
	}
After:
	func (x *MyMessage) GetField() remotepb.FooEnum {
		...
		return remotepb.FooEnum(0) // always 0 for proto3 and often 0 for proto2
	}

Change-Id: I3a06cd553f2eaf6124666f6c36c196d500d35718
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/319649
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
2021-05-24 22:21:56 +00:00
Joe Tsai
31610fd913 cmd/protoc-gen-go: remove generation of the ExtensionRangeArray method
The ExtensionRangeArray was a pseudo-internal API that was used by
the protobuf runtime implementation to know what the extension ranges are.
For some time now, the runtime implementation does not make use of this.

According to all the latest modules known by the module proxy,
no code depends directly on this method being present.

In the extremely rare case where some user code is depending on this method,
the user can place a .go file next to the generated .pb.go file that
injects this method.

Change-Id: Iae40a9a33b8c9b5a243d48db14f25b05ca24e3dc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/305574
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
2021-03-29 20:25:28 +00:00
Joe Tsai
85f47621e0 all: update protobuf toolchain dependency
This updates the dependency on the protobuf toolchain to an unreleased
version that has the go_package option for relevant .proto files in
that repository to reference this module.

Change-Id: Ie1ac8f81a323285efbd14a17d02c7105e810af2d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/235283
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
2021-03-03 01:59:11 +00:00
cybrcodr
a709e31e5d types/known/fieldmaskpb: repeated and map fields are only valid in the last position of a path
Fixes golang/protobuf#1179.

Change-Id: Ia40c287036889c798305cdaf7c8930811d25f957
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/246097
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2020-08-05 23:11:51 +00:00
Joe Tsai
1a290e9a0e reflect/protoreflect: add MessageFieldTypes
The MessageFieldTypes interface (if implemented by a MessageType)
provides Go type information about the fields if they are
an enum or message type.

Change-Id: I68b20f5726377f6b0f2c20a8b6e45f9802b43f67
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/236777
Reviewed-by: Damien Neil <dneil@google.com>
2020-07-23 21:24:58 +00:00
Joe Tsai
5a96da2901 cmd/protoc-gen-go: generate package documentation for well-known types
Change-Id: Ia079805e24d3490301355e09693255b157cabaed
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/239167
Reviewed-by: Damien Neil <dneil@google.com>
2020-06-22 19:32:21 +00:00
Joe Tsai
5f5c066b7f types/known: minor adjustments to error handling
Trivial adjustments to error handling to reduce differences between
the generated helpers and the legacy ptypes package.

Change-Id: Ib3022eb50d9a9c0906b7809fe7e8011ee9399b0a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/238009
Reviewed-by: Damien Neil <dneil@google.com>
2020-06-17 18:52:16 +00:00
Joe Tsai
a0d1c75183 types/known: handle nil in IsValid and CheckValid methods
Many usages of the legacy ptypes.Timestamp and ptypes.Duration rely
on the functions implicitly returning an error for a nil message.
In order to migrate previous usages of the ptypes constructors to
the new API, we modify CheckValid to preserve similar semantics.

There is some consistency with this change in that:
	(*M)(nil).IsValid() == (*M)(nil).ProtoReflect().IsValid()
where M is Duration, Timestamp, and FieldMask.

Change-Id: I05ae152511f9875ea034e78aefb7aab18d17a318
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/238007
Reviewed-by: Damien Neil <dneil@google.com>
2020-06-16 21:18:46 +00:00
Joe Tsai
0084168af8 types/known/anypb: add New constructor
While the MarshalFrom method and function are  more flexible since
it permits the reuse of an existing Any, the most common use case
by far is to construct a new Any. Optimize for that use case
by providing a constructor that directly returns a new Any instance.
We do not provide a variant that accepts a MarshalOptions since
the more flexible MarshalFrom function can be used for that.

Change-Id: Iab0219229c7d3aaa7390a2340a4f248002995293
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/237997
Reviewed-by: Damien Neil <dneil@google.com>
2020-06-16 00:35:17 +00:00
Joe Tsai
bfc31022a5 types/known/anypb: gracefully handle nil sources
Add checks so that MarshalFrom and UnmarshalTo consistently matches
the behavior of the proto package with regard to nil messages.
In particular, using a nil message as the destination results in
a panic (which is already the current behavior), while using a
nil message as the source does not panic (it is usually treated as
an untyped empty message). Since an untyped message has no
sensible meaning in the context of Any, return an error.

Change-Id: I99e86c2cdfbd8be57cc039efd550458dc56aadbc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/237920
Reviewed-by: Damien Neil <dneil@google.com>
2020-06-15 18:37:58 +00:00
Damien Neil
7e8b902d03 types/dynamicpb: add NewEnumType
Add support for creating dynamic EnumTypes.

Change-Id: Ic9f5b73630734848b29cc436f6c179549a8ea74a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/237219
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Joe Tsai <joetsai@google.com>
2020-06-10 19:22:58 +00:00
Joe Tsai
f49fd502d3 all: implement first-class WKT support
This CL introduces generation of specialized APIs directly into the
generated packages for certain well-known types. This follows the pattern
set forth by the other language implementations that have specialized
generated support for certain well-known types.

Overview of new API:
  package anypb
    func MarshalFrom(*Any, proto.Message, proto.MarshalOptions) error
    func UnmarshalTo(*Any, proto.Message, proto.UnmarshalOptions) error
    func UnmarshalNew(*Any, proto.UnmarshalOptions) (proto.Message, error)
    func (*Any) MessageIs(proto.Message) bool
    func (*Any) MessageName() protoreflect.FullName
    func (*Any) MarshalFrom(proto.Message) error
    func (*Any) UnmarshalTo(proto.Message) error
    func (*Any) UnmarshalNew() (proto.Message, error)

  package timestamppb
    func Now() *Timestamp
    func New(time.Time) *Timestamp
    func (*Timestamp) AsTime() time.Time
    func (*Timestamp) IsValid() bool
    func (*Timestamp) CheckValid() error

  package durationpb
    func New(time.Duration) *Duration
    func (*Duration) AsDuration() time.Duration
    func (*Duration) IsValid() bool
    func (*Duration) CheckValid() error

  package structpb
    func NewStruct(map[string]interface{}) (*Struct, error)
    func (*Struct) AsMap() map[string]interface{}
    func (*Struct) MarshalJSON() ([]byte, error)
    func (*Struct) UnmarshalJSON(b []byte) error

    func NewList([]interface{}) (*ListValue, error)
    func (*ListValue) AsSlice() []interface{}
    func (*ListValue) MarshalJSON() ([]byte, error)
    func (*ListValue) UnmarshalJSON(b []byte) error

    func NewValue(interface{}) (*Value, error)
    func NewNullValue() *Value
    func NewBoolValue(bool) *Value
    func NewNumberValue(float64) *Value
    func NewStringValue(string) *Value
    func NewStructValue(*Struct) *Value
    func NewListValue(*ListValue) *Value
    func (*Value) AsInterface() interface{}
    func (*Value) MarshalJSON() ([]byte, error)
    func (*Value) UnmarshalJSON(b []byte) error

  package fieldmaskpb
    func New(proto.Message, ...string) (*FieldMask, error)
    func Union(*FieldMask, *FieldMask, ...*FieldMask) *FieldMask
    func Intersect(*FieldMask, *FieldMask, ...*FieldMask) *FieldMask
    func (*FieldMask) IsValid(proto.Message) bool
    func (*FieldMask) Append(proto.Message, ...string) error
    func (*FieldMask) Normalize()

  package wrapperspb
    func Bool(bool) *BoolValue
    func Int32(int32) *Int32Value
    func Int64(int64) *Int64Value
    func UInt32(uint32) *UInt32Value
    func UInt64(uint64) *UInt64Value
    func Float(float32) *FloatValue
    func Double(float64) *DoubleValue
    func String(string) *StringValue
    func Bytes([]byte) *BytesValue

This functionality expands upon and supersedes the
older github.com/golang/protobuf/ptypes package,
which provided helpers for Any, Timestamp, and Duration.

Comparison with older ptypes package:

* ptypes.AnyMessageName is replaced by anypb.Any.MessageName.
  The former returned an error for malformed type URLs,
  while the latter simply returns an empty string.

* ptypes.Is is replaced by anypb.Any.MessageIs.

* ptypes.Empty has no direct replacement as it is equivalent to:
	mt, err := protoregistry.GlobalTypes.FindMessageByURL(any.GetTypeUrl())
	if err != nil {
		return nil, err
	}
	return mt.New().Interface(), nil
  Analysis of user code revealed that this function is seldom used.

* ptypes.MarshalAny is replaced by anypb.Any.MarshalFrom.
  The former creates a new Any message and returns it,
  while the latter is a method that modifies the receiver.

* ptypes.UnmarshalAny is replaced by anypb.Any.UnmarshalTo.

* ptypes.DynamicAny is loosely replaced by anypb.Any.UnmarshalNew.
  The DynamicAny type is a custom proto.Message that is special
  to ptypes.UnmarshalAny where it would allocate a new message
  and store it into the DynamicAny instance. The UnmarshalNew method
  accomplishes the equivalent functionality in a more direct fashion.

* ptypes.TimestampNow is replaced by timestamppb.Now.

* ptypes.TimestampProto is replaced by timestamppb.New.
  The former returned an error if the timestamp was outside the
  10000-year range recommended by timestamp.proto,
  while the latter always succeeded. To preserve the behavior of
  the former validation check, the replacement can additionally
  call the timestamppb.Timestamp.CheckValid method.

* ptypes.Timestamp is replaced by timestamppb.Timestamp.AsTime.
  The former returned an error if the timestamp was outside the
  10000-year range recommended by timestamp.proto,
  while the latter always succeeded. To preserve the behavior of
  the former validation check, the replacement can additionally
  call the timestamppb.Timestamp.CheckValid method.

* ptypes.TimestampString has no direct replacement as it is equivalent to:
	ts.AsTime().Format(time.RFC3339Nano)

* ptypes.DurationProto is replaced by durationpb.New.

* ptypes.Duration is replaced by durationpb.Duration.AsDuration.
  The former returned an error if the duration would overflow
  when converting to a time.Duration, while the latter uses
  saturation arithmetic (similiar to the time package itself).
  Underflow resulted in time.Duration(math.MinInt64), while
  overflow resulted in time.Duration(math.MaxInt64).
  To preserve the behavior of former validation checks,
  the replacement can call the durationpb.Duration.CheckValid method
  and check whether the duration is fixed to one of the overflow values.

Change-Id: Ia996b1037a1fcafced7c7e10e9408ef7fa22863a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/225298
Reviewed-by: Damien Neil <dneil@google.com>
2020-06-09 21:21:54 +00:00
Joe Tsai
81db48ad09 all: move well-known types
This is step 2 of 6 in a multi-stage migration
to move the well-known types
from the google.golang.org/genproto module
to the google.golang.org/protobuf module.

The generated Go packages for field_mask.proto, api.proto,
type.proto, and source_context.proto are being moved over
to this module alongside all the other well-known types.

In order to move these types between two modules,
there needs to be a sequence of changes submitted in
decently rapid succession. It is impossible to atomically
make these changes, so a brief breakage is inevitable.
The steps are as follows:

Step 1: Submit a change to cloud.google.com/go/internal/gapicgen
to avoid generating the well-known types. Otherwise, the tool
will undo the changes made in step 3.
See https://code-review.googlesource.com/c/gocloud/+/56810

Step 2: Submit a change to google.golang.org/protobuf that
adds the generated well-known types being migrated to that module.
In order to prevent the situation where a user links in
too old a version of the genproto module such that
duplicate registration occurs for the well-known types,
the registry is specially modified to provide an error
message that instructs users to upgrade the genproto module.
See https://golang.org/cl/234937

Step 3: Submit a change to google.golang.org/genproto that
switches all generated well-known types to be aliases to the
ones declared in google.golang.org/protobuf from the previous step.
This will cause the genproto module to incur an dependency
on an unreleased version of the protobuf module.
See https://github.com/googleapis/go-genproto/pull/372

Step 4: Submit a change to google.golang.org/protobuf that
adds a weak module depdency on the genproto module at the
revision from the previous step.

Step 5: Release google.golang.org/protobuf@v1.24.0.

Step 6: Submit a change to google.golang.org/genproto that
updates the protobuf module dependency to v1.24.0.

Change-Id: I36a19049d2240b67a37dfad20e154505aee7c784
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/234937
Reviewed-by: Damien Neil <dneil@google.com>
2020-05-26 19:51:55 +00:00
Joe Tsai
387873dd53 all: implement support for proto3 optional semantics
In the upcoming 3.12.x release of protoc, the proto3 language will be
amended to support true presence for scalars. This CL adds support
to both the generator and runtime to support these semantics.

Newly added public API:
	protogen.Plugin.SupportedFeatures
	protoreflect.FieldDescriptor.HasPresence
	protoreflect.FieldDescriptor.HasOptionalKeyword
	protoreflect.OneofDescriptor.IsSynthetic

Change-Id: I7c86bf66d0ae56642109beb5f2132184593747ad
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/230698
Reviewed-by: Damien Neil <dneil@google.com>
2020-04-29 20:02:24 +00:00
Joe Tsai
b57aae9def all: improve extension validation
Changes made:
* Ensure protoreflect.ExtensionType.IsValidInterface never panics,
especially if given a nil interface value.
* Have protoreflect.ExtensionType.IsValid{Interface,Value} only
perform type-checks. It does not do value checks (i.e., whether the
value itself is valid). Value validity is left to when an actual
protoreflect.Message.Set operation is performed.
* Add special-casing on proto.SetExtension to treat an invalid
message or list as functionally equivalent to Clear. This is to
be more consistent with the legacy SetExtension implementation
which never panicked when given such values.
* Add special-casing on proto.HasExtension to treat a mismatched
extension descriptor as simply not being present in the message.
This is also to be more consistent with the legacy HasExtension
implementation which did the same thing.

Change-Id: Idf0419abf27b9f85d9b92bd2ff8088e25b7990cc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/229558
Reviewed-by: Damien Neil <dneil@google.com>
2020-04-23 06:01:13 +00:00
Edward McFarlane
9d397869d8 types/dynamicpb: fix message Zero return type to read-only
Changes the message Zero return type to be read-only by omitting internal known map.

Change-Id: I1c1191a125df74251be3d8bb70f4b06c1ff57070
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/223857
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2020-03-20 19:41:50 +00:00
Joe Tsai
b738ac9285 all: regenerate remote .proto files
This fixes the conformance test failures which occur because the
conformance protos have not been regenerated.
The generator script has been modified with a sanity check that
files do not exist outside the expected sub-tree.

Change-Id: I473efec4a016f6bc96ddf7e20d54bcf5ff9b55fe
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/223538
Reviewed-by: Damien Neil <dneil@google.com>
2020-03-18 20:17:11 +00:00
Joe Tsai
29677a9c11 types/dynamic: make Message implement legacy message interface
It's annoying for some users that they can't directly pass
*dynamicpb.Message to APIs that expect the legacy message interfaces.
The proto.MessageV1 wrappers work, but is somewhat suspect since they
were originally designed to wrap legacy generated messages.

Change-Id: I0f9900dcd1c9865c754551f8763680c9bb904813
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/223817
Reviewed-by: Damien Neil <dneil@google.com>
2020-03-17 23:08:45 +00:00
Damien Neil
56786dc5df testing/prototest: refactor prototest API
For consistency with other options types in the protobuf module, make
the test function a method of the options.

Drop the ExtensionTypes option and just look up the extension types to
test with in the provided resolver.

Change-Id: I7918bd10b7c003e4af56d27521d30218653d5b4d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/219142
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2020-02-20 22:47:27 +00:00
Damien Neil
3dbd95a558 reflect/protoreflect: add List.AppendMutable and Map.Mutable
Add methods to add a new, mutable message to a list or map, matching the
existing Message.Mutable.

These methods are purely a convenience, as each can be implemented in
terms of the existing interface.

Change-Id: I889c20fe37ea0f2a566555212e99e6378fb9fe1d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/220117
Reviewed-by: Joe Tsai <joetsai@google.com>
2020-02-19 20:35:39 +00:00
Damien Neil
c4051cd4ec types/known: remove packages present in genproto
Remove the generated proto packages that already exist in
google.golang.org/genproto. We want to eventually move these
packages here, but it doesn't need to happen yet.

Add a local copy of fieldmaskpb for use in tests.

Refactor proto generation to override import paths using the
M<source>=<import_path> compiler option instead of by patching the
source files.

Change-Id: I8d31f67e931d70140182f19f3e0106111f71c4b4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/219598
Reviewed-by: Joe Tsai <joetsai@google.com>
2020-02-18 22:04:05 +00:00
Joe Tsai
6320bdfa7e types/dynamicpb: minor doc change
Change-Id: I13439b810e18142d85f4466be006d3abf3a66c30
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/219502
Reviewed-by: Damien Neil <dneil@google.com>
2020-02-14 20:42:33 +00:00
Damien Neil
3c5fb5f879 all: make .proto file names relative to module root
Change the protoc flags such that when one of our test .proto files
imports another, the filename is consistently specified relative to the
module root.

Change-Id: I690282795cef23347c8794c1c6357e4fe9560d8a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/217762
Reviewed-by: Joe Tsai <joetsai@google.com>
2020-02-04 23:20:20 +00:00
Damien Neil
f12fb45fd6 all: add ProtoMethods method to protoreflect.Message
Promote the fast-path magic ProtoMethods method to first-class citizen
of the protoreflect.Message interface.

To avoid polluting the protoreflect package with the various types
required by this method, make the necessary protoiface types unnamed and
duplicate them in protoreflect.

Updates golang/protobuf#1022.

Change-Id: I9595bae40b3bc7536d727fb6f99b3bce8f73da87
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/215718
Reviewed-by: Joe Tsai <joetsai@google.com>
2020-01-21 21:05:54 +00:00
Damien Neil
9f1165c3bf all: fix reflection behavior for empty lists and maps
The protoreflect documentation states that Get on an empty repeated or
map field returns an invalid (empty, read-only) List or Map. We weren't
doing this; fix it.

The documentation also states that an invalid List or Map may not be
assigned via Set. We were permitting Set with an invalid map; fix this.

Add tests for this behavior in prototest.

Change-Id: I4678af532e192210af0bde7c96a1439a4cd26efa
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/209019
Reviewed-by: Joe Tsai <joetsai@google.com>
2019-11-26 22:58:51 +00:00
Damien Neil
82886da2b9 reflect/protoreflect: add {Message,List,Map}.IsValid
Various protoreflect methods can return an "empty, read-only" message,
list, or map value. Provide a method to test if a value is one of these.

Fixes golang/protobuf#966

Change-Id: I793d8426d6e2201755983c06f024412a7e09bc4c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/209018
Reviewed-by: Joe Tsai <joetsai@google.com>
2019-11-26 22:56:18 +00:00
Damien Neil
a0a54b8005 reflect/protoreflect: remove nullability from repeated extension fields
Remove repeated extension fields from the set of nullable fields,
so that Has reports false and Range does not visit a a zero-length
repeated extension field.

This corrects a fuzzer-detected case where unmarshaling and remarshaling
a wire-format message could result in a semantic change. For a repeated
extension field in non-packed encoding, unmarshaling a packed
representation of the field would result in a message which Has the
extension. Remarshaling it would discard the the field.

Fixes golang.org/protobuf#975

Change-Id: Ie836559c93d218db5b5201742a3b8ebbaacf54ed
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/204897
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Joe Tsai <joetsai@google.com>
2019-11-05 18:31:30 +00:00
Joe Tsai
ae313d4af3 cmd/protoc-gen-go: fix Reset method
The MessageInfo cache, once set, must not be cleared, otherwise
there exists a *messageState value where the MessageInfo value is nil.
Fix the generation of the Reset method to avoid clearing this value.

Change-Id: Ic84ca8b2640a43e967c36993da1ccd3f2b7096c4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/201478
Reviewed-by: Damien Neil <dneil@google.com>
2019-10-17 17:10:14 +00:00
Damien Neil
3b65992c65 all: remove {Enum,Message,Extension}Type.GoType methods
These methods are difficult or impossible to use correctly; types
created by the dynamicpb package, for example, all have the same GoType.

Fixes golang/protobuf#938

Change-Id: I33d4ef381579ff18569b11df501d0ba7f38a6b5c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/199060
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-10-07 21:08:26 +00:00
Joe Tsai
84177c9bf3 all: use typed variant of protoreflect.ValueOf
Change-Id: I7479632b57e7c8efade12a2eb2b855e9c321adb1
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/196037
Reviewed-by: Damien Neil <dneil@google.com>
2019-09-17 21:33:16 +00:00
Damien Neil
2eaed36627 all: remove NewMessage methods
Superseded by New{Field,Element,Value}.

Change-Id: Iad09f3b0c6bac7b75cf28925eef2982d45267317
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/193598
Reviewed-by: Joe Tsai <joetsai@google.com>
2019-09-05 17:49:46 +00:00
Damien Neil
dfae831ab7 types/dynamicpb: New->NewMessage, add NewMessageType
Rename New to NewMessage, since messages are not the only type in this
package.

Add NewMessageType to allow creating a MessageType without first
creating a Message.

Fixes golang/protobuf#934

Change-Id: I9c59594a436dec75d3e26078b0055a4aa08e0169
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/193157
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-09-04 04:34:35 +00:00
Damien Neil
290ceea663 types/dynamicpb: support dynamic extensions
Add a dynamicpb.NewExtensionType function to permit creating extension
types from descriptors.

Also fix a some bugs around extension field handling:
When creating a new value for an extension field, use the
ExtensionType's Zero or New method to create the value.

Ensure that prototest exercises true zero-values of fields. (i.e.,
getting a list, map, or message from an empty message rather than
creating a new empty one with NewField.)

Change-Id: Idb8e87cdc92692610e12a4b8a68c34b129fae617
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186180
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-09-03 21:40:45 +00:00
Damien Neil
835b271169 reflect/protoreflect: add ExtensionType IsValid{Interface,Value} methods
Add a way to typecheck a Value or interface{} without converting it to
the other form.  This permits implementations which store field values as
a Value (such as dynamicpb, or (soon) extensions in generated messages)
to validate inputs without an unnecessary conversion.

Fixes golang/protobuf#905

Change-Id: I1b78612b22ae832efbb55f81ae420871729e3a02
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/192457
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-09-03 20:58:16 +00:00
Joe Tsai
abd06a8b30 all: update to protobuf-v3.9.1 and go-v1.11.13 and go-v1.12.9
A conformance test has been added for the text format.
Update our conformance runner to handle that format and
update with the list of current failures.

Change-Id: I36a271fcfbe328c4ee8c870caff4661659ad27ef
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/190517
Reviewed-by: Herbie Ong <herbie@google.com>
2019-08-16 18:12:10 +00:00
Joe Tsai
7328839f81 cmd/protoc-gen-go: annotate depIdxs list with index comments
Generate the current index into depIdxs for easier human debugging.

Change-Id: Ida42aa95137b2044a4dc267c31cebec5023bdfb1
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/190278
Reviewed-by: Herbie Ong <herbie@google.com>
2019-08-15 22:47:44 +00:00
Damien Neil
16163b4f67 all: drop reflect/prototype package
Remove the remaining uses of the prototype package.

The most significant change is to impl.MessageInfo, which now directly
implements the MessageType interface. This involves two notable changes
to exported fields of MessageInfo:

  - PBType is now Desc.
  - GoType is now GoReflectType. (Name changed to avoid a conflict with
    the GoType method of the MessageType interface.)

Fixes golang/protobuf#911

Change-Id: Ie2aa4766d6887ceaa9cf06b1f109aa6e6e2a208f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189340
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-08-09 19:06:30 +00:00
Damien Neil
4401a0de4b cmd/protoc-gen-go, internal/filetype: clean up EnumType construction
Drop the dependency from generated files on prototype.Enum: Generated
code should only depend on runtime/proto{iface,impl}.

Drop the Enums, Messages, and Extensions returns form
filetype.Builder.Build. Of these, only Enums was used by generated code.
Change the generated init function to pass the builder a slice of values
to fill in (as is done for messages and extensions).

Remove the filetype dependency on prototype in preparation for
eventually dropping the prototype package entirely.

Change-Id: I28a3420f5dfcc13fed531a64ef07b9afddfd9d55
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189200
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-08-09 19:04:41 +00:00
Damien Neil
92f76189a3 all: refactor extensions, add proto.GetExtension etc.
Change protoiface.ExtensionDescV1 to implement protoreflect.ExtensionType.

ExtensionDescV1's Name field conflicts with the Descriptor Name method,
so change the protoreflect.{Message,Enum,Extension}Type types to no
longer implement the corresponding Descriptor interface. This also leads
to a clearer distinction between the two types.

Introduce a protoreflect.ExtensionTypeDescriptor type which bridges
between ExtensionType and ExtensionDescriptor.

Add extension accessor functions to the proto package:
proto.{Has,Clear,Get,Set}Extension. These functions take a
protoreflect.ExtensionType parameter, which allows writing the
same function call using either the old or new API:

  proto.GetExtension(message, somepb.E_ExtensionFoo)

Fixes golang/protobuf#908

Change-Id: Ibc65d12a46666297849114fd3aefbc4a597d9f08
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189199
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-08-08 18:20:51 +00:00
Joe Tsai
bab3d4084e runtime/protoimpl, cmd/protoc-gen-go: support release versioning
In order for protoc-gen-go to output the current version,
it needs to know what version it is currently running as.
However, we cannot rely on the git tags since the tags are not
made until *after* the commit has been submitted.
Instead, we manually encode the version into the code and
make sure that git tags match up with the version in the code.

The version.go file in runtime/protoimpl contains instructions
for how to make a release. Essentially:
* Every non-release commit has a version string with "devel" in it.
* Every release commit must not have "devel" in it and must be unique.
* The "release process" involves submitting two CLs.
The first CL creates a version string without "devel",
which is the commit that a git tag will actually reference.
The second CL follows immediately and re-introduces "devel"
into the version string.

The following example shows a possible sequence of VersionStrings
for git commits in time-ascending order:
	v1.19.0-devel      (this CL)
	v1.19.0-devel
	v1.19.0-devel
	v1.19.0-devel
	v1.20.0-rc.1       <- tagged
	v1.20.0-rc.1.devel
	v1.20.0-rc.1.devel
	v1.20.0-rc.1.devel
	v1.20.0-rc.2       <- tagged
	v1.20.0-rc.2.devel
	v1.20.0            <- tagged (future public release)
	v1.20.0-devel
	v1.20.0-devel
	v1.20.0-devel
	v1.20.0-devel
	v1.20.1            <- tagged
	v1.20.1-devel
	v1.20.1-devel
	v1.21.0            <- tagged
	v1.21.0-devel

Note that we start today with v1.19.0-devel, which means that our initial
release will be v1.20.0. This number was intentionally chosen since
1) the number 20 has some correlation to the fact that we keep calling
the new implementation the "v2" implementation, and
2) the set of tagged versions for github.com/golang/protobuf
and google.golang.org/protobuf are unlikely to ever overlap.
This way, the version of protoc-gen-go is never ambiguous which module
it was built from.

Now that we have version information, we add support for generating .pb.go
files with the version information recorded. However, we do not emit
these for .pb.go files in our own repository since they are always guaranteed
to be at the right version (enforced by integration_test.go).

Updates golang/protobuf#524

Change-Id: I25495a45042c2aa39a39cb7e7738ae8e831a9d26
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186117
Reviewed-by: Damien Neil <dneil@google.com>
2019-08-07 22:59:30 +00:00
Joe Tsai
8d5e6d6927 cmd/protoc-gen-go: improve generation of comments
The following improvements were made:
* All standalone comments above the "syntax" marker are preserved
similar to Java and some other generators.
* All standalone comments above the "package" marker are preserved
to be consistent with our former behavior.
* Leading comments are now generated for enums and extension fields.
* Single-line trailing comments are now generated for
enum values, message fields, and extension fields.
* The leading comments for each field that is part of a oneof are now
generated with the wrapper types rather than being shoved into the
comment for the oneof itself in an unreadable way.
* The deprecation marker is always generated as being above the declaration
rather than sometimes being an inlined comment.
* The deprecation marker is now properly generated for weak field setters.

Updates golang/protobuf#666

Change-Id: I7fd832dd4f86d15bfff70d7c22c6ba4934c05fcf
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189238
Reviewed-by: Damien Neil <dneil@google.com>
2019-08-07 17:33:08 +00:00
Damien Neil
f5274511fe all: add NewField, NewElement, NewValue
Add methods to protoreflect.{Message,List,Map} to constrict values
assignable to a message field, list element, or map value. These
methods return the default value for scalar fields, the zero value for
scalar list elements and map values, and an empty, mutable value for
messages, lists, and maps.

Deprecate the NewMessage methods on these types, which are superseded.

Updates golang/protobuf#879

Change-Id: I0f064f60c89a239330ccea81523f559f14fd2c4f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/188997
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-08-06 21:54:30 +00:00
Joe Tsai
9b8a433283 cmd/protoc-gen-go: group enum map vars in blocks
Since the enum maps are here to stay, group the declarations together
in a var block for better readability in godoc.

Change-Id: I9a313266539e9a60781f98b80a5293379f82607b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189077
Reviewed-by: Damien Neil <dneil@google.com>
2019-08-06 21:33:36 +00:00
Joe Tsai
d29a71bff0 cmd/protoc-gen-go: group default consts and vars in blocks
Group the default constant and variable declarations together in a block
for better readability in godoc.

Change-Id: I6b62f5374f0302d0f7cb224cbe34102359c8c51d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189057
Reviewed-by: Damien Neil <dneil@google.com>
2019-08-06 21:23:53 +00:00
Joe Tsai
38b6196f13 cmd/protoc-gen-go: group generation of internal fields together
These were originally kept separate to assist Google-internal patches,
but it turns out that Google-internal patches do not use the
genMessageInternalFields function.

Change-Id: Idfa962b943d3bede9982b5b0875ba90c86c6d181
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/188979
Reviewed-by: Damien Neil <dneil@google.com>
2019-08-05 20:58:04 +00:00
Joe Tsai
fb3ff727f0 cmd/protoc-gen-go: refactor logic to be more compartmentalized
This CL makes no feature changes except to move code around.
The only change to the actual generated code is the placement of
the default constants and variables. They move because the new logic
generates all methods together, while previously the constants
were interspersed in-between.

Change-Id: I45932d5aeec5ba45180fb255ea17898beb6c3bd2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186878
Reviewed-by: Herbie Ong <herbie@google.com>
2019-07-19 08:16:51 +00:00
Joe Tsai
43761bdfe7 cmd/protoc-gen-go: update deprecation warning
Change-Id: Ie1a854bf6f47d4ca9e941b6ccc64dc24ff32bd19
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186657
Reviewed-by: Damien Neil <dneil@google.com>
2019-07-18 02:26:18 +00:00
Joe Tsai
af57087245 reflect/protoregistry: provide more informative errors for conflicts
The v2 implementation strictly enforces that there are no conflicts at
all in the protobuf namespace unlike the prior v1 implementation.
This change is almost certainly going to cause loud failures for users
that were unknowingly tolerating registration conflicts.

We modify internal/filedesc to be able to record the Go package path
that the file descriptor is declared within. This information is used
by reflect/protoregistry to print both the previous Go package that
registered some declaration, and current Go package that is attempting
to register some declaration.

Change-Id: Ib5eb21c1c98495afc51aa08bd4404bd9d64b5b57
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186177
Reviewed-by: Damien Neil <dneil@google.com>
2019-07-15 20:39:24 +00:00