Commit Graph

144 Commits

Author SHA1 Message Date
Lasse Folger
f9eb6c71e7 go_features.proto: move .proto file next to generated .pb.go file
Change-Id: Ia24ca1ad2186bcc95681081d2719ee834b385728
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/565955
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
2024-02-22 08:56:15 +00:00
Lasse Folger
e39100579e reflect/protodesc: propagate field_presence legacy_required properly
Change-Id: I3f067eb564349bc1ff867a03d9f1aa94b3428236
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/563735
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
2024-02-13 15:44:19 +00:00
Lasse Folger
020ec10f7b internal/filedesc: implement runtime editions support
This change add parsing editions specific descriptor parts to
internal/filedesc which makes it possible to load file descriptors of
protos using editions at runtime.

Change-Id: I7a0c23303acddd2cff115859d4cf82bf0102b14c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/563615
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-02-13 12:20:49 +00:00
Lasse Folger
f387b036d8 editions: add some test protos for editions
I'll add the runtime implementations and tests in a followup change to
make it easier to review.

Change-Id: I2917e0d40a99e81799d89a66584a680ed7e8dbf7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/561939
Auto-Submit: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-02-12 14:14:45 +00:00
Lasse Folger
2040e8671b all: implement Go-specific GenerateLegacyUnmarshalJSON
This change adds the first Go specific editions feature and the
associated proto. The feature is used to control if the legacy
UnmarshalJSON method should be generated for enums. This change only
affects protos using editions.

More tests will be added in a followup change.

Change-Id: Ifb62454d7568bd6d90d0b93f8953adcfe46c45fd
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/561938
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
2024-02-12 13:54:51 +00:00
Cody Schroeder
cf06b0c33c compiler/protogen: add Semantic.SET to setter annotations
Provide an API to add the GeneratedCodeInfo.Annotation.Semantic enum to
annotations.

Change-Id: I92ab30619a94a117679a0eb16d8cb5b3a1352586
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/489795
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-05-01 15:43:20 +00:00
googleberg
eba8b0975f cmd/protoc-gen-go: support protobuf retention feature
This change strips out all descriptor option fields marked with [retention = RETENTION_SOURCE] before writing the FileDescriptor to the generated go code bindings.

Change-Id: Ie624d9d4b4f211a256661d80a04199ae8401662b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/472696
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
2023-03-07 09:24:00 +00:00
Josh Humphries
bc1253ad37 types/descriptorpb: regenerate using latest protobuf v22.0 release
This updates all generated code to match the contents of the latest
v22.0 release of Protobuf.

This involved a couple of changes to the script that does the sync'ing:
  1. The new Protobuf version no longer includes autoconf configuration
     and instead requires using bazel to build things.
  2. The new Protobuf release does not have an artifact named
     "protobuf-all-${VERSION}.tar.gz", but the one named
     "protobuf-${VERSION}.tar.gz" has all of the sources and was
     sufficient for the regenerate.bash script to complete.

This change does NOT regenerate the protos related to benchmarks.
The Protobuf repo no longer includes benchmarks. The CL removing them
says they are superceded by google/fleetbench:
    83c499de86
But that project's proto benchmark files are very different:
    https://github.com/google/fleetbench/tree/main/fleetbench/proto
So I commented out those steps in the generation code since the benchmarks
will need some work to reconcile with fleetbench.

This code adds known failing tests for conformance. New test cases were
added in https://github.com/protocolbuffers/protobuf/pull/9534, but the
Go protojson package does not behave according to the new tests. There
is an existing issue in GitHub about this:
    https://github.com/golang/protobuf/issues/1208

Change-Id: Iad796ec7889bc2a74b58db5224facf850cd1a1cd
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/469255
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-02-22 09:33:03 +00:00
Michael Stapelberg
a9481185b3 cmd/protoc-gen-go: make deprecated messages more descriptive
Also respect the file-wide deprecation option by treating every
identifier within the file as deprecated.

Change-Id: Ic3a5e84c2d90bbcbefc0a1b3272cc50554b94218
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/462315
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-01-18 09:34:59 +00:00
Joe Tsai
acaef6adb7 cmd/protoc-gen-go: avoid referencing remote enum values by name
The Go generator has historically always prefixed an enum value
with the name of the enum type, when it was unnecessary to do so.

For example:
	enum Status {
		STATUS_FAILED = 0;
		STATUS_PASSED = 1;
	}
would be generated as:
	type Status int32
	const (
		Status_STATUS_FAILED Status = 0
		Status_STATUS_PASSED Status = 1
	)

It is common for the enum values to be manually prefixed by the
enum type since protobuf enums use C++ namespace rules where
enum types and enum values are in the same namespace scope.
Thus, having the Go generator add a prefix is redundant.
See https://github.com/golang/protobuf/issues/513.

Some custom Go generators like protoc-gen-gogo allow removing
the prefix with the gogoproto.goproto_enum_prefix feature.
However, this leads to interoperability issues between
protoc-gen-go and protoc-gen-gogo, where the enum value names
cannot be accurately inferred.

Avoid this problem by just hard-coding the enum value number
for values declared in other packages. This provides benefits
in interoperability at the small cost of enum values possibly
being stale if their value were ever changed in a remote package.
However, this would only occur with use of proto2 enums and
default values, which seems to be an exceptionally rare situation.

Before:
	Default_MyMessage_MyField = remotepb.FooEnum_FOO_ENUM
After:
	Default_MyMessage_MyField = remotepb.FooEnum(4) // remotepb.FooEnum_FOO_ENUM

Before:
	func (x *MyMessage) GetField() remotepb.FooEnum {
		...
		return remotepb.FooEnum_FOO_ZERO
	}
After:
	func (x *MyMessage) GetField() remotepb.FooEnum {
		...
		return remotepb.FooEnum(0) // always 0 for proto3 and often 0 for proto2
	}

Change-Id: I3a06cd553f2eaf6124666f6c36c196d500d35718
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/319649
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
2021-05-24 22:21:56 +00:00
Joe Tsai
31610fd913 cmd/protoc-gen-go: remove generation of the ExtensionRangeArray method
The ExtensionRangeArray was a pseudo-internal API that was used by
the protobuf runtime implementation to know what the extension ranges are.
For some time now, the runtime implementation does not make use of this.

According to all the latest modules known by the module proxy,
no code depends directly on this method being present.

In the extremely rare case where some user code is depending on this method,
the user can place a .go file next to the generated .pb.go file that
injects this method.

Change-Id: Iae40a9a33b8c9b5a243d48db14f25b05ca24e3dc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/305574
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
2021-03-29 20:25:28 +00:00
Joe Tsai
e7c3f89377 cmd/protoc-gen-go: remove name mangling for MessageSet extensions
MessageSet are a proto1 feature only used by Google.
We no longer rely on the name mangling logic performed here,
so remove it.

Change-Id: I5d66ebd86875894632f0d0c1e9816ae47ee0d5f4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/242657
Reviewed-by: Herbie Ong <herbie@google.com>
2020-07-14 22:22:58 +00:00
Damien Neil
c92333953e internal/cmd/generate-protos: use module= generator option
Now we'll get a generator error if a file is missing a go_package
option.

Change-Id: I89eec716f86956e6c164a61f5531c140b74bc099
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/222378
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2020-03-20 21:35:56 +00:00
Joe Tsai
bb941fe881 cmd/protoc-gen-go/testdata: remove go.mod file
In the distant past this was necessary because:
1) the generated code depended on github.com/golang/protobuf,
but now it doesn't.
2) protoc-gen-go-grpc had a dependency on google.golang.org/grpc,
and the seperate go.mod file allowed us to isolate the dependency,
but that binary doesn't exist here anymore.

Drop the go.mod file for cmd/protoc-gen-go/testdata.

Change-Id: I7341d78dc346bd82e5c68ed48cee3275b6296249
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/220502
Reviewed-by: Damien Neil <dneil@google.com>
2020-02-24 22:15:38 +00:00
Damien Neil
3a18560b88 all: add a test for copyright headers
Fix a few files with no or unusual copyright headers.

Manually add copyright headers to the legacy testprotos, which will
(probably) never be regenerated.

Change-Id: Ifb52fa0eb59bf9d2de40b8df7e581a3c9731c883
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/220498
Reviewed-by: Joe Tsai <joetsai@google.com>
2020-02-21 18:41:46 +00:00
Damien Neil
0232edc1d7 all: remove stray "." from license headers
Remove a stray bit of punctuation that crept into one of the license
headers and got copied around everywhere.

Change-Id: Iebe4e882650ab6dab28f132b5e324e2ab0b99a73
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/220339
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2020-02-20 18:54:38 +00:00
Damien Neil
7059483cfd all: update go.mod in submodules
Change-Id: Ibeabaecd3ba55b03dea554fc9fd28f647e9c66fd
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/219398
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2020-02-14 21:20:35 +00:00
Damien Neil
3c5fb5f879 all: make .proto file names relative to module root
Change the protoc flags such that when one of our test .proto files
imports another, the filename is consistently specified relative to the
module root.

Change-Id: I690282795cef23347c8794c1c6357e4fe9560d8a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/217762
Reviewed-by: Joe Tsai <joetsai@google.com>
2020-02-04 23:20:20 +00:00
Damien Neil
c975a7097d reflect/protoregistry: remove deprecated APIs
Remove previously deprecated types, functions, and methods.

Update github.com/golang/protobuf module version to one which does not
depend on any deprecated APIs.

Fixexs golang/protobuf#963

Change-Id: Ida451ef5ef3f34830808f737cc0d1c98f32ce76a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/206017
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-11-08 01:03:52 +00:00
Joe Tsai
ae313d4af3 cmd/protoc-gen-go: fix Reset method
The MessageInfo cache, once set, must not be cleared, otherwise
there exists a *messageState value where the MessageInfo value is nil.
Fix the generation of the Reset method to avoid clearing this value.

Change-Id: Ic84ca8b2640a43e967c36993da1ccd3f2b7096c4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/201478
Reviewed-by: Damien Neil <dneil@google.com>
2019-10-17 17:10:14 +00:00
Damien Neil
95d75b7143 go.mod: update github.com/golang/protobuf
Drop use of the MessageType.GoType method.

Change-Id: I50278e54b04556fe6e2830756a5689be1edc9235
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/198997
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-10-04 17:38:12 +00:00
Damien Neil
3cda377ed2 all: rename ExtensionType Descriptor method to TypeDescriptor (2/2)
Remove the ExtensionType Descriptor method.

Change-Id: I89c985c45f2a5abc4e4e3770f9652bc2a444251e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/192141
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-08-28 19:46:47 +00:00
Joe Tsai
08ff730048 all: add go1.13rc1 to list of supported Go versions
Change-Id: I2ba52055bcb0e434b66dbcd4763afd2fce5b4264
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/191179
Reviewed-by: Herbie Ong <herbie@google.com>
2019-08-21 21:14:39 +00:00
Joe Tsai
945a1706d0 all: rely on message_set_extension name mangling in ExtensionDescV1
Too much code inside Google rely on ExtensionDescV1.Name being the
target message name rather than the real name of the extension.
Double down on this assumption consistently across our codebase.

Change-Id: If0b7cd16dd1f2f7acfb2e562dbf1a2b5487162b5
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186980
Reviewed-by: Herbie Ong <herbie@google.com>
2019-08-21 02:47:33 +00:00
Damien Neil
f247a903b9 internal/impl: clean up obsolete ExtensionInfo fields/funcs
This is change 5/5 in a series of commits changing protoV1.ExtensionDesc
to directly implement protoreflect.ExtensionType.

1. [v2] Add protoimpl.ExtensionInfo as an alias for
   protoiface.ExtensionDescV1.

2. [v1] Update references to protoimpl.ExtensionInfo to use
   protoiface.ExtensionInfo.

3. [v2] Create protoimpl.ExtensionInfo (an alias to a new type in
   the impl package) and remove protoiface.ExtensionDescV1.

4. [v1] Remove unneeded explicit conversions between ExtensionDesc and
   ExtensionType (since the former now directly implements the latter).

5. [v2] Remove stub conversion functions.

Change-Id: I1fdfb18c481e72a362a6f9ee0e440f8f909790ca
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189564
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-08-20 22:03:04 +00:00
Damien Neil
f1e905b042 all: unify protoV1.ExtensionDesc and proto.ExtensionType
Change protoV1.ExtensionDesc to directly implement ExtensionType
rather than delegating to one.

Unify the previous types protoiface.ExtensionDescV1 and
filetype.Extension in impl.ExtensionInfo. The protoV1.ExtensionDesc
type becomes an alias to ExtensionInfo.

This gives us:

  - Just one implementation of ExtensionType.
  - Generated foopb.E_Ext vars are canonical ExtensionTypes.
  - Generated foopb.E_Ext vars are also v1.ExtensionDescs for backwards
    compatibility.
  - Conversion between legacy and modern representations happens
    transparently when lazily initializing an ExtensionInfo.

Overall, a simplification for users of generated code, since they can
mostly ignore the ExtensionDesc/ExtentionType distinction and use the
same value in either the old or new API.

This is change 3/5 in a series of commits changing protoV1.ExtensionDesc
to directly implement protoreflect.ExtensionType.

1. [v2] Add protoimpl.ExtensionInfo as an alias for
   protoiface.ExtensionDescV1.

2. [v1] Update references to protoimpl.ExtensionInfo to use
   protoiface.ExtensionInfo.

3. [v2] Create protoimpl.ExtensionInfo (an alias to a new type in
   the impl package) and remove protoiface.ExtensionDescV1.

4. [v1] Remove unneeded explicit conversions between ExtensionDesc and
   ExtensionType (since the former now directly implements the latter).

5. [v2] Remove stub conversion functions.

Change-Id: I96ee890541ec11b2412e1a72c9d7b96e4d7f66b4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189563
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-08-20 21:32:57 +00:00
Joe Tsai
7328839f81 cmd/protoc-gen-go: annotate depIdxs list with index comments
Generate the current index into depIdxs for easier human debugging.

Change-Id: Ida42aa95137b2044a4dc267c31cebec5023bdfb1
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/190278
Reviewed-by: Herbie Ong <herbie@google.com>
2019-08-15 22:47:44 +00:00
Damien Neil
4401a0de4b cmd/protoc-gen-go, internal/filetype: clean up EnumType construction
Drop the dependency from generated files on prototype.Enum: Generated
code should only depend on runtime/proto{iface,impl}.

Drop the Enums, Messages, and Extensions returns form
filetype.Builder.Build. Of these, only Enums was used by generated code.
Change the generated init function to pass the builder a slice of values
to fill in (as is done for messages and extensions).

Remove the filetype dependency on prototype in preparation for
eventually dropping the prototype package entirely.

Change-Id: I28a3420f5dfcc13fed531a64ef07b9afddfd9d55
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189200
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-08-09 19:04:41 +00:00
Damien Neil
92f76189a3 all: refactor extensions, add proto.GetExtension etc.
Change protoiface.ExtensionDescV1 to implement protoreflect.ExtensionType.

ExtensionDescV1's Name field conflicts with the Descriptor Name method,
so change the protoreflect.{Message,Enum,Extension}Type types to no
longer implement the corresponding Descriptor interface. This also leads
to a clearer distinction between the two types.

Introduce a protoreflect.ExtensionTypeDescriptor type which bridges
between ExtensionType and ExtensionDescriptor.

Add extension accessor functions to the proto package:
proto.{Has,Clear,Get,Set}Extension. These functions take a
protoreflect.ExtensionType parameter, which allows writing the
same function call using either the old or new API:

  proto.GetExtension(message, somepb.E_ExtensionFoo)

Fixes golang/protobuf#908

Change-Id: Ibc65d12a46666297849114fd3aefbc4a597d9f08
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189199
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-08-08 18:20:51 +00:00
Joe Tsai
bab3d4084e runtime/protoimpl, cmd/protoc-gen-go: support release versioning
In order for protoc-gen-go to output the current version,
it needs to know what version it is currently running as.
However, we cannot rely on the git tags since the tags are not
made until *after* the commit has been submitted.
Instead, we manually encode the version into the code and
make sure that git tags match up with the version in the code.

The version.go file in runtime/protoimpl contains instructions
for how to make a release. Essentially:
* Every non-release commit has a version string with "devel" in it.
* Every release commit must not have "devel" in it and must be unique.
* The "release process" involves submitting two CLs.
The first CL creates a version string without "devel",
which is the commit that a git tag will actually reference.
The second CL follows immediately and re-introduces "devel"
into the version string.

The following example shows a possible sequence of VersionStrings
for git commits in time-ascending order:
	v1.19.0-devel      (this CL)
	v1.19.0-devel
	v1.19.0-devel
	v1.19.0-devel
	v1.20.0-rc.1       <- tagged
	v1.20.0-rc.1.devel
	v1.20.0-rc.1.devel
	v1.20.0-rc.1.devel
	v1.20.0-rc.2       <- tagged
	v1.20.0-rc.2.devel
	v1.20.0            <- tagged (future public release)
	v1.20.0-devel
	v1.20.0-devel
	v1.20.0-devel
	v1.20.0-devel
	v1.20.1            <- tagged
	v1.20.1-devel
	v1.20.1-devel
	v1.21.0            <- tagged
	v1.21.0-devel

Note that we start today with v1.19.0-devel, which means that our initial
release will be v1.20.0. This number was intentionally chosen since
1) the number 20 has some correlation to the fact that we keep calling
the new implementation the "v2" implementation, and
2) the set of tagged versions for github.com/golang/protobuf
and google.golang.org/protobuf are unlikely to ever overlap.
This way, the version of protoc-gen-go is never ambiguous which module
it was built from.

Now that we have version information, we add support for generating .pb.go
files with the version information recorded. However, we do not emit
these for .pb.go files in our own repository since they are always guaranteed
to be at the right version (enforced by integration_test.go).

Updates golang/protobuf#524

Change-Id: I25495a45042c2aa39a39cb7e7738ae8e831a9d26
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186117
Reviewed-by: Damien Neil <dneil@google.com>
2019-08-07 22:59:30 +00:00
Joe Tsai
8d5e6d6927 cmd/protoc-gen-go: improve generation of comments
The following improvements were made:
* All standalone comments above the "syntax" marker are preserved
similar to Java and some other generators.
* All standalone comments above the "package" marker are preserved
to be consistent with our former behavior.
* Leading comments are now generated for enums and extension fields.
* Single-line trailing comments are now generated for
enum values, message fields, and extension fields.
* The leading comments for each field that is part of a oneof are now
generated with the wrapper types rather than being shoved into the
comment for the oneof itself in an unreadable way.
* The deprecation marker is always generated as being above the declaration
rather than sometimes being an inlined comment.
* The deprecation marker is now properly generated for weak field setters.

Updates golang/protobuf#666

Change-Id: I7fd832dd4f86d15bfff70d7c22c6ba4934c05fcf
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189238
Reviewed-by: Damien Neil <dneil@google.com>
2019-08-07 17:33:08 +00:00
Joe Tsai
4a7d633604 cmd/protoc-gen-go: group extension variable declarations
For better readability in godoc, group extension fields by the
target message that they are extending.

Change-Id: Icc0a247b37639e3dbf7a107810923b8ca8294724
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189257
Reviewed-by: Damien Neil <dneil@google.com>
2019-08-07 04:26:34 +00:00
Joe Tsai
9b8a433283 cmd/protoc-gen-go: group enum map vars in blocks
Since the enum maps are here to stay, group the declarations together
in a var block for better readability in godoc.

Change-Id: I9a313266539e9a60781f98b80a5293379f82607b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189077
Reviewed-by: Damien Neil <dneil@google.com>
2019-08-06 21:33:36 +00:00
Joe Tsai
d29a71bff0 cmd/protoc-gen-go: group default consts and vars in blocks
Group the default constant and variable declarations together in a block
for better readability in godoc.

Change-Id: I6b62f5374f0302d0f7cb224cbe34102359c8c51d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189057
Reviewed-by: Damien Neil <dneil@google.com>
2019-08-06 21:23:53 +00:00
Joe Tsai
38b6196f13 cmd/protoc-gen-go: group generation of internal fields together
These were originally kept separate to assist Google-internal patches,
but it turns out that Google-internal patches do not use the
genMessageInternalFields function.

Change-Id: Idfa962b943d3bede9982b5b0875ba90c86c6d181
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/188979
Reviewed-by: Damien Neil <dneil@google.com>
2019-08-05 20:58:04 +00:00
Joe Tsai
fb3ff727f0 cmd/protoc-gen-go: refactor logic to be more compartmentalized
This CL makes no feature changes except to move code around.
The only change to the actual generated code is the placement of
the default constants and variables. They move because the new logic
generates all methods together, while previously the constants
were interspersed in-between.

Change-Id: I45932d5aeec5ba45180fb255ea17898beb6c3bd2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186878
Reviewed-by: Herbie Ong <herbie@google.com>
2019-07-19 08:16:51 +00:00
Joe Tsai
43761bdfe7 cmd/protoc-gen-go: update deprecation warning
Change-Id: Ie1a854bf6f47d4ca9e941b6ccc64dc24ff32bd19
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186657
Reviewed-by: Damien Neil <dneil@google.com>
2019-07-18 02:26:18 +00:00
Joe Tsai
ad8dff3eec all: update to github.com/golang/protobuf@b9f5089f
Change-Id: I6178e0552630f2eb11f0ad95d77f23527708e5ec
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186638
Reviewed-by: Damien Neil <dneil@google.com>
2019-07-18 00:01:40 +00:00
Joe Tsai
af57087245 reflect/protoregistry: provide more informative errors for conflicts
The v2 implementation strictly enforces that there are no conflicts at
all in the protobuf namespace unlike the prior v1 implementation.
This change is almost certainly going to cause loud failures for users
that were unknowingly tolerating registration conflicts.

We modify internal/filedesc to be able to record the Go package path
that the file descriptor is declared within. This information is used
by reflect/protoregistry to print both the previous Go package that
registered some declaration, and current Go package that is attempting
to register some declaration.

Change-Id: Ib5eb21c1c98495afc51aa08bd4404bd9d64b5b57
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186177
Reviewed-by: Damien Neil <dneil@google.com>
2019-07-15 20:39:24 +00:00
Joe Tsai
d421150c3b reflect/protoreflect: add Enum.Type and Message.Type
CL/174938 removed these methods in favor of a method that returned
only the descriptors. This CL adds back in the Type methods alongside
the Descriptor methods.

In a vast majority of protobuf usages, only the descriptor information
is needed. However, there is a small percentage that legitimately needs
the Go type information. We should provide both, but document that the
descriptor-only information is preferred.

Change-Id: Ia0a098997fb1bd009994940ae8ea5257ccd87cae
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184578
Reviewed-by: Damien Neil <dneil@google.com>
2019-07-12 23:43:21 +00:00
Joe Tsai
0f81b38d61 runtime/protoiface: move and rename XXX_Methods
This CL moves and renames the protoreflect.ProtoMessage.XXX_Methods
to protoreflect.Message.ProtoMethods.

Since one needs to obtain a protoreflect.Message now to get at
the fast-path methods, we modify the method signatures to take
in a protoreflect.Message instead of protoreflect.ProtoMessage.
Doing so also avoids the wrapper hack that was formerly done on
impl.messageReflectWrapper.

After this change the new protoc-gen-go no longer generates
any XXX fields or methods. All internal fields and methods are truly
hidden from the end-user.

name                                     old time/op    new time/op    delta
Wire/Unmarshal/google_message1_proto2-4    1.50µs ±10%    1.50µs ± 2%    ~     (p=0.483 n=10+9)
Wire/Unmarshal/google_message1_proto3-4    1.06µs ± 6%    1.06µs ± 4%    ~     (p=0.814 n=9+9)
Wire/Unmarshal/google_message2-4            734µs ±22%     689µs ±13%    ~     (p=0.133 n=10+9)
Wire/Marshal/google_message1_proto2-4       790ns ±46%     652ns ± 8%    ~     (p=0.590 n=10+9)
Wire/Marshal/google_message1_proto3-4       872ns ± 4%     857ns ± 3%    ~     (p=0.168 n=9+9)
Wire/Marshal/google_message2-4              232µs ±16%     221µs ± 3%  -4.75%  (p=0.014 n=9+9)
Wire/Size/google_message1_proto2-4          164ns ± 2%     167ns ± 4%  +1.87%  (p=0.046 n=9+10)
Wire/Size/google_message1_proto3-4          240ns ± 9%     229ns ± 1%  -4.81%  (p=0.000 n=9+8)
Wire/Size/google_message2-4                58.9µs ± 9%    59.6µs ± 2%  +1.23%  (p=0.040 n=9+9)

name                                     old alloc/op   new alloc/op   delta
Wire/Unmarshal/google_message1_proto2-4      912B ± 0%      912B ± 0%    ~     (all equal)
Wire/Unmarshal/google_message1_proto3-4      688B ± 0%      688B ± 0%    ~     (all equal)
Wire/Unmarshal/google_message2-4            470kB ± 0%     470kB ± 0%    ~     (p=0.215 n=10+10)
Wire/Marshal/google_message1_proto2-4        240B ± 0%      240B ± 0%    ~     (all equal)
Wire/Marshal/google_message1_proto3-4        224B ± 0%      224B ± 0%    ~     (all equal)
Wire/Marshal/google_message2-4             90.1kB ± 0%    90.1kB ± 0%    ~     (all equal)
Wire/Size/google_message1_proto2-4          0.00B          0.00B         ~     (all equal)
Wire/Size/google_message1_proto3-4          0.00B          0.00B         ~     (all equal)
Wire/Size/google_message2-4                 0.00B          0.00B         ~     (all equal)

name                                     old allocs/op  new allocs/op  delta
Wire/Unmarshal/google_message1_proto2-4      24.0 ± 0%      24.0 ± 0%    ~     (all equal)
Wire/Unmarshal/google_message1_proto3-4      6.00 ± 0%      6.00 ± 0%    ~     (all equal)
Wire/Unmarshal/google_message2-4            8.49k ± 0%     8.49k ± 0%    ~     (all equal)
Wire/Marshal/google_message1_proto2-4        1.00 ± 0%      1.00 ± 0%    ~     (all equal)
Wire/Marshal/google_message1_proto3-4        1.00 ± 0%      1.00 ± 0%    ~     (all equal)
Wire/Marshal/google_message2-4               1.00 ± 0%      1.00 ± 0%    ~     (all equal)
Wire/Size/google_message1_proto2-4           0.00           0.00         ~     (all equal)
Wire/Size/google_message1_proto3-4           0.00           0.00         ~     (all equal)
Wire/Size/google_message2-4                  0.00           0.00         ~     (all equal)

Change-Id: Ibf3263ad0f293326695c22020a92a6b938ef4f65
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185697
Reviewed-by: Damien Neil <dneil@google.com>
2019-07-12 19:31:58 +00:00
Joe Tsai
82760ceffa internal/impl: add MessageState to every generated message
We define MessageState, which is essentially an atomically set *MessageInfo.
By nesting this as the first field in every generated message, we can
implement the reflective methods on a *MessageState when obtained by
unsafe casting a concrete message pointer as a *MessageState.
The MessageInfo held by MessageState provides additional Go type information
to interpret the memory that comes after the contents of the MessageState.

Since we are nesting a MessageState in every message,
the memory use of every message instance grows by 8B.

On average, the body of ProtoReflect grows from 133B to 202B (+50%).
However, this is offset by XXX_Methods, which is 108B and
will be removed in a future CL. Taking into account the eventual removal
of XXX_Methods, this is a net reduction of 25%.

name          old time/op    new time/op    delta
Name/Value-4    70.3ns ± 2%    17.5ns ± 6%   -75.08%  (p=0.000 n=10+10)
Name/Nil-4      70.6ns ± 3%    33.4ns ± 2%   -52.66%  (p=0.000 n=10+10)

name          old alloc/op   new alloc/op   delta
Name/Value-4     16.0B ± 0%      0.0B       -100.00%  (p=0.000 n=10+10)
Name/Nil-4       16.0B ± 0%      0.0B       -100.00%  (p=0.000 n=10+10)

name          old allocs/op  new allocs/op  delta
Name/Value-4      1.00 ± 0%      0.00       -100.00%  (p=0.000 n=10+10)
Name/Nil-4        1.00 ± 0%      0.00       -100.00%  (p=0.000 n=10+10)

Change-Id: I92bd58dc681c57c92612fd5ba7fc066aea34e95a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185460
Reviewed-by: Damien Neil <dneil@google.com>
2019-07-10 19:44:24 +00:00
Joe Tsai
6ceeaab1ba cmd/protoc-gen-go: remove MessageSet hackery
The encoding/prototext and encoding/protojson are implemented entirely
in terms of protobuf reflection, which side-steps this information.
Remove the hacks in the generator to special-case MessageSet.

Change-Id: I708c4636b77672545a103b7ab686f103b9dfc514
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185240
Reviewed-by: Herbie Ong <herbie@google.com>
2019-07-08 21:17:17 +00:00
Joe Tsai
c0e4bb2054 cmd/protoc-gen-go: unexport implementation-specific XXX fields
We modify protoc-gen-go to stop generating exported XXX fields.
The unsafe implementation is unaffected by this change since unsafe
can access fields regardless of visibility. However, for the purego
implementation, we need to respect Go visibility rules as enforced
by the reflect package.

We work around this by generating a exporter function that given
a reference to the message and the field to export, returns a reference
to the unexported field value. This exporter function is protected by
a constant such that it is not linked into the final binary in non-purego
build environment.

Updates golang/protobuf#276

Change-Id: Idf5c1f158973fa1c61187ff41440acb21c5dac94
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185141
Reviewed-by: Damien Neil <dneil@google.com>
2019-07-08 20:45:09 +00:00
Joe Tsai
09912274cc cmd/protoc-gen-go: remove generation of XXX_OneofWrappers
Associate the oneof wrapper types with a message by conveying that
information to the associated MessageInfo.

Change-Id: Iabfca593850e1d6a89498a37eacbf22dbb73bd20
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185239
Reviewed-by: Damien Neil <dneil@google.com>
2019-07-08 20:36:12 +00:00
Joe Tsai
7198fb6edf cmd/protoc-gen-go/testdata: add registry test
This test verifies that the inialization logic does not accidentally
execute the lazy initialization logic that is intended to only evaluate
upon first use.

Change-Id: I5e9dea17ce88081c78195af0e86dd38223689f63
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184259
Reviewed-by: Damien Neil <dneil@google.com>
2019-07-08 20:23:18 +00:00
Joe Tsai
7ea76d2a4c cmd/protoc-gen-go: un-deprecate enum maps
If we were starting from scratch, we would not have added the enum maps.
However, they already exist and see a fair amount of usage.
The effort to remove them is not worth it. Thus, remove the deprecation
warning since they are here to stay.

Note that the generated code does not refer to the generated enum maps.
One day, the linker should be able to elide them if unused by the user.
However, https://golang.org/issue/2559 would need to be resolved first.

Change-Id: Ia8b9b1812b5d8462ca2fa1d543170e4a09ff9e4f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/183177
Reviewed-by: Damien Neil <dneil@google.com>
2019-06-20 19:36:37 +00:00
Joe Tsai
dfe160316f internal/fileinit: remove package
This package is no longer used.

Change-Id: Ib6d50490b07d6c5005f9e7e6c4b9cdc6ce6722fb
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/183101
Reviewed-by: Damien Neil <dneil@google.com>
2019-06-20 19:35:23 +00:00
Joe Tsai
d888139e7b internal/filedesc, internal/filetype: initial commit
The internal/fileinit package is split apart into two packages:
* internal/filedesc constructs descriptors from the raw proto.
It is very similar to the previous internal/fileinit package.
* internal/filetype wraps descriptors with Go type information

Overview:
* The internal/fileinit package will be deleted in a future CL.
It is kept around since the v1 repo currently depends on it.
* The internal/prototype package is deleted. All former usages of it
are now using internal/filedesc instead. Most significantly,
the reflect/protodesc package was almost entirely re-written.
* The internal/impl package drops support for messages that do not
have a Descriptor method (pre-2016). This removes a significant amount
of technical debt.
filedesc.Builder to parse raw descriptors.
* The internal/encoding/defval package now handles enum values by name.

Change-Id: I3957bcc8588a70470fd6c7de1122216b80615ab7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/182360
Reviewed-by: Damien Neil <dneil@google.com>
2019-06-20 02:06:11 +00:00
Joe Tsai
7a5e02648a reflect/protoreflect: remove {Enum,Message}.Type methods
This is a breaking change in light of new methods added in CL/174918.

This removes:
	Enum.Type: replacement is Descriptor
	Message.Type: replacement is Descriptor and New

Change-Id: Iaa5328795407c8401ef14ed038bd5ace19d8e03b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/174938
Reviewed-by: Damien Neil <dneil@google.com>
2019-06-17 19:43:39 +00:00