38 Commits

Author SHA1 Message Date
Joe Tsai
d7e97bc71b cmd/protoc-gen-go: generate XXX_OneofWrappers instead of XXX_OneofFuncs
The marshaler, unmarshaler, and sizer functions are unused ever since
the underlying implementation was switched to be table-driven.
Change the function to only return the wrapper structs.

This change:
* enables generated protos to drop dependencies on certain proto types
* reduces the size of generated protos
* simplifies the implementation of oneofs in protoc-gen-go

Updates #708

Change-Id: I845c9009bc0236d1b51d34b014dc3e184303c0f2
Reviewed-on: https://go-review.googlesource.com/c/151357
Reviewed-by: Damien Neil <dneil@google.com>
2018-11-27 19:36:27 +00:00
Damien Neil
6238696abc cmd/protoc-gen-go: replicate v1 generator behavior for MessageSet extensions
Given:

  package foo
  extend proto2.bridge.MessageSet {
    optional Message message_set_extension = 100;
  }

Register the extension as a message set extension and give it the name
"foo.".

We really shouldn't do this in this case; the special-case treatment of
extensions to MessageSet is only for extensions nested in a parent
message. However, this is consistent with the behavior of the v1 generator.
Match that for now.

Change-Id: I919c409605a197904fd3227efc920192d484f431
Reviewed-on: https://go-review.googlesource.com/c/145957
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-30 18:46:20 +00:00
Damien Neil
6b54131a89 cmd/protoc-gen-go: generate forwarding decls for ExtensionDescs
When publicly importing a package with extension definitions, generate
forwarding declarations for the "E_..." ExtensionDesc var:

    var E_ExtensionField = publicimport.E_ExtensionField

Change-Id: Ifd57c487c3a44f303c2c098a42ea249b219b734f
Reviewed-on: https://go-review.googlesource.com/c/145498
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-29 17:01:50 +00:00
Damien Neil
a485fbd353 reflect/protoreflect: add FieldDescriptor.DefaultEnumValue method
Default values for enums are specified by name, not number. An enum
may contain multiple values with different names but the same number.
Representing the default as a protoreflect.Value containing an EnumNumber
can discard information.

Add a method returning the EnumValueDescriptor.

Change-Id: If8beee3f81d41c4f9af45423252603b86949c7a5
Reviewed-on: https://go-review.googlesource.com/c/145158
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-29 15:59:23 +00:00
Damien Neil
44000a1bea cmd/protoc-gen-go: treat extensions fields as proto2 in some places
The v1 generator doesn't include a "proto3" tag on extension fields,
even when the field is defined in a proto3 file. Match that behavior for
consistency. We can probably change this later if we want to; it's
unlikely anyone is depending on this behavior.

The v1 generator uses pointer types for extension fields, even when the
field is defined in a proto3 file. (e.g., *FooEnum instead of FooEnum.)
Match this behavior. We can't change this without breaking compatibility
in the generated code.

Change-Id: I4072f3dd1c915bf9ab89f1d5198e0144cb4de20f
Reviewed-on: https://go-review.googlesource.com/c/144282
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-24 19:46:45 +00:00
Joe Tsai
bda671fa93 cmd/protoc-gen-go: re-escape default byte values
Historically, protoc-gen-go outputted the escaped form of bytes as provided by
protoc verbatim. This behavior is buggy, but nothing really uses this tag
since default values are properties of getters instead of serialization.
Rather than fixing it, just preserve prior behavior. Otherwise, logic depending
on the old legacy behavior will not be able to distinguish between the unescaped
or the escaped forms.

Furthermore, since protoc-gen-go historically copied the protoc output verbatim,
we will need to escape the default bytes in a way that is identical to the
CEscape function from strutil.cc of the protoc source code.

Change-Id: I0ab55e220ae430dd123ad050406e285788f6cb40
Reviewed-on: https://go-review.googlesource.com/c/143543
Reviewed-by: Damien Neil <dneil@google.com>
2018-10-22 17:14:19 +00:00
Damien Neil
162c12703c protogen: generate .meta file with code annotations
When the generator parameter 'annotate_code' is provided, generate a .meta
file containing a GeneratedCodeInfo message describing the generated code's
relation to the source .proto file.

Annotations are added with (*protogen.GeneratedFile).Annotate, which takes the
name of a Go identifier (e.g., "SomeMessage" or "SomeMessage.GetField") and an
associated source location. The generator examines the generated AST to
determine source offsets for the symbols.

Change the []int32 "Path" in protogen types to a "Location", which also captures
the source file name.

Change-Id: Icd2340875831f40a1f91d495e3bd7ea381475c77
Reviewed-on: https://go-review.googlesource.com/c/139759
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-16 17:22:41 +00:00
Damien Neil
2193e8df3b cmd/protoc-gen-go: import public: forward default const/vars
Generate forwarders for default value const/vars defined in public
imports:

    const Default_Message_Field = pubimport.Default_Message_Field

Change-Id: Ife09e38ae6a674b4460dd6613a8264e23f30b277
Reviewed-on: https://go-review.googlesource.com/c/140897
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-16 16:53:07 +00:00
Damien Neil
21f62f4d53 reflect/prototype: fix name resolution when file has no package name
A .proto source file with no 'package' statement may still contain
references to descriptors within the file.

Change-Id: I86e942c9c06e5a2915e9722162e0455ffa9ba2ab
Reviewed-on: https://go-review.googlesource.com/c/140899
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-16 16:16:46 +00:00
Damien Neil
daa4fad986 cmd/protoc-gen-go: remove spurious . from enums with no package
Enums, for historical reasons, are registered with the proto package
under the name "<proto_package>.<go_type_name>". Don't include the dot
if there is no package statement in the .proto source file.

Change-Id: I6fb57d0803506668f60123a29fa06ae87fec523b
Reviewed-on: https://go-review.googlesource.com/c/140657
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-16 16:04:35 +00:00
Damien Neil
09f89d0246 protoc-gen-go: generate per-field comments for oneofs
Comment location and formatting matches previous generator.

Change-Id: Id51489bb8ca6e6bbd2c7bc2129a7bc170e56ed44
Reviewed-on: https://go-review.googlesource.com/c/140617
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-16 16:04:15 +00:00
Damien Neil
b9781cc938 cmd/protoc-gen-go: convert field number to uint64 before shifting
Avoid generating invalid tag and wire size for large oneof field numbers
which overflow int32.

Change-Id: I005fe32aba4944b33b6b6ba83ef0ddd4d6e5863b
Reviewed-on: https://go-review.googlesource.com/138519
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-28 22:01:48 +00:00
Damien Neil
ccf3fa6cad cmd/protoc-gen-go: don't generate default values for [default=""].
The previous generator considers a default value of "" (valid only for
string and bytes fields) to be unset, and does not generate a const or
var to hold the default value.

e.g.,

  message M {
    optional F string = 1 [default=""];
  }

does not generate this constant, even though the field has a default
value:

  const Default_M_F string = ""

Maintain consistent output.

Change-Id: Ib172b02d59c15c05e19a7056d05ce1c619a2fa40
Reviewed-on: https://go-review.googlesource.com/138518
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-28 21:56:28 +00:00
Damien Neil
982684b22e cmd/protoc-gen-go: format default float values as float32
Printing default float values as float64 results in e.g., "3.14"
becoming "3.140000104904175".

Change-Id: I5b21a1f94084b6d9c7b015b34d911972c4a2a1b2
Reviewed-on: https://go-review.googlesource.com/138516
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-28 21:20:49 +00:00
Damien Neil
154da98aa5 cmd/protoc-gen-go: special cases for MessageSet extensions
Add special-case handling for extension fields named
"message_set_extension" that extend a message_set_wire_format message.

Support special cases for a proto1 feature that was superseded by proto2
extensions.

Change-Id: Icbdb711111c66be547bf8d6f37ab3079c320e2a1
Reviewed-on: https://go-review.googlesource.com/136536
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-24 17:25:31 +00:00
Joe Tsai
01ab29648e go.mod: rename google.golang.org/proto as github.com/golang/protobuf/v2
This change was created by running:
	git ls-files | xargs sed -i "s|google.golang.org/proto|github.com/golang/protobuf/v2|g"

This change is *not* an endorsement of "github.com/golang/protobuf/v2" as the
final import path when the v2 API is eventually released as stable.
We continue to reserve the right to make breaking changes as we see fit.

This change enables us to host the v2 API on a repository that is go-gettable
(since go.googlesource.com is not a known host by the "go get" tool;
and google.golang.org/proto was just a stub URL that is not currently served).
Thus, we can start work on a forked version of the v1 API that explores
what it would take to implement v1 in terms of v2 in a backwards compatible way.

Change-Id: Ia3ebc41ac4238af62ee140200d3158b53ac9ec48
Reviewed-on: https://go-review.googlesource.com/136736
Reviewed-by: Damien Neil <dneil@google.com>
2018-09-24 16:11:50 +00:00
Damien Neil
a6c374a8f1 cmd/protoc-gen-go: add test data for proto3 extensions
Change-Id: I2d47db2c177553dc2a3dacc66c70db579c1145fd
Reviewed-on: https://go-review.googlesource.com/136535
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-20 19:56:44 +00:00
Damien Neil
d4803f5a4f cmd/protoc-gen-go: add "packed" encoding tag to fields
Change-Id: I423039d6c9edb50a07781a6e76336ed364f21866
Reviewed-on: https://go-review.googlesource.com/136359
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-20 17:08:44 +00:00
Damien Neil
55fe1c0178 cmd/protoc-gen-go: generate deprecation comments
Generate deprecation comments on packages, enums, enum values, messages,
and fields.

Change-Id: I8a94aff535078d33d1cc6104cff17e062cbfe94f
Reviewed-on: https://go-review.googlesource.com/136355
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-19 22:46:31 +00:00
Damien Neil
81d6d83a09 cmd/protoc-gen-go: additional conflict resolution for oneof field types
This is purely for consistent output with the previous protoc-gen-go.

Consider this message:

  message M {
    oneof union { string conflict = 1; }
    message Conflict {}
  }

The type for the wrapper of M.conflict will have the same name as the
embedded message type for M.Conflict.

The previous protoc-gen-go performs a disambiguation step where it adds
_s to the names of oneof field types until they have no conflicts with
nested messages or enums in the same message as the field.

There are a number of ways in which this can fail, of course. Preserve
the behavior for now.

Change-Id: I78a1c6588b577324e003b8bc337b75bb363432ba
Reviewed-on: https://go-review.googlesource.com/136357
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-19 21:15:24 +00:00
Damien Neil
2e0c3da5a4 cmd/protoc-gen-go: generate blank imports for unused proto dependencies
Generate Go imports for all packages imported by the .proto source file,
even if they are not referenced.

Change-Id: I116bdf460ab441d205b42606b2b05b315ed68954
Reviewed-on: https://go-review.googlesource.com/136358
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-19 21:12:48 +00:00
Damien Neil
73ac885dab cmd/protoc-gen-go: generate public imports
Generate forwarding declarations for all types, consts, and vars in
publicly imported files.

Change-Id: I2f1041fa91b0ae18b1e123935951618b8d594f84
Reviewed-on: https://go-review.googlesource.com/136175
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-19 15:37:56 +00:00
Damien Neil
993c04dcf2 cmd/protoc-gen-go: generate extensions
Generate everything related to extensions: extension descriptors,
XXX_InternalExtensions fields, etc.

Tweak the order in which we generate code for descriptors to ensure
consistent output with the previous protoc-gen-go.

Change Field.ContainingType to Field.ParentMessage, since we need to get
at the parent of both message fields and extensions (and the "containing
type" of an extension field is the extended message, under existing
terminology).

Change-Id: I5d045ca80536436e7c987bca3d8fb8c1e1521e55
Reviewed-on: https://go-review.googlesource.com/136155
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-18 22:49:53 +00:00
Damien Neil
1fa78d8a35 cmd/protoc-gen-go: generate oneofs
Generate everything related to oneofs: Message struct fields, wrapper
types, XXX_OneofFuncs.

Change-Id: I409040e0deb5716afabf59186eeaae21757d29f1
Reviewed-on: https://go-review.googlesource.com/135535
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-18 21:10:04 +00:00
Damien Neil
1ec3315873 protoc-gen-go: generate imports consistent with previous protoc-gen-go
Include references to the proto, fmt, and math packages, consistent with
the ones in the previous generator:

	// Reference imports to suppress errors if they are not otherwise used.
	var _ = proto.Marshal
	var _ = fmt.Errorf
	var _ = math.Inf

Sort imports with ast.SortImports to ensure gofmt-compatible output.

Copy the protoc-gen-go/testdata/imports test files from
github.com/golang/protobuf.

Change-Id: I263e2ac750d03596ae9cb23609049d5ef1016205
Reviewed-on: https://go-review.googlesource.com/135361
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-14 22:40:55 +00:00
Damien Neil
ce36f8d1b4 cmd/protoc-gen-go: register messages and map field types
Move generation of the init function that registers all the types in a
file into a single function.

Take some care to generate the registrations in the same order as the
previous protoc-gen-go, to make it easier to catch unintended
differences in output.

For the same reason, adjust the order of generation to generate all
enums before all messages (matches previous behavior).

Change-Id: Ie0d574004d01a16f8d7b10be3882719a3c41676e
Reviewed-on: https://go-review.googlesource.com/135359
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-14 22:26:09 +00:00
Damien Neil
0bd5a38474 cmd/protoc-gen-go: add support for map fields
Generate the proper map[key]value type for map fields.
Include the protobuf_key and protobuf_val field tags.
Do not generate the map entry structs.

Fix an initialization order bug in protogen: While proto files cannot
contain circular dependencies, a single file can contain dependency
cycles. First create types for all the descriptors in a file, then add
in references (currently just field->message and field->enum) in a
second pass.

Change-Id: Ifedfa657d8dbb00413ba493adee1119b19c1b773
Reviewed-on: https://go-review.googlesource.com/135355
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-13 22:20:09 +00:00
Damien Neil
77f82fe6bc cmd/protoc-gen-go: generate getters
Refactor fieldGoType to return the non-pointer type for scalar types
(i.e., "int", not "*int"), and a bool indicating whether the struct
field is a pointer to that type.

Change-Id: Ic80220e92f0b190e41ead847440d57af5c6cc919
Reviewed-on: https://go-review.googlesource.com/135336
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-13 20:51:57 +00:00
Damien Neil
ebc699d099 cmd/protoc-gen-go: generate def= protobuf field tag, default constants
Change-Id: Id5cc34f0c1a5eb72f19e648844a5480827bcbda3
Reviewed-on: https://go-review.googlesource.com/135256
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-13 19:34:28 +00:00
Damien Neil
a1c6abc906 cmd/protoc-gen-go: generate common message methods
Generate the standard Reset, String, ProtoMessage, and Descriptor
methods for each message struct.

Generate the table-driven support methods (XXX_Marshal, et al.).

Change-Id: I8a5ed8af1433b481e2458ad534456692d6f74c1f
Reviewed-on: https://go-review.googlesource.com/134996
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-12 22:18:32 +00:00
Damien Neil
d412792d34 cmd/protoc-gen-go: reference proto.ProtoPackageIsVersionX
Include the compile-time assertion that the generated file is compatible
with the proto package it is being compiled against.

Change-Id: Iefa27ee3a99a6669a93303b6674b0033794ba0be
Reviewed-on: https://go-review.googlesource.com/134995
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-12 20:08:45 +00:00
Damien Neil
658051ba78 cmd/protoc-gen-go: generate message fields
This produces exactly the same output as github.com/golang/protobuf.

Change-Id: I01aacc9277c5cb5b4cc295f5ee8af12b4a524781
Reviewed-on: https://go-review.googlesource.com/134955
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-12 18:08:39 +00:00
Damien Neil
46abb57549 cmd/protoc-gen-go: generate enums
This produces exactly the same output (to the best of my ability to
determine) as github.com/golang/protobuf.

Change-Id: Ib60e7a836efb1eb0e5167b30458049ec239e7903
Reviewed-on: https://go-review.googlesource.com/134695
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-11 21:42:46 +00:00
Damien Neil
cab8dfee90 protogen: include source comments in generated code
Add initial handling of location paths (arrays of integers identifying
an entity in a .proto source file).

Expose path info in protogen; each descriptor has a Path field containing
its location path.

Format comments in protoc-gen-go. This contains one change from
github.com/golang/protobuf: Package comments are now included before the
package statement (but not attached to it) and use // comment syntax
instead of /* */. e.g.,

Before:

	package test

	/*
	This package contains interesting messages.
	*/

After:

	// This package contains interesting messages.

	package test

Change-Id: Ieee13ae77b3584f7562183100554d3df732348aa
Reviewed-on: https://go-review.googlesource.com/133915
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-11 17:19:32 +00:00
Damien Neil
7779e05b0a cmd/protoc-gen-go: register FileDescriptorProto
Generate the gzipped FileDescriptorProto var, and register it with
proto.RegisterFile at init time.

Change-Id: Ie232f20412ca9cd7bde91aaba7127dc181e9758c
Reviewed-on: https://go-review.googlesource.com/134118
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-10 20:31:17 +00:00
Damien Neil
082ce923d3 protogen: compute package names, import paths, generated filenames
Copy/duplicate the logic in github.com/golang/protobuf for computing
package names and import paths and the names of generated files.

This is all sufficiently complicated that the code is the best
documentation. In practice, users should always set a go_package option
containing an import path in every file and pass the
paths=source_relative generator flag to get reasonable behavior.

Change-Id: I34ae38fcc8db6909a4b25b16c73b982a7bad0463
Reviewed-on: https://go-review.googlesource.com/133876
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-07 17:58:10 +00:00
Damien Neil
c7d07d9ba5 protogen: generate message skeletons
Copy generator.CamelCase for camel-casing names, with one change: Convert
'.' in names to '_'. This removes the need for the CamelCaseSlice function
which operates on a []string representing a name split along '.'s.

Add protogen.Message.

Reformat generated code.

Add regenerate.bash, largely copied from regenerate.sh.

Change-Id: Iecf0bfc43b552f53e458499a328b933b0c9c5f82
Reviewed-on: https://go-review.googlesource.com/130915
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-08-23 22:10:13 +00:00
Damien Neil
220c20246b protogen, cmd/protoc-gen-go: initial commit
Package protogen provides support for writing protoc plugins.
A "plugin" in this case is a program run by protoc to generate output.

The protoc-gen-go command is a protoc plugin to generate Go code.

cmd/protoc-gen-go/golden_test.go is mostly a straight copy from
the golden test in github.com/golang/protobuf.

Change-Id: I332d0df1e4b60bb8cd926320b8721e16b99a4b71
Reviewed-on: https://go-review.googlesource.com/130175
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-08-22 17:08:04 +00:00