Commit Graph

28 Commits

Author SHA1 Message Date
Joe Tsai
411f33952f protogen: clarify that GoIdent.GoName is a single idenfitier
While we're at it, also add the license header for the file.

Change-Id: If33f1764111aff96efcdb5a4e88c2a70ae28c09c
Reviewed-on: https://go-review.googlesource.com/c/150060
Reviewed-by: Damien Neil <dneil@google.com>
2018-11-27 19:56:42 +00:00
Joe Tsai
c1c17aa013 protogen: add GoImportPath.Ident helper
The GoImportPath.Ident helper creates a GoIdent using the receiver
as the GoImportPath in the GoIdent. This helper helps with the construction
of qualified identifiers.

Example usage:
	const protoPackage = protogen.GoImportPath("github.com/golang/protobuf/proto")
	protoPackage.Ident("ExtensionRange") // produces "proto.ExtensionRange"

The advantage of this helper is that usage of it looks similar to how
the identifier will eventually be rendered.

This is significantly more readable than the current approach:
	protogen.GoIdent{
		GoImportPath: protoPackage,
		GoName: "ExtensionRange",
	}

Change-Id: If7ecd7e60fad12bc491eee0dcb05f8fdebc9c94e
Reviewed-on: https://go-review.googlesource.com/c/150058
Reviewed-by: Damien Neil <dneil@google.com>
2018-11-16 19:34:14 +00:00
Damien Neil
ba1159f426 protogen: move comment generation into protogen
Most plugins need to copy comments from .proto source files into the
generated code. Move this functionality into protogen to avoid
duplicating it everywhere.

Change-Id: I48a96ba794192e7ddc00281342afd4805ef6fe0f
Reviewed-on: https://go-review.googlesource.com/c/142890
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-17 21:03:49 +00:00
Damien Neil
162c12703c protogen: generate .meta file with code annotations
When the generator parameter 'annotate_code' is provided, generate a .meta
file containing a GeneratedCodeInfo message describing the generated code's
relation to the source .proto file.

Annotations are added with (*protogen.GeneratedFile).Annotate, which takes the
name of a Go identifier (e.g., "SomeMessage" or "SomeMessage.GetField") and an
associated source location. The generator examines the generated AST to
determine source offsets for the symbols.

Change the []int32 "Path" in protogen types to a "Location", which also captures
the source file name.

Change-Id: Icd2340875831f40a1f91d495e3bd7ea381475c77
Reviewed-on: https://go-review.googlesource.com/c/139759
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-16 17:22:41 +00:00
Damien Neil
d277b5236a protogen: produce an error if len(os.Args) > 1
Users sometimes run protoc-gen-go directly (rather than via protoc
--go_out) and are then confused when it sits and does nothing,
waiting for input from stdin. Print an error if any command-line
arguments are passed to it.

Change-Id: I10a059a82ec71b61c4eb6aeecc1e47e2446feabb
Reviewed-on: https://go-review.googlesource.com/c/139877
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-16 16:53:21 +00:00
Joe Tsai
42fa50da34 test.bash: do not run golden tests by default
The golden tests are sensitive to the exact version used:
* Protobuf toolchain since the generated Descriptor is dependendent on protoc
(for example, default json_names were auto-populated in later versions of protoc)
* Go toolchain since the generated .pb.go files is dependent on the exact output
of gofmt and the gzip compression used.

There are other areas where it depends on unstable output, but the above are the
major causes.

Since test.bash ensures that the golden tests are run with exact versions of the
protobuf and Go toolchains, we can ensure that the tests are reproducible across
different developer workstations.

Thus, we add a "golden" build tag to disable them for normal invocations of
"go test ./..." and only run them if test.bash is run.

If test.bash is ever updated with newer versions, the golden testdata can be
updated at the same time.

Change-Id: Ia2b7b039aad2ddaef7652e332215bf9403a6d856
Reviewed-on: https://go-review.googlesource.com/c/142458
Reviewed-by: Damien Neil <dneil@google.com>
2018-10-16 16:49:18 +00:00
Damien Neil
3863ee56b5 protogen: camel-case "Foo.bar" as "FooBar" instead of "Foo_Bar".
Given:

  message Parent {
    message Child1 {}
    message child2 {}
  }

Historic behavior is to generate child messages named:

  Parent_Child1
  ParentChild2

Avoid adding an _ in the latter case.

Change-Id: I49a6732655d64967b8c7bb7ad358ae54d294f7b4
Reviewed-on: https://go-review.googlesource.com/c/140898
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-16 16:18:01 +00:00
Damien Neil
bbbd38f9f9 protogen: avoid an obscure package name consistency check failure
When performing consistency checks on package names, ignore files that
are mentioned on the command line (M<filename>=<import_path>) but which
do not appear in the CodeGeneratorRequest.

Change-Id: I51f1eeda5eaaac14f4a0a975163f65d7774e7212
Reviewed-on: https://go-review.googlesource.com/c/140717
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-16 16:04:50 +00:00
Damien Neil
329b75ece9 protogen: rewrite _ imports as well
Call the import-rewriting function on _ imports as well as named ones.

Change-Id: I2fe2c90fe4b201e96a243f231b54387c6b65f7a5
Reviewed-on: https://go-review.googlesource.com/c/139878
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-16 16:03:55 +00:00
Damien Neil
872146653d protogen: conflict resolve local package names against predefined idents
Avoid importing packages with a local name that conflicts with a predefined
identifier (e.g., "string").

Change-Id: I51164635351895e8a060355e59d718240e26ef2e
Reviewed-on: https://go-review.googlesource.com/c/140178
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-05 19:58:09 +00:00
Damien Neil
1fa8ab0ed5 protogen: add an option to rewrite import paths
This allows us to implement the import_prefix parameter in the v1
protoc-gen-go.

Drop support for import_prefix in protogen, and explicitly produce an
error if it is used in the v2 protoc-gen-go.

Change-Id: I66136b6b3affa3c0e9a93dc565619c90c42c0ecc
Reviewed-on: https://go-review.googlesource.com/138257
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-28 21:19:59 +00:00
Damien Neil
2dc6718b59 cmd/protoc-gen-go-grpc: add gRPC code generator
This is a straight translation of the v1 API gRPC "plugin" to protogen.

Add a protoc-gen-go-grpc command. The preferred way to generate gRPC
services is to invoke both plugins separately:

  protoc --go_out=. --go-grpc_out=. foo.proto

When invoked in this fashion, the generators will produce separate
foo.pb.go and foo_grpc.pb.go files.

Change-Id: Ie180385dab3da7063db96f7c2f9de3abbd749f63
Reviewed-on: https://go-review.googlesource.com/137037
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-27 19:32:34 +00:00
Joe Tsai
01ab29648e go.mod: rename google.golang.org/proto as github.com/golang/protobuf/v2
This change was created by running:
	git ls-files | xargs sed -i "s|google.golang.org/proto|github.com/golang/protobuf/v2|g"

This change is *not* an endorsement of "github.com/golang/protobuf/v2" as the
final import path when the v2 API is eventually released as stable.
We continue to reserve the right to make breaking changes as we see fit.

This change enables us to host the v2 API on a repository that is go-gettable
(since go.googlesource.com is not a known host by the "go get" tool;
and google.golang.org/proto was just a stub URL that is not currently served).
Thus, we can start work on a forked version of the v1 API that explores
what it would take to implement v1 in terms of v2 in a backwards compatible way.

Change-Id: Ia3ebc41ac4238af62ee140200d3158b53ac9ec48
Reviewed-on: https://go-review.googlesource.com/136736
Reviewed-by: Damien Neil <dneil@google.com>
2018-09-24 16:11:50 +00:00
Damien Neil
2e0c3da5a4 cmd/protoc-gen-go: generate blank imports for unused proto dependencies
Generate Go imports for all packages imported by the .proto source file,
even if they are not referenced.

Change-Id: I116bdf460ab441d205b42606b2b05b315ed68954
Reviewed-on: https://go-review.googlesource.com/136358
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-19 21:12:48 +00:00
Damien Neil
993c04dcf2 cmd/protoc-gen-go: generate extensions
Generate everything related to extensions: extension descriptors,
XXX_InternalExtensions fields, etc.

Tweak the order in which we generate code for descriptors to ensure
consistent output with the previous protoc-gen-go.

Change Field.ContainingType to Field.ParentMessage, since we need to get
at the parent of both message fields and extensions (and the "containing
type" of an extension field is the extended message, under existing
terminology).

Change-Id: I5d045ca80536436e7c987bca3d8fb8c1e1521e55
Reviewed-on: https://go-review.googlesource.com/136155
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-18 22:49:53 +00:00
Damien Neil
1fa78d8a35 cmd/protoc-gen-go: generate oneofs
Generate everything related to oneofs: Message struct fields, wrapper
types, XXX_OneofFuncs.

Change-Id: I409040e0deb5716afabf59186eeaae21757d29f1
Reviewed-on: https://go-review.googlesource.com/135535
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-18 21:10:04 +00:00
Damien Neil
1ec3315873 protoc-gen-go: generate imports consistent with previous protoc-gen-go
Include references to the proto, fmt, and math packages, consistent with
the ones in the previous generator:

	// Reference imports to suppress errors if they are not otherwise used.
	var _ = proto.Marshal
	var _ = fmt.Errorf
	var _ = math.Inf

Sort imports with ast.SortImports to ensure gofmt-compatible output.

Copy the protoc-gen-go/testdata/imports test files from
github.com/golang/protobuf.

Change-Id: I263e2ac750d03596ae9cb23609049d5ef1016205
Reviewed-on: https://go-review.googlesource.com/135361
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-14 22:40:55 +00:00
Damien Neil
0bd5a38474 cmd/protoc-gen-go: add support for map fields
Generate the proper map[key]value type for map fields.
Include the protobuf_key and protobuf_val field tags.
Do not generate the map entry structs.

Fix an initialization order bug in protogen: While proto files cannot
contain circular dependencies, a single file can contain dependency
cycles. First create types for all the descriptors in a file, then add
in references (currently just field->message and field->enum) in a
second pass.

Change-Id: Ifedfa657d8dbb00413ba493adee1119b19c1b773
Reviewed-on: https://go-review.googlesource.com/135355
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-13 22:20:09 +00:00
Damien Neil
658051ba78 cmd/protoc-gen-go: generate message fields
This produces exactly the same output as github.com/golang/protobuf.

Change-Id: I01aacc9277c5cb5b4cc295f5ee8af12b4a524781
Reviewed-on: https://go-review.googlesource.com/134955
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-12 18:08:39 +00:00
Damien Neil
3cf6e62f69 protogen: support passing command-line parameters to plugins
Add a protogen.Options struct for future expansion. Include a FlagSet
which will be populated with parameters passed to the plugin.

Change-Id: I26a13bbde7ce011135b9c151edd160f3b51b7f9a
Reviewed-on: https://go-review.googlesource.com/134696
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-11 22:07:12 +00:00
Damien Neil
46abb57549 cmd/protoc-gen-go: generate enums
This produces exactly the same output (to the best of my ability to
determine) as github.com/golang/protobuf.

Change-Id: Ib60e7a836efb1eb0e5167b30458049ec239e7903
Reviewed-on: https://go-review.googlesource.com/134695
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-11 21:42:46 +00:00
Damien Neil
cab8dfee90 protogen: include source comments in generated code
Add initial handling of location paths (arrays of integers identifying
an entity in a .proto source file).

Expose path info in protogen; each descriptor has a Path field containing
its location path.

Format comments in protoc-gen-go. This contains one change from
github.com/golang/protobuf: Package comments are now included before the
package statement (but not attached to it) and use // comment syntax
instead of /* */. e.g.,

Before:

	package test

	/*
	This package contains interesting messages.
	*/

After:

	// This package contains interesting messages.

	package test

Change-Id: Ieee13ae77b3584f7562183100554d3df732348aa
Reviewed-on: https://go-review.googlesource.com/133915
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-11 17:19:32 +00:00
Damien Neil
7779e05b0a cmd/protoc-gen-go: register FileDescriptorProto
Generate the gzipped FileDescriptorProto var, and register it with
proto.RegisterFile at init time.

Change-Id: Ie232f20412ca9cd7bde91aaba7127dc181e9758c
Reviewed-on: https://go-review.googlesource.com/134118
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-10 20:31:17 +00:00
Damien Neil
082ce923d3 protogen: compute package names, import paths, generated filenames
Copy/duplicate the logic in github.com/golang/protobuf for computing
package names and import paths and the names of generated files.

This is all sufficiently complicated that the code is the best
documentation. In practice, users should always set a go_package option
containing an import path in every file and pass the
paths=source_relative generator flag to get reasonable behavior.

Change-Id: I34ae38fcc8db6909a4b25b16c73b982a7bad0463
Reviewed-on: https://go-review.googlesource.com/133876
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-07 17:58:10 +00:00
Damien Neil
abc6fc1ff9 protogen: use protoreflect descriptors
Change the protogen types wrapping FileDescriptorProto et al. to use
protoreflect descriptors instead.

Change-Id: I99fe83b995a0a6f4fc88f03a6e4b827109a2ec80
Reviewed-on: https://go-review.googlesource.com/133815
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-06 17:45:34 +00:00
Damien Neil
d901677135 protogen: automatic handling of imports
The GoIdent type is now a tuple of import path and name. Generated files
have an associated import path. Writing a GoIdent to a generated file
qualifies the name if the identifier is from a different package.
All necessary imports are automatically added to generated Go files.

Change-Id: I839e0b7aa8ec967ce178aea4ffb960b62779cf74
Reviewed-on: https://go-review.googlesource.com/133635
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-05 23:14:28 +00:00
Damien Neil
c7d07d9ba5 protogen: generate message skeletons
Copy generator.CamelCase for camel-casing names, with one change: Convert
'.' in names to '_'. This removes the need for the CamelCaseSlice function
which operates on a []string representing a name split along '.'s.

Add protogen.Message.

Reformat generated code.

Add regenerate.bash, largely copied from regenerate.sh.

Change-Id: Iecf0bfc43b552f53e458499a328b933b0c9c5f82
Reviewed-on: https://go-review.googlesource.com/130915
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-08-23 22:10:13 +00:00
Damien Neil
220c20246b protogen, cmd/protoc-gen-go: initial commit
Package protogen provides support for writing protoc plugins.
A "plugin" in this case is a program run by protoc to generate output.

The protoc-gen-go command is a protoc plugin to generate Go code.

cmd/protoc-gen-go/golden_test.go is mostly a straight copy from
the golden test in github.com/golang/protobuf.

Change-Id: I332d0df1e4b60bb8cd926320b8721e16b99a4b71
Reviewed-on: https://go-review.googlesource.com/130175
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-08-22 17:08:04 +00:00