Add fields to the Message and Field builder structs which hold the value
of MessageOptions.map_entry, FieldOptions.packed, and FieldOptions.weak
options. Remove all access to the contents of options messages from the
prototype package.
Change IsPacked to always return false for unpackable field types,
which is consistent with the equivalent C++ API.
This change helps avoid dependency cycles between prototype and the
options messages. (Previously this was resolved by accessing options
with reflection, but just breaking the dependency from prototype to the
options message is cleaner and simpler.)
Change-Id: I756aefe2e04cfa8fea31eaaaa0b5a99d4ac9e851
Reviewed-on: https://go-review.googlesource.com/c/153517
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Fix a few points of divergence between the v1 and v2 generators around
when to apply camel-casing to service and method names.
Change-Id: I862f89c0995c540e4862013316d7af772e1ab0d8
Reviewed-on: https://go-review.googlesource.com/c/153658
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Change the service and method names in grpc.proto to not use camel-case,
to make it explicit where rewriting occurs in the generated code.
Change-Id: I9e4a851097b0ee14817a589f5f959adcc5a14fe3
Reviewed-on: https://go-review.googlesource.com/c/153657
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Remove the Mutable methods from KnownFields, List, and Map, replacing
them with methods which return a new, empty message value without adding
that value to the collection.
The new API is simpler, since it clearly applies only to message values,
and more orthogonal, since it provides a way to create a value without
mutating the collection. This latter point is particularly useful in
map deserialization, where the key may be unknown at the time the value
is deserialized.
Drop the Mutable interface, since it is no longer necessary.
Change-Id: Ic5f3d06a2aa331a5d5cd2b4e670a3dba4a74f77c
Reviewed-on: https://go-review.googlesource.com/c/153278
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
These properties of descriptors are currently missing and makes it impossible
to convert a FileDescriptorProto into one of the structured Go representations
and convert it back to a proto message without loss of information.
Furthermore, ReservedRanges and ReservedNames has semantic importance
to text serialization.
Change-Id: Ic33c30020ad51912b143156b95f47a4fb8da3503
Reviewed-on: https://go-review.googlesource.com/c/153019
Reviewed-by: Damien Neil <dneil@google.com>
Add pseudo-hidden functions to register the concrete Go type used for
the optional types. Also, augment protoc-gen-go to specially generate the
descriptor proto to register these types.
This change does not add validation logic yet to ensure that the correct option
types are passed to the API.
Change-Id: I5decc897e14b4bf570a61cf17b57a066a2a0f9d7
Reviewed-on: https://go-review.googlesource.com/c/153017
Reviewed-by: Damien Neil <dneil@google.com>
The following TODOs were addressed:
* Consistently collect all enums, messages, and extensions in a breadth-first order.
The practical affect of this is that the declaration order in a Go file may change.
This simplifies reflection generation, which relies on consistent ordering.
* Removal of placeholder declarations (e.g., "var _ = proto.Marshal") since
protogen is intelligent about including imports as necessary.
* Always generate a default variable or constant for explicit empty strings.
The practical effect of this is the addition of new declarations in some cases.
However, it simplifies our logic such that it matches the protobuf data model.
* Generate the registration statements in a consistent order.
Change-Id: I627bb72589432bb65d53b50965ea88e5f7983977
Reviewed-on: https://go-review.googlesource.com/c/152778
Reviewed-by: Damien Neil <dneil@google.com>
In CL/152020, we checked in pre-generated versions of descriptor and plugin.
This CL makes it such that they are generated by protoc-gen-go.
We modify protoc-gen-go to avoid reflection support by default
since the binary size increase is still an issue to investigate.
Reflection support is temporarily enabled by setting a special
PROTOC_GEN_GO_ENABLE_REFLECT environment variable.
Reflection support is always enabled for descriptor and plugin.
Furthermore, we change descriptor to depend on the protoapi package
instead of the proto package. The reason we do not switch to protoapi
for all generated protos is because we still depend on v1 proto
for the table-driven InternalMessageInfo type. Dropping it from descriptor
is semantically correct, but does incur slight performance cost.
It does not seem appropriate to drop it for all generated messages.
We could move InternalMessageInfo to protoapi, but the logic behind that
is significant.
Change-Id: I5c3fff7f6eab1a5a2399049d42fa6bf42d4c93f9
Reviewed-on: https://go-review.googlesource.com/c/152547
Reviewed-by: Damien Neil <dneil@google.com>
Implement support in protoc-gen-go for generating messages and enums
that satisfy the v2 protobuf reflection interfaces. Specifically, the following
are added:
* top-level variable representing the file descriptor
* ProtoReflect method on enums (to implement protoV2.Enum)
* ProtoReflect method on messages (to implement protoV2.Message)
The following are not supported yet:
* resolving transitive dependencies for file imports
* Extension descriptors
* Service descriptors
The implementation approach creates a single array for all the message and enum
declarations and references sections of that array to complete dependencies.
Since protobuf declarations can form a graph (a message may depend on itself),
it is difficult to construct a graph as a single literal. One way is to use
placeholder descriptors, but that is not efficient as it requires encoding
the full name of each dependent enum and message and then later resolving it;
thus, both expanding the binary size and also increasing initialization cost.
Instead, we add a prototype.{Enum,Message}.Reference method to obtain a
descriptor reference for the purposes for satisfying dependencies.
As such, nested declarations and dependencies are populated in an init function.
Other changes to support the implementation:
* Added a protoimpl package to expose the MessageType type and also the
MessageTypeOf and EnumTypeOf helper functions.
* Added a protogen.File.GoIdent field to provide a suggested variable name
for the file descriptor.
* Added prototype.{Enum,Message}.Reference that provides a descriptor reference
for the purposes for satisfying cyclic dependencies.
* Added protoreflect.{Syntax,Cardinality,Kind}.GoString to obtain a Go source
identifier that represents the given constant.
Change-Id: I9455764882dee6ad10f251901e7d419091e8bf1d
Reviewed-on: https://go-review.googlesource.com/c/150074
Reviewed-by: Damien Neil <dneil@google.com>
In order to generate descriptor.proto, the generated code would want to depend
on the prototype package to construct the reflection data structures.
However, this is a problem since descriptor itself is one of the dependencies
for prototype. To break this dependency, we do the following:
* Avoid using concrete *descriptorpb.XOptions messages in the public API, and
instead just use protoreflect.ProtoMessage. We do lose some type safety here
as a result.
* Use protobuf reflection to interpret the Options message.
* Split out NewFileFromDescriptorProto into a separate protodesc package since
constructing protobuf reflection from the descriptor proto obviously depends
on the descriptor protos themselves.
As part of this CL, we check in a pre-generated version of descriptor and plugin
that supports protobuf reflection natively and switchover all usages of those
protos to the new definitions. These files were generated by protoc-gen-go
from CL/150074, but hand-modified to remove dependencies on the v1 proto runtime.
Change-Id: I81e03c42eeab480b03764e2fcbe1aae0e058fc57
Reviewed-on: https://go-review.googlesource.com/c/152020
Reviewed-by: Damien Neil <dneil@google.com>
Also replace the ident function with the handy GoImportPath.Ident method.
Change-Id: Ie4e820556fb83e659ab7e7af98f27fc24cdcd760
Reviewed-on: https://go-review.googlesource.com/c/152177
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Floating-point defaults of -inf, +inf, and NaN must be generated as
vars rather than consts. Fix the forwarding declaration for public
imports of these vars. (Was "const", is now "var".)
Change-Id: Ic6dc90ab7f88378ba477bff39e047de5f5193c35
Reviewed-on: https://go-review.googlesource.com/c/151757
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Add the scalar package to reduce dependencies on the v1 proto runtime package.
It may very well be the case that these functions should be exposed in the
public API of v2, but that is not a decision we need to make now.
Change-Id: Ifbc6d15311ba5837909ac72af47c630a80a142ef
Reviewed-on: https://go-review.googlesource.com/c/151402
Reviewed-by: Herbie Ong <herbie@google.com>
PR#741 in the v1 repository deprecated this behavior.
Change-Id: Ife48f1d586f178d875b9b3002a88b3336a2cd3b4
Reviewed-on: https://go-review.googlesource.com/c/151401
Reviewed-by: Damien Neil <dneil@google.com>
PR#741 in the v1 repository deprecated this behavior.
Change-Id: Idffa3884d43f9cc5528ce8d9f303676e0e501b67
Reviewed-on: https://go-review.googlesource.com/c/151400
Reviewed-by: Damien Neil <dneil@google.com>
The marshaler, unmarshaler, and sizer functions are unused ever since
the underlying implementation was switched to be table-driven.
Change the function to only return the wrapper structs.
This change:
* enables generated protos to drop dependencies on certain proto types
* reduces the size of generated protos
* simplifies the implementation of oneofs in protoc-gen-go
Updates #708
Change-Id: I845c9009bc0236d1b51d34b014dc3e184303c0f2
Reviewed-on: https://go-review.googlesource.com/c/151357
Reviewed-by: Damien Neil <dneil@google.com>
The GoImportPath.Ident helper creates a GoIdent using the receiver
as the GoImportPath in the GoIdent. This helper helps with the construction
of qualified identifiers.
Example usage:
const protoPackage = protogen.GoImportPath("github.com/golang/protobuf/proto")
protoPackage.Ident("ExtensionRange") // produces "proto.ExtensionRange"
The advantage of this helper is that usage of it looks similar to how
the identifier will eventually be rendered.
This is significantly more readable than the current approach:
protogen.GoIdent{
GoImportPath: protoPackage,
GoName: "ExtensionRange",
}
Change-Id: If7ecd7e60fad12bc491eee0dcb05f8fdebc9c94e
Reviewed-on: https://go-review.googlesource.com/c/150058
Reviewed-by: Damien Neil <dneil@google.com>
The bespoke text-serialization of field descriptors in protoc-gen-go is also
used in the legacy implementation of protobuf reflection to derive a
protoreflect.FieldDescriptor from legacy messages and also to convert to/from
protoreflect.ExtensionDescriptor and protoV1.ExtensionDesc.
Centralize this logic in a single place:
* to avoid reimplementing the same logic in internal/impl
* to keep the marshal and unmarshal logic co-located
Change-Id: I634c5afbb9dc6eda91d6cb6b0e68dbd724cb1ccb
Reviewed-on: https://go-review.googlesource.com/c/146758
Reviewed-by: Herbie Ong <herbie@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
A oneof is represented by a single struct field of interface type.
Pull out the decision of what to name that field into a separate
function.
The function is trivial (return oneof.GoName), but factoring out the
field name like this makes it a bit easier to experiment with changes to
the oneof implementation in the future.
Change-Id: I1114b68c85cb6608852fa1c6bf4103ff58fd5de6
Reviewed-on: https://go-review.googlesource.com/c/146397
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Given:
package foo
extend proto2.bridge.MessageSet {
optional Message message_set_extension = 100;
}
Register the extension as a message set extension and give it the name
"foo.".
We really shouldn't do this in this case; the special-case treatment of
extensions to MessageSet is only for extensions nested in a parent
message. However, this is consistent with the behavior of the v1 generator.
Match that for now.
Change-Id: I919c409605a197904fd3227efc920192d484f431
Reviewed-on: https://go-review.googlesource.com/c/145957
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
When publicly importing a package with extension definitions, generate
forwarding declarations for the "E_..." ExtensionDesc var:
var E_ExtensionField = publicimport.E_ExtensionField
Change-Id: Ifd57c487c3a44f303c2c098a42ea249b219b734f
Reviewed-on: https://go-review.googlesource.com/c/145498
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Don't generate a blank import for unused weak imports.
Full support for weak imports would involve not importing the package at
all. This just avoids generating an import when we don't need one.
Change-Id: I7e8491f415dc8333a2837db5225256b959921be2
Reviewed-on: https://go-review.googlesource.com/c/145497
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Default values for enums are specified by name, not number. An enum
may contain multiple values with different names but the same number.
Representing the default as a protoreflect.Value containing an EnumNumber
can discard information.
Add a method returning the EnumValueDescriptor.
Change-Id: If8beee3f81d41c4f9af45423252603b86949c7a5
Reviewed-on: https://go-review.googlesource.com/c/145158
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The v1 generator doesn't include a "proto3" tag on extension fields,
even when the field is defined in a proto3 file. Match that behavior for
consistency. We can probably change this later if we want to; it's
unlikely anyone is depending on this behavior.
The v1 generator uses pointer types for extension fields, even when the
field is defined in a proto3 file. (e.g., *FooEnum instead of FooEnum.)
Match this behavior. We can't change this without breaking compatibility
in the generated code.
Change-Id: I4072f3dd1c915bf9ab89f1d5198e0144cb4de20f
Reviewed-on: https://go-review.googlesource.com/c/144282
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Add a method to fetch descriptor options. Since options are proto
messages (e.g., google.protobuf.FieldOptions), and proto message
packages depend on the protoreflect package, returning the actual option
type would cause a dependency cycle. Instead, we return an interface
value which can be type asserted to the appropriate concrete type.
Add options support to the prototype package.
Some of the prototype constructors included fields (such as
Field.IsPacked) which represent information from the options
(such as google.protobuf.FieldOptions.packed). To avoid confusion about
the canonical source of information, drop these fields in favor of the
options.
Drop the unimplemented Descriptor.DescriptorOptionsProto.
Change-Id: I66579b6a7d10d99eb6977402a247306a78913e74
Reviewed-on: https://go-review.googlesource.com/c/144277
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Historically, protoc-gen-go outputted the escaped form of bytes as provided by
protoc verbatim. This behavior is buggy, but nothing really uses this tag
since default values are properties of getters instead of serialization.
Rather than fixing it, just preserve prior behavior. Otherwise, logic depending
on the old legacy behavior will not be able to distinguish between the unescaped
or the escaped forms.
Furthermore, since protoc-gen-go historically copied the protoc output verbatim,
we will need to escape the default bytes in a way that is identical to the
CEscape function from strutil.cc of the protoc source code.
Change-Id: I0ab55e220ae430dd123ad050406e285788f6cb40
Reviewed-on: https://go-review.googlesource.com/c/143543
Reviewed-by: Damien Neil <dneil@google.com>
Most plugins need to copy comments from .proto source files into the
generated code. Move this functionality into protogen to avoid
duplicating it everywhere.
Change-Id: I48a96ba794192e7ddc00281342afd4805ef6fe0f
Reviewed-on: https://go-review.googlesource.com/c/142890
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
When the generator parameter 'annotate_code' is provided, generate a .meta
file containing a GeneratedCodeInfo message describing the generated code's
relation to the source .proto file.
Annotations are added with (*protogen.GeneratedFile).Annotate, which takes the
name of a Go identifier (e.g., "SomeMessage" or "SomeMessage.GetField") and an
associated source location. The generator examines the generated AST to
determine source offsets for the symbols.
Change the []int32 "Path" in protogen types to a "Location", which also captures
the source file name.
Change-Id: Icd2340875831f40a1f91d495e3bd7ea381475c77
Reviewed-on: https://go-review.googlesource.com/c/139759
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Generate forwarders for default value const/vars defined in public
imports:
const Default_Message_Field = pubimport.Default_Message_Field
Change-Id: Ife09e38ae6a674b4460dd6613a8264e23f30b277
Reviewed-on: https://go-review.googlesource.com/c/140897
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The golden tests are sensitive to the exact version used:
* Protobuf toolchain since the generated Descriptor is dependendent on protoc
(for example, default json_names were auto-populated in later versions of protoc)
* Go toolchain since the generated .pb.go files is dependent on the exact output
of gofmt and the gzip compression used.
There are other areas where it depends on unstable output, but the above are the
major causes.
Since test.bash ensures that the golden tests are run with exact versions of the
protobuf and Go toolchains, we can ensure that the tests are reproducible across
different developer workstations.
Thus, we add a "golden" build tag to disable them for normal invocations of
"go test ./..." and only run them if test.bash is run.
If test.bash is ever updated with newer versions, the golden testdata can be
updated at the same time.
Change-Id: Ia2b7b039aad2ddaef7652e332215bf9403a6d856
Reviewed-on: https://go-review.googlesource.com/c/142458
Reviewed-by: Damien Neil <dneil@google.com>
A .proto source file with no 'package' statement may still contain
references to descriptors within the file.
Change-Id: I86e942c9c06e5a2915e9722162e0455ffa9ba2ab
Reviewed-on: https://go-review.googlesource.com/c/140899
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Enums, for historical reasons, are registered with the proto package
under the name "<proto_package>.<go_type_name>". Don't include the dot
if there is no package statement in the .proto source file.
Change-Id: I6fb57d0803506668f60123a29fa06ae87fec523b
Reviewed-on: https://go-review.googlesource.com/c/140657
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
From:
func (* T) isT_F() {}
To:
func (*T) isT_F() {}
Formatting the file removes the space, but the presence or absence of
the space affects the formatter's decision on whether to split the {}
into {\n} or not.
Change-Id: I794c855a3115f9ae1b5f048728d8cad7a5f03e69
Reviewed-on: https://go-review.googlesource.com/c/140637
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Avoid generating invalid tag and wire size for large oneof field numbers
which overflow int32.
Change-Id: I005fe32aba4944b33b6b6ba83ef0ddd4d6e5863b
Reviewed-on: https://go-review.googlesource.com/138519
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The previous generator considers a default value of "" (valid only for
string and bytes fields) to be unset, and does not generate a const or
var to hold the default value.
e.g.,
message M {
optional F string = 1 [default=""];
}
does not generate this constant, even though the field has a default
value:
const Default_M_F string = ""
Maintain consistent output.
Change-Id: Ib172b02d59c15c05e19a7056d05ce1c619a2fa40
Reviewed-on: https://go-review.googlesource.com/138518
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This allows us to implement the import_prefix parameter in the v1
protoc-gen-go.
Drop support for import_prefix in protogen, and explicitly produce an
error if it is used in the v2 protoc-gen-go.
Change-Id: I66136b6b3affa3c0e9a93dc565619c90c42c0ecc
Reviewed-on: https://go-review.googlesource.com/138257
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Move things around a little bit to allow the v1 protoc-gen-go to support
plugins=grpc.
Change-Id: I98d1bb86828450afe7915b1fefaf22bb7915cf44
Reviewed-on: https://go-review.googlesource.com/138256
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This is a straight translation of the v1 API gRPC "plugin" to protogen.
Add a protoc-gen-go-grpc command. The preferred way to generate gRPC
services is to invoke both plugins separately:
protoc --go_out=. --go-grpc_out=. foo.proto
When invoked in this fashion, the generators will produce separate
foo.pb.go and foo_grpc.pb.go files.
Change-Id: Ie180385dab3da7063db96f7c2f9de3abbd749f63
Reviewed-on: https://go-review.googlesource.com/137037
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Name protoc-gen-go.go was temporarily used to keep git from getting
confused about the revision history of main.go.
Change-Id: I3b20ff93f750eaf54dcf3c5c3488d465fba5eaa1
Reviewed-on: https://go-review.googlesource.com/137036
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
To permit the api-v1 branch to import the protoc-gen-go internals, move
them into a quasi-internal package.
Change-Id: I64e50ee299b99da1f648f7abb6cd0347a13e06e3
Reviewed-on: https://go-review.googlesource.com/137035
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
In preparation for factoring out the protoc-gen-go implementation into an
importable package, change exported names (just File) to unexported ones.
Change-Id: I2c65913332447e75c0dc7622a56f5dc09ff90116
Reviewed-on: https://go-review.googlesource.com/137017
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Add special-case handling for extension fields named
"message_set_extension" that extend a message_set_wire_format message.
Support special cases for a proto1 feature that was superseded by proto2
extensions.
Change-Id: Icbdb711111c66be547bf8d6f37ab3079c320e2a1
Reviewed-on: https://go-review.googlesource.com/136536
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This change was created by running:
git ls-files | xargs sed -i "s|google.golang.org/proto|github.com/golang/protobuf/v2|g"
This change is *not* an endorsement of "github.com/golang/protobuf/v2" as the
final import path when the v2 API is eventually released as stable.
We continue to reserve the right to make breaking changes as we see fit.
This change enables us to host the v2 API on a repository that is go-gettable
(since go.googlesource.com is not a known host by the "go get" tool;
and google.golang.org/proto was just a stub URL that is not currently served).
Thus, we can start work on a forked version of the v1 API that explores
what it would take to implement v1 in terms of v2 in a backwards compatible way.
Change-Id: Ia3ebc41ac4238af62ee140200d3158b53ac9ec48
Reviewed-on: https://go-review.googlesource.com/136736
Reviewed-by: Damien Neil <dneil@google.com>
