Without resolving the question of whether an extension field is always proto2
or sometimes proto3, just make it impossible to construct a standalone extension
field of the proto3 syntax.
By dropping the syntax field from the constructor, we can just assume that the
syntax is always proto2. There is no benefit for creating a proto3 standalone
extension, so there is no loss of functionality.
In the unlikely future where there needs to be distinction, we can add the field
back into the descriptor. The zero value of protoreflect.Syntax is invalid,
so we can determine if the user did not set that field.
Change-Id: Ie93fa55de96f29553fb04ff6649bbea79a144407
Reviewed-on: https://go-review.googlesource.com/c/144279
Reviewed-by: Damien Neil <dneil@google.com>
Add a method to fetch descriptor options. Since options are proto
messages (e.g., google.protobuf.FieldOptions), and proto message
packages depend on the protoreflect package, returning the actual option
type would cause a dependency cycle. Instead, we return an interface
value which can be type asserted to the appropriate concrete type.
Add options support to the prototype package.
Some of the prototype constructors included fields (such as
Field.IsPacked) which represent information from the options
(such as google.protobuf.FieldOptions.packed). To avoid confusion about
the canonical source of information, drop these fields in favor of the
options.
Drop the unimplemented Descriptor.DescriptorOptionsProto.
Change-Id: I66579b6a7d10d99eb6977402a247306a78913e74
Reviewed-on: https://go-review.googlesource.com/c/144277
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
In order for the v2 rollout to be as seamless as possible, we need to support
the situation where a v2 message depends on some other generated v1 message that
may be stale and does not support the v2 API. In such a situation, there needs
to be some way to wrap a legacy message or enum in such a way that it satisfies
the v2 API.
This wrapping is comprised of two parts:
1) Deriving an enum or message descriptor
2) Providing an reflection implementation for messages
This CL addresses part 1 (while part 2 has already been partially implemented,
since the implementation applies to both v1 and v2).
To derive the enum and message descriptor we rely on a mixture of parsing the
raw descriptor proto and also introspection on the fields in the message.
Methods for obtaining the raw descriptor protos were added in February, 2016,
and so has not always been available. For that reason, we attempt to derive
as much information from the Go type as possible.
As part of this change, we modify prototype to be able to create multiple
standalone messages as a set. This is needed since cyclic dependencies is allowed
between messages within a single proto file.
Change-Id: I71aaf5f977faf9fba03c370b1ee17b3758ce60a6
Reviewed-on: https://go-review.googlesource.com/c/143539
Reviewed-by: Damien Neil <dneil@google.com>
Historically, protoc-gen-go outputted the escaped form of bytes as provided by
protoc verbatim. This behavior is buggy, but nothing really uses this tag
since default values are properties of getters instead of serialization.
Rather than fixing it, just preserve prior behavior. Otherwise, logic depending
on the old legacy behavior will not be able to distinguish between the unescaped
or the escaped forms.
Furthermore, since protoc-gen-go historically copied the protoc output verbatim,
we will need to escape the default bytes in a way that is identical to the
CEscape function from strutil.cc of the protoc source code.
Change-Id: I0ab55e220ae430dd123ad050406e285788f6cb40
Reviewed-on: https://go-review.googlesource.com/c/143543
Reviewed-by: Damien Neil <dneil@google.com>
Dynamically generate functions for handling individual fields within an oneof.
This implementation uses Go reflection to interact with the currently generated
approach, which uses an interface that can only be set by a limited set of
wrapper structs.
Change-Id: Ic848df922d6547411a15c4a20bfbbcae362da5c0
Reviewed-on: https://go-review.googlesource.com/c/142895
Reviewed-by: Damien Neil <dneil@google.com>
Most plugins need to copy comments from .proto source files into the
generated code. Move this functionality into protogen to avoid
duplicating it everywhere.
Change-Id: I48a96ba794192e7ddc00281342afd4805ef6fe0f
Reviewed-on: https://go-review.googlesource.com/c/142890
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Remove List from KnownFields, UnknownFields, ExtensionFieldTypes, and Map.
Rationale:
* Each of those interfaces already have a Range method, which provides
a superset of the functionality of List. Furthermore, Range is more expressive
as it allows you to terminate iteration and provides both keys and values.
* List must always allocate a slice and populates it.
* Range is allocation free in some cases. For example, if you simply wanted to
iterate over all the populated fields to clear them, there is no need for a
closure, so a static version of the function can be directly referenced
(i.e., there is no need to create a stub function header that references the
closed-over variables).
* In the cases where a closure is needed, the allocation cost is O(1) since
there are a finite number of variables being closed over.
* In highly performance sensitive cases, the closured function could close over
a struct, such that the function and struct are stored in a sync.Pool when not
in use. For example:
type MapLister struct {
Entries []struct{K MapKey; V Value}
f func(MapKey, Value) true
}
func (m *MapLister) Ranger() func(MapKey, Value) bool {
if m.f != nil {
m.f = func(k MapKey, v Value) bool {
m.Entries = append(m.Entries, ...)
return true
}
}
m.Entries = m.Entries[:0]
return m.f
}
The main benefit of List is the ease of use:
for _, num := range knownFields.List() {
...
}
as opposed to:
knownFields.Range(func(n FieldNumber, v Value) bool {
...
return true
})
However, this is a marginal benefit.
Thus, remove List as it mostly inferior to Range.
Change-Id: I25586c6ea07a4706072ba06b1cf25cb6efb5e8a7
Reviewed-on: https://go-review.googlesource.com/c/142888
Reviewed-by: Damien Neil <dneil@google.com>
Generate functions for wrapping map[K]V to implement protoreflect.Map.
This implementation uses Go reflection instead to provide a single implementation
that can handle all Go map types.
Change-Id: Idcb8069ef836614a88e5df12ef7c5044e8aa3dea
Reviewed-on: https://go-review.googlesource.com/c/142778
Reviewed-by: Damien Neil <dneil@google.com>
Generate functions for wrapping []T to implement protoreflect.Vector.
This implementation uses Go reflection instead to provide a single implementation
that can handle all Go slice types.
The test harness was greatly expanded to be able to test vectors (in addition
to messages and maps in the near future).
Change-Id: I0106c175f84a1e7e0a0a5b0e02e2489b70b0d177
Reviewed-on: https://go-review.googlesource.com/c/135339
Reviewed-by: Damien Neil <dneil@google.com>
When the generator parameter 'annotate_code' is provided, generate a .meta
file containing a GeneratedCodeInfo message describing the generated code's
relation to the source .proto file.
Annotations are added with (*protogen.GeneratedFile).Annotate, which takes the
name of a Go identifier (e.g., "SomeMessage" or "SomeMessage.GetField") and an
associated source location. The generator examines the generated AST to
determine source offsets for the symbols.
Change the []int32 "Path" in protogen types to a "Location", which also captures
the source file name.
Change-Id: Icd2340875831f40a1f91d495e3bd7ea381475c77
Reviewed-on: https://go-review.googlesource.com/c/139759
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Users sometimes run protoc-gen-go directly (rather than via protoc
--go_out) and are then confused when it sits and does nothing,
waiting for input from stdin. Print an error if any command-line
arguments are passed to it.
Change-Id: I10a059a82ec71b61c4eb6aeecc1e47e2446feabb
Reviewed-on: https://go-review.googlesource.com/c/139877
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Generate forwarders for default value const/vars defined in public
imports:
const Default_Message_Field = pubimport.Default_Message_Field
Change-Id: Ife09e38ae6a674b4460dd6613a8264e23f30b277
Reviewed-on: https://go-review.googlesource.com/c/140897
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The golden tests are sensitive to the exact version used:
* Protobuf toolchain since the generated Descriptor is dependendent on protoc
(for example, default json_names were auto-populated in later versions of protoc)
* Go toolchain since the generated .pb.go files is dependent on the exact output
of gofmt and the gzip compression used.
There are other areas where it depends on unstable output, but the above are the
major causes.
Since test.bash ensures that the golden tests are run with exact versions of the
protobuf and Go toolchains, we can ensure that the tests are reproducible across
different developer workstations.
Thus, we add a "golden" build tag to disable them for normal invocations of
"go test ./..." and only run them if test.bash is run.
If test.bash is ever updated with newer versions, the golden testdata can be
updated at the same time.
Change-Id: Ia2b7b039aad2ddaef7652e332215bf9403a6d856
Reviewed-on: https://go-review.googlesource.com/c/142458
Reviewed-by: Damien Neil <dneil@google.com>
A .proto source file with no 'package' statement may still contain
references to descriptors within the file.
Change-Id: I86e942c9c06e5a2915e9722162e0455ffa9ba2ab
Reviewed-on: https://go-review.googlesource.com/c/140899
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
When performing consistency checks on package names, ignore files that
are mentioned on the command line (M<filename>=<import_path>) but which
do not appear in the CodeGeneratorRequest.
Change-Id: I51f1eeda5eaaac14f4a0a975163f65d7774e7212
Reviewed-on: https://go-review.googlesource.com/c/140717
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Enums, for historical reasons, are registered with the proto package
under the name "<proto_package>.<go_type_name>". Don't include the dot
if there is no package statement in the .proto source file.
Change-Id: I6fb57d0803506668f60123a29fa06ae87fec523b
Reviewed-on: https://go-review.googlesource.com/c/140657
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Call the import-rewriting function on _ imports as well as named ones.
Change-Id: I2fe2c90fe4b201e96a243f231b54387c6b65f7a5
Reviewed-on: https://go-review.googlesource.com/c/139878
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
From:
func (* T) isT_F() {}
To:
func (*T) isT_F() {}
Formatting the file removes the space, but the presence or absence of
the space affects the formatter's decision on whether to split the {}
into {\n} or not.
Change-Id: I794c855a3115f9ae1b5f048728d8cad7a5f03e69
Reviewed-on: https://go-review.googlesource.com/c/140637
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Avoid importing packages with a local name that conflicts with a predefined
identifier (e.g., "string").
Change-Id: I51164635351895e8a060355e59d718240e26ef2e
Reviewed-on: https://go-review.googlesource.com/c/140178
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Given a pointer to a Go struct (that is well-formed according to the v1
struct field layout), wrap the type such that it implements the v2
protoreflect.Message interface.
Change-Id: I5987cad0d22e53970c613cdbbb1cfd4210897f69
Reviewed-on: https://go-review.googlesource.com/c/138897
Reviewed-by: Damien Neil <dneil@google.com>
Avoid generating invalid tag and wire size for large oneof field numbers
which overflow int32.
Change-Id: I005fe32aba4944b33b6b6ba83ef0ddd4d6e5863b
Reviewed-on: https://go-review.googlesource.com/138519
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The previous generator considers a default value of "" (valid only for
string and bytes fields) to be unset, and does not generate a const or
var to hold the default value.
e.g.,
message M {
optional F string = 1 [default=""];
}
does not generate this constant, even though the field has a default
value:
const Default_M_F string = ""
Maintain consistent output.
Change-Id: Ib172b02d59c15c05e19a7056d05ce1c619a2fa40
Reviewed-on: https://go-review.googlesource.com/138518
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This allows us to implement the import_prefix parameter in the v1
protoc-gen-go.
Drop support for import_prefix in protogen, and explicitly produce an
error if it is used in the v2 protoc-gen-go.
Change-Id: I66136b6b3affa3c0e9a93dc565619c90c42c0ecc
Reviewed-on: https://go-review.googlesource.com/138257
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
According to google.protobuf.FieldDescriptorProto.default_value documentation:
>>>
For strings, contains the default text contents (not escaped in any way).
For bytes, contains the C escaped value. All bytes >= 128 are escaped.
<<<
Thus, use the default value ad-verbatim if the kind is a string.
For the bytes kind, we use the text format hack to unescape the string,
since the text format uses the same unescape syntax as C due to its heritage.
Change-Id: I79e6d4d7e8f0d1ebedccecb9424a49384485589d
Reviewed-on: https://go-review.googlesource.com/138515
Reviewed-by: Damien Neil <dneil@google.com>
Move things around a little bit to allow the v1 protoc-gen-go to support
plugins=grpc.
Change-Id: I98d1bb86828450afe7915b1fefaf22bb7915cf44
Reviewed-on: https://go-review.googlesource.com/138256
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This is a straight translation of the v1 API gRPC "plugin" to protogen.
Add a protoc-gen-go-grpc command. The preferred way to generate gRPC
services is to invoke both plugins separately:
protoc --go_out=. --go-grpc_out=. foo.proto
When invoked in this fashion, the generators will produce separate
foo.pb.go and foo_grpc.pb.go files.
Change-Id: Ie180385dab3da7063db96f7c2f9de3abbd749f63
Reviewed-on: https://go-review.googlesource.com/137037
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Name protoc-gen-go.go was temporarily used to keep git from getting
confused about the revision history of main.go.
Change-Id: I3b20ff93f750eaf54dcf3c5c3488d465fba5eaa1
Reviewed-on: https://go-review.googlesource.com/137036
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
To permit the api-v1 branch to import the protoc-gen-go internals, move
them into a quasi-internal package.
Change-Id: I64e50ee299b99da1f648f7abb6cd0347a13e06e3
Reviewed-on: https://go-review.googlesource.com/137035
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
In preparation for factoring out the protoc-gen-go implementation into an
importable package, change exported names (just File) to unexported ones.
Change-Id: I2c65913332447e75c0dc7622a56f5dc09ff90116
Reviewed-on: https://go-review.googlesource.com/137017
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Add special-case handling for extension fields named
"message_set_extension" that extend a message_set_wire_format message.
Support special cases for a proto1 feature that was superseded by proto2
extensions.
Change-Id: Icbdb711111c66be547bf8d6f37ab3079c320e2a1
Reviewed-on: https://go-review.googlesource.com/136536
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This change was created by running:
git ls-files | xargs sed -i "s|google.golang.org/proto|github.com/golang/protobuf/v2|g"
This change is *not* an endorsement of "github.com/golang/protobuf/v2" as the
final import path when the v2 API is eventually released as stable.
We continue to reserve the right to make breaking changes as we see fit.
This change enables us to host the v2 API on a repository that is go-gettable
(since go.googlesource.com is not a known host by the "go get" tool;
and google.golang.org/proto was just a stub URL that is not currently served).
Thus, we can start work on a forked version of the v1 API that explores
what it would take to implement v1 in terms of v2 in a backwards compatible way.
Change-Id: Ia3ebc41ac4238af62ee140200d3158b53ac9ec48
Reviewed-on: https://go-review.googlesource.com/136736
Reviewed-by: Damien Neil <dneil@google.com>
The "go list" command is used only to get the module name.
However, just invoking the command results in a series of network IO
fetching package information about dependencies.
That information is then placed in the default GOPATH,
which previously was the system's default.
This is not generally a problem, but results in additional time spent
in Travis since the system's GOPATH is not cached between test runs.
By setting the GOPATH beforehand to be within TEST_DIR, we can ensure
that the result is cached and preserved between runs.
Change-Id: Ib750a9a31b9891b354fe9a70b3f592d949186ba3
Reviewed-on: https://go-review.googlesource.com/136738
Reviewed-by: Damien Neil <dneil@google.com>
The minimum supported version is currently Go1.10 since we use strings.Builder
for a fairly significant optimization when constructing all of the descriptor
full names.
The strings.Builder implementation is not particularly complicated,
so just fork it into our code. The golang/protobuf and the golang/go
projects share the same authors and copyright license.
Change-Id: Ibb9519dbe756327a07369f10f80c15761002b5e7
Reviewed-on: https://go-review.googlesource.com/136735
Reviewed-by: Damien Neil <dneil@google.com>
This is purely for consistent output with the previous protoc-gen-go.
Consider this message:
message M {
oneof union { string conflict = 1; }
message Conflict {}
}
The type for the wrapper of M.conflict will have the same name as the
embedded message type for M.Conflict.
The previous protoc-gen-go performs a disambiguation step where it adds
_s to the names of oneof field types until they have no conflicts with
nested messages or enums in the same message as the field.
There are a number of ways in which this can fail, of course. Preserve
the behavior for now.
Change-Id: I78a1c6588b577324e003b8bc337b75bb363432ba
Reviewed-on: https://go-review.googlesource.com/136357
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Generate Go imports for all packages imported by the .proto source file,
even if they are not referenced.
Change-Id: I116bdf460ab441d205b42606b2b05b315ed68954
Reviewed-on: https://go-review.googlesource.com/136358
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Generate forwarding declarations for all types, consts, and vars in
publicly imported files.
Change-Id: I2f1041fa91b0ae18b1e123935951618b8d594f84
Reviewed-on: https://go-review.googlesource.com/136175
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Generate everything related to extensions: extension descriptors,
XXX_InternalExtensions fields, etc.
Tweak the order in which we generate code for descriptors to ensure
consistent output with the previous protoc-gen-go.
Change Field.ContainingType to Field.ParentMessage, since we need to get
at the parent of both message fields and extensions (and the "containing
type" of an extension field is the extended message, under existing
terminology).
Change-Id: I5d045ca80536436e7c987bca3d8fb8c1e1521e55
Reviewed-on: https://go-review.googlesource.com/136155
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Include references to the proto, fmt, and math packages, consistent with
the ones in the previous generator:
// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf
Sort imports with ast.SortImports to ensure gofmt-compatible output.
Copy the protoc-gen-go/testdata/imports test files from
github.com/golang/protobuf.
Change-Id: I263e2ac750d03596ae9cb23609049d5ef1016205
Reviewed-on: https://go-review.googlesource.com/135361
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Move generation of the init function that registers all the types in a
file into a single function.
Take some care to generate the registrations in the same order as the
previous protoc-gen-go, to make it easier to catch unintended
differences in output.
For the same reason, adjust the order of generation to generate all
enums before all messages (matches previous behavior).
Change-Id: Ie0d574004d01a16f8d7b10be3882719a3c41676e
Reviewed-on: https://go-review.googlesource.com/135359
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Derive the compact string from the multi string output.
This makes is such that we don't have to update two sets of golden data.
Change-Id: I94b94f1094b448539f8bee234be83ac7186fdd30
Reviewed-on: https://go-review.googlesource.com/135335
Reviewed-by: Herbie Ong <herbie@google.com>
