Include references to the proto, fmt, and math packages, consistent with
the ones in the previous generator:
// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf
Sort imports with ast.SortImports to ensure gofmt-compatible output.
Copy the protoc-gen-go/testdata/imports test files from
github.com/golang/protobuf.
Change-Id: I263e2ac750d03596ae9cb23609049d5ef1016205
Reviewed-on: https://go-review.googlesource.com/135361
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Move generation of the init function that registers all the types in a
file into a single function.
Take some care to generate the registrations in the same order as the
previous protoc-gen-go, to make it easier to catch unintended
differences in output.
For the same reason, adjust the order of generation to generate all
enums before all messages (matches previous behavior).
Change-Id: Ie0d574004d01a16f8d7b10be3882719a3c41676e
Reviewed-on: https://go-review.googlesource.com/135359
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Derive the compact string from the multi string output.
This makes is such that we don't have to update two sets of golden data.
Change-Id: I94b94f1094b448539f8bee234be83ac7186fdd30
Reviewed-on: https://go-review.googlesource.com/135335
Reviewed-by: Herbie Ong <herbie@google.com>
Add constructors for creating protoreflect.{Enum,Message}Type,
which extend the protobuf descriptor with Go type information.
Rationale for API:
* NewGoEnum and NewGoMessage take in a struct instead of multiple
function parameters to provide future flexibility in adding more parameters.
* NewGoEnum and NewGoMessage take in a pointer to a struct so that generated
code can batch allocate these structs and convert them to them to EnumTypes
or MessageTypes efficiently. In other words, allocations are entirely in the
control of the caller.
* The user provides a New function as opposed to a reflect.Type or pointer to
the concrete type to support dynamic protobufs, where a single Go type may be
used to represent multiple protobuf types.
* The New functions take in an EnumType or MessageType since an implementation
for dynamic protobufs would need to close over these types. All statically
generated messages can ignore that argument.
There are no tests added since these are just wrappers;
they will be heavily exercised by other code.
Change-Id: I9fca44e06f3388375be7eaca94f8fd75c17233a3
Reviewed-on: https://go-review.googlesource.com/135356
Reviewed-by: Damien Neil <dneil@google.com>
Generate the proper map[key]value type for map fields.
Include the protobuf_key and protobuf_val field tags.
Do not generate the map entry structs.
Fix an initialization order bug in protogen: While proto files cannot
contain circular dependencies, a single file can contain dependency
cycles. First create types for all the descriptors in a file, then add
in references (currently just field->message and field->enum) in a
second pass.
Change-Id: Ifedfa657d8dbb00413ba493adee1119b19c1b773
Reviewed-on: https://go-review.googlesource.com/135355
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Refactor fieldGoType to return the non-pointer type for scalar types
(i.e., "int", not "*int"), and a bool indicating whether the struct
field is a pointer to that type.
Change-Id: Ic80220e92f0b190e41ead847440d57af5c6cc919
Reviewed-on: https://go-review.googlesource.com/135336
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This provides an implementation of the has, get, set, clear methods for each
field in a message. The approach taken here is similar to the table-driven
implementation in the current v1 proto package.
The pointer_reflect.go and pointer_unsafe.go files are a simplified version of
the same files in the v1 implementation. They provide a pointer abstraction
that enables a high-efficiency approach in a non-purego environment.
The unsafe fast-path is not implemented in this commit.
This commit only implements the accessor methods for scalars using pure
Go reflection.
Change-Id: Icdf707e9d4e3385e55434f93b30a341a7680ae11
Reviewed-on: https://go-review.googlesource.com/135136
Reviewed-by: Damien Neil <dneil@google.com>
protoreflect.Value.IsNull was inverted to become IsValid in CL 134998.
However, that CL was submitted without a rebase,
which broke the build since CL 135255 added a few more usages of IsNull.
Change-Id: I5e52fb880ffc92e9d8f6cab30848e65b2d6dcbdc
Reviewed-on: https://go-review.googlesource.com/135337
Reviewed-by: Damien Neil <dneil@google.com>
Change the API to use explicit Has/Clear methods instead of relying on the
"invalid" form of Value to represent nullability.
There are several reasons for this change:
* Much of the ecosystem around protobufs do not care about nullability alone.
For example, for the encoder to determine whether to emit a field:
it needs to first check if a field is nulled, and if not, it still needs to go
through a series of type-assertion to check whether the value is the zero value.
It is much easier to be able to just call Has.
* It enables representing the default value as part of the value API, rather
than only as part of the descriptor API.
* The C++ API also uses explicit has and clear methods. However, we differ from
them by defining Has for proto3 scalars (while C++ panics instead).
For internal consistency, we also use a Has/Clear API for Maps.
Change-Id: I30eda482c959d3e1454d72d9fc761c761ace27a6
Reviewed-on: https://go-review.googlesource.com/134998
Reviewed-by: Damien Neil <dneil@google.com>
Provide a way to distinguish between a field with a zero-value
default and one with no default.
Change-Id: I4b1231de2d1bb141099cb1a415b35184dd198f93
Reviewed-on: https://go-review.googlesource.com/135255
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Generate the standard Reset, String, ProtoMessage, and Descriptor
methods for each message struct.
Generate the table-driven support methods (XXX_Marshal, et al.).
Change-Id: I8a5ed8af1433b481e2458ad534456692d6f74c1f
Reviewed-on: https://go-review.googlesource.com/134996
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The Value and MayKey types are defined in value.go, but the methods on
these types are defined in value_union.go. It's not immediately clear
to the reader of the code that these types have methods, or where they
are defined.
Move the type definitions to value_union.go to keep the entire type in a
single place.
Change-Id: I7b3f3fc219a24a3b0236c2c3335e5d46f9086d25
Reviewed-on: https://go-review.googlesource.com/134997
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Include the compile-time assertion that the generated file is compatible
with the proto package it is being compiled against.
Change-Id: Iefa27ee3a99a6669a93303b6674b0033794ba0be
Reviewed-on: https://go-review.googlesource.com/134995
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This produces exactly the same output as github.com/golang/protobuf.
Change-Id: I01aacc9277c5cb5b4cc295f5ee8af12b4a524781
Reviewed-on: https://go-review.googlesource.com/134955
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Implement Format methods to provide the ability to pretty-print the descriptor
in a humanly readable fashion. While this functionality is not strictly necessary
for the operation of Go protobuf, it is a useful aid for humans.
Change-Id: I88807b38b5be713867f2f2aab5a0843fc147dc35
Reviewed-on: https://go-review.googlesource.com/131255
Reviewed-by: Herbie Ong <herbie@google.com>
Add a protogen.Options struct for future expansion. Include a FlagSet
which will be populated with parameters passed to the plugin.
Change-Id: I26a13bbde7ce011135b9c151edd160f3b51b7f9a
Reviewed-on: https://go-review.googlesource.com/134696
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This produces exactly the same output (to the best of my ability to
determine) as github.com/golang/protobuf.
Change-Id: Ib60e7a836efb1eb0e5167b30458049ec239e7903
Reviewed-on: https://go-review.googlesource.com/134695
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Add initial handling of location paths (arrays of integers identifying
an entity in a .proto source file).
Expose path info in protogen; each descriptor has a Path field containing
its location path.
Format comments in protoc-gen-go. This contains one change from
github.com/golang/protobuf: Package comments are now included before the
package statement (but not attached to it) and use // comment syntax
instead of /* */. e.g.,
Before:
package test
/*
This package contains interesting messages.
*/
After:
// This package contains interesting messages.
package test
Change-Id: Ieee13ae77b3584f7562183100554d3df732348aa
Reviewed-on: https://go-review.googlesource.com/133915
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
According to linguistics, this is actually a concrete syntax tree, rather
than an abstract syntax tree since it perfectly represents the grammatical
structure of the original raw input.
On the other hand, an abstract syntax tree (AST) loses some
grammatical structure and is only concerned with preserving syntax.
See https://eli.thegreenplace.net/2009/02/16/abstract-vs-concrete-syntax-trees/
Change-Id: Ia3fdb407d2b15c5431984956b7d74921891c2ad9
Reviewed-on: https://go-review.googlesource.com/133995
Reviewed-by: Herbie Ong <herbie@google.com>
Generate the gzipped FileDescriptorProto var, and register it with
proto.RegisterFile at init time.
Change-Id: Ie232f20412ca9cd7bde91aaba7127dc181e9758c
Reviewed-on: https://go-review.googlesource.com/134118
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Copy/duplicate the logic in github.com/golang/protobuf for computing
package names and import paths and the names of generated files.
This is all sufficiently complicated that the code is the best
documentation. In practice, users should always set a go_package option
containing an import path in every file and pass the
paths=source_relative generator flag to get reasonable behavior.
Change-Id: I34ae38fcc8db6909a4b25b16c73b982a7bad0463
Reviewed-on: https://go-review.googlesource.com/133876
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Useful for dealing with SourceCodeInfo location paths, which identify
entities by their index.
Change-Id: I2034fc06b14c9b29b26e356fad21e106f63fbd14
Reviewed-on: https://go-review.googlesource.com/134115
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The v2 API already assumes that type aliases are available elsewhere.
Thus, we can drop this dynamic check for type alias support.
Change-Id: Ia4deab8f22384a982642da4a9aa5166870822b7d
Reviewed-on: https://go-review.googlesource.com/133877
Reviewed-by: Damien Neil <dneil@google.com>
Use a local ".cache" directory in the repository root for the cache
instead of a directory in /tmp. On many operating systems, files are arbitrarily
purged as part of the OS' cleanup procedures, leading to obscure failures.
Since the build cache is in the repo root, we adjust the invocation of gofmt
to only act upon the set of Go files tracked by git.
Change-Id: I10cd0ea0de3370ea25fcd3870451fb41c4204a4b
Reviewed-on: https://go-review.googlesource.com/133875
Reviewed-by: Damien Neil <dneil@google.com>
Minor changes:
* Use spaces instead of tabs in help printout since each system may have
a different tab width.
* Reduce repetition for field flags.
* Print to stdout instead of stderr for flag usages as is customary for
most other Unix command line tools.
Change-Id: I4544dbf46ee150d552a8fdfe72683d0a1aa7f0ba
Reviewed-on: https://go-review.googlesource.com/133755
Reviewed-by: Herbie Ong <herbie@google.com>
Change the protogen types wrapping FileDescriptorProto et al. to use
protoreflect descriptors instead.
Change-Id: I99fe83b995a0a6f4fc88f03a6e4b827109a2ec80
Reviewed-on: https://go-review.googlesource.com/133815
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
protoc leaves FileDescriptorProto.Syntax unset when the syntax is
"proto2".
Change-Id: Id1370145fc9c5c67344f5a998d118fd0f9f65f24
Reviewed-on: https://go-review.googlesource.com/133636
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The GoIdent type is now a tuple of import path and name. Generated files
have an associated import path. Writing a GoIdent to a generated file
qualifies the name if the identifier is from a different package.
All necessary imports are automatically added to generated Go files.
Change-Id: I839e0b7aa8ec967ce178aea4ffb960b62779cf74
Reviewed-on: https://go-review.googlesource.com/133635
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Implement NewFileFromDescriptorProto, which constructs a
protoreflect.FileDescriptor from a provided descriptor.FileDescriptorProto.
Some other minor changes:
* Allow calling Find and Range methods on nil *protoregistry.Files to match
the behavior of maps where index operations are permitted, but store panics.
* Switch protoregistry test to be protoregistry_test to avoid cyclic dependency.
Change-Id: I5536901ef5096014a3421e63bc4a9dfcad335ea4
Reviewed-on: https://go-review.googlesource.com/132455
Reviewed-by: Damien Neil <dneil@google.com>
The pbdump binary is an internal-only command line tool that is similar
to the hexdump tool in Linux, but is tailored towards the protocol buffer
wire format. This tool is not necessary for the new API, but is very useful
for debugging the wire format.
Change-Id: Ie688338d33b01aee5dd88a63606ec0ffce57741d
Reviewed-on: https://go-review.googlesource.com/129405
Reviewed-by: Herbie Ong <herbie@google.com>
Package pack enables manual encoding and decoding of protobuf wire data.
This package is intended only for testing and debugging purposes.
Message.Marshal is useful for hand-crafting raw wire testdata in tests
in a readable form.
Message.Unmarshal is useful for parsing raw wire data for debugging.
For that reason, effort was put into trying to get its string formatted
output look humanly readable.
High-level API:
type Number = wire.Number
const MinValidNumber Number = wire.MinValidNumber ...
type Type = wire.Type
const VarintType Type = wire.VarintType ...
type Token token
type Tag struct { ... }
type Bool bool
type Varint int64
type Svarint int64
type Uvarint uint64
type Int32 int32
type Uint32 uint32
type Float32 float32
type Int64 int64
type Uint64 uint64
type Float64 float64
type String string
type Bytes []byte
type LengthPrefix Message
type Denormalized struct { ... }
type Raw []byte
type Message []Token
func (Message) Size() int
func (Message) Marshal() []byte
func (*Message) Unmarshal(in []byte)
func (*Message) UnmarshalDescriptor(in []byte, desc protoreflect.MessageDescriptor)
func (Message) Format(s fmt.State, r rune)
Change-Id: Id99b340971a09c8a040838b155782a5d32b548bc
Reviewed-on: https://go-review.googlesource.com/129404
Reviewed-by: Herbie Ong <herbie@google.com>
Copy generator.CamelCase for camel-casing names, with one change: Convert
'.' in names to '_'. This removes the need for the CamelCaseSlice function
which operates on a []string representing a name split along '.'s.
Add protogen.Message.
Reformat generated code.
Add regenerate.bash, largely copied from regenerate.sh.
Change-Id: Iecf0bfc43b552f53e458499a328b933b0c9c5f82
Reviewed-on: https://go-review.googlesource.com/130915
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Package protogen provides support for writing protoc plugins.
A "plugin" in this case is a program run by protoc to generate output.
The protoc-gen-go command is a protoc plugin to generate Go code.
cmd/protoc-gen-go/golden_test.go is mostly a straight copy from
the golden test in github.com/golang/protobuf.
Change-Id: I332d0df1e4b60bb8cd926320b8721e16b99a4b71
Reviewed-on: https://go-review.googlesource.com/130175
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Changes made:
* create a /bin directory for symlinks and prepend it to the PATH
* include the "go" prefix in the Go toolchain versions
* move setting up GOPATH before calling "go mod vendor" since modules dump their
results inside a special directory within the GOPATH
* setup a per-version Go cache to 1) work around a bug in the toolchain and
2) to allow Travis to also cache this state
* factor out common logic into a check function that ensures stdout is empty
Change-Id: I9400452b85a66f019cb616aaded33610f2c1eb2f
Reviewed-on: https://go-review.googlesource.com/129402
Reviewed-by: Herbie Ong <herbie@google.com>
The prototype package provides constructors to create protobuf types that
implement the interfaces defined in the protoreflect package.
High-level API:
func NewFile(t *File) (protoreflect.FileDescriptor, error)
type File struct{ ... }
type Message struct{ ... }
type Field struct{ ... }
type Oneof struct{ ... }
type Enum struct{ ... }
type EnumValue struct{ ... }
type Extension struct{ ... }
type Service struct{ ... }
type Method struct{ ... }
func NewEnum(t *StandaloneEnum) (protoreflect.EnumDescriptor, error)
func NewMessage(t *StandaloneMessage) (protoreflect.MessageDescriptor, error)
func NewExtension(t *StandaloneExtension) (protoreflect.ExtensionDescriptor, error)
type StandaloneEnum struct{ ... }
type StandaloneMessage struct{ ... }
type StandaloneExtension struct{ ... }
func PlaceholderFile(path string, pkg protoreflect.FullName) protoreflect.FileDescriptor
func PlaceholderEnum(name protoreflect.FullName) protoreflect.EnumDescriptor
func PlaceholderMessage(name protoreflect.FullName) protoreflect.MessageDescriptor
This CL is missing some features that are to be added later:
* The stringer methods are not implemented, providing no way to print the
descriptors in a humanly readable manner.
* There is no support for proto options or retrieving the raw descriptor.
* There are constructors for Go specific types (e.g., protoreflect.MessageType).
We drop go1.9 support since we use strings.Builder.
We switch to go.11rc1 to obtain some bug fixes for modules.
Change-Id: Ieac9a2530afc81e5a5bb9ab5816804372f652b18
Reviewed-on: https://go-review.googlesource.com/129057
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Herbie Ong <herbie@google.com>
Package protoreflect provides APIs for programatically interacting with messages
according to the protobuf type system.
Change-Id: I11fa3874e4b3f94771514c69efb2ae8bb812d7fa
Reviewed-on: https://go-review.googlesource.com/127823
Reviewed-by: Damien Neil <dneil@google.com>
Package set provides simple set data structures for uint64 and string types.
High-level API:
type Set(T {}) xxx
func (Set) Len() int
func (Set) Has(T) bool
func (Set) Set(T)
func (Set) Clear(T)
These data structures are useful for implementing required fields efficiently
or ensuring that protobuf identifiers do not conflict.
Change-Id: If846630a9034909a43121b3e0f6720275f4b7aaf
Reviewed-on: https://go-review.googlesource.com/128898
Reviewed-by: Chris Manghane <cmang@golang.org>
Modify the test harness to run Go tests concurrently.
This vastly speeds up execution time on machines with many cores.
Other changes:
* Some stylistic changes to make the output colored and more readable.
* Add protoc and conformance-test-runner to the $PATH
* Check that Go source files were properly formatted
Change-Id: I5d7e14defeef32c587e6f8e2f66367143413a41f
Reviewed-on: https://go-review.googlesource.com/128415
Reviewed-by: Herbie Ong <herbie@google.com>
Now that we have a test.bash script that does comprehensive testing,
add instructions to CONTRIBUTING.md for how to install the hook.
We use a pre-push hook, rather than a pre-commit hook (which is more common)
since we're more interested in making sure the changes are correct just prior
to a user sending out code for review.
It is common for a user to continually add new files and/or update the commit
message. It would be annoying running the tests for every one of those operations.
Change-Id: I084a74fdb7f601d8075b3e349c7ee854b3b9394b
Reviewed-on: https://go-review.googlesource.com/128364
Reviewed-by: Herbie Ong <herbie@google.com>
Package text provides a parser and serializer for the proto text format.
This focuses on the grammar of the format and is agnostic towards specific
semantics of protobuf types.
High-level API:
func Marshal(v Value, indent string, delims [2]byte, outputASCII bool) ([]byte, error)
func Unmarshal(b []byte) (Value, error)
type Type uint8
const Bool Type ...
type Value struct{ ... }
func ValueOf(v interface{}) Value
func (v Value) Type() Type
func (v Value) Bool() (x bool, ok bool)
func (v Value) Int(b64 bool) (x int64, ok bool)
func (v Value) Uint(b64 bool) (x uint64, ok bool)
func (v Value) Float(b64 bool) (x float64, ok bool)
func (v Value) Name() (protoreflect.Name, bool)
func (v Value) String() string
func (v Value) List() []Value
func (v Value) Message() [][2]Value
func (v Value) Raw() []byte
Change-Id: I4a78ec4474c160d0de4d32120651edd931ea2c1e
Reviewed-on: https://go-review.googlesource.com/127455
Reviewed-by: Herbie Ong <herbie@google.com>
Package json provides a parser and serializer for the JSON format.
This focuses on the grammar of the format and is agnostic towards specific
semantics of protobuf types.
High-level API:
func Marshal(v Value, indent string) ([]byte, error)
func Unmarshal(b []byte) (Value, error)
type Type uint8
const Null Type ...
type Value struct{ ... }
func ValueOf(v interface{}) Value
func (v Value) Type() Type
func (v Value) Bool() bool
func (v Value) Number() float64
func (v Value) String() string
func (v Value) Array() []Value
func (v Value) Object() [][2]Value
func (v Value) Raw() []byte
Change-Id: I26422f6b3881ef1a11b8aa95160645b1384b27b8
Reviewed-on: https://go-review.googlesource.com/127824
Reviewed-by: Herbie Ong <herbie@google.com>
Change-Id: Iedcda3e5819c6cdab8fa33d4994be1f0011fa3a9
Reviewed-on: https://go-review.googlesource.com/127917
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
When Travis-CI runs the test.bash, it unnecessarily pollutes the environment
with a GOROOT environment variable, which confuses each of the custom invocations
of different Go toolchains. Rather than setting the GOROOT prior to each
invocation of the Go toolchain, just clear the variable once, and let each
toolchain figure out the root for themselves (which all recent versions of Go
know how to do so correctly).
Change-Id: I45ae9f9f98f5573ff42502b7244c90bf1569f66e
Reviewed-on: https://go-review.googlesource.com/128363
Reviewed-by: Herbie Ong <herbie@google.com>
Use module support in Go1.11 to download the exact version of dependencies
as specified in the go.mod file, this is contrary to "go get -u", which grabs
the latest version, making reproducible builds and tests difficult.
In order for Go1.9 and Go1.10 to work, we also emit a vendor directory for
pre-module support.
Lastly, check whether the go.mod or go.sum files changed
(by shelling to "git diff"). This provides protection in case
a new dependency was added and the go.mod file was not updated.
Change-Id: Iac4e9b224ca9188dc9e63be720f188bfb5ee56ef
Reviewed-on: https://go-review.googlesource.com/127916
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Package pragma provides certain constructs that can be used to statically enforce
certain language properties.
Change-Id: I58a87bf7429f3be1b5aae418595fadda32f9d039
Reviewed-on: https://go-review.googlesource.com/127822
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Herbie Ong <herbie@google.com>
Add a simple .travis.yml file that shells out testing to the test.bash script.
Caching the toolchain directories (Go toolchain and protobuf toolchain) saves
around 14min on each run.
This script only runs after Gerrit changes have been pushed to GitHub and acts
as a form of post-submit tests.
Change-Id: Id7d73d2c5fdc0de259d93cceb341ce921f71951d
Reviewed-on: https://go-review.googlesource.com/127915
Reviewed-by: Herbie Ong <herbie@google.com>
