Running "go build ./..." does not descend into testdata directories.
However, the testdata in this repository is source code that is
intended to build properly. We could rename the directory, but that does
not test whether the generated packages can initialize properly.
Thus, we generate a trivial test that simply blank imports all packages.
Doing this reveals that some of the generated files have incorrect imports,
leading to registration conflicts.
To avoid introducing a dependency on gRPC from our go.mod file, we put
the testdata directories in their own module. Also, we avoid running
internal/testprotos through the grpc plugin because the servie and method
definitions in that directory are more for testing proto file initialization
rather than testing grpc generation.
Change-Id: Iaa6a06449787a085200e31bc7606e3ac904d3180
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/164917
Reviewed-by: Damien Neil <dneil@google.com>
To keep the dependency tree of Go protobufs as small as possible,
avoid depending on astutil. Most of the complexity of astutil.AddNamedImport
was for identifying an existing import block to insert imports into,
which is not relevant for our use-case.
Assuming that we always create a new import block after the package
statement, the logic for doing the AST manipulation is relatively simple.
This re-write properly handles an inline comment after the
package statement, which astutil.AddNamedImport (see golang.org/issue/30724)
currently fails to do.
Change-Id: I894e733aa82a241719b6f0c23de8d2fbfb67b778
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/166522
Reviewed-by: Damien Neil <dneil@google.com>
Create a single binary for handling generation of protos.
This replaces previous logic spread throughout the repo in:
* regenerate.bash
* cmd/protoc-gen-go/golden_test.go
* cmd/protoc-gen-go-grpc/golden_test.go
* (indirectly) internal/protogen/goldentest
One of the problems with the former approaches is that they relied on
a version of protoc that was specific to a developer's workstation.
This meant that the result of generation was not hermetic.
To address this, we rely on the hard-coded version of protobuf specified
in the test.bash script.
A summary of changes in this CL are:
* The internal_gengo.GenerateFile and internal_gengogrpc.GenerateFile
functions are unified to have consistent signatures. It seems that the
former accepted a *protogen.GeneratedFile to support v1 where gRPC code
was generated into the same file as the base .pb.go file. However, the
same functionality can be achieved by having the function return
the generated file object.
* The test.bash script patches the protobuf toolchain to have properly
specified go_package options in each proto source file.
* The test.bash script accepts a "-regenerate" argument.
* Add generation for the well-known types. Contrary to how these were
laid out in the v1 repo, all the well-known types are placed in the
same Go package.
* Add generation for the conformance proto.
* Remove regenerate.bash
* Remove internal/protogen
* Remove cmd/protoc-gen-go/golden_test.go
* Remove cmd/protoc-gen-go-grpc/golden_test.go
* Add cmd/protoc-gen-go/annotation_test.go
Change-Id: I4a1a97ae6f66e2fabcf4e4d292c95ab2a2db0248
Reviewed-on: https://go-review.googlesource.com/c/164477
Reviewed-by: Damien Neil <dneil@google.com>
Calling a helper function directly should reduce binary bloat slightly.
Change-Id: I6068dc4cd00c8d90d2e6e6d99633b81388bc8781
Reviewed-on: https://go-review.googlesource.com/c/164679
Reviewed-by: Damien Neil <dneil@google.com>
When encoding/textpb marshals out float32 values, it was previously
formatting it as float64 bitsize since both float types are stored as
float64 and internal/encoding/text only has one Float type. A
consequence of this is that the output may display a different value
than expected, e.g. 1.02 becomes 1.0199999809265137.
This CL splits Float type into Float32 and Float64 to keep track of
which bitsize to use when formatting. Values of both types are still
stored as float64 to keep the logic simple.
Decoding will always use Float64, but users can ask for a float32 value
from it.
Change-Id: Iea5b14b283fec2236a0c3946fac34d4d79b95274
Reviewed-on: https://go-review.googlesource.com/c/158497
Reviewed-by: Damien Neil <dneil@google.com>
In CL/152020, we checked in pre-generated versions of descriptor and plugin.
This CL makes it such that they are generated by protoc-gen-go.
We modify protoc-gen-go to avoid reflection support by default
since the binary size increase is still an issue to investigate.
Reflection support is temporarily enabled by setting a special
PROTOC_GEN_GO_ENABLE_REFLECT environment variable.
Reflection support is always enabled for descriptor and plugin.
Furthermore, we change descriptor to depend on the protoapi package
instead of the proto package. The reason we do not switch to protoapi
for all generated protos is because we still depend on v1 proto
for the table-driven InternalMessageInfo type. Dropping it from descriptor
is semantically correct, but does incur slight performance cost.
It does not seem appropriate to drop it for all generated messages.
We could move InternalMessageInfo to protoapi, but the logic behind that
is significant.
Change-Id: I5c3fff7f6eab1a5a2399049d42fa6bf42d4c93f9
Reviewed-on: https://go-review.googlesource.com/c/152547
Reviewed-by: Damien Neil <dneil@google.com>
In order to generate descriptor.proto, the generated code would want to depend
on the prototype package to construct the reflection data structures.
However, this is a problem since descriptor itself is one of the dependencies
for prototype. To break this dependency, we do the following:
* Avoid using concrete *descriptorpb.XOptions messages in the public API, and
instead just use protoreflect.ProtoMessage. We do lose some type safety here
as a result.
* Use protobuf reflection to interpret the Options message.
* Split out NewFileFromDescriptorProto into a separate protodesc package since
constructing protobuf reflection from the descriptor proto obviously depends
on the descriptor protos themselves.
As part of this CL, we check in a pre-generated version of descriptor and plugin
that supports protobuf reflection natively and switchover all usages of those
protos to the new definitions. These files were generated by protoc-gen-go
from CL/150074, but hand-modified to remove dependencies on the v1 proto runtime.
Change-Id: I81e03c42eeab480b03764e2fcbe1aae0e058fc57
Reviewed-on: https://go-review.googlesource.com/c/152020
Reviewed-by: Damien Neil <dneil@google.com>
This initial implementation covers marshaling Message without use
of extensions, Any expansion, weak yet.
Change-Id: Ic787939c1d2a4e70e40c3a1654c6e7073052b7d3
Reviewed-on: https://go-review.googlesource.com/c/151677
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The new v1 protoapi package enables:
* Referencing types in the protoapi package instead of protoV1, which further
reduces the number of situations where we need to depend on protoV1.
This is for the goal of eventually breaking all cases where the v2 implementation
relies on v1, so that in the near future, proto v1 can rely on proto v2 instead.
* Removes the need for legacy_extension_hack.go since that functionality has now
been exported into the protoapi package.
Change-Id: If71002d9ec711bfabfe494636829df9abf19e23e
Reviewed-on: https://go-review.googlesource.com/c/151403
Reviewed-by: Herbie Ong <herbie@google.com>
The marshaler, unmarshaler, and sizer functions are unused ever since
the underlying implementation was switched to be table-driven.
Change the function to only return the wrapper structs.
This change:
* enables generated protos to drop dependencies on certain proto types
* reduces the size of generated protos
* simplifies the implementation of oneofs in protoc-gen-go
Updates #708
Change-Id: I845c9009bc0236d1b51d34b014dc3e184303c0f2
Reviewed-on: https://go-review.googlesource.com/c/151357
Reviewed-by: Damien Neil <dneil@google.com>
Make output deliberately unstable so users don't rely on exactness.
For multi-line output, add another extra random space after <key>: for
at most one field per message.
-- example --
key1: field1
key2: {
foo: bar
}
For single-line output, add another extra random space after a field per
message.
-- example --
key1:field1 key2:{foo:bar}
Change-Id: I3ab25d4d970fdebb88bbd9dd8fa6d73af84338ea
Reviewed-on: https://go-review.googlesource.com/c/150977
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The GoIdent type is now a tuple of import path and name. Generated files
have an associated import path. Writing a GoIdent to a generated file
qualifies the name if the identifier is from a different package.
All necessary imports are automatically added to generated Go files.
Change-Id: I839e0b7aa8ec967ce178aea4ffb960b62779cf74
Reviewed-on: https://go-review.googlesource.com/133635
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Package protogen provides support for writing protoc plugins.
A "plugin" in this case is a program run by protoc to generate output.
The protoc-gen-go command is a protoc plugin to generate Go code.
cmd/protoc-gen-go/golden_test.go is mostly a straight copy from
the golden test in github.com/golang/protobuf.
Change-Id: I332d0df1e4b60bb8cd926320b8721e16b99a4b71
Reviewed-on: https://go-review.googlesource.com/130175
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Package json provides a parser and serializer for the JSON format.
This focuses on the grammar of the format and is agnostic towards specific
semantics of protobuf types.
High-level API:
func Marshal(v Value, indent string) ([]byte, error)
func Unmarshal(b []byte) (Value, error)
type Type uint8
const Null Type ...
type Value struct{ ... }
func ValueOf(v interface{}) Value
func (v Value) Type() Type
func (v Value) Bool() bool
func (v Value) Number() float64
func (v Value) String() string
func (v Value) Array() []Value
func (v Value) Object() [][2]Value
func (v Value) Raw() []byte
Change-Id: I26422f6b3881ef1a11b8aa95160645b1384b27b8
Reviewed-on: https://go-review.googlesource.com/127824
Reviewed-by: Herbie Ong <herbie@google.com>
Use module support in Go1.11 to download the exact version of dependencies
as specified in the go.mod file, this is contrary to "go get -u", which grabs
the latest version, making reproducible builds and tests difficult.
In order for Go1.9 and Go1.10 to work, we also emit a vendor directory for
pre-module support.
Lastly, check whether the go.mod or go.sum files changed
(by shelling to "git diff"). This provides protection in case
a new dependency was added and the go.mod file was not updated.
Change-Id: Iac4e9b224ca9188dc9e63be720f188bfb5ee56ef
Reviewed-on: https://go-review.googlesource.com/127916
Reviewed-by: Bryan C. Mills <bcmills@google.com>
