13 Commits

Author SHA1 Message Date
Joe Tsai
cf81e67b53 cmd/protoc-gen-go: use protoapi.CompressGZIP
Calling a helper function directly should reduce binary bloat slightly.

Change-Id: I6068dc4cd00c8d90d2e6e6d99633b81388bc8781
Reviewed-on: https://go-review.googlesource.com/c/164679
Reviewed-by: Damien Neil <dneil@google.com>
2019-02-28 22:28:32 +00:00
Herbie Ong
6a0e0722e7 go.sum: update to fix test.bash failure
Change-Id: I7216e7816275b0b6a278e6b16a5c04869259b5cc
Reviewed-on: https://go-review.googlesource.com/c/159937
Reviewed-by: Damien Neil <dneil@google.com>
2019-01-28 22:30:47 +00:00
Herbie Ong
84f0960b04 internal/encoding/text: format using 32 bitsize when encoding float32
When encoding/textpb marshals out float32 values, it was previously
formatting it as float64 bitsize since both float types are stored as
float64 and internal/encoding/text only has one Float type.  A
consequence of this is that the output may display a different value
than expected, e.g.  1.02 becomes 1.0199999809265137.

This CL splits Float type into Float32 and Float64 to keep track of
which bitsize to use when formatting.  Values of both types are still
stored as float64 to keep the logic simple.

Decoding will always use Float64, but users can ask for a float32 value
from it.

Change-Id: Iea5b14b283fec2236a0c3946fac34d4d79b95274
Reviewed-on: https://go-review.googlesource.com/c/158497
Reviewed-by: Damien Neil <dneil@google.com>
2019-01-18 17:54:23 +00:00
Joe Tsai
24ceb2b095 cmd/protoc-gen-go: generate descriptor and plugin with reflection
In CL/152020, we checked in pre-generated versions of descriptor and plugin.
This CL makes it such that they are generated by protoc-gen-go.

We modify protoc-gen-go to avoid reflection support by default
since the binary size increase is still an issue to investigate.
Reflection support is temporarily enabled by setting a special
PROTOC_GEN_GO_ENABLE_REFLECT environment variable.

Reflection support is always enabled for descriptor and plugin.
Furthermore, we change descriptor to depend on the protoapi package
instead of the proto package. The reason we do not switch to protoapi
for all generated protos is because we still depend on v1 proto
for the table-driven InternalMessageInfo type. Dropping it from descriptor
is semantically correct, but does incur slight performance cost.
It does not seem appropriate to drop it for all generated messages.

We could move InternalMessageInfo to protoapi, but the logic behind that
is significant.

Change-Id: I5c3fff7f6eab1a5a2399049d42fa6bf42d4c93f9
Reviewed-on: https://go-review.googlesource.com/c/152547
Reviewed-by: Damien Neil <dneil@google.com>
2018-12-06 17:47:27 +00:00
Joe Tsai
e1f8d50e17 reflect/protodesc: split descriptor related functionality from prototype
In order to generate descriptor.proto, the generated code would want to depend
on the prototype package to construct the reflection data structures.
However, this is a problem since descriptor itself is one of the dependencies
for prototype. To break this dependency, we do the following:
* Avoid using concrete *descriptorpb.XOptions messages in the public API, and
instead just use protoreflect.ProtoMessage. We do lose some type safety here
as a result.
* Use protobuf reflection to interpret the Options message.
* Split out NewFileFromDescriptorProto into a separate protodesc package since
constructing protobuf reflection from the descriptor proto obviously depends
on the descriptor protos themselves.

As part of this CL, we check in a pre-generated version of descriptor and plugin
that supports protobuf reflection natively and switchover all usages of those
protos to the new definitions. These files were generated by protoc-gen-go
from CL/150074, but hand-modified to remove dependencies on the v1 proto runtime.

Change-Id: I81e03c42eeab480b03764e2fcbe1aae0e058fc57
Reviewed-on: https://go-review.googlesource.com/c/152020
Reviewed-by: Damien Neil <dneil@google.com>
2018-12-05 00:38:30 +00:00
Herbie Ong
cddf8195e1 encoding/textpb: initial implementation of textproto marshaling
This initial implementation covers marshaling Message without use
of extensions, Any expansion, weak yet.

Change-Id: Ic787939c1d2a4e70e40c3a1654c6e7073052b7d3
Reviewed-on: https://go-review.googlesource.com/c/151677
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-11-29 23:06:35 +00:00
Joe Tsai
f18ab539ab all: make use of the protoapi package in v1
The new v1 protoapi package enables:
* Referencing types in the protoapi package instead of protoV1, which further
reduces the number of situations where we need to depend on protoV1.
This is for the goal of eventually breaking all cases where the v2 implementation
relies on v1, so that in the near future, proto v1 can rely on proto v2 instead.
* Removes the need for legacy_extension_hack.go since that functionality has now
been exported into the protoapi package.

Change-Id: If71002d9ec711bfabfe494636829df9abf19e23e
Reviewed-on: https://go-review.googlesource.com/c/151403
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-29 22:46:29 +00:00
Joe Tsai
d7e97bc71b cmd/protoc-gen-go: generate XXX_OneofWrappers instead of XXX_OneofFuncs
The marshaler, unmarshaler, and sizer functions are unused ever since
the underlying implementation was switched to be table-driven.
Change the function to only return the wrapper structs.

This change:
* enables generated protos to drop dependencies on certain proto types
* reduces the size of generated protos
* simplifies the implementation of oneofs in protoc-gen-go

Updates #708

Change-Id: I845c9009bc0236d1b51d34b014dc3e184303c0f2
Reviewed-on: https://go-review.googlesource.com/c/151357
Reviewed-by: Damien Neil <dneil@google.com>
2018-11-27 19:36:27 +00:00
Herbie Ong
c3f4d48629 internal/encoding/text: add extra random space to make output unstable.
Make output deliberately unstable so users don't rely on exactness.

For multi-line output, add another extra random space after <key>: for
at most one field per message.

-- example --
key1: field1
key2:  {
    foo:  bar
}

For single-line output, add another extra random space after a field per
message.

-- example --
key1:field1  key2:{foo:bar}

Change-Id: I3ab25d4d970fdebb88bbd9dd8fa6d73af84338ea
Reviewed-on: https://go-review.googlesource.com/c/150977
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-11-27 00:46:16 +00:00
Damien Neil
d901677135 protogen: automatic handling of imports
The GoIdent type is now a tuple of import path and name. Generated files
have an associated import path. Writing a GoIdent to a generated file
qualifies the name if the identifier is from a different package.
All necessary imports are automatically added to generated Go files.

Change-Id: I839e0b7aa8ec967ce178aea4ffb960b62779cf74
Reviewed-on: https://go-review.googlesource.com/133635
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-05 23:14:28 +00:00
Damien Neil
220c20246b protogen, cmd/protoc-gen-go: initial commit
Package protogen provides support for writing protoc plugins.
A "plugin" in this case is a program run by protoc to generate output.

The protoc-gen-go command is a protoc plugin to generate Go code.

cmd/protoc-gen-go/golden_test.go is mostly a straight copy from
the golden test in github.com/golang/protobuf.

Change-Id: I332d0df1e4b60bb8cd926320b8721e16b99a4b71
Reviewed-on: https://go-review.googlesource.com/130175
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-08-22 17:08:04 +00:00
Joe Tsai
879b18d902 internal/encoding/json: initial commit of JSON parser/serializer
Package json provides a parser and serializer for the JSON format.
This focuses on the grammar of the format and is agnostic towards specific
semantics of protobuf types.

High-level API:
	func Marshal(v Value, indent string) ([]byte, error)
	func Unmarshal(b []byte) (Value, error)
	type Type uint8
	    const Null Type ...
	type Value struct{ ... }
	    func ValueOf(v interface{}) Value
		func (v Value) Type() Type
		func (v Value) Bool() bool
		func (v Value) Number() float64
		func (v Value) String() string
		func (v Value) Array() []Value
		func (v Value) Object() [][2]Value
		func (v Value) Raw() []byte

Change-Id: I26422f6b3881ef1a11b8aa95160645b1384b27b8
Reviewed-on: https://go-review.googlesource.com/127824
Reviewed-by: Herbie Ong <herbie@google.com>
2018-08-07 22:40:28 +00:00
Joe Tsai
ed2fbe0328 test.bash: download dependencies
Use module support in Go1.11 to download the exact version of dependencies
as specified in the go.mod file, this is contrary to "go get -u", which grabs
the latest version, making reproducible builds and tests difficult.

In order for Go1.9 and Go1.10 to work, we also emit a vendor directory for
pre-module support.

Lastly, check whether the go.mod or go.sum files changed
(by shelling to "git diff"). This provides protection in case
a new dependency was added and the go.mod file was not updated.

Change-Id: Iac4e9b224ca9188dc9e63be720f188bfb5ee56ef
Reviewed-on: https://go-review.googlesource.com/127916
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2018-08-07 21:43:44 +00:00