This updates all generated code to match the contents of the latest
v24.0-rc2 release candidate of Protobuf. This provides access to the
various new fields and types defined in google/protobuf/descriptor.proto.
Change-Id: I8ec56b308dd7eed52fb9b17b258a45d0669c47ff
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/512795
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Before this change, synthetic oneofs were ignored in the inOneof() helper
function, but not in the remainder of the LegacyFieldOrder function.
Change-Id: Ia60e5244ae1000e98bbba9dd26f1d1583337fab4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/497935
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
Adds MarshalAppend methods to allow for byte slices to be reused.
Copies signature from the binary encoding.
Small changes to internal json and text libraries to use strconv
AppendInt and AppendUint for number encoding.
Change-Id: Ife7c8979c1c153a0a0bf9b70b296b8158d38dffc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/489615
Reviewed-by: Edward McFarlane <emcfarlane000@gmail.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Update to a protobuf version that contains a fix for
https://github.com/protocolbuffers/protobuf/issues/12173
(macOS build error).
Set "CC=clang" when building the protobuf repo on macOS,
which fixes build errors on my machine (macOS 13.3.1, M1).
Change-Id: Id8abb054479454b7ccbded5a6fa4d74f3d3c236d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/489315
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Fix a panic when parsing the incomplete negative number "- ".
Fixesgolang/protobuf#1530
Change-Id: Iba5e8ee68d5f7255c28f1a74f31beee36c9ed847
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/475995
Reviewed-by: Lasse Folger <lassefolger@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
The text format specification[1] indicates that whitespace and comments
may appear after a minus sign and before the subsequent numeric component
in negative number literals. But the Go implementation does not allow
this.
This brings the Go implementation info conformance with this aspect.
Fixesgolang/protobuf#1526
[1] https://protobuf.dev/reference/protobuf/textformat-spec/#parsing
Change-Id: I3996c89ee9d37cf2b7502fc6736d6e2ed6dbcf43
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/473015
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
This updates all generated code to match the contents of the latest
v22.0 release of Protobuf.
This involved a couple of changes to the script that does the sync'ing:
1. The new Protobuf version no longer includes autoconf configuration
and instead requires using bazel to build things.
2. The new Protobuf release does not have an artifact named
"protobuf-all-${VERSION}.tar.gz", but the one named
"protobuf-${VERSION}.tar.gz" has all of the sources and was
sufficient for the regenerate.bash script to complete.
This change does NOT regenerate the protos related to benchmarks.
The Protobuf repo no longer includes benchmarks. The CL removing them
says they are superceded by google/fleetbench:
83c499de86
But that project's proto benchmark files are very different:
https://github.com/google/fleetbench/tree/main/fleetbench/proto
So I commented out those steps in the generation code since the benchmarks
will need some work to reconcile with fleetbench.
This code adds known failing tests for conformance. New test cases were
added in https://github.com/protocolbuffers/protobuf/pull/9534, but the
Go protojson package does not behave according to the new tests. There
is an existing issue in GitHub about this:
https://github.com/golang/protobuf/issues/1208
Change-Id: Iad796ec7889bc2a74b58db5224facf850cd1a1cd
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/469255
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Fix typos in comments and unexported function name.
Change-Id: I239fff85bc68d7f0ef6f8853acdc9bbe19d86304
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/466535
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
The grow() method in func did create a new internal byte slice with a
specific size instead of a specific capacity. As the call following
grow() is append() we ended up with a slice larger than needed.
It affects the memory consumption of programs that import generated
protobuf files. As an example with the following:
-- go.mod --
module x
go 1.20
require cloud.google.com/go/appengine v1.6.0
-- main.go --
package main
import _ "cloud.google.com/go/appengine/apiv1/appenginepb"
func main() {}
Running the following a few times:
$ go mod tidy && GODEBUG=inittrace=1 go run . 2>&1 | grep appenginepb
Before:
init cloud.google.com/go/appengine/apiv1/appenginepb @3.4 ms, 0.23 ms clock, 204624 bytes, 231 allocs
init cloud.google.com/go/appengine/apiv1/appenginepb @3.2 ms, 0.17 ms clock, 204688 bytes, 231 allocs
init cloud.google.com/go/appengine/apiv1/appenginepb @2.5 ms, 0.15 ms clock, 204400 bytes, 230 allocs
init cloud.google.com/go/appengine/apiv1/appenginepb @2.5 ms, 0.16 ms clock, 205552 bytes, 234 allocs
After:
init cloud.google.com/go/appengine/apiv1/appenginepb @3.3 ms, 0.19 ms clock, 143440 bytes, 226 allocs
init cloud.google.com/go/appengine/apiv1/appenginepb @2.7 ms, 0.16 ms clock, 144368 bytes, 229 allocs
init cloud.google.com/go/appengine/apiv1/appenginepb @2.8 ms, 0.16 ms clock, 144304 bytes, 229 allocs
init cloud.google.com/go/appengine/apiv1/appenginepb @3.1 ms, 0.16 ms clock, 142864 bytes, 224 allocs
Change-Id: If4ece5d70d6bd9de8a758cb29ce9dffc741c4951
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/465115
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Also respect the file-wide deprecation option by treating every
identifier within the file as deprecated.
Change-Id: Ic3a5e84c2d90bbcbefc0a1b3272cc50554b94218
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/462315
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
I updated the protoc version to 21.5 and ran regenerate.bash.
This updates the various *.pb.go files to use the sources from
the latest protobuf release. In particular, this updates the
descriptorpb, so it includes some changes to descriptor.proto
in the past year and a half since this was last re-generated.
The change also pulled over several other generated changes.
Change-Id: Icd9c4e5d47ba85e1f4bae1d0a025f8dec15a3014
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/426054
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Calling the ProtoReflect method of the newly-constructed
message avoids an allocation in MessageInfo.MessageOf in
the common case of a generated message with an optimized
ProtoReflect method.
Benchmark for creating an empty message, darwin/arm64 M1 laptop:
name old time/op new time/op delta
EmptyMessage/New-10 32.1ns ± 2% 23.7ns ± 2% -26.06% (p=0.000 n=10+9)
name old alloc/op new alloc/op delta
EmptyMessage/New-10 64.0B ± 0% 48.0B ± 0% -25.00% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
EmptyMessage/New-10 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=10+10)
Change-Id: Ifa3c3ffa8edc76f78399306d0f4964eae4aacd28
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/418677
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Apply go1.19 gofmt to non-generated files.
Generated .pb.go files are created by generate.bash using Go 1.18,
so leave them unchanged for now.
Change-Id: Ied36c83cf99704988d059bf0412e677f0fbc71b0
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/418676
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Correctly distinguish between 0 and -0 in tests.
Change-Id: I41f6702a903ca0cf9edf9e95be812a3a29930978
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/406917
Reviewed-by: Lasse Folger <lassefolger@google.com>
Avoid very long errors returned by limiting the length of what errId
returns to 32 bytes (the value is chosen so that the error will not
be too long yet useful).
Append ellipsis to the returned value to denote that it was truncated.
Change-Id: I232d5192a2d9ad675daa0be0fe0c8518489c2953
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/406694
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
This eliminates the last user of the regexp package, which should save
about 130K from the resulting stripped binary importing this package
(unless, of course, regexp is brought in directly of via another
dependency).
Added some new cases to TestDecoder to test the new function.
Benchmark (not included) shows the following results, comparing to
old implementation using regexp.Find:
name old time/op new time/op delta
ErrId-4 1.93µs ± 1% 0.21µs ± 1% -89.20% (p=0.002 n=6+6)
name old alloc/op new alloc/op delta
ErrId-4 128B ± 0% 0B -100.00% (p=0.002 n=6+6)
name old allocs/op new allocs/op delta
ErrId-4 13.0 ± 0% 0.0 -100.00% (p=0.002 n=6+6)
Change-Id: I5569a47580f41cc60f92c444e8d43bb3f26faa4e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/402774
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
+ This change introduce a default and configurable depth limit for
proto.Unmarshal. If a message is nested deeper than the limit,
unmarshaling will fail. There are two ways to nest messages. Either by
having fields which are message types itself or by using groups.
+ The default limit is 10,000 for now. This might change in the future
to align it with other language implementation (C++ and Java use 100
as limit).
+ If pure groups (groups that don't contain message fields) are nested
deeper than the default limit the unmarshaling fails with:
proto: cannot parse invalid wire-format data
+ Note: the configured limit does not apply to pure groups.
+ This change is introduced to improve security and robustness. Because
unmarshaling is implemented using recursion it can lead to stack overflows
for certain inputs. The introduced limit protects against this.
+ A secondary motivation for this limit is the alignment with other
languages. Protocol buffers are a language interoperability mechanism
and thus either all implementations should accept the input or all
implementation should reject the input.
Change-Id: I14bdb44d06e4bd1aa90d6336c2cf6446003b2037
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/385854
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Damien Neil <dneil@google.com>
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
Previously, this call strconv.FormatInt with base 32,
when the intention is to call it with base 10.
Change-Id: I31cdd2415b3a80936cdcdeb5612a486204404ecb
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/331149
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
The Go generator has historically always prefixed an enum value
with the name of the enum type, when it was unnecessary to do so.
For example:
enum Status {
STATUS_FAILED = 0;
STATUS_PASSED = 1;
}
would be generated as:
type Status int32
const (
Status_STATUS_FAILED Status = 0
Status_STATUS_PASSED Status = 1
)
It is common for the enum values to be manually prefixed by the
enum type since protobuf enums use C++ namespace rules where
enum types and enum values are in the same namespace scope.
Thus, having the Go generator add a prefix is redundant.
See https://github.com/golang/protobuf/issues/513.
Some custom Go generators like protoc-gen-gogo allow removing
the prefix with the gogoproto.goproto_enum_prefix feature.
However, this leads to interoperability issues between
protoc-gen-go and protoc-gen-gogo, where the enum value names
cannot be accurately inferred.
Avoid this problem by just hard-coding the enum value number
for values declared in other packages. This provides benefits
in interoperability at the small cost of enum values possibly
being stale if their value were ever changed in a remote package.
However, this would only occur with use of proto2 enums and
default values, which seems to be an exceptionally rare situation.
Before:
Default_MyMessage_MyField = remotepb.FooEnum_FOO_ENUM
After:
Default_MyMessage_MyField = remotepb.FooEnum(4) // remotepb.FooEnum_FOO_ENUM
Before:
func (x *MyMessage) GetField() remotepb.FooEnum {
...
return remotepb.FooEnum_FOO_ZERO
}
After:
func (x *MyMessage) GetField() remotepb.FooEnum {
...
return remotepb.FooEnum(0) // always 0 for proto3 and often 0 for proto2
}
Change-Id: I3a06cd553f2eaf6124666f6c36c196d500d35718
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/319649
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
When merging aberrant messages with legacy Marshal and Unmarshal
methods, check for a typed nil source before calling Marshal.
Add an aberrant message with Marshal/Unmarshal methods to
internal/testprotos/nullable and use it to test the internal/impl
support for these methods.
Fixesgolang/protobuf#1324
Change-Id: Ib6ce85b30b46e3392a226ca6abe411932a371f02
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/321529
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The protobuf wire format is insufficiently self-decribing such that
it is impossible to know whether for sure whether an unknown bytes value
is a sub-message or not. However, protopack is primarily used for debugging
where a best-effort guess is still very useful.
The Message.UnmarshalAbductive unmarshals an unknown bytes value as a message
if it is syntactically well-formed. Otherwise, it is left as is.
Change-Id: I5e2b4b995e2b5eb60942a242558bf4cea1da9891
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/309669
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
The protopath package provides a means to programmatically represent
a sequence of protobuf reflection operations.
The protorange package traverses through a message and
calls a user-provided function as it iterates.
This feature sets the groundwork for the often requested feature
of being able to exclude certain fields when merging or serializing.
package protopath
type Path []Step
type Step struct{ ... }
func Root(protoreflect.MessageDescriptor) Step
func FieldAccess(protoreflect.FieldDescriptor) Step
func UnknownAccess() Step
func ListIndex(int) Step
func MapIndex(protoreflect.MapKey) Step
func AnyExpand(protoreflect.MessageDescriptor) Step
func (Step) Kind() StepKind
func (Step) FieldDescriptor() protoreflect.FieldDescriptor
func (Step) MessageDescriptor() protoreflect.MessageDescriptor
func (Step) ListIndex() int
func (Step) MapIndex() protoreflect.MapKey
func (Step) String() string
type StepKind int
const RootStep StepKind
const FieldAccessStep StepKind
const UnknownAccessStep StepKind
const ListIndexStep StepKind
const MapIndexStep StepKind
const AnyExpandStep StepKind
type Values struct {
Path Path
Values []protoreflect.Value
}
func (Values) Index(int) (out struct{ ... })
func (Values) Len() int
func (Values) String() string
package protorange
var Break error
var Terminate error
func Range(protoreflect.Message, func(protopath.Values) error) error
type Options struct {
Stable bool
Resolver interface { ... }
}
func (Options) Range(m protoreflect.Message, push, pop func(protopath.Values) error) error
Change-Id: I29cbd5142fe169d78367d54a95d37801888b64f4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/236540
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
The ExtensionRangeArray was a pseudo-internal API that was used by
the protobuf runtime implementation to know what the extension ranges are.
For some time now, the runtime implementation does not make use of this.
According to all the latest modules known by the module proxy,
no code depends directly on this method being present.
In the extremely rare case where some user code is depending on this method,
the user can place a .go file next to the generated .pb.go file that
injects this method.
Change-Id: Iae40a9a33b8c9b5a243d48db14f25b05ca24e3dc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/305574
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
Add back the weak dependency on github.com/golang/protobuf.
Change-Id: I2e6fabbe2cac3b694fb56a86bb3aa85cc8615b42
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/262684
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
