This change strips out all descriptor option fields marked with [retention = RETENTION_SOURCE] before writing the FileDescriptor to the generated go code bindings.
Change-Id: Ie624d9d4b4f211a256661d80a04199ae8401662b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/472696
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
The text format specification[1] indicates that whitespace and comments
may appear after a minus sign and before the subsequent numeric component
in negative number literals. But the Go implementation does not allow
this.
This brings the Go implementation info conformance with this aspect.
Fixesgolang/protobuf#1526
[1] https://protobuf.dev/reference/protobuf/textformat-spec/#parsing
Change-Id: I3996c89ee9d37cf2b7502fc6736d6e2ed6dbcf43
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/473015
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
This updates all generated code to match the contents of the latest
v22.0 release of Protobuf.
This involved a couple of changes to the script that does the sync'ing:
1. The new Protobuf version no longer includes autoconf configuration
and instead requires using bazel to build things.
2. The new Protobuf release does not have an artifact named
"protobuf-all-${VERSION}.tar.gz", but the one named
"protobuf-${VERSION}.tar.gz" has all of the sources and was
sufficient for the regenerate.bash script to complete.
This change does NOT regenerate the protos related to benchmarks.
The Protobuf repo no longer includes benchmarks. The CL removing them
says they are superceded by google/fleetbench:
83c499de86
But that project's proto benchmark files are very different:
https://github.com/google/fleetbench/tree/main/fleetbench/proto
So I commented out those steps in the generation code since the benchmarks
will need some work to reconcile with fleetbench.
This code adds known failing tests for conformance. New test cases were
added in https://github.com/protocolbuffers/protobuf/pull/9534, but the
Go protojson package does not behave according to the new tests. There
is an existing issue in GitHub about this:
https://github.com/golang/protobuf/issues/1208
Change-Id: Iad796ec7889bc2a74b58db5224facf850cd1a1cd
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/469255
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Fix typos in comments and unexported function name.
Change-Id: I239fff85bc68d7f0ef6f8853acdc9bbe19d86304
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/466535
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
The doc comment of proto.Equal now explicitly describes the behavior
in the case when one message is valid and other is invalid.
Change-Id: Ifeba40277c7da25193d46ad496e910f9b63eb795
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/464275
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Chressie Himpel <chressie@google.com>
The grow() method in func did create a new internal byte slice with a
specific size instead of a specific capacity. As the call following
grow() is append() we ended up with a slice larger than needed.
It affects the memory consumption of programs that import generated
protobuf files. As an example with the following:
-- go.mod --
module x
go 1.20
require cloud.google.com/go/appengine v1.6.0
-- main.go --
package main
import _ "cloud.google.com/go/appengine/apiv1/appenginepb"
func main() {}
Running the following a few times:
$ go mod tidy && GODEBUG=inittrace=1 go run . 2>&1 | grep appenginepb
Before:
init cloud.google.com/go/appengine/apiv1/appenginepb @3.4 ms, 0.23 ms clock, 204624 bytes, 231 allocs
init cloud.google.com/go/appengine/apiv1/appenginepb @3.2 ms, 0.17 ms clock, 204688 bytes, 231 allocs
init cloud.google.com/go/appengine/apiv1/appenginepb @2.5 ms, 0.15 ms clock, 204400 bytes, 230 allocs
init cloud.google.com/go/appengine/apiv1/appenginepb @2.5 ms, 0.16 ms clock, 205552 bytes, 234 allocs
After:
init cloud.google.com/go/appengine/apiv1/appenginepb @3.3 ms, 0.19 ms clock, 143440 bytes, 226 allocs
init cloud.google.com/go/appengine/apiv1/appenginepb @2.7 ms, 0.16 ms clock, 144368 bytes, 229 allocs
init cloud.google.com/go/appengine/apiv1/appenginepb @2.8 ms, 0.16 ms clock, 144304 bytes, 229 allocs
init cloud.google.com/go/appengine/apiv1/appenginepb @3.1 ms, 0.16 ms clock, 142864 bytes, 224 allocs
Change-Id: If4ece5d70d6bd9de8a758cb29ce9dffc741c4951
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/465115
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Also respect the file-wide deprecation option by treating every
identifier within the file as deprecated.
Change-Id: Ic3a5e84c2d90bbcbefc0a1b3272cc50554b94218
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/462315
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Field numbers in the reserved range are semantically valid,
although the protocol buffer compiler may reject attempts to
define fields in this range.
Change-Id: Ib939ed5f4fef09b29047b509829b2f01494bfb6a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/461238
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
The Value.Equal method compares Values in a way that is
analogous to the reflect.DeepEqual function.
Most of the implementation is moved from the "proto" package,
with proto.Equal implemented in terms of protoreflect.Value.Equal.
Change-Id: Ie9d5f6c073dc49fa59f78385c441db8137de5ec5
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/450775
TryBot-Bypass: Heschi Kreinick <heschi@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Oleksii Prokopchuk <prokopchuk@google.com>
Reviewed-by: Christian Höppner <hoeppi@google.com>
Run-TryBot: Lasse Folger <lassefolger@google.com>
This reverts commit 4d6d2135eba33eaef1c020129188ce7e82aa4270.
Post-submit discussion on CL 448876 suggests that this API should
change; revert it pending a decision.
Change-Id: I2e81be847c62ea7de5dfef36bb10ab193132e612
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/450955
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Run-TryBot: Lasse Folger <lassefolger@google.com>
TryBot-Bypass: Michael Stapelberg <stapelberg@google.com>
There is a demand for comparing protobuf.Value-s.
One use case is when one wants to compare just one field via reflection.
Another is when one wants to iterate over fields and check for diffs.
Existing diff packages are recommended only for tests as they panic on error.
Exposing equalField will prevent excessive branching of proto/equal.go code.
Change-Id: Iec8843dae96c9ae3c45858e8b97e7aa963473ce2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/448876
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Run-TryBot: Michael Stapelberg <stapelberg@google.com>
TryBot-Bypass: Michael Stapelberg <stapelberg@google.com>
The Timestamp message uses a subset of RFC 3339.
RFC 3339, section 5.6 specifies that the subsecond field can have
any non-zero number of digits.
On the other hand, the protobuf documentation specifies that Timestamp
uses RFC 3339 with a few restrictions. In other words,
protobuf does NOT use RFC 3339, but rather a subset of it.
An upstream change https://go.dev/cl/425037 modifies the time package
to be internally consistent about handling of extra subsecond digits
allowing it to be more in line with RFC 3339.
Make a corresponding change here to ensure we remain compliant
with protobuf's restricted use of RFC 3339.
Change-Id: Ic145c68492fb41a5f7b79b653f3246dd9091d5d8
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/425554
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
I updated the protoc version to 21.5 and ran regenerate.bash.
This updates the various *.pb.go files to use the sources from
the latest protobuf release. In particular, this updates the
descriptorpb, so it includes some changes to descriptor.proto
in the past year and a half since this was last re-generated.
The change also pulled over several other generated changes.
Change-Id: Icd9c4e5d47ba85e1f4bae1d0a025f8dec15a3014
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/426054
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
In 2009, Google's open-source lawyers asked us to create the AUTHORS
file to define "The Go Authors", and the CONTRIBUTORS file was in
keeping with open source best practices of the time.
Re-reviewing our repos now in 2022, the open-source lawyers are
comfortable with source control history taking the place of the
AUTHORS file, and most open source projects no longer maintain
CONTRIBUTORS files.
To ease maintenance, remove AUTHORS and CONTRIBUTORS from all repos.
For #53961.
Change-Id: I6f3becd863672356de571303ff31434f517d959e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/418902
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Calling the ProtoReflect method of the newly-constructed
message avoids an allocation in MessageInfo.MessageOf in
the common case of a generated message with an optimized
ProtoReflect method.
Benchmark for creating an empty message, darwin/arm64 M1 laptop:
name old time/op new time/op delta
EmptyMessage/New-10 32.1ns ± 2% 23.7ns ± 2% -26.06% (p=0.000 n=10+9)
name old alloc/op new alloc/op delta
EmptyMessage/New-10 64.0B ± 0% 48.0B ± 0% -25.00% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
EmptyMessage/New-10 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=10+10)
Change-Id: Ifa3c3ffa8edc76f78399306d0f4964eae4aacd28
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/418677
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Apply go1.19 gofmt to non-generated files.
Generated .pb.go files are created by generate.bash using Go 1.18,
so leave them unchanged for now.
Change-Id: Ied36c83cf99704988d059bf0412e677f0fbc71b0
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/418676
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Skip tests with Go versions prior to 1.16.
Bump test version for Go 1.17 and 1.18 to latest.
Change-Id: I740bbc63bf35362fcfad3cf0e07ea4bd0ee56748
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/418675
Reviewed-by: Lasse Folger <lassefolger@google.com>
reflect.Pointer introduced on go1.18. Support 1.17 and earlier.
Change-Id: I62dcdb580a7976068e86df432bc44ee21c2cda81
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/412354
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Update
- actions/setup-go from v2 to v3;
- actions/checkout from v2 to v3;
- actions/cache from v2 to v3.0.2.
Tested at https://github.com/kolyshkin/protobuf-go/runs/6480937412
Change-Id: Ibeda7c83b1dfaf69dff1dab358b977958a48e51d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/406846
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
1. Switch from Go 1.16 to Go 1.x (which should now resolve to Go 1.18.1).
2. Add a comment explaining that the above version is not the actual Go
version used for testing.
3. Drop -mod=vendor as this is the unconditional default since Go 1.16
(unless GO111MODULES overrides this).
4. While at it, fix the options order in go test command, putting options
before the file name.
Tested at https://github.com/kolyshkin/protobuf-go/actions/runs/2342081396
Change-Id: Icc9aa47ddc0d04abb69b48fcb6b021e3e8a98df3
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/406954
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Lasse Folger <lassefolger@google.com>
The staticcheck version being used (2020.1.4) can no longer
work with latest Go, spitting out errors like this one:
> could not load export data: cannot import
> "unicode" (unknown iexport format version 2), export data is newer
> version - update tool (compile)
Update to latest version, 2022.1.2, to fix this.
Change-Id: I745482561a46baebc70931443008b95bec2fa377
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/406842
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Correctly distinguish between 0 and -0 in tests.
Change-Id: I41f6702a903ca0cf9edf9e95be812a3a29930978
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/406917
Reviewed-by: Lasse Folger <lassefolger@google.com>
Avoid very long errors returned by limiting the length of what errId
returns to 32 bytes (the value is chosen so that the error will not
be too long yet useful).
Append ellipsis to the returned value to denote that it was truncated.
Change-Id: I232d5192a2d9ad675daa0be0fe0c8518489c2953
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/406694
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
This eliminates the last user of the regexp package, which should save
about 130K from the resulting stripped binary importing this package
(unless, of course, regexp is brought in directly of via another
dependency).
Added some new cases to TestDecoder to test the new function.
Benchmark (not included) shows the following results, comparing to
old implementation using regexp.Find:
name old time/op new time/op delta
ErrId-4 1.93µs ± 1% 0.21µs ± 1% -89.20% (p=0.002 n=6+6)
name old alloc/op new alloc/op delta
ErrId-4 128B ± 0% 0B -100.00% (p=0.002 n=6+6)
name old allocs/op new allocs/op delta
ErrId-4 13.0 ± 0% 0.0 -100.00% (p=0.002 n=6+6)
Change-Id: I5569a47580f41cc60f92c444e8d43bb3f26faa4e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/402774
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
There are a number of message types, which can be confusing.
Expand the documentation. In particular, use Go's reflection model
as an analogy, which helps users if they are at least familiar
with how Go reflection works. After all, protobuf reflection
was modeled after Go reflection to a large degree.
Change-Id: I388ae41c792905145ccc86dc938b901a2b79f408
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/399654
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
+ This change introduce a default and configurable depth limit for
proto.Unmarshal. If a message is nested deeper than the limit,
unmarshaling will fail. There are two ways to nest messages. Either by
having fields which are message types itself or by using groups.
+ The default limit is 10,000 for now. This might change in the future
to align it with other language implementation (C++ and Java use 100
as limit).
+ If pure groups (groups that don't contain message fields) are nested
deeper than the default limit the unmarshaling fails with:
proto: cannot parse invalid wire-format data
+ Note: the configured limit does not apply to pure groups.
+ This change is introduced to improve security and robustness. Because
unmarshaling is implemented using recursion it can lead to stack overflows
for certain inputs. The introduced limit protects against this.
+ A secondary motivation for this limit is the alignment with other
languages. Protocol buffers are a language interoperability mechanism
and thus either all implementations should accept the input or all
implementation should reject the input.
Change-Id: I14bdb44d06e4bd1aa90d6336c2cf6446003b2037
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/385854
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Damien Neil <dneil@google.com>
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
Go 1.9 and 1.10 produce binaries that don't run on the most recent version
of macOS. Drop these versions from the tested set.
Update minor revision where a newer one is available:
1.15.9 -> 1.15.15
1.16.1 -> 1.16.10
Add 1.17.3.
Bump go.mod version to go1.11.
Fixesgolang/protobuf#1394.
Change-Id: I7850b86819d356891917d2987f080d3e7dbd7cc5
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/370055
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Florian Zenker <floriank@google.com>
