This is a no-op change in preparation of removing support for weak fields.
It allows users to keep enabling ProtoLegacy in general,
but also disable weak field support independently.
For golang/protobuf#1666
Change-Id: Ic3cb90d937e21a817ddbbb36029274be8b5f2513
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/641635
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This is the equivalent of CL 638495, but for the Opaque API.
While the behavior is the same for non-synthetic oneofs between
the Open Struct API and Opaque API, the behavior for synthetic oneofs
is not the same: Because the Opaque API uses a bitfield to store
presence, we need to use the Opaque presence check in WhichOneof().
This CL extends the testproto generator to use the protoc
command-line flag for the test3 testproto (which specifies
syntax = "proto3";), as the in-file api_level is only available
in .proto files using editions (but editions does not use synthetic
oneofs for optional fields, only proto3 does).
For golang/protobuf#1659
Change-Id: I0a1fd6e5fc6f96eeab043f966728ce2a14dbd446
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/638515
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
This reverts change https://go.dev/cl/632735, in which
I misunderstood what the Protobuf documentation wanted to convey:
The quoted docs in CL 632735 refer to reflection for proto3 optional
fields, not to reflection for proto3 synthetic oneofs.
Synthetic oneofs should remain visible through reflection.
For the Open API, this change restores the old behavior.
For the Opaque API, another fix is needed and will be sent
in a separate, follow-up CL (follow golang/protobuf#1659).
For golang/protobuf#1659
Thanks to Joshua Humphries for the report and reproducer!
Change-Id: I3a924eed6d2425581adc65651395a68fc1576f4d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/638495
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
For golang/protobuf#1657
Change-Id: I7b2b0c30506706015ce278e6054439c9ad9ef727
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/634815
TryBot-Bypass: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
This required renaming the extension fields to have distinct names, as
makeMessages() uses the protobuild package, which works with field names.
The makeMessages() function can be centrally extended with other variants of
testprotos (e.g. editions).
Change-Id: I68ef28baee674c6b53da8629c5ff7b2e1b92d4ff
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/624415
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
This is important for test coverage: the makeMessages() function is where we
declare the various different test message types. Using this central function is
better than explicitly having to spell out all the various cases in the test
table.
This CL is roughly neutral in terms of lines changed, but actually increases
coverage: in the existing table, not every group of tests had coverage for all
cases (proto2, proto3, editions). This was difficult to spot previously, but now
the different test message types are now clearly spelled out (allTypesNoProto3).
Also, the test now prints the description and message types as subtest name.
This makes it easy to verify coverage by just reading the test output.
Change-Id: I08f541548d087233cbb4686785c456674371c2a3
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/623115
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Also adds better benchmark cases for large message where some fields are
actually populated.
This change was previously done in Google internal cl/660848520.
Change-Id: I682aae0c9c2850bfe7638de29ab743ad7d7b119a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/609035
Reviewed-by: Christian Höppner <hoeppi@google.com>
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Most of our imports already comply with the style, but this change makes it
consistent across the entire repository.
I suspect that these unnecessary imports are left-overs from differences between
Google-internal (where many imports are renamed) and Open Source.
Change-Id: Id5f6eabd22bcc72dd4e36dd018b6cc5138df7696
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/609875
Reviewed-by: Christian Höppner <hoeppi@google.com>
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This change was originally done by Patrik Nyblom in Google-internal cl/308876443
(The corresponding test is TestDecodeFastCheckInitialized in methods_test.go.)
Change-Id: I312dff53db4d6f87decf152eb20ec4dd280f03c3
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/608256
Reviewed-by: Christian Höppner <hoeppi@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Document the type mapping and the recommended pattern for working with
extensions. This documentation was Google-internal (cl/335451491), presumably
because of the reference to static analysis tools that don’t exist outside.
Compared to the internal version, this CL uses “should” instead of “must”.
Change-Id: I9293c10b2f6f91c9546d145c94b52e84f3a7a9c9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/607995
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Christian Höppner <hoeppi@google.com>
related to golang/protobuf#1613
Change-Id: Ie4255c24c1b79b13aab763a75125836191088d26
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/585096
Reviewed-by: Lasse Folger <lassefolger@google.com>
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This used to not be necessary, but a subsequent change will change behavior
based on MarshalOptions.Deterministic, so it is important that we do not
accidentally drop MarshalOptions anywhere.
related to golang/protobuf#1609
Change-Id: I6b53352707d93939642a627eb41c930f8ac3157f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/579995
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
There currently is no risk of producing invalid wire format,
but that will change with subsequent changes regarding lazy decoding.
We have been running this change in production for about a month,
without ever triggering the check (until lazy decoding is involved).
related to golang/protobuf#1609
Change-Id: I3c5c956aee2fa81f99dea03ed2a977a1547081fc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/579595
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
When I implemented this initially, I thought the parent of an extension is the
extendee. This is incorrect. The parent is the scope in which the extension is
defined. This CL changes the code to use the correct parent. This also allows
us to reduce some complexity in the implementation because we don't need to
wait until the extendee is resolved before we can resolve the features.
Change-Id: I6d7012f7502ef95457ab96f3e8abc4ab763d5bcb
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/579275
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
Hopefully this gives users a better understanding of the MarshalAppend
entrypoint and what it can be used for, as well as the typical Size usage.
Change-Id: I26c9705c3d1dbfea5f30820d41ccabbb88fbb772
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/573361
Reviewed-by: Lasse Folger <lassefolger@google.com>
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Extensions are unmarshaled lazily if protolegacy is true. The current
implementation of proto.HasExtension forces this unmarshaling to happen.
Change that.
Lazy message extensions are unmarshaled on first access, see
(*ExtensionField).Value. This leads to an (expensive) unmarshal
operation even if the user only wanted to know whether the extension is
present.
Granted, in most cases a HasExtension returning true will be followed by
a GetExtension. Due to memoization (see (*ExtensionField).lazyInit), the
cost will just shift from HasExtension to GetExtension. But, this CL
allows building cheaper functionality that only needs to know about
extension existence.
Why can this validation be removed?
- All tests pass.
- This check was added in CL 229558. The author (Joe Tsai) noted:
> Technically this shouldn't be needed, but I couldn't adequately reason
> whether a nil message value would ever be set through non-reflection
> means.
Like the author, I believe it's not needed:
- `proto.SetExtension` does not allow setting invalid messages (see
proto/extension.go).
- Likewise, (*extensionMap).Set panics when attempting to set an
invalid value.
- Unmarshaling does not produce submessages for which `IsValid` is
false.
The added test fails without the fix:
$ go test -tags=protolegacy -test.v -test.run=TestHasExtensionNoAlloc proto/extension_test.go
=== RUN TestHasExtensionNoAlloc
=== RUN TestHasExtensionNoAlloc/Nil
=== RUN TestHasExtensionNoAlloc/Eager
=== RUN TestHasExtensionNoAlloc/Lazy
extension_test.go:156: proto.HasExtension should not allocate, but allocated 3.00B per run
--- FAIL: TestHasExtensionNoAlloc (0.00s)
--- PASS: TestHasExtensionNoAlloc/Nil (0.00s)
--- PASS: TestHasExtensionNoAlloc/Eager (0.00s)
--- FAIL: TestHasExtensionNoAlloc/Lazy (0.00s)
FAIL
FAIL command-line-arguments 0.018s
The tests are disabled in race mode because the race instrumentation for
closures et al. always allocates. The protolegacy tests were previously
only run in race mode. I added a non-race variant in
integration_test.go.
Change-Id: Idbc67c1cf0aea8833a2735ca7bfc8d2466ceaf44
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/575035
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Auto-Submit: Nicolas Hillegeer <aktau@google.com>
This example uses the same protobuf and wire format encoding
as the corresponding Marshal example added in commit
https://go.googlesource.com/protobuf/+/c69658e23457d4e09
Change-Id: Ifd64a93a14589595cbe9b218235b57fb15d423c2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/574635
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The example has been written so that it can be run, meaning it only uses
packages that are included in the protobuf module (durationpb specifically).
I included an Output: comment in the example so that pkgsite displays the
program output even without having to run it (but running it is of course
possible).
I added a brief tip to protoscope, which is often mentioned in the protobuf.dev
docs for illustrative purposes, and I think it really makes much clearer to the
reader how the protobuf wire format looks like and what information survives the
encoding process (field numbers and their values, but not field names like in
JSON).
The struct literal contains only one field so that we don’t need to marshal this
message deterministically for stable wire format, which is not the point of the
example and would be distracting.
The value was chosen such that the wire format hex representation contains at
least one byte that is clearly identifiable as hexadecimal, to avoid confusion.
Change-Id: I86103abfd7d5b3f654aca3bfbb452f8ef7e49828
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/574455
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This change removes most usages of Syntax() from the repository and uses
edition features for instead. The appropriate edition feature defaults are
loaded for proto2/proto3 when the initialization of the descriptors
start.
All of these changes were tested on the Google corpus.
Change-Id: Ieca076a2b38ca8e50e084cd32e725b7b3dcb4171
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/572435
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
This makes it a little easier to track down test failures.
Also add a note to TestMarshalAppendAllocations to explain what it tests.
Change-Id: Ie35f3ddd7c7d5cb300294f0fe665c6711d45d186
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/569775
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
fuzztest seeds are executed as part of `go test` so there is no need for
an explicit test and having this as a seed in the fuzztest is sufficient
to get test coverage for this case.
Change-Id: Ia4a2d721e544e1a1ad0ad3ef9aa9d0af6bfe2db8
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/567755
Auto-Submit: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Before this change the implementation would ignore editions options set on
extension declarations.
This change also add much more coverage for valid messages.
Change-Id: I0e7029b1430a39859cdd4a93667ac20c7e0ff20e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/565396
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
This fixes more minor issue when constructing the descriptor in that
delimited encoded message fields are not considered properly as groups.
Change-Id: I714a227a0f8d256fa7430c526844e8e94acdda33
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/564816
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
There are several minor fixes:
* fix the unmarshaler for the field options (it used the wrong field numbers).
* fix the unmarshaler to always initialize the file options with the appropriate defaults.
* fix the coder selection to work properly for fields with implicit presence tracking (they used the coders for explicit tracking).
* fix the dynamic message created from a descriptor for an editions using proto to use HasPresence instead of a syntax check for proto3
Change-Id: Ic13bc22a71aa9f93b476e5edd650d4d2ab5dcb98
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/564455
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Modernize the documentation across the entire module
to make use of the newer ability to linkify declarations.
Change-Id: I440f9a3f025ec6fadfd9cba59b1dfb57028bf3f1
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/309430
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
The doc comment of proto.Equal now explicitly describes the behavior
in the case when one message is valid and other is invalid.
Change-Id: Ifeba40277c7da25193d46ad496e910f9b63eb795
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/464275
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Chressie Himpel <chressie@google.com>
The Value.Equal method compares Values in a way that is
analogous to the reflect.DeepEqual function.
Most of the implementation is moved from the "proto" package,
with proto.Equal implemented in terms of protoreflect.Value.Equal.
Change-Id: Ie9d5f6c073dc49fa59f78385c441db8137de5ec5
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/450775
TryBot-Bypass: Heschi Kreinick <heschi@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Oleksii Prokopchuk <prokopchuk@google.com>
Reviewed-by: Christian Höppner <hoeppi@google.com>
Run-TryBot: Lasse Folger <lassefolger@google.com>
This reverts commit 4d6d2135eba33eaef1c020129188ce7e82aa4270.
Post-submit discussion on CL 448876 suggests that this API should
change; revert it pending a decision.
Change-Id: I2e81be847c62ea7de5dfef36bb10ab193132e612
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/450955
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Run-TryBot: Lasse Folger <lassefolger@google.com>
TryBot-Bypass: Michael Stapelberg <stapelberg@google.com>
There is a demand for comparing protobuf.Value-s.
One use case is when one wants to compare just one field via reflection.
Another is when one wants to iterate over fields and check for diffs.
Existing diff packages are recommended only for tests as they panic on error.
Exposing equalField will prevent excessive branching of proto/equal.go code.
Change-Id: Iec8843dae96c9ae3c45858e8b97e7aa963473ce2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/448876
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Run-TryBot: Michael Stapelberg <stapelberg@google.com>
TryBot-Bypass: Michael Stapelberg <stapelberg@google.com>
Apply go1.19 gofmt to non-generated files.
Generated .pb.go files are created by generate.bash using Go 1.18,
so leave them unchanged for now.
Change-Id: Ied36c83cf99704988d059bf0412e677f0fbc71b0
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/418676
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
reflect.Pointer introduced on go1.18. Support 1.17 and earlier.
Change-Id: I62dcdb580a7976068e86df432bc44ee21c2cda81
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/412354
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Lasse Folger <lassefolger@google.com>
+ This change introduce a default and configurable depth limit for
proto.Unmarshal. If a message is nested deeper than the limit,
unmarshaling will fail. There are two ways to nest messages. Either by
having fields which are message types itself or by using groups.
+ The default limit is 10,000 for now. This might change in the future
to align it with other language implementation (C++ and Java use 100
as limit).
+ If pure groups (groups that don't contain message fields) are nested
deeper than the default limit the unmarshaling fails with:
proto: cannot parse invalid wire-format data
+ Note: the configured limit does not apply to pure groups.
+ This change is introduced to improve security and robustness. Because
unmarshaling is implemented using recursion it can lead to stack overflows
for certain inputs. The introduced limit protects against this.
+ A secondary motivation for this limit is the alignment with other
languages. Protocol buffers are a language interoperability mechanism
and thus either all implementations should accept the input or all
implementation should reject the input.
Change-Id: I14bdb44d06e4bd1aa90d6336c2cf6446003b2037
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/385854
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Damien Neil <dneil@google.com>
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
