Use unsafeBytes in rawDescGZIP() as well, which is safe because our
CompressGZIP() does not write to the byte slice.
Store the result of CompressGZIP as a byte slice so that subsequent
calls do not cause any more allocations.
In http://go.dev/cl/638135, I asserted that rawDescGZIP() is rarely-used,
but after rolling out the change Google-internally, affected users made
me aware of a few programs that heavily access these descriptors.
Change-Id: Ieb5010ddc7b9ac6be88970321ff01a3d29e846bf
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/643276
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Nicolas Hillegeer <aktau@google.com>
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
Commit-Queue: Nicolas Hillegeer <aktau@google.com>
Improves on CL 642575 with a few additional checks to make
sure a panic doesn't occur even under unexpected conditions.
Fixesgolang/protobuf#1671
Change-Id: I25bf1cfdf0b35b381cab603f9e06581b3b630d73
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/642975
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
It tourns out that Piper (the revision control system Google uses)
has a check that requires marking files containing non-UTF8 bytes
as binary, preventing textual review. We wanted to avoid this issue
in http://go.dev/cl/638135 and added \r to the escape set for Gerrit,
but we did not anticipate how strict other systems are in practice.
(I did not notice this issue earlier because the Piper check
does not trigger when sending a CL for Google-wide testing,
only when mailing a CL for review.)
It stands to reason that if our revision control and review systems
do not like string literals with non-UTF8 content, other systems
probably behave similarly. Hence, let’s revert that part of the change.
The key part of the change was to switch the type of the embedded
descriptor bytes from mutable []byte to immutable string.
I verified that the string literal remains in .rodata:
% (cd internal/reflection_test && \
go test -c && \
objdump -s -j .rodata reflection_test.test | grep '0a46696e')
8e9fd0 0a46696e 7465726e 616c2f74 65737470 .Finternal/testp
The bytes printed above only occur once and match the bytes from
file_internal_testprotos_testeditions_testeditions_hybrid_test_hybrid_proto_rawDesc
Change-Id: I8e1bfe53a5bbf65abe7861a749ace37b215a8e28
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/642857
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
This means our generated code requires Go 1.20+.
(Go Protobuf currently requires at least Go 1.21.)
Change-Id: Ie65be553bcb5912991d590104ff6b7c6a82b9c38
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/642055
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
In Go, []byte literals go into the writable .data (or .noptrdata) section,
but string literals goes into the read-only .rodata section.
I verified that the contents move from .noptrdata to .rodata:
% (cd internal/reflection_test && \
go test -c && \
objdump -s -j .rodata reflection_test.test | grep '0a4669 6e')
88bfd0 6e7370e8 070a4669 6e746572 6e616c2f nsp...Finternal/
Change-Id: I87e190b41c02235abea1967eddca2f0262060ed9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/638135
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Thanks to Joshua Humphries and Edward McFarlane for
the excellent bug report, reproducer and fix suggestion!
Fixesgolang/protobuf#1669
Change-Id: I03df76f789e6e11b53396396a1f6b58bb3fb840b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/642575
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
This is a no-op change in preparation of removing support for weak fields.
It allows users to keep enabling ProtoLegacy in general,
but also disable weak field support independently.
For golang/protobuf#1666
Change-Id: Ic3cb90d937e21a817ddbbb36029274be8b5f2513
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/641635
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This parameter became unused with golang/protobuf#1640.
Change-Id: Iab50ffb757f5b6f017bc32c37b25294894c1c6d9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/641019
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This commit does not change behavior because the encoding/json
package ignores unexported fields. To encode a Protobuf message
into JSON, users should reach for protobuf/encoding/protojson.
With this change, the generated code (.pb.go files) passes go vet.
I also made the integration test run go vet so that we do not regress.
For golang/protobuf#1657Fixesgolang/protobuf#1658
Change-Id: I0a677d09e4517b2649bfcfe76a49194c7ccbb823
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/641036
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This is the equivalent of CL 638495, but for the Opaque API.
While the behavior is the same for non-synthetic oneofs between
the Open Struct API and Opaque API, the behavior for synthetic oneofs
is not the same: Because the Opaque API uses a bitfield to store
presence, we need to use the Opaque presence check in WhichOneof().
This CL extends the testproto generator to use the protoc
command-line flag for the test3 testproto (which specifies
syntax = "proto3";), as the in-file api_level is only available
in .proto files using editions (but editions does not use synthetic
oneofs for optional fields, only proto3 does).
For golang/protobuf#1659
Change-Id: I0a1fd6e5fc6f96eeab043f966728ce2a14dbd446
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/638515
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
This reverts change https://go.dev/cl/632735, in which
I misunderstood what the Protobuf documentation wanted to convey:
The quoted docs in CL 632735 refer to reflection for proto3 optional
fields, not to reflection for proto3 synthetic oneofs.
Synthetic oneofs should remain visible through reflection.
For the Open API, this change restores the old behavior.
For the Opaque API, another fix is needed and will be sent
in a separate, follow-up CL (follow golang/protobuf#1659).
For golang/protobuf#1659
Thanks to Joshua Humphries for the report and reproducer!
Change-Id: I3a924eed6d2425581adc65651395a68fc1576f4d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/638495
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
While godoc does not render comments within interface definitions,
using the [] syntax regardless helps with the replacement rules
we use to import Go Protobuf back into Google.
Change-Id: I0287ec3df5ba74261f563ce1117ab1e1be1fc270
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/635138
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
For golang/protobuf#1657
In order to change the default API level, one can now specify:
protoc […] --go_opt=default_api_level=API_HYBRID
To override the default API level for a specific file, use
the apilevelM mapping flag (similar to the M flag for import paths):
protoc […] --go_opt=apilevelMhello.proto=API_HYBRID
(Similar to the M option.)
Change-Id: I44590e9aa4c034a5bb9c93ae32f4b11188e684a0
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/634818
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
For golang/protobuf#1657
Change-Id: I7b2b0c30506706015ce278e6054439c9ad9ef727
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/634815
TryBot-Bypass: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
This updates all generated code to match the contents of the latest
v29.1 release of Protobuf. The main significant change is that
SourceCodeInfo is now extendable (though in a very limited way).
This release of Protobuf includes new conformance tests that do not
pass with the latest version of this protobuf-go module. These have
been added to the list of known-failing tests. However all of them
are marked "required" in the conformance suite so should be likely
be addressed as soon as possible. (This change makes no attempt to
fix any of them.)
Change-Id: Ib554d393d7742b7a616af998eeab84a080a85e0e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/634015
Reviewed-by: Chressie Himpel <chressie@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Calling WhichOneof should not be possible for synthetic oneofs;
on the reflection level, these fields should work as if they
were regular fields, not as if they were oneofs:
> Reflection for proto3 optional fields should work properly. For example, a
> method like Reflection::HasField() should know to look for the hasbit for a
> proto3 optional field. It should not be fooled by the synthetic oneof into
> thinking that there is a case member for the oneof.
From the Protobuf docs at:
https://github.com/protocolbuffers/protobuf/blob/main/docs/implementing_proto3_presence.md#updating-reflection
This change was tested Google-internally as CL 701866153.
Change-Id: Id9500d4aa492acf4ebc6d2d84be07ed81201d2aa
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/632735
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This test was accidentally migrated from proto.GetExtension to
proto.HasExtension in a semi-automated change and nobody noticed.
proto.HasExtension does not actually trigger lazy extension decoding.
I verified this using the dlv debugger:
% go test -tags protolegacy -c -gcflags "all=-N -l"
% dlv exec ./impl.test
Type 'help' for list of commands.
(dlv) r -test.run=TestMarshalMessageSetLazyRace -test.v
Process restarted with PID 3586699
(dlv) b TestMarshalMessageSetLazyRace
Breakpoint 1 set at 0xac9bf6 for google.golang.org/protobuf/internal/impl_test.TestMarshalMessageSetLazyRace() ./lazy_test.go:500
(dlv) c
=== RUN TestMarshalMessageSetLazyRace
> [Breakpoint 1] google.golang.org/protobuf/internal/impl_test.TestMarshalMessageSetLazyRace() ./lazy_test.go:500 (hits goroutine(32):1 total:1) (PC: 0xac9bf6)
=> 500: func TestMarshalMessageSetLazyRace(t *testing.T) {
(dlv) b ExtensionField.lazyInit
Breakpoint 2 set at 0x8da076 for google.golang.org/protobuf/internal/impl.(*ExtensionField).lazyInit() ./codec_extension.go:123
(dlv) c
--- PASS: TestMarshalMessageSetLazyRace (4.07s)
PASS
Process 3586699 has exited with status 0
(dlv) exit
Note how breakpoint 2 is unexpectedly not hit!
Here is how the output changes with my fix:
% go test -tags protolegacy -c -gcflags "all=-N -l"
% dlv --check-go-version=false exec ./impl.test
Type 'help' for list of commands.
(dlv) r -test.run=TestMarshalMessageSetLazyRace -test.v
Process restarted with PID 3587344
(dlv) b TestMarshalMessageSetLazyRace
Breakpoint 1 set at 0xac9bf6 for google.golang.org/protobuf/internal/impl_test.TestMarshalMessageSetLazyRace() ./lazy_test.go:500
(dlv) c
=== RUN TestMarshalMessageSetLazyRace
> [Breakpoint 1] google.golang.org/protobuf/internal/impl_test.TestMarshalMessageSetLazyRace() ./lazy_test.go:500 (hits goroutine(28):1 total:1) (PC: 0xac9bf6)
=> 500: func TestMarshalMessageSetLazyRace(t *testing.T) {
(dlv) b ExtensionField.lazyInit
Breakpoint 2 set at 0x8da076 for google.golang.org/protobuf/internal/impl.(*ExtensionField).lazyInit() ./codec_extension.go:123
(dlv) c
> [Breakpoint 2] google.golang.org/protobuf/internal/impl.(*ExtensionField).lazyInit() ./codec_extension.go:123 (hits goroutine(68):1 total:4) (PC: 0x8da076)
> [Breakpoint 2] google.golang.org/protobuf/internal/impl.(*ExtensionField).lazyInit() ./codec_extension.go:123 (hits goroutine(70):1 total:4) (PC: 0x8da076)
> [Breakpoint 2] google.golang.org/protobuf/internal/impl.(*ExtensionField).lazyInit() ./codec_extension.go:123 (hits goroutine(58):1 total:4) (PC: 0x8da076)
> [Breakpoint 2] google.golang.org/protobuf/internal/impl.(*ExtensionField).lazyInit() ./codec_extension.go:123 (hits goroutine(54):1 total:4) (PC: 0x8da076)
=> 123: func (f *ExtensionField) lazyInit() {
(dlv) bt
0 0x00000000008da076 in google.golang.org/protobuf/internal/impl.(*ExtensionField).lazyInit
at ./codec_extension.go:123
1 0x00000000008dac3e in google.golang.org/protobuf/internal/impl.(*ExtensionField).Value
at ./codec_extension.go:180
2 0x000000000094e5d9 in google.golang.org/protobuf/internal/impl.(*extensionMap).Get
at ./message_reflect.go:276
3 0x000000000095a2ee in google.golang.org/protobuf/internal/impl.(*messageState).Get
at ./message_reflect_gen.go:90
4 0x0000000000890099 in google.golang.org/protobuf/proto.GetExtension
at /usr/local/google/home/stapelberg/protobuf/proto/extension.go:90
5 0x0000000000aca7d4 in google.golang.org/protobuf/internal/impl_test.TestMarshalMessageSetLazyRace.func2.1
at ./lazy_test.go:545
6 0x0000000000aca625 in google.golang.org/protobuf/internal/impl_test.TestMarshalMessageSetLazyRace.func2
at ./lazy_test.go:550
7 0x00000000006001c1 in runtime.goexit
at /usr/lib/google-golang/src/runtime/asm_amd64.s:1700
Change-Id: Ie7a8621064af412a1db7efb3ad6b8473f9db58e8
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/624416
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
This required renaming the extension fields to have distinct names, as
makeMessages() uses the protobuild package, which works with field names.
The makeMessages() function can be centrally extended with other variants of
testprotos (e.g. editions).
Change-Id: I68ef28baee674c6b53da8629c5ff7b2e1b92d4ff
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/624415
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
This will be useful for a few different upcoming tests,
to be submitted in a follow-up CL.
Change-Id: I1fde9f4567b149141737a74c8448c2ee14946ae4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/623117
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
With this change, we establish a similar mechanism in encoding/prototext as
already exists in the proto package: a testmessages_test.go which serves as an
extension point where one can inject more variants into the corpus of test
messages.
Change-Id: I9f83735a44e5fd2e4736f888aaca428e4cc8ed42
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/623116
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This is important for test coverage: the makeMessages() function is where we
declare the various different test message types. Using this central function is
better than explicitly having to spell out all the various cases in the test
table.
This CL is roughly neutral in terms of lines changed, but actually increases
coverage: in the existing table, not every group of tests had coverage for all
cases (proto2, proto3, editions). This was difficult to spot previously, but now
the different test message types are now clearly spelled out (allTypesNoProto3).
Also, the test now prints the description and message types as subtest name.
This makes it easy to verify coverage by just reading the test output.
Change-Id: I08f541548d087233cbb4686785c456674371c2a3
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/623115
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This change required updating editions_defaults.binpb with
--edition_defaults_maximum=2024 so that we can use edition 2024 in our
testdata/enumprefix.proto test file.
For end users, this feature will only be available once:
1. protoc declares support for edition 2024 (expected early 2025).
2. protoc-gen-go declares support for edition 2024.
related to golang/protobuf#513
Change-Id: Ib8daeecae39ef32eff942279707d256c312f2a53
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/618979
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Mike Kruskal <mkruskal@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Some other implementations do not send the value field when encoding
an Any representing a message of the Empty Well-Known-Type.
Fixesgolang/protobuf#1620
Change-Id: I89bffd5f92656ba3ac462c0b6365ed4b49e6189d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/618395
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Also adds better benchmark cases for large message where some fields are
actually populated.
This change was previously done in Google internal cl/660848520.
Change-Id: I682aae0c9c2850bfe7638de29ab743ad7d7b119a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/609035
Reviewed-by: Christian Höppner <hoeppi@google.com>
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
