The changes to integration_test.go and internal/filedesc/editions.go
were manual, the rest is the result of ./regenerate.bash.
Note that while Protobuf v30 supports Bazel 8, the targets we need
("test and release targets") still require Bazel 7. An easy way
to work with Bazel 7 is to use bazelisk:
mkdir /tmp/wrap
cat > /tmp/wrap/bazel <<'EOT'
USE_BAZEL_VERSION=7.x bazelisk "$@"
EOT
chmod +x /tmp/wrap/bazel
export PATH=/tmp/wrap:$PATH
./regenerate.bash
Change-Id: Idfa8d149995025ec5194711ca60648cee50b0766
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/654955
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
Go 1.24 was released recently.
Change-Id: I4e2c83c6cc90a3c7e996f8e6ba6ee7f8589596ee
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/649135
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This turns the nameclash testprotos ending in 3 into .proto
files which use syntax = "proto3". Prior to the Opaque API release,
we changed these .proto files to use editions, but using editions
with proto3 semantics is not exactly the same as using proto3,
so we need these .proto files to use syntax = "proto3";
Fixesgolang/protobuf#1675
Change-Id: I0ebb37d4e1bf0fc830bbe613148429ec6ff5c46d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/644437
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Weak fields were the predecessor to extensions (many many years ago) and were
entirely removed from Google’s production usage by now. (The corresponding field
in descriptor.proto was always documented as “// For Google-internal migration
only. Do not use.”)
Before this change, Go Protobuf still contained support for weak fields behind
the `protolegacy` build tag.
The `protolegacy` build tag was always documented as not being part of the
compatibility agreement:
// WARNING: The compatibility agreement covers nothing provided by this flag.
// As such, functionality may suddenly be removed or changed at our discretion.
const ProtoLegacy = protoLegacy
Fixesgolang/protobuf#1666
Change-Id: Ie1675424bc80d9f44345ccb96a858ef847ee1018
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/641655
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Use unsafeBytes in rawDescGZIP() as well, which is safe because our
CompressGZIP() does not write to the byte slice.
Store the result of CompressGZIP as a byte slice so that subsequent
calls do not cause any more allocations.
In http://go.dev/cl/638135, I asserted that rawDescGZIP() is rarely-used,
but after rolling out the change Google-internally, affected users made
me aware of a few programs that heavily access these descriptors.
Change-Id: Ieb5010ddc7b9ac6be88970321ff01a3d29e846bf
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/643276
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Nicolas Hillegeer <aktau@google.com>
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
Commit-Queue: Nicolas Hillegeer <aktau@google.com>
It tourns out that Piper (the revision control system Google uses)
has a check that requires marking files containing non-UTF8 bytes
as binary, preventing textual review. We wanted to avoid this issue
in http://go.dev/cl/638135 and added \r to the escape set for Gerrit,
but we did not anticipate how strict other systems are in practice.
(I did not notice this issue earlier because the Piper check
does not trigger when sending a CL for Google-wide testing,
only when mailing a CL for review.)
It stands to reason that if our revision control and review systems
do not like string literals with non-UTF8 content, other systems
probably behave similarly. Hence, let’s revert that part of the change.
The key part of the change was to switch the type of the embedded
descriptor bytes from mutable []byte to immutable string.
I verified that the string literal remains in .rodata:
% (cd internal/reflection_test && \
go test -c && \
objdump -s -j .rodata reflection_test.test | grep '0a46696e')
8e9fd0 0a46696e 7465726e 616c2f74 65737470 .Finternal/testp
The bytes printed above only occur once and match the bytes from
file_internal_testprotos_testeditions_testeditions_hybrid_test_hybrid_proto_rawDesc
Change-Id: I8e1bfe53a5bbf65abe7861a749ace37b215a8e28
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/642857
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
This means our generated code requires Go 1.20+.
(Go Protobuf currently requires at least Go 1.21.)
Change-Id: Ie65be553bcb5912991d590104ff6b7c6a82b9c38
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/642055
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
In Go, []byte literals go into the writable .data (or .noptrdata) section,
but string literals goes into the read-only .rodata section.
I verified that the contents move from .noptrdata to .rodata:
% (cd internal/reflection_test && \
go test -c && \
objdump -s -j .rodata reflection_test.test | grep '0a4669 6e')
88bfd0 6e7370e8 070a4669 6e746572 6e616c2f nsp...Finternal/
Change-Id: I87e190b41c02235abea1967eddca2f0262060ed9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/638135
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
This is a no-op change in preparation of removing support for weak fields.
It allows users to keep enabling ProtoLegacy in general,
but also disable weak field support independently.
For golang/protobuf#1666
Change-Id: Ic3cb90d937e21a817ddbbb36029274be8b5f2513
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/641635
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This parameter became unused with golang/protobuf#1640.
Change-Id: Iab50ffb757f5b6f017bc32c37b25294894c1c6d9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/641019
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This commit does not change behavior because the encoding/json
package ignores unexported fields. To encode a Protobuf message
into JSON, users should reach for protobuf/encoding/protojson.
With this change, the generated code (.pb.go files) passes go vet.
I also made the integration test run go vet so that we do not regress.
For golang/protobuf#1657Fixesgolang/protobuf#1658
Change-Id: I0a677d09e4517b2649bfcfe76a49194c7ccbb823
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/641036
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This is the equivalent of CL 638495, but for the Opaque API.
While the behavior is the same for non-synthetic oneofs between
the Open Struct API and Opaque API, the behavior for synthetic oneofs
is not the same: Because the Opaque API uses a bitfield to store
presence, we need to use the Opaque presence check in WhichOneof().
This CL extends the testproto generator to use the protoc
command-line flag for the test3 testproto (which specifies
syntax = "proto3";), as the in-file api_level is only available
in .proto files using editions (but editions does not use synthetic
oneofs for optional fields, only proto3 does).
For golang/protobuf#1659
Change-Id: I0a1fd6e5fc6f96eeab043f966728ce2a14dbd446
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/638515
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
This reverts change https://go.dev/cl/632735, in which
I misunderstood what the Protobuf documentation wanted to convey:
The quoted docs in CL 632735 refer to reflection for proto3 optional
fields, not to reflection for proto3 synthetic oneofs.
Synthetic oneofs should remain visible through reflection.
For the Open API, this change restores the old behavior.
For the Opaque API, another fix is needed and will be sent
in a separate, follow-up CL (follow golang/protobuf#1659).
For golang/protobuf#1659
Thanks to Joshua Humphries for the report and reproducer!
Change-Id: I3a924eed6d2425581adc65651395a68fc1576f4d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/638495
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
For golang/protobuf#1657
Change-Id: I7b2b0c30506706015ce278e6054439c9ad9ef727
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/634815
TryBot-Bypass: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
This updates all generated code to match the contents of the latest
v29.1 release of Protobuf. The main significant change is that
SourceCodeInfo is now extendable (though in a very limited way).
This release of Protobuf includes new conformance tests that do not
pass with the latest version of this protobuf-go module. These have
been added to the list of known-failing tests. However all of them
are marked "required" in the conformance suite so should be likely
be addressed as soon as possible. (This change makes no attempt to
fix any of them.)
Change-Id: Ib554d393d7742b7a616af998eeab84a080a85e0e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/634015
Reviewed-by: Chressie Himpel <chressie@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Calling WhichOneof should not be possible for synthetic oneofs;
on the reflection level, these fields should work as if they
were regular fields, not as if they were oneofs:
> Reflection for proto3 optional fields should work properly. For example, a
> method like Reflection::HasField() should know to look for the hasbit for a
> proto3 optional field. It should not be fooled by the synthetic oneof into
> thinking that there is a case member for the oneof.
From the Protobuf docs at:
https://github.com/protocolbuffers/protobuf/blob/main/docs/implementing_proto3_presence.md#updating-reflection
This change was tested Google-internally as CL 701866153.
Change-Id: Id9500d4aa492acf4ebc6d2d84be07ed81201d2aa
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/632735
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This test was accidentally migrated from proto.GetExtension to
proto.HasExtension in a semi-automated change and nobody noticed.
proto.HasExtension does not actually trigger lazy extension decoding.
I verified this using the dlv debugger:
% go test -tags protolegacy -c -gcflags "all=-N -l"
% dlv exec ./impl.test
Type 'help' for list of commands.
(dlv) r -test.run=TestMarshalMessageSetLazyRace -test.v
Process restarted with PID 3586699
(dlv) b TestMarshalMessageSetLazyRace
Breakpoint 1 set at 0xac9bf6 for google.golang.org/protobuf/internal/impl_test.TestMarshalMessageSetLazyRace() ./lazy_test.go:500
(dlv) c
=== RUN TestMarshalMessageSetLazyRace
> [Breakpoint 1] google.golang.org/protobuf/internal/impl_test.TestMarshalMessageSetLazyRace() ./lazy_test.go:500 (hits goroutine(32):1 total:1) (PC: 0xac9bf6)
=> 500: func TestMarshalMessageSetLazyRace(t *testing.T) {
(dlv) b ExtensionField.lazyInit
Breakpoint 2 set at 0x8da076 for google.golang.org/protobuf/internal/impl.(*ExtensionField).lazyInit() ./codec_extension.go:123
(dlv) c
--- PASS: TestMarshalMessageSetLazyRace (4.07s)
PASS
Process 3586699 has exited with status 0
(dlv) exit
Note how breakpoint 2 is unexpectedly not hit!
Here is how the output changes with my fix:
% go test -tags protolegacy -c -gcflags "all=-N -l"
% dlv --check-go-version=false exec ./impl.test
Type 'help' for list of commands.
(dlv) r -test.run=TestMarshalMessageSetLazyRace -test.v
Process restarted with PID 3587344
(dlv) b TestMarshalMessageSetLazyRace
Breakpoint 1 set at 0xac9bf6 for google.golang.org/protobuf/internal/impl_test.TestMarshalMessageSetLazyRace() ./lazy_test.go:500
(dlv) c
=== RUN TestMarshalMessageSetLazyRace
> [Breakpoint 1] google.golang.org/protobuf/internal/impl_test.TestMarshalMessageSetLazyRace() ./lazy_test.go:500 (hits goroutine(28):1 total:1) (PC: 0xac9bf6)
=> 500: func TestMarshalMessageSetLazyRace(t *testing.T) {
(dlv) b ExtensionField.lazyInit
Breakpoint 2 set at 0x8da076 for google.golang.org/protobuf/internal/impl.(*ExtensionField).lazyInit() ./codec_extension.go:123
(dlv) c
> [Breakpoint 2] google.golang.org/protobuf/internal/impl.(*ExtensionField).lazyInit() ./codec_extension.go:123 (hits goroutine(68):1 total:4) (PC: 0x8da076)
> [Breakpoint 2] google.golang.org/protobuf/internal/impl.(*ExtensionField).lazyInit() ./codec_extension.go:123 (hits goroutine(70):1 total:4) (PC: 0x8da076)
> [Breakpoint 2] google.golang.org/protobuf/internal/impl.(*ExtensionField).lazyInit() ./codec_extension.go:123 (hits goroutine(58):1 total:4) (PC: 0x8da076)
> [Breakpoint 2] google.golang.org/protobuf/internal/impl.(*ExtensionField).lazyInit() ./codec_extension.go:123 (hits goroutine(54):1 total:4) (PC: 0x8da076)
=> 123: func (f *ExtensionField) lazyInit() {
(dlv) bt
0 0x00000000008da076 in google.golang.org/protobuf/internal/impl.(*ExtensionField).lazyInit
at ./codec_extension.go:123
1 0x00000000008dac3e in google.golang.org/protobuf/internal/impl.(*ExtensionField).Value
at ./codec_extension.go:180
2 0x000000000094e5d9 in google.golang.org/protobuf/internal/impl.(*extensionMap).Get
at ./message_reflect.go:276
3 0x000000000095a2ee in google.golang.org/protobuf/internal/impl.(*messageState).Get
at ./message_reflect_gen.go:90
4 0x0000000000890099 in google.golang.org/protobuf/proto.GetExtension
at /usr/local/google/home/stapelberg/protobuf/proto/extension.go:90
5 0x0000000000aca7d4 in google.golang.org/protobuf/internal/impl_test.TestMarshalMessageSetLazyRace.func2.1
at ./lazy_test.go:545
6 0x0000000000aca625 in google.golang.org/protobuf/internal/impl_test.TestMarshalMessageSetLazyRace.func2
at ./lazy_test.go:550
7 0x00000000006001c1 in runtime.goexit
at /usr/lib/google-golang/src/runtime/asm_amd64.s:1700
Change-Id: Ie7a8621064af412a1db7efb3ad6b8473f9db58e8
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/624416
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
This required renaming the extension fields to have distinct names, as
makeMessages() uses the protobuild package, which works with field names.
The makeMessages() function can be centrally extended with other variants of
testprotos (e.g. editions).
Change-Id: I68ef28baee674c6b53da8629c5ff7b2e1b92d4ff
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/624415
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
This will be useful for a few different upcoming tests,
to be submitted in a follow-up CL.
Change-Id: I1fde9f4567b149141737a74c8448c2ee14946ae4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/623117
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
With this change, we establish a similar mechanism in encoding/prototext as
already exists in the proto package: a testmessages_test.go which serves as an
extension point where one can inject more variants into the corpus of test
messages.
Change-Id: I9f83735a44e5fd2e4736f888aaca428e4cc8ed42
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/623116
Reviewed-by: Chressie Himpel <chressie@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This change required updating editions_defaults.binpb with
--edition_defaults_maximum=2024 so that we can use edition 2024 in our
testdata/enumprefix.proto test file.
For end users, this feature will only be available once:
1. protoc declares support for edition 2024 (expected early 2025).
2. protoc-gen-go declares support for edition 2024.
related to golang/protobuf#513
Change-Id: Ib8daeecae39ef32eff942279707d256c312f2a53
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/618979
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Mike Kruskal <mkruskal@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Also adds better benchmark cases for large message where some fields are
actually populated.
This change was previously done in Google internal cl/660848520.
Change-Id: I682aae0c9c2850bfe7638de29ab743ad7d7b119a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/609035
Reviewed-by: Christian Höppner <hoeppi@google.com>
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Most of our imports already comply with the style, but this change makes it
consistent across the entire repository.
I suspect that these unnecessary imports are left-overs from differences between
Google-internal (where many imports are renamed) and Open Source.
Change-Id: Id5f6eabd22bcc72dd4e36dd018b6cc5138df7696
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/609875
Reviewed-by: Christian Höppner <hoeppi@google.com>
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This change was originally done by Mike Kruskal in Google-internal cl/520045664.
(Undeclared extensions are no longer accepted Google-internally.)
Change-Id: I24d24ff2c7dc85920bd5dbe86724186d98783a18
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/609555
Reviewed-by: Christian Höppner <hoeppi@google.com>
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This change was originally done by Patrik Nyblom in Google-internal cl/308876443
(The corresponding test is TestDecodeFastCheckInitialized in methods_test.go.)
Change-Id: I312dff53db4d6f87decf152eb20ec4dd280f03c3
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/608256
Reviewed-by: Christian Höppner <hoeppi@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This change was originally done in Google-internal cl/512993970.
Change-Id: I66ea95cd975b160892ed9f931312f1767cbc9fc9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/608315
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Christian Höppner <hoeppi@google.com>
