In scope of this change, I had to fix the `IsPacked()` implementation
for field descriptors because it was not returning the correct values
for edition protos.
Change-Id: Ic1ba9d0b3552ddf16360a80336c14632f2ce6f16
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/566039
Auto-Submit: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Before this change the implementation would ignore editions options set on
extension declarations.
This change also add much more coverage for valid messages.
Change-Id: I0e7029b1430a39859cdd4a93667ac20c7e0ff20e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/565396
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
This fixes more minor issue when constructing the descriptor in that
delimited encoded message fields are not considered properly as groups.
Change-Id: I714a227a0f8d256fa7430c526844e8e94acdda33
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/564816
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
There are several minor fixes:
* fix the unmarshaler for the field options (it used the wrong field numbers).
* fix the unmarshaler to always initialize the file options with the appropriate defaults.
* fix the coder selection to work properly for fields with implicit presence tracking (they used the coders for explicit tracking).
* fix the dynamic message created from a descriptor for an editions using proto to use HasPresence instead of a syntax check for proto3
Change-Id: Ic13bc22a71aa9f93b476e5edd650d4d2ab5dcb98
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/564455
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
This change add parsing editions specific descriptor parts to
internal/filedesc which makes it possible to load file descriptors of
protos using editions at runtime.
Change-Id: I7a0c23303acddd2cff115859d4cf82bf0102b14c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/563615
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
I'll add the runtime implementations and tests in a followup change to
make it easier to review.
Change-Id: I2917e0d40a99e81799d89a66584a680ed7e8dbf7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/561939
Auto-Submit: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This change adds the first Go specific editions feature and the
associated proto. The feature is used to control if the legacy
UnmarshalJSON method should be generated for enums. This change only
affects protos using editions.
More tests will be added in a followup change.
Change-Id: Ifb62454d7568bd6d90d0b93f8953adcfe46c45fd
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/561938
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
Resolvesgolang/protobuf#1521
In the internal/impl package, a helper computes metadata used for
serialization and deserialization of extension values. It features a
package var of type sync.Map that is used as cache. This ostensibly
was for performance, however it has never worked, because the code
that updates the cache inserts entries into the wrong map. (These
erroneous entries do not cause any issues because they are keys that
never conflict with those used in valid queries.)
Instead of a one-line fix to have this code update the correct cache,
this change removes the cache altogether. The existence of the cache
means that once a protoreflect.ExtensionType is used to serialize or
deserialize a message, it can *never* be garbage collected. Workloads
that are long-lived servers using dynamic messages and extensions
based on user requests will exhibit unbounded growth in memory usage
as the cache only gets larger and larger.
Since the cache has never worked, any advantages it ostensibly
conferred have not been missed. So this fixes the unbounded memory
growth instead.
Change-Id: I15957fd8521852f9f7f9f89db7ebfd7170d85202
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/560095
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
+ The previous commit forgot to resolve the LegacyRequired feature
Change-Id: I05a2847be3d52277c65df004363ea3c87944364e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/555976
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Before this CL we only used features where they were applicable, e.g.
file features were only set on the file descriptor and field features
only on the fields descriptor. After this CL all descriptors (file,
message and field) contain always all features and when initializing
either of them we merge the features specified by options with the
features of the parent descriptor and use the result as the effective
feature. This way we initialize all the descriptors the same way and the
proto spec can change on which level features can be specified without
us having to change the code (at the moment most features can only be
specified on file or field level, in the future message might also be
possible).
Change-Id: I822202bf6a2bba5167bf9622c91a65c4523e78f9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/555975
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Before this change, calling dynamicpb.NewMessage() on a
protoreflect.MessageDescriptor that uses Editions would fail with:
message field "[…].relatedqueries" is not packable
(While Go Protobuf does not yet support Editions, C++ programs
can save descriptors to a file that is later loaded by a Go program.)
This was an oversight in commit e8baad6b6c.
Change-Id: I45d2b07a85ee40cb7c028098b81c4e76bf2ff555
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/554896
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
Reviewed-by: Christian Höppner <hoeppi@google.com>
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
Before this change, calling dynamicpb.NewMessage() on a
protoreflect.MessageDescriptor that uses Editions would fail with:
message field "[…].relatedqueries" is an invalid group: proto: invalid under
proto2 semantics
(While Go Protobuf does not yet support Editions, C++ programs
can save descriptors to a file that is later loaded by a Go program.)
Given that groups are only supported in proto2, but not in proto3
(see https://protobuf.dev/editions/features/#message_encoding),
I think the conditional meant to say "proto3 semantics".
Change-Id: I00368e4433ca60bb883b726e34ee6b8862cfd1e7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/554955
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
Reviewed-by: Christian Höppner <hoeppi@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The composites x/tools/analysis pass errors on this use of
reflect.ValueError with:
```
internal/impl/message_reflect_field.go:541:10: reflect.ValueError struct literal uses unkeyed fields (composites)
```
This patch adds the missing field names.
Change-Id: I1a175aad6e9724c27f2469adc1f3a9b1e8c61e85
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/553915
Reviewed-by: Christian Höppner <hoeppi@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This should help direct reports to the correct channel.
Change-Id: I468fc5ecaf81d9c8b95765f1fd7de9c5b5adaf2e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/553576
Reviewed-by: Christian Höppner <hoeppi@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Fixesgolang/protobuf#1583 and golang/protobuf#1584
Limits the level of recursion when parsing JSON to avoid
fatal stack overflow errors if input uses pathologically
deep nesting. This is already a feature of the binary
format, and this adds that feature to the JSON format.
This also re-implements how JSON values are discarded
to be more efficient (and not use recursion).
Change-Id: I4026b739abe0335387209a43645f65e4b6e43409
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/552255
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Bazel 7 is the latest release, but it cannot build Protobuf 25.1:
ERROR: no such package '@@com_google_protobuf//': The repository
'@@com_google_protobuf' could not be resolved: Repository
'@@com_google_protobuf' is not defined
Change-Id: I5a43ac6ea157056b6e40f1ccc2621b7808613b55
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/552075
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Remove the 'ignore' build constraint from integration_test.go and
arrange things such that the integration test runs on longtest
(but not longtest-race, for now anyway) builders. The existing
scripts for running the invocation test locally are preserved
and can be used as before.
This change relies on the builders being configured to place pre-built
protoc and conformance_test_runner binaries in $PATH (CL 547116); thanks
to Michael for making them available via a CIPD package.
Apply a few improvements to downloadFile and downloadArchive that came
about from earlier prototypes that used them and uncovered some issues.
Fixesgolang/go#64066.
Change-Id: I48b163e3ea9c441b748071da340d3e37282cc22b
Co-authored-by: Michael Stapelberg <stapelberg@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/541123
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
This is the first of multiple changes to add protobuf editions support
to the Go implementation.
This change includes the feature resolutions for protobuf editions.
The plugin does not yet report editions support and protoc would fail
if it invoked this plugin with a proto that uses editions.
Change-Id: I7d31909366c3433b21abab74ec92263e08145434
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/547315
Auto-Submit: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Commit-Queue: Lasse Folger <lassefolger@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Drop -mod=vendor because it's not needed by now. There are multiple
reasons for this: 1) integration_test.go has no external dependencies,
2) there's no checked-in vendor directory, 3) CL 544855 deleted the
last 'go mod vendor' invocation in code, and 4) now that the go.mod
has a go directive at 1.14+ (CL 547337), automatic vendoring¹ is on.
For golang/go#64066.
¹ https://go.dev/doc/go1.14#vendor, https://go.dev/ref/mod#vendoring
Change-Id: If8f3ea85d560b2796a2ccdb4ae7dbc38f0760982
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/548115
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
This should be slightly more efficient and insulate this code from
changes to the internal representation/alignment of slices and strings.
The interface header lacks an unsafe constructor but is also tweaked
to follow any future alignment changes, if they should occur.
Change-Id: I54a3a93f248d3eef883d5ebf9ed75cc1f5c20977
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/547336
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: David Chase <drchase@google.com>
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
1.17 is the oldest version we are testing and thus this is the oldest
version we can guarantee this repository works for. We also plan to
use features of newer Go version (up to 1.17) in following CLs.
Other changed to the go.mod file were introduced by `go mod tidy`
Change-Id: I79034d0a5f9c9ac9608308d01bc7e35725b2b592
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/547337
Commit-Queue: Michael Stapelberg <stapelberg@google.com>
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Prior to this change, one could either use go generate or
./test.bash -regenerate to re-generate the files — but only
the latter way would work.
With this change, we make ./test.bash -regenerate call go generate
so that the two ways cannot diverge again.
Change-Id: I940df66ebc6a4200847cf751929b3611309b67bc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/547256
Reviewed-by: Lasse Folger <lassefolger@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This is slightly more efficient and removes explicit dependence
on string layout and alignment. Build tags control use.
Because of what is arguably a bug in go 1.20's treatment of build tags,
is build-tagged to go1.21, not go1.20. (1.20 sees that it matches the
build tag and tries to compile the file, but then demands that it only use
language features from the version mentioned in go.mod. Go 1.21 and 1.22
behave more sensibly.)
Change-Id: I0cfc3b2e5651595edd3a82d9071d918700f961cd
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/545195
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The oldest version we currently support is Go 1.17,
so all pre-module code can be removed now.
Change-Id: Ief2834f3941f36e4f531fc14a82a820ecb6af105
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/544855
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
Fixesgolang/protobuf#1575
When converting a protoreflect.FileDescriptor
to a FileDescriptorProto, don't emit an invalid syntax.
This can happen for placeholder files, which represent
missing dependencies of a file.
Change-Id: I1d2ae1bd9f89c8a92e751871ce5c02727505b5e2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/540455
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
This updates all generated code to match the contents of the latest
v25.0-rc2 release candidate of Protobuf. This provides access to the
various new fields and types defined in google/protobuf/descriptor.proto.
Change-Id: I209f8509f2d3e9a0ac7d9e134caef38c48a0d16b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/538075
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
If `DiscardUnknown` was enabled, previously this syntax was rejected
with an error "unexpected token: ]":
unknown_field: [
{}
]
This is because the parser short-circuited after parsing the {}, and
didn't handle the closing square brace. Now it no longer short-circuits.
Change-Id: I6230b56766632752a5cc8822b17ed8262873d6cc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/530616
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Introduce the EmitDefaultValues in addition to the existing
EmitUnpopulated option.
EmitDefaultValues is added to emit json messages more compatible with
the `always_print_primitive_fields` option of the cpp protobuf library.
EmitUnpopulated overrides EmitDefaultValues since the former generates
a strict superset of the latter.
See descussion:
https://github.com/golang/protobuf/issues/1536
Change-Id: Ib29b69d630fa3e8d8fdeb0de43b5683f30152151
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/521215
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Testing with Go 1.21 also requires updating staticcheck.
Also remove Go 1.13, 1.14, 1.15 and 1.16 from the version matrix
and describe the version policy in a comment.
Change-Id: I28093d405b26e8e34e17119bddc5c990321ec348
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/527895
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
fixes https://github.com/golang/protobuf/issues/1562
Update to the latest version of the actions we use while we’re at it.
For setup-go@v4 this means we get caching based on go.{mod,sum}.
For actions/cache@v3 this means we get zstd compression.
Change-Id: I4ddb1a3e15eaafe1e9c946c3f0214da4fa1424df
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/526955
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
