258 Commits

Author SHA1 Message Date
Lasse Folger
b3f1c7a8f5 reflect/protodesc: remove obsolete JSON name check from desc validator
This check was removed from protoc in [1] and the comment in
desc_validate.go mentioned that it was there to emulate the protoc
behavior.

[1] 535069ec1b

fixes golang/protobuf#1616


Change-Id: I8cd6a28a4b2f2b807cdd4432b096cfce8e1f28c8
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/585736
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
2024-05-16 12:32:14 +00:00
Michael Stapelberg
cbc3dd69c1 all: replace interface{} by any now that we are on Go 1.21
I generated this change using:

  % sed -i 's,interface{},any,g' **/*.go
  % git checkout -- **/*.pb.go
  % $EDITOR cmd/protoc-gen-go/internal_gengo/well_known_types.go
  % ./regenerate.bash

Change-Id: I728f4b69c87ffc8f3b19bf807bf9bf1479bdbab4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/585735
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
2024-05-15 12:42:15 +00:00
Lasse Folger
6c3ebca21f all: update to protobuf 27.0-rc1 and regenerate protos
This change required some changes to the editions default handling code
because the descriptor.proto changed upstream [2]. The defaults are no
longer one feature set but are split into overridable and
not-overridable features which have to be merged.

I had to do bootstraping in 4 phases but the results should be correct:

1. generate everything depending on descriptor.proto
2. generate new defaults binary proto
3. adjust all code that works with defaults (*/edition.go files)
4. generate everything else

The was required because 1. is a prerequisite for 3. while 2. and 3. are
a prerequisite for 4. (2. and 3. can probably be done in parallel).

The new release also introduced new conformance tests. The go
implementation is not yet conformant and the tests will be fixed in a
follow up change because they require changes to the protojson and
protoext encoders.

[1] e5502c746e

Change-Id: Iddf248f6582a0402ab31256f6e64755d870ed82c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/582635
Auto-Submit: Lasse Folger <lassefolger@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
2024-05-03 08:08:45 +00:00
Lasse Folger
671c2db939 [proto] use the correct parent when resolving features for extensions
When I implemented this initially, I thought the parent of an extension is the
extendee. This is incorrect. The parent is the scope in which the extension is
defined. This CL changes the code to use the correct parent. This also allows
us to reduce some complexity in the implementation because we don't need to
wait until the extendee is resolved before we can resolve the features.

Change-Id: I6d7012f7502ef95457ab96f3e8abc4ab763d5bcb
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/579275
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
2024-04-16 13:33:29 +00:00
Mike Kruskal
b30b634cb8 protobuf: fix required/group bug in descriptor proto output of editions files
These need to be converted back to the appropriate label/type enums to produce valid descriptor protos under editions.

Change-Id: Ife04c4c556ffb06d1bc477725ff49058928a75ca
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/575916
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
2024-04-05 07:47:23 +00:00
Mike Kruskal
a18684df74 protobuf: fix delimited fields under editions in go
This brings go into conformance with other implementations.  Group-like message fields with delimited encoding will continue to use the type name for text-format, but everything else will use the field name.

Change-Id: Ib6d07f19ccfa853ce0370392c89fd24fb7148793
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/575896
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Commit-Queue: Michael Stapelberg <stapelberg@google.com>
2024-04-04 07:31:06 +00:00
Lasse Folger
dea00b5e2a all: enable editions support
Change-Id: I30e08a1610e11187a7632a409bd001f3bff1ba4a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/573355
Auto-Submit: Lasse Folger <lassefolger@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
2024-03-22 09:09:14 +00:00
Lasse Folger
59034d830f reflect/protodesc: restore edition in protodesc.ToDescriptorProto
I have choosen to implement this via interace assertion so that
other implementers of protoreflect.FileDescriptor can implement
this as well.

Change-Id: Ib907895044e89bdba1009cc09129d3dd1224561f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/573055
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-21 13:45:49 +00:00
Lasse Folger
3039476726 all: implement proto2/proto3 as editions [2/2]
This change removes the remaining usages of Syntax() from Go Protobuf
and uses edition features instead.

It also adds a new function to the EnumDescriptor interface checking if
the enum is using a Closed semantics.

All of these changes were tested on the Google corpus.

Change-Id: I7a8110f6f3b6ed24bf7ece500b4942371302c56c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/572695
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-19 12:54:36 +00:00
Lasse Folger
7259b46773 all: implement proto2/proto3 as editions [1/2]
This change removes most usages of Syntax() from the repository and uses
edition features for instead. The appropriate edition feature defaults are
loaded for proto2/proto3 when the initialization of the descriptors
start.

All of these changes were tested on the Google corpus.

Change-Id: Ieca076a2b38ca8e50e084cd32e725b7b3dcb4171
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/572435
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
2024-03-19 09:40:08 +00:00
Lasse Folger
f2cb7f136e encoding/protojson: add protojson editions tests including fuzztests
Change-Id: I478bf6a945cb2c86c71fd20b64dedb4b2e585f1d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/566035
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
2024-02-23 08:35:31 +00:00
Lasse Folger
f9eb6c71e7 go_features.proto: move .proto file next to generated .pb.go file
Change-Id: Ia24ca1ad2186bcc95681081d2719ee834b385728
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/565955
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
2024-02-22 08:56:15 +00:00
Lasse Folger
997075a7f5 reflect/protodesc: add editions support
Change-Id: Ie7fee936bdb480802cbaca34cf63c0dad34590b8
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/564815
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
2024-02-19 07:03:53 +00:00
Lasse Folger
e39100579e reflect/protodesc: propagate field_presence legacy_required properly
Change-Id: I3f067eb564349bc1ff867a03d9f1aa94b3428236
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/563735
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
2024-02-13 15:44:19 +00:00
Lasse Folger
020ec10f7b internal/filedesc: implement runtime editions support
This change add parsing editions specific descriptor parts to
internal/filedesc which makes it possible to load file descriptors of
protos using editions at runtime.

Change-Id: I7a0c23303acddd2cff115859d4cf82bf0102b14c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/563615
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-02-13 12:20:49 +00:00
Lasse Folger
f387b036d8 editions: add some test protos for editions
I'll add the runtime implementations and tests in a followup change to
make it easier to review.

Change-Id: I2917e0d40a99e81799d89a66584a680ed7e8dbf7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/561939
Auto-Submit: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-02-12 14:14:45 +00:00
Lasse Folger
2040e8671b all: implement Go-specific GenerateLegacyUnmarshalJSON
This change adds the first Go specific editions feature and the
associated proto. The feature is used to control if the legacy
UnmarshalJSON method should be generated for enums. This change only
affects protos using editions.

More tests will be added in a followup change.

Change-Id: Ifb62454d7568bd6d90d0b93f8953adcfe46c45fd
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/561938
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
2024-02-12 13:54:51 +00:00
Lasse Folger
99e193e5dd all: update to protobuf-26.0-rc2 to implement go edition features
Change-Id: I190b1550e3e2e630392fed155716af9d651c74bc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/562615
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
2024-02-08 09:23:48 +00:00
Lasse Folger
b70f02be53 editions: minor fix to resolve editions features correctly
+ The previous commit forgot to resolve the LegacyRequired feature

Change-Id: I05a2847be3d52277c65df004363ea3c87944364e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/555976
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-16 15:43:50 +00:00
Lasse Folger
9e454d6514 internal/filedesc: refactor editions feature resolution
Before this CL we only used features where they were applicable, e.g.
file features were only set on the file descriptor and field features
only on the fields descriptor. After this CL all descriptors (file,
message and field) contain always all features and when initializing
either of them we merge the features specified by options with the
features of the parent descriptor and use the result as the effective
feature. This way we initialize all the descriptors the same way and the
proto spec can change on which level features can be specified without
us having to change the code (at the moment most features can only be
specified on file or field level, in the future message might also be
possible).

Change-Id: I822202bf6a2bba5167bf9622c91a65c4523e78f9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/555975
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
2024-01-16 12:12:51 +00:00
Michael Stapelberg
26a52f39da reflect/protodesc: fix packed field conditional
Before this change, calling dynamicpb.NewMessage() on a
protoreflect.MessageDescriptor that uses Editions would fail with:

message field "[…].relatedqueries" is not packable

(While Go Protobuf does not yet support Editions, C++ programs
can save descriptors to a file that is later loaded by a Go program.)

This was an oversight in commit e8baad6b6c9e2bb1c48e4963866248d4f35d4fd7.

Change-Id: I45d2b07a85ee40cb7c028098b81c4e76bf2ff555
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/554896
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
Reviewed-by: Christian Höppner <hoeppi@google.com>
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
2024-01-09 15:05:22 +00:00
Michael Stapelberg
7c85df2652 reflect/protodesc: fix group validation: not supported in proto3
Before this change, calling dynamicpb.NewMessage() on a
protoreflect.MessageDescriptor that uses Editions would fail with:

message field "[…].relatedqueries" is an invalid group: proto: invalid under
proto2 semantics

(While Go Protobuf does not yet support Editions, C++ programs
can save descriptors to a file that is later loaded by a Go program.)

Given that groups are only supported in proto2, but not in proto3
(see https://protobuf.dev/editions/features/#message_encoding),
I think the conditional meant to say "proto3 semantics".

Change-Id: I00368e4433ca60bb883b726e34ee6b8862cfd1e7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/554955
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
Reviewed-by: Christian Höppner <hoeppi@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-09 14:54:38 +00:00
Lasse Folger
e8baad6b6c reflect/protodesc: support editions feature resolution
This is the first of multiple changes to add protobuf editions support
to the Go implementation.
This change includes the feature resolutions for protobuf editions.
The plugin does not yet report editions support and protoc would fail
if it invoked this plugin with a proto that uses editions.

Change-Id: I7d31909366c3433b21abab74ec92263e08145434
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/547315
Auto-Submit: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Commit-Queue: Lasse Folger <lassefolger@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
2023-12-08 13:18:14 +00:00
David Chase
c9d255971a reflect/protoreflect: use go1.20 unsafe interface for strings and slices
This should be slightly more efficient and insulate this code from
changes to the internal representation/alignment of slices and strings.

The interface header lacks an unsafe constructor but is also tweaked
to follow any future alignment changes, if they should occur.

Change-Id: I54a3a93f248d3eef883d5ebf9ed75cc1f5c20977
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/547336
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: David Chase <drchase@google.com>
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
2023-12-06 13:41:21 +00:00
Josh Humphries
a8317fb7c5 reflect/protodesc: don't generate invalid syntax values
Fixes golang/protobuf#1575

When converting a protoreflect.FileDescriptor
to a FileDescriptorProto, don't emit an invalid syntax.

This can happen for placeholder files, which represent
missing dependencies of a file.

Change-Id: I1d2ae1bd9f89c8a92e751871ce5c02727505b5e2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/540455
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-11-08 07:09:48 +00:00
Josh Humphries
f4a6c1f6e5 types/descriptorpb: regenerate using latest protobuf v25.0-rc2 release candidate
This updates all generated code to match the contents of the latest
v25.0-rc2 release candidate of Protobuf. This provides access to the
various new fields and types defined in google/protobuf/descriptor.proto.

Change-Id: I209f8509f2d3e9a0ac7d9e134caef38c48a0d16b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/538075
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
2023-10-27 08:25:48 +00:00
Joe Tsai
f9212a8dfa all: modernize documentation
Modernize the documentation across the entire module
to make use of the newer ability to linkify declarations.

Change-Id: I440f9a3f025ec6fadfd9cba59b1dfb57028bf3f1
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/309430
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-09-05 14:55:28 +00:00
Josh Humphries
6d0a5dbd95 types/descriptorpb: regenerate using latest protobuf v24.0-rc2 release candidate
This updates all generated code to match the contents of the latest
v24.0-rc2 release candidate of Protobuf. This provides access to the
various new fields and types defined in google/protobuf/descriptor.proto.

Change-Id: I8ec56b308dd7eed52fb9b17b258a45d0669c47ff
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/512795
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
2023-07-27 12:38:59 +00:00
Damien Neil
736947c4de all: update protobuf release version to fix macOS tests
Update to a protobuf version that contains a fix for
https://github.com/protocolbuffers/protobuf/issues/12173
(macOS build error).

Set "CC=clang" when building the protobuf repo on macOS,
which fixes build errors on my machine (macOS 13.3.1, M1).

Change-Id: Id8abb054479454b7ccbded5a6fa4d74f3d3c236d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/489315
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Lasse Folger <lassefolger@google.com>
2023-04-26 20:34:47 +00:00
Josh Humphries
bc1253ad37 types/descriptorpb: regenerate using latest protobuf v22.0 release
This updates all generated code to match the contents of the latest
v22.0 release of Protobuf.

This involved a couple of changes to the script that does the sync'ing:
  1. The new Protobuf version no longer includes autoconf configuration
     and instead requires using bazel to build things.
  2. The new Protobuf release does not have an artifact named
     "protobuf-all-${VERSION}.tar.gz", but the one named
     "protobuf-${VERSION}.tar.gz" has all of the sources and was
     sufficient for the regenerate.bash script to complete.

This change does NOT regenerate the protos related to benchmarks.
The Protobuf repo no longer includes benchmarks. The CL removing them
says they are superceded by google/fleetbench:
    83c499de86
But that project's proto benchmark files are very different:
    https://github.com/google/fleetbench/tree/main/fleetbench/proto
So I commented out those steps in the generation code since the benchmarks
will need some work to reconcile with fleetbench.

This code adds known failing tests for conformance. New test cases were
added in https://github.com/protocolbuffers/protobuf/pull/9534, but the
Go protojson package does not behave according to the new tests. There
is an existing issue in GitHub about this:
    https://github.com/golang/protobuf/issues/1208

Change-Id: Iad796ec7889bc2a74b58db5224facf850cd1a1cd
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/469255
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-02-22 09:33:03 +00:00
Oleksandr Redko
246af0d153 internal,reflect: correct typos
Fix typos in comments and unexported function name.

Change-Id: I239fff85bc68d7f0ef6f8853acdc9bbe19d86304
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/466535
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
2023-02-09 09:27:51 +00:00
Oleksandr Redko
49eaa78c6c all: update links to Protocol Buffer documentation
https://developers.google.com/protocol-buffers/ states that
this site will be deprecated January 2023 and provides
https://protobuf.dev/ as a link to the new location.

Generated .pb.go files are created by generate.bash using Go 1.18,
so leave them unchanged for now.

Fixes golang/protobuf#1520.

Change-Id: I1ddf6f329c96ba05d7c7d8562ad2945f479ee09b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/466375
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
2023-02-08 13:52:20 +00:00
Joe Tsai
b2a7dfe48b reflect/protoreflect: add Value.Equal method
The Value.Equal method compares Values in a way that is
analogous to the reflect.DeepEqual function.

Most of the implementation is moved from the "proto" package,
with proto.Equal implemented in terms of protoreflect.Value.Equal.

Change-Id: Ie9d5f6c073dc49fa59f78385c441db8137de5ec5
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/450775
TryBot-Bypass: Heschi Kreinick <heschi@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Oleksii Prokopchuk <prokopchuk@google.com>
Reviewed-by: Christian Höppner <hoeppi@google.com>
Run-TryBot: Lasse Folger <lassefolger@google.com>
2022-12-01 15:36:18 +00:00
Michael Stapelberg
7a48e2b662 protoreflect: fix list example code
Change-Id: I2c8afba104ae97be5e575226a47f6a7f20198abe
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/431798
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
2022-09-20 08:06:00 +00:00
Josh Humphries
692f4a24f8 types/descriptorpb: update *.pb.go to use latest protoc release, 21.5
I updated the protoc version to 21.5 and ran regenerate.bash.

This updates the various *.pb.go files to use the sources from
the latest protobuf release. In particular, this updates the
descriptorpb, so it includes some changes to descriptor.proto
in the past year and a half since this was last re-generated.
The change also pulled over several other generated changes.

Change-Id: Icd9c4e5d47ba85e1f4bae1d0a025f8dec15a3014
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/426054
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
2022-08-27 06:10:12 +00:00
Damien Neil
b0a944684d all: reformat with go1.19 gofmt
Apply go1.19 gofmt to non-generated files.

Generated .pb.go files are created by generate.bash using Go 1.18,
so leave them unchanged for now.

Change-Id: Ied36c83cf99704988d059bf0412e677f0fbc71b0
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/418676
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
2022-07-21 16:01:13 +00:00
Koichi Shiraishi
784c482554 all: remove shorthand import aliases
Change-Id: I62ec79419354a2c65ac69e4bcc1cc864a6a40a1a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/407934
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2022-05-24 20:05:50 +00:00
Joe Tsai
8a7ba0762c reflect/protoreflect: improve documentation for message types
There are a number of message types, which can be confusing.
Expand the documentation. In particular, use Go's reflection model
as an analogy, which helps users if they are at least familiar
with how Go reflection works. After all, protobuf reflection
was modeled after Go reflection to a large degree.

Change-Id: I388ae41c792905145ccc86dc938b901a2b79f408
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/399654
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
2022-04-12 06:05:53 +00:00
Lasse Folger
3992ea83a2 all: implement depth limit for unmarshaling
+ This change introduce a default and configurable depth limit for
  proto.Unmarshal. If a message is nested deeper than the limit,
  unmarshaling will fail. There are two ways to nest messages. Either by
  having fields which are message types itself or by using groups.
+ The default limit is 10,000 for now. This might change in the future
  to align it with other language implementation (C++ and Java use 100
  as limit).
+ If pure groups (groups that don't contain message fields) are nested
  deeper than the default limit the unmarshaling fails with:
  proto: cannot parse invalid wire-format data
+ Note: the configured limit does not apply to pure groups.
+ This change is introduced to improve security and robustness. Because
  unmarshaling is implemented using recursion it can lead to stack overflows
  for certain inputs. The introduced limit protects against this.
+ A secondary motivation for this limit is the alignment with other
  languages. Protocol buffers are a language interoperability mechanism
  and thus either all implementations should accept the input or all
  implementation should reject the input.

Change-Id: I14bdb44d06e4bd1aa90d6336c2cf6446003b2037
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/385854
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Damien Neil <dneil@google.com>
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
2022-02-17 17:07:31 +00:00
Damien Neil
3a9e1dc314 all: gofmt all
Change-Id: I1d42c8d784440a0b7e40b4f0b8da54be0fb338a6
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/370054
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Florian Zenker <floriank@google.com>
2022-01-04 22:57:54 +00:00
Damien Neil
05be61fde3 reflect/protoreflect: add more docs on Value aliasing
Fixes golang/protobuf#1346

Change-Id: I9deb36e1c5e4f28c1c7e99ca64260c7a86af4ea1
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/339569
Trust: Damien Neil <dneil@google.com>
Trust: Joe Tsai <joetsai@digital-static.net>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joe Tsai <joetsai@digital-static.net>
2021-08-05 18:00:43 +00:00
Damien Neil
21e33cc910 reflect/protoregistry: restore conflicting file names check
There are inconsistencies between implementations on what happens when
a single program contains generated code for multiple files with the
same source path. At least one canonical implementation (C++) will fail
at link time. Others print warnings. Some silently resolve the registry
conflict in favor of one file or the other. The protobuf maintainers
agree, however, that the desired behavior is for this condition to be an
error.

Updates golang/protobuf#1122

Change-Id: I716708f16ef90210bdfceb0888691e47783df172
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/322729
Trust: Damien Neil <dneil@google.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2021-06-02 01:04:00 +00:00
Damien Neil
febffdd88e reflect/protoregistry: permit conflicting file names
Remove the requirement that all registered files have unique names.

Files.FindFileByPath now returns an error if multiple files match
the path. Files.RangeFiles iterates over all files, including ones with
the same name.

Unlike descriptor names, file names are not part of any of the wire
encodings. Conflicts in file names only affect file lookups by path,
which is an uncommon operation.

The Files.RegisterFile documentation already states (inaccurately),
"It is permitted for multiple files to have the same file path."
This change brings the behavior of RegisterFile in line with the
documentation.

Fixes golang/protobuf#1122

Change-Id: I2750e35736fd34def23fa6ab6d6a829095a65a26
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/321531
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2021-05-25 00:53:49 +00:00
sryoya
0e358a402f reflect/protorange: fix typo of func name in comment
Change-Id: I80ffc34e29757e0cbdccdd530404fa215cfaf75f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/317989
Reviewed-by: Alberto Donizetti <alb.donizetti@gmail.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
2021-05-07 17:05:00 +00:00
Joe Tsai
e4fcb9f7e7 reflect: add protopath and protorange packages
The protopath package provides a means to programmatically represent
a sequence of protobuf reflection operations.
The protorange package traverses through a message and
calls a user-provided function as it iterates.

This feature sets the groundwork for the often requested feature
of being able to exclude certain fields when merging or serializing.

package protopath
    type Path []Step
    type Step struct{ ... }
        func Root(protoreflect.MessageDescriptor) Step
        func FieldAccess(protoreflect.FieldDescriptor) Step
        func UnknownAccess() Step
        func ListIndex(int) Step
        func MapIndex(protoreflect.MapKey) Step
        func AnyExpand(protoreflect.MessageDescriptor) Step
        func (Step) Kind() StepKind
        func (Step) FieldDescriptor() protoreflect.FieldDescriptor
        func (Step) MessageDescriptor() protoreflect.MessageDescriptor
        func (Step) ListIndex() int
        func (Step) MapIndex() protoreflect.MapKey
        func (Step) String() string
    type StepKind int
        const RootStep StepKind
        const FieldAccessStep StepKind
        const UnknownAccessStep StepKind
        const ListIndexStep StepKind
        const MapIndexStep StepKind
        const AnyExpandStep StepKind
    type Values struct {
        Path   Path
        Values []protoreflect.Value
    }
    func (Values) Index(int) (out struct{ ... })
    func (Values) Len() int
    func (Values) String() string

package protorange
    var Break error
    var Terminate error
    func Range(protoreflect.Message, func(protopath.Values) error) error
    type Options struct {
        Stable bool
        Resolver interface { ... }
    }
    func (Options) Range(m protoreflect.Message, push, pop func(protopath.Values) error) error

Change-Id: I29cbd5142fe169d78367d54a95d37801888b64f4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/236540
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
2021-04-01 23:09:19 +00:00
Joe Tsai
77dfd04893 reflect/protoregistry: add compile-time opt-out for registration conflicts
The next release of this module will panic on registration conflicts.
While we have been warning that this will become a fatal error for about
a year now, it's almost certainly the case that some users still have
ambiguous registrations.

In addition to the GOLANG_PROTOBUF_REGISTRATION_CONFLICT environment variable,
also define a linker-initialized variable that users can set.

Change-Id: I3325d841b9b6c692e2862100d81aebdcfda069fc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/295349
Trust: Joe Tsai <joetsai@digital-static.net>
Trust: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
2021-03-02 23:29:07 +00:00
Joe Tsai
aa45c46752 reflect/protoregistry: panic upon registration conflicts
In this CL, we switch the behavior of registration conflicts
from simply logging a warning to panicking.
This is a change that we have warned about ever since
the release of v1.20.0 in 2020-03-02.

For users that have not fixed their dependencies or are unable to,
they may set the GOLANG_PROTOBUF_REGISTRATION_CONFLICT environment
to "warn" in order to obtain the prior behavior of simply logging
and ignoring subsequent conflicts (first wins).

Change-Id: Ied7937f0fd5cb5d875858411714ebeb5ce145574
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/235298
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
2020-10-16 22:00:47 +00:00
Joe Tsai
9712361eda all: rely on dynamic dependency check for genproto
The genproto module incurs an unfortunate amount of dependency bloat
since it has a relatively large set of transitive dependencies.
Even though these are brought in as a weak dependency
(i.e., there is no hard code dependency on genproto),
it complicates dependency inspection since users need to vet
modules that they may not actually be using.

Avoid this weak dependency and instead rely on a dynamic check
to ensure that a sufficiently new version of genproto is used.
While it can't statically prevent old versions of genproto from being
linked into a binary, it guarantees that it will panic at init
with a helpful message on what's wrong.

Change-Id: I2b5a9759a5cd4c4c57aced54278502b1b6352ba7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/262679
Trust: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
2020-10-15 21:46:17 +00:00
Joe Tsai
687df2333b reflect/protodesc: fix round-tripping for package field
If a .proto file does not have a package statement,
then protoc will not populate the package field.
Emulate the same behavior for ToFileDescriptorProto.

Fixes golang/protobuf#1195

Change-Id: I617102e50c96f0fa1bac6a6bc4de1cc485750bb9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/259899
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Joe Tsai <thebrokentoaster@gmail.com>
2020-10-07 22:44:53 +00:00
Joe Tsai
42cc4c592f reflect/protoreflect: improve source information usability
Added API:
  SourceLocations.ByPath
  SourceLocations.ByDescriptor
  SourceLocation.Next
  SourcePath.String
  SourcePath.Equal

We modify compiler/protogen to use SourceLocations.ByDescriptor.
In retrospect, if this had existed during the development of protogen,
we would not have exposed protogen.Location and related fields.

Change-Id: I58f17e59f90b9ba16f0982c4b71c2542e4ff6e75
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/238000
Reviewed-by: Damien Neil <dneil@google.com>
2020-07-23 21:50:46 +00:00