238 Commits

Author SHA1 Message Date
Michael Stapelberg
26a52f39da reflect/protodesc: fix packed field conditional
Before this change, calling dynamicpb.NewMessage() on a
protoreflect.MessageDescriptor that uses Editions would fail with:

message field "[…].relatedqueries" is not packable

(While Go Protobuf does not yet support Editions, C++ programs
can save descriptors to a file that is later loaded by a Go program.)

This was an oversight in commit e8baad6b6c9e2bb1c48e4963866248d4f35d4fd7.

Change-Id: I45d2b07a85ee40cb7c028098b81c4e76bf2ff555
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/554896
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
Reviewed-by: Christian Höppner <hoeppi@google.com>
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
2024-01-09 15:05:22 +00:00
Michael Stapelberg
7c85df2652 reflect/protodesc: fix group validation: not supported in proto3
Before this change, calling dynamicpb.NewMessage() on a
protoreflect.MessageDescriptor that uses Editions would fail with:

message field "[…].relatedqueries" is an invalid group: proto: invalid under
proto2 semantics

(While Go Protobuf does not yet support Editions, C++ programs
can save descriptors to a file that is later loaded by a Go program.)

Given that groups are only supported in proto2, but not in proto3
(see https://protobuf.dev/editions/features/#message_encoding),
I think the conditional meant to say "proto3 semantics".

Change-Id: I00368e4433ca60bb883b726e34ee6b8862cfd1e7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/554955
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
Reviewed-by: Christian Höppner <hoeppi@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-09 14:54:38 +00:00
Lasse Folger
e8baad6b6c reflect/protodesc: support editions feature resolution
This is the first of multiple changes to add protobuf editions support
to the Go implementation.
This change includes the feature resolutions for protobuf editions.
The plugin does not yet report editions support and protoc would fail
if it invoked this plugin with a proto that uses editions.

Change-Id: I7d31909366c3433b21abab74ec92263e08145434
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/547315
Auto-Submit: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Commit-Queue: Lasse Folger <lassefolger@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
2023-12-08 13:18:14 +00:00
David Chase
c9d255971a reflect/protoreflect: use go1.20 unsafe interface for strings and slices
This should be slightly more efficient and insulate this code from
changes to the internal representation/alignment of slices and strings.

The interface header lacks an unsafe constructor but is also tweaked
to follow any future alignment changes, if they should occur.

Change-Id: I54a3a93f248d3eef883d5ebf9ed75cc1f5c20977
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/547336
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: David Chase <drchase@google.com>
Auto-Submit: Michael Stapelberg <stapelberg@google.com>
2023-12-06 13:41:21 +00:00
Josh Humphries
a8317fb7c5 reflect/protodesc: don't generate invalid syntax values
Fixes golang/protobuf#1575

When converting a protoreflect.FileDescriptor
to a FileDescriptorProto, don't emit an invalid syntax.

This can happen for placeholder files, which represent
missing dependencies of a file.

Change-Id: I1d2ae1bd9f89c8a92e751871ce5c02727505b5e2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/540455
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-11-08 07:09:48 +00:00
Josh Humphries
f4a6c1f6e5 types/descriptorpb: regenerate using latest protobuf v25.0-rc2 release candidate
This updates all generated code to match the contents of the latest
v25.0-rc2 release candidate of Protobuf. This provides access to the
various new fields and types defined in google/protobuf/descriptor.proto.

Change-Id: I209f8509f2d3e9a0ac7d9e134caef38c48a0d16b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/538075
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
2023-10-27 08:25:48 +00:00
Joe Tsai
f9212a8dfa all: modernize documentation
Modernize the documentation across the entire module
to make use of the newer ability to linkify declarations.

Change-Id: I440f9a3f025ec6fadfd9cba59b1dfb57028bf3f1
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/309430
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-09-05 14:55:28 +00:00
Josh Humphries
6d0a5dbd95 types/descriptorpb: regenerate using latest protobuf v24.0-rc2 release candidate
This updates all generated code to match the contents of the latest
v24.0-rc2 release candidate of Protobuf. This provides access to the
various new fields and types defined in google/protobuf/descriptor.proto.

Change-Id: I8ec56b308dd7eed52fb9b17b258a45d0669c47ff
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/512795
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
2023-07-27 12:38:59 +00:00
Damien Neil
736947c4de all: update protobuf release version to fix macOS tests
Update to a protobuf version that contains a fix for
https://github.com/protocolbuffers/protobuf/issues/12173
(macOS build error).

Set "CC=clang" when building the protobuf repo on macOS,
which fixes build errors on my machine (macOS 13.3.1, M1).

Change-Id: Id8abb054479454b7ccbded5a6fa4d74f3d3c236d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/489315
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Lasse Folger <lassefolger@google.com>
2023-04-26 20:34:47 +00:00
Josh Humphries
bc1253ad37 types/descriptorpb: regenerate using latest protobuf v22.0 release
This updates all generated code to match the contents of the latest
v22.0 release of Protobuf.

This involved a couple of changes to the script that does the sync'ing:
  1. The new Protobuf version no longer includes autoconf configuration
     and instead requires using bazel to build things.
  2. The new Protobuf release does not have an artifact named
     "protobuf-all-${VERSION}.tar.gz", but the one named
     "protobuf-${VERSION}.tar.gz" has all of the sources and was
     sufficient for the regenerate.bash script to complete.

This change does NOT regenerate the protos related to benchmarks.
The Protobuf repo no longer includes benchmarks. The CL removing them
says they are superceded by google/fleetbench:
    83c499de86
But that project's proto benchmark files are very different:
    https://github.com/google/fleetbench/tree/main/fleetbench/proto
So I commented out those steps in the generation code since the benchmarks
will need some work to reconcile with fleetbench.

This code adds known failing tests for conformance. New test cases were
added in https://github.com/protocolbuffers/protobuf/pull/9534, but the
Go protojson package does not behave according to the new tests. There
is an existing issue in GitHub about this:
    https://github.com/golang/protobuf/issues/1208

Change-Id: Iad796ec7889bc2a74b58db5224facf850cd1a1cd
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/469255
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-02-22 09:33:03 +00:00
Oleksandr Redko
246af0d153 internal,reflect: correct typos
Fix typos in comments and unexported function name.

Change-Id: I239fff85bc68d7f0ef6f8853acdc9bbe19d86304
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/466535
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
2023-02-09 09:27:51 +00:00
Oleksandr Redko
49eaa78c6c all: update links to Protocol Buffer documentation
https://developers.google.com/protocol-buffers/ states that
this site will be deprecated January 2023 and provides
https://protobuf.dev/ as a link to the new location.

Generated .pb.go files are created by generate.bash using Go 1.18,
so leave them unchanged for now.

Fixes golang/protobuf#1520.

Change-Id: I1ddf6f329c96ba05d7c7d8562ad2945f479ee09b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/466375
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
2023-02-08 13:52:20 +00:00
Joe Tsai
b2a7dfe48b reflect/protoreflect: add Value.Equal method
The Value.Equal method compares Values in a way that is
analogous to the reflect.DeepEqual function.

Most of the implementation is moved from the "proto" package,
with proto.Equal implemented in terms of protoreflect.Value.Equal.

Change-Id: Ie9d5f6c073dc49fa59f78385c441db8137de5ec5
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/450775
TryBot-Bypass: Heschi Kreinick <heschi@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Oleksii Prokopchuk <prokopchuk@google.com>
Reviewed-by: Christian Höppner <hoeppi@google.com>
Run-TryBot: Lasse Folger <lassefolger@google.com>
2022-12-01 15:36:18 +00:00
Michael Stapelberg
7a48e2b662 protoreflect: fix list example code
Change-Id: I2c8afba104ae97be5e575226a47f6a7f20198abe
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/431798
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
2022-09-20 08:06:00 +00:00
Josh Humphries
692f4a24f8 types/descriptorpb: update *.pb.go to use latest protoc release, 21.5
I updated the protoc version to 21.5 and ran regenerate.bash.

This updates the various *.pb.go files to use the sources from
the latest protobuf release. In particular, this updates the
descriptorpb, so it includes some changes to descriptor.proto
in the past year and a half since this was last re-generated.
The change also pulled over several other generated changes.

Change-Id: Icd9c4e5d47ba85e1f4bae1d0a025f8dec15a3014
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/426054
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
2022-08-27 06:10:12 +00:00
Damien Neil
b0a944684d all: reformat with go1.19 gofmt
Apply go1.19 gofmt to non-generated files.

Generated .pb.go files are created by generate.bash using Go 1.18,
so leave them unchanged for now.

Change-Id: Ied36c83cf99704988d059bf0412e677f0fbc71b0
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/418676
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
2022-07-21 16:01:13 +00:00
Koichi Shiraishi
784c482554 all: remove shorthand import aliases
Change-Id: I62ec79419354a2c65ac69e4bcc1cc864a6a40a1a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/407934
Reviewed-by: Lasse Folger <lassefolger@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2022-05-24 20:05:50 +00:00
Joe Tsai
8a7ba0762c reflect/protoreflect: improve documentation for message types
There are a number of message types, which can be confusing.
Expand the documentation. In particular, use Go's reflection model
as an analogy, which helps users if they are at least familiar
with how Go reflection works. After all, protobuf reflection
was modeled after Go reflection to a large degree.

Change-Id: I388ae41c792905145ccc86dc938b901a2b79f408
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/399654
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
2022-04-12 06:05:53 +00:00
Lasse Folger
3992ea83a2 all: implement depth limit for unmarshaling
+ This change introduce a default and configurable depth limit for
  proto.Unmarshal. If a message is nested deeper than the limit,
  unmarshaling will fail. There are two ways to nest messages. Either by
  having fields which are message types itself or by using groups.
+ The default limit is 10,000 for now. This might change in the future
  to align it with other language implementation (C++ and Java use 100
  as limit).
+ If pure groups (groups that don't contain message fields) are nested
  deeper than the default limit the unmarshaling fails with:
  proto: cannot parse invalid wire-format data
+ Note: the configured limit does not apply to pure groups.
+ This change is introduced to improve security and robustness. Because
  unmarshaling is implemented using recursion it can lead to stack overflows
  for certain inputs. The introduced limit protects against this.
+ A secondary motivation for this limit is the alignment with other
  languages. Protocol buffers are a language interoperability mechanism
  and thus either all implementations should accept the input or all
  implementation should reject the input.

Change-Id: I14bdb44d06e4bd1aa90d6336c2cf6446003b2037
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/385854
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Damien Neil <dneil@google.com>
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
2022-02-17 17:07:31 +00:00
Damien Neil
3a9e1dc314 all: gofmt all
Change-Id: I1d42c8d784440a0b7e40b4f0b8da54be0fb338a6
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/370054
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Florian Zenker <floriank@google.com>
2022-01-04 22:57:54 +00:00
Damien Neil
05be61fde3 reflect/protoreflect: add more docs on Value aliasing
Fixes golang/protobuf#1346

Change-Id: I9deb36e1c5e4f28c1c7e99ca64260c7a86af4ea1
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/339569
Trust: Damien Neil <dneil@google.com>
Trust: Joe Tsai <joetsai@digital-static.net>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joe Tsai <joetsai@digital-static.net>
2021-08-05 18:00:43 +00:00
Damien Neil
21e33cc910 reflect/protoregistry: restore conflicting file names check
There are inconsistencies between implementations on what happens when
a single program contains generated code for multiple files with the
same source path. At least one canonical implementation (C++) will fail
at link time. Others print warnings. Some silently resolve the registry
conflict in favor of one file or the other. The protobuf maintainers
agree, however, that the desired behavior is for this condition to be an
error.

Updates golang/protobuf#1122

Change-Id: I716708f16ef90210bdfceb0888691e47783df172
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/322729
Trust: Damien Neil <dneil@google.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2021-06-02 01:04:00 +00:00
Damien Neil
febffdd88e reflect/protoregistry: permit conflicting file names
Remove the requirement that all registered files have unique names.

Files.FindFileByPath now returns an error if multiple files match
the path. Files.RangeFiles iterates over all files, including ones with
the same name.

Unlike descriptor names, file names are not part of any of the wire
encodings. Conflicts in file names only affect file lookups by path,
which is an uncommon operation.

The Files.RegisterFile documentation already states (inaccurately),
"It is permitted for multiple files to have the same file path."
This change brings the behavior of RegisterFile in line with the
documentation.

Fixes golang/protobuf#1122

Change-Id: I2750e35736fd34def23fa6ab6d6a829095a65a26
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/321531
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2021-05-25 00:53:49 +00:00
sryoya
0e358a402f reflect/protorange: fix typo of func name in comment
Change-Id: I80ffc34e29757e0cbdccdd530404fa215cfaf75f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/317989
Reviewed-by: Alberto Donizetti <alb.donizetti@gmail.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
2021-05-07 17:05:00 +00:00
Joe Tsai
e4fcb9f7e7 reflect: add protopath and protorange packages
The protopath package provides a means to programmatically represent
a sequence of protobuf reflection operations.
The protorange package traverses through a message and
calls a user-provided function as it iterates.

This feature sets the groundwork for the often requested feature
of being able to exclude certain fields when merging or serializing.

package protopath
    type Path []Step
    type Step struct{ ... }
        func Root(protoreflect.MessageDescriptor) Step
        func FieldAccess(protoreflect.FieldDescriptor) Step
        func UnknownAccess() Step
        func ListIndex(int) Step
        func MapIndex(protoreflect.MapKey) Step
        func AnyExpand(protoreflect.MessageDescriptor) Step
        func (Step) Kind() StepKind
        func (Step) FieldDescriptor() protoreflect.FieldDescriptor
        func (Step) MessageDescriptor() protoreflect.MessageDescriptor
        func (Step) ListIndex() int
        func (Step) MapIndex() protoreflect.MapKey
        func (Step) String() string
    type StepKind int
        const RootStep StepKind
        const FieldAccessStep StepKind
        const UnknownAccessStep StepKind
        const ListIndexStep StepKind
        const MapIndexStep StepKind
        const AnyExpandStep StepKind
    type Values struct {
        Path   Path
        Values []protoreflect.Value
    }
    func (Values) Index(int) (out struct{ ... })
    func (Values) Len() int
    func (Values) String() string

package protorange
    var Break error
    var Terminate error
    func Range(protoreflect.Message, func(protopath.Values) error) error
    type Options struct {
        Stable bool
        Resolver interface { ... }
    }
    func (Options) Range(m protoreflect.Message, push, pop func(protopath.Values) error) error

Change-Id: I29cbd5142fe169d78367d54a95d37801888b64f4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/236540
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
2021-04-01 23:09:19 +00:00
Joe Tsai
77dfd04893 reflect/protoregistry: add compile-time opt-out for registration conflicts
The next release of this module will panic on registration conflicts.
While we have been warning that this will become a fatal error for about
a year now, it's almost certainly the case that some users still have
ambiguous registrations.

In addition to the GOLANG_PROTOBUF_REGISTRATION_CONFLICT environment variable,
also define a linker-initialized variable that users can set.

Change-Id: I3325d841b9b6c692e2862100d81aebdcfda069fc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/295349
Trust: Joe Tsai <joetsai@digital-static.net>
Trust: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
2021-03-02 23:29:07 +00:00
Joe Tsai
aa45c46752 reflect/protoregistry: panic upon registration conflicts
In this CL, we switch the behavior of registration conflicts
from simply logging a warning to panicking.
This is a change that we have warned about ever since
the release of v1.20.0 in 2020-03-02.

For users that have not fixed their dependencies or are unable to,
they may set the GOLANG_PROTOBUF_REGISTRATION_CONFLICT environment
to "warn" in order to obtain the prior behavior of simply logging
and ignoring subsequent conflicts (first wins).

Change-Id: Ied7937f0fd5cb5d875858411714ebeb5ce145574
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/235298
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
2020-10-16 22:00:47 +00:00
Joe Tsai
9712361eda all: rely on dynamic dependency check for genproto
The genproto module incurs an unfortunate amount of dependency bloat
since it has a relatively large set of transitive dependencies.
Even though these are brought in as a weak dependency
(i.e., there is no hard code dependency on genproto),
it complicates dependency inspection since users need to vet
modules that they may not actually be using.

Avoid this weak dependency and instead rely on a dynamic check
to ensure that a sufficiently new version of genproto is used.
While it can't statically prevent old versions of genproto from being
linked into a binary, it guarantees that it will panic at init
with a helpful message on what's wrong.

Change-Id: I2b5a9759a5cd4c4c57aced54278502b1b6352ba7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/262679
Trust: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
2020-10-15 21:46:17 +00:00
Joe Tsai
687df2333b reflect/protodesc: fix round-tripping for package field
If a .proto file does not have a package statement,
then protoc will not populate the package field.
Emulate the same behavior for ToFileDescriptorProto.

Fixes golang/protobuf#1195

Change-Id: I617102e50c96f0fa1bac6a6bc4de1cc485750bb9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/259899
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Joe Tsai <thebrokentoaster@gmail.com>
2020-10-07 22:44:53 +00:00
Joe Tsai
42cc4c592f reflect/protoreflect: improve source information usability
Added API:
  SourceLocations.ByPath
  SourceLocations.ByDescriptor
  SourceLocation.Next
  SourcePath.String
  SourcePath.Equal

We modify compiler/protogen to use SourceLocations.ByDescriptor.
In retrospect, if this had existed during the development of protogen,
we would not have exposed protogen.Location and related fields.

Change-Id: I58f17e59f90b9ba16f0982c4b71c2542e4ff6e75
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/238000
Reviewed-by: Damien Neil <dneil@google.com>
2020-07-23 21:50:46 +00:00
Joe Tsai
1a290e9a0e reflect/protoreflect: add MessageFieldTypes
The MessageFieldTypes interface (if implemented by a MessageType)
provides Go type information about the fields if they are
an enum or message type.

Change-Id: I68b20f5726377f6b0f2c20a8b6e45f9802b43f67
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/236777
Reviewed-by: Damien Neil <dneil@google.com>
2020-07-23 21:24:58 +00:00
Joe Tsai
e14d6b3cdc reflect/protoreflect: add FieldDescriptor.TextName
Add a new TextName accessor that returns the field name that should
be used for the text format. It is usually just the field name, except:
1) it uses the inlined message name for groups,
2) uses the full name surrounded by brackets for extensions, and
3) strips the "message_set_extension" for well-formed extensions
to the proto1 MessageSet.

We make similar adjustments to the JSONName accessor so that it applies
similar semantics for extensions.

The two changes simplifies all logic that wants the humanly readable
name for a field.

Change-Id: I524b6e017fb955146db81819270fe197f8f97980
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/239838
Reviewed-by: Herbie Ong <herbie@google.com>
2020-07-08 23:23:57 +00:00
Martin Möhrmann
91eb7d3a63 protoregistry: avoid checking for '/' in FindMessageByName
Profiling shows FindMessageByName spending considerable time
in strings.LastIndexByte in FindMessageByURL.

This overhead can be avoided by giving FindMessageByName
its own implementation not pruning the name string up
to the last '/' which can not appear in a protobuf name.

Change-Id: Ifec07689f72790ead8b7552cd04dd99fa67b61e4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/240378
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2020-07-01 19:55:25 +00:00
Joe Tsai
b78321453d reflect/protoregistry: centralize MessageSet extension resolution logic
Centralize the MessageSet extension resolution logic in the registry.
This avoids needless replication of this exact logic in multiple places
(for JSON and text) and elsewhere.

Change-Id: I70bfea899e295e8c589f418965bf0dd099f93628
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/240077
Reviewed-by: Herbie Ong <herbie@google.com>
2020-07-01 17:56:02 +00:00
Joe Tsai
44e4150b30 reflect/protoreflect: optimize Name.IsValid and FullName.IsValid
For simplicity, IsValid was implemented in terms of regular expressions,
which are useful for verifying a string according to a strict grammar,
but performs poorly. Implement the check in terms of hand-written code,
which provides a 20x improvement to validate the name "google.protobuf.Any".

name               old time/op  new time/op  delta
FullNameIsValid-8   683ns ± 2%    35ns ± 1%  -94.86%  (p=0.000 n=10+10)

Change-Id: I980403befca0b72cea22acd274064a46cb02644b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/238002
Reviewed-by: Damien Neil <dneil@google.com>
2020-06-16 19:13:20 +00:00
Joe Tsai
e0b77db13b internal/genid: add new package for generated identifiers
The genid package unifies the genname, fieldnum, and detectknown
packages into a single package.

Whenever possible use the generated constants rather than
hard-coded literals. This makes it easier to search the entire
module for special logic that deal with well-known types.

Change-Id: I13beff1f4149444a0c0b9e607ebf759657f000f4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/235301
Reviewed-by: Herbie Ong <herbie@google.com>
2020-05-29 07:08:23 +00:00
Joe Tsai
27ea64ff2c reflect/protoreflect: adjust ValueOf panic message
A common mistake is to pass proto.Message to protoreflect.ValueOf
instead of a protoreflect.Message. Specialize for this case
and provide a panic message that is more helpful.

Change-Id: I5def43341aa9607182edde309dea2823e61c1fdb
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/234117
Reviewed-by: Damien Neil <dneil@google.com>
2020-05-26 22:22:26 +00:00
Joe Tsai
0567a55b47 all: add weak dependency on google.golang.org/genproto module
This is step 4 of 6 in a multi-stage migration
to move the well-known types
from the google.golang.org/genproto module
to the google.golang.org/protobuf module.

The generated Go packages for field_mask.proto, api.proto,
type.proto, and source_context.proto are being moved over
to this module alongside all the other well-known types.

In order to move these types between two modules,
there needs to be a sequence of changes submitted in
decently rapid succession. It is impossible to atomically
make these changes, so a brief breakage is inevitable.
The steps are as follows:

Step 1: Submit a change to cloud.google.com/go/internal/gapicgen
to avoid generating the well-known types. Otherwise, the tool
will undo the changes made in step 3.
See https://code-review.googlesource.com/c/gocloud/+/56810

Step 2: Submit a change to google.golang.org/protobuf that
adds the generated well-known types being migrated to that module.
In order to prevent the situation where a user links in
too old a version of the genproto module such that
duplicate registration occurs for the well-known types,
the registry is specially modified to provide an error
message that instructs users to upgrade the genproto module.
See https://golang.org/cl/234937

Step 3: Submit a change to google.golang.org/genproto that
switches all generated well-known types to be aliases to the
ones declared in google.golang.org/protobuf from the previous step.
This will cause the genproto module to incur an dependency
on an unreleased version of the protobuf module.
See https://github.com/googleapis/go-genproto/pull/372

Step 4: Submit a change to google.golang.org/protobuf that
adds a weak module depdency on the genproto module at the
revision from the previous step.
See https://golang.org/cl/235297

Step 5: Release google.golang.org/protobuf@v1.24.0.
See https://golang.org/cl/235299

Step 6: Submit a change to google.golang.org/genproto that
updates the protobuf module dependency to v1.24.0.

Change-Id: I22d496e7496446becad742eab05b169abb735ae2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/235297
Reviewed-by: Damien Neil <dneil@google.com>
2020-05-26 22:06:12 +00:00
Joe Tsai
81db48ad09 all: move well-known types
This is step 2 of 6 in a multi-stage migration
to move the well-known types
from the google.golang.org/genproto module
to the google.golang.org/protobuf module.

The generated Go packages for field_mask.proto, api.proto,
type.proto, and source_context.proto are being moved over
to this module alongside all the other well-known types.

In order to move these types between two modules,
there needs to be a sequence of changes submitted in
decently rapid succession. It is impossible to atomically
make these changes, so a brief breakage is inevitable.
The steps are as follows:

Step 1: Submit a change to cloud.google.com/go/internal/gapicgen
to avoid generating the well-known types. Otherwise, the tool
will undo the changes made in step 3.
See https://code-review.googlesource.com/c/gocloud/+/56810

Step 2: Submit a change to google.golang.org/protobuf that
adds the generated well-known types being migrated to that module.
In order to prevent the situation where a user links in
too old a version of the genproto module such that
duplicate registration occurs for the well-known types,
the registry is specially modified to provide an error
message that instructs users to upgrade the genproto module.
See https://golang.org/cl/234937

Step 3: Submit a change to google.golang.org/genproto that
switches all generated well-known types to be aliases to the
ones declared in google.golang.org/protobuf from the previous step.
This will cause the genproto module to incur an dependency
on an unreleased version of the protobuf module.
See https://github.com/googleapis/go-genproto/pull/372

Step 4: Submit a change to google.golang.org/protobuf that
adds a weak module depdency on the genproto module at the
revision from the previous step.

Step 5: Release google.golang.org/protobuf@v1.24.0.

Step 6: Submit a change to google.golang.org/genproto that
updates the protobuf module dependency to v1.24.0.

Change-Id: I36a19049d2240b67a37dfad20e154505aee7c784
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/234937
Reviewed-by: Damien Neil <dneil@google.com>
2020-05-26 19:51:55 +00:00
Joe Tsai
1f5b6fe645 all: improve panic messages for better debugability
Change-Id: If3e505e715d5ce2c9a81249c868d26226a25b724
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/232339
Reviewed-by: Damien Neil <dneil@google.com>
2020-05-05 21:10:54 +00:00
Joe Tsai
90042a5531 reflect/protoreflect: fix typo in FieldDescriptor.HasOptionalKeyword
Change-Id: I9b1dac5d10bbac4618bda174ef04135b81c81d7e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/231777
Reviewed-by: Damien Neil <dneil@google.com>
2020-05-04 18:59:39 +00:00
Joe Tsai
387873dd53 all: implement support for proto3 optional semantics
In the upcoming 3.12.x release of protoc, the proto3 language will be
amended to support true presence for scalars. This CL adds support
to both the generator and runtime to support these semantics.

Newly added public API:
	protogen.Plugin.SupportedFeatures
	protoreflect.FieldDescriptor.HasPresence
	protoreflect.FieldDescriptor.HasOptionalKeyword
	protoreflect.OneofDescriptor.IsSynthetic

Change-Id: I7c86bf66d0ae56642109beb5f2132184593747ad
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/230698
Reviewed-by: Damien Neil <dneil@google.com>
2020-04-29 20:02:24 +00:00
Joe Tsai
d824637fdf reflect/protoreflect: mention bidirectional conversion of protodesc
Expand the mention of the protodesc package to indicate that it can
construct protoreflect.Descriptors from the descriptor messages, but
also convert a protoreflect.Descriptor back to the message form.

Change-Id: I9f3d75c6050fff603655ee0a4e8e29d0a2c0cf84
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/221611
Reviewed-by: Damien Neil <dneil@google.com>
2020-04-21 00:35:05 +00:00
Joe Tsai
7650ca0905 reflect/protoreflect: cleanup wording on Message.NewField
Change-Id: Id2200e335fb5caf8e7e1574524081fed36aab268
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/227817
Reviewed-by: Damien Neil <dneil@google.com>
2020-04-21 00:34:41 +00:00
Damien Neil
188e702e78 reflect/protodesc: add NewFiles
Add a function that takes a protoreflect.FileDescriptorSet and returns
a protoregistry.Files.

Updates golang/protobuf##1065.

Change-Id: I2715d042053ef7d3f1bfcee1866c20cac423e327
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/226237
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2020-03-31 19:56:16 +00:00
Joe Tsai
cd108d00a8 encoding/protowire: make package publicly available
Change-Id: I95e293c208e787a91d50e29817620535dfeaa7f2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/219838
Reviewed-by: Damien Neil <dneil@google.com>
2020-03-20 07:42:18 +00:00
Damien Neil
13e7f145cc reflect/protoregistry: avoid deadlock in registration
Under some rare circumstances registration can deadlock
when lazy descriptor initialization consults the registry.

Move the call triggering the lazy init out of the critical section.

Fixes golang/protobuf#1052.

Change-Id: Ic266e06b0db99fea65e797b879ce53e5342fff95
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/204804
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2020-03-05 17:35:16 +00:00
Damien Neil
0f2b33b67a reflect/protoregistry: add link to namespace conflict FAQ
Updates golang/protobuf#1039.

Change-Id: Ia9f80da5dab1e6cd3cf9ad512b78b63b32065c31
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/221782
Reviewed-by: Herbie Ong <herbie@google.com>
2020-03-02 18:04:46 +00:00
Damien Neil
6e36304a32 reflect/protoreflect: ExtensionRangeOptions never returns nil
Documentation only change.

Change-Id: Ib620efc99bdaab90eeb5404554ae442f345d6f47
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/221560
Reviewed-by: Joe Tsai <joetsai@google.com>
2020-02-28 23:53:49 +00:00
Joe Tsai
f26a9e7e30 all: rename IsInitialized as CheckInitialized
An Is prefix implies it returns a boolean.
A Check prefix better suggests that it could return an error.

Change-Id: I6ffcb32099a944c656c07654c294a0980efb2d0e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/220338
Reviewed-by: Damien Neil <dneil@google.com>
2020-02-20 20:45:40 +00:00