Implement support in the protobuf runtime to better understand
message types that are not generated by the official generator.
In particular:
* Add a best-effort implementation of protobuf reflection for
"non-nullable" fields which are supposed to be represented by *T,
but are instead represented by a T. "Non-nullable" message fields
report presence based on whether the message is the zero Go value.
* We do NOT implement support for "non-nullable" fields in the
table-driven implementation since we assume that the aberrant messages
that we care about have a Marshal and Unmarshal method.
* We better handle custom messages that implement Marshal and Unmarshal,
but do NOT implement Merge. In that case, we implement merge in terms of
a back-to-back marshal and unmarshal.
* We better tolerate the situations where a protobuf message field
cannot be mapped to a Go struct field since the latter is missing.
In such cases, reflection treats the field as if it were unpopulated.
Setting such fields will panic.
This change allows the runtime to handle all message types declared
in the "go.etcd.io/etcd" and "k8s.io" modules where protobuf reflection,
Marshal, Unmarshal, Reset, Merge, and Equal all work.
The only types that still do not fully work are:
* "k8s.io/api/authentication/v1".ExtraValue
* "k8s.io/api/authentication/v1beta1".ExtraValue
* "k8s.io/api/authorization/v1".ExtraValue
* "k8s.io/api/authorization/v1beta1".ExtraValue
* "k8s.io/api/certificates/v1".ExtraValue
* "k8s.io/api/certificates/v1beta1".ExtraValue
* "k8s.io/apimachinery/pkg/apis/meta/v1".MicroTime
* "k8s.io/apimachinery/pkg/apis/meta/v1".Time
* "k8s.io/apimachinery/pkg/apis/meta/v1".Verbs
While Marshal, Unmarshal, Reset, and Merge continue to work,
protobuf reflection and any functionality that depends on it
(e.g., prototext, protojson, Equal, etc.) will not work.
Change-Id: I67a9d2f1bec35248045ad0c16220d02fc2e0e172
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/300869
Trust: Joe Tsai <joetsai@digital-static.net>
Trust: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
In the upcoming 3.12.x release of protoc, the proto3 language will be
amended to support true presence for scalars. This CL adds support
to both the generator and runtime to support these semantics.
Newly added public API:
protogen.Plugin.SupportedFeatures
protoreflect.FieldDescriptor.HasPresence
protoreflect.FieldDescriptor.HasOptionalKeyword
protoreflect.OneofDescriptor.IsSynthetic
Change-Id: I7c86bf66d0ae56642109beb5f2132184593747ad
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/230698
Reviewed-by: Damien Neil <dneil@google.com>
The protoreflect documentation states that Get on an empty repeated or
map field returns an invalid (empty, read-only) List or Map. We weren't
doing this; fix it.
The documentation also states that an invalid List or Map may not be
assigned via Set. We were permitting Set with an invalid map; fix this.
Add tests for this behavior in prototest.
Change-Id: I4678af532e192210af0bde7c96a1439a4cd26efa
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/209019
Reviewed-by: Joe Tsai <joetsai@google.com>
Fix a reversed error check in impl.Export{}.WeakNil.
Check to see if we have a type for the weak field on marshal/size.
Treat a typed nil valued in XXX_Weak as not indicating presence for
the field.
Change-Id: Id667ac7eb4f53236be9e181017082bd8cd21d115
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/198717
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This change performs two optimizations:
* It uses a pre-constructed rangeInfos slice to iterate over
all the fields. This is more performant since iterating over a slice
is faster than iterating over a map. Furthermore, this slice
does not contain fields that are part of a oneof. If a oneof has
N fields, the time to check presence on the oneof is now O(1)
instead of O(N).
* It uses a dense field info slice that is optmized for the common
case where the field number is relatively low and close in value
to the index itself.
We also fix a minor bug in the construction of oneofInfo where
it wasn't treating a typed nil pointer to a wrapper struct as if
it were unset. This ensures WhichOneof and Has always agree.
name old time/op new time/op delta
Reflect/Has-4 7.81µs ± 3% 6.74µs ± 3% -13.61% (p=0.000 n=9+9)
Reflect/Get-4 12.7µs ± 1% 11.3µs ± 4% -10.85% (p=0.000 n=8+10)
Reflect/Set-4 19.5µs ± 5% 17.8µs ± 2% -8.99% (p=0.000 n=10+10)
Reflect/Clear-4 12.0µs ± 4% 10.2µs ± 3% -14.86% (p=0.000 n=9+10)
Reflect/Range-4 6.58µs ± 1% 4.17µs ± 2% -36.65% (p=0.000 n=8+9)
Change-Id: I2c48b4d3fb6103ab238924950529ded0d37f8c8a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/196358
Reviewed-by: Damien Neil <dneil@google.com>
This interface is only ever used within the impl package.
Unexport it to avoid potential abuse by users.
Change-Id: I58d0cc2e8cb01c8c9d36cdec469c8d2196c4f836
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/193197
Reviewed-by: Damien Neil <dneil@google.com>
The old implementation had the behavior where a nil wrapper value:
m := new(foopb.Message)
m.OneofField = (*foopb.Message_OneofUint32)(nil)
was functionally equivalent to it being directly set to nil:
m := new(foopb.Message)
m.OneofField = nil
preserve this semantic in both the table-drive implementation
and the reflection implementation.
Change-Id: Ie44d51e044d4822e61d0e646fbc44aa8d9b90c1f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189559
Reviewed-by: Herbie Ong <herbie@google.com>
Changes made:
* Move MessageInfo.{methods,extensionFieldInfosMu,extensionFieldInfos}
to the coderMessageInfo type since they are fields all related to the
fast-path implementation, which is what the coderMessageInfo is for.
* Rename message_field.go -> message_reflect_field.go to make it obvious
from the file name that this only deals with the reflection implementation.
* Rename message_test.go -> message_reflect_test.go.
* Move reflection-specific implementation functions from message.go
to message_reflect.go. The intention is to make it such that message.go
is the entry point to message implementations and is agnostic towards
whether we are implementing reflection or the table-driven fast path.
Overall, there is no semantic changes. Just code movement.
Change-Id: I7743c39ba84dc63cd2a02934c3319285e16d6b1c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/190100
Reviewed-by: Herbie Ong <herbie@google.com>
