Extensions should be checked based on ContainingMessage,
rather than the Parent. Add tests to ensure this works.
Change-Id: Iaf257f65197fb8d332039bc77a192753f8c4159f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/221426
Reviewed-by: Damien Neil <dneil@google.com>
For user convenience, automatically transform message values by
shallow copying them if necessary. Storing messages as values is
frowned upon, but is sometimes done by APIs that a user does not own.
Change-Id: I7e927d1a1e050bf4cea1aa889f56d23e99355f26
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/221423
Reviewed-by: Damien Neil <dneil@google.com>
By having the Message type implement proto.Message, it can be passed
to other general-purpose protobuf functions such as proto.Merge.
This provides a convenient way to convert the Message back into a
concrete form that may be easier to work with.
A minor consequence of this change is that invalid messages are
converted to an invalid Message that preserves type information.
Previously, they were simply transformed to a nil Message.
Change-Id: I6fca8a0879408c7f44a99d52734613302fa23f70
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/221422
Reviewed-by: Damien Neil <dneil@google.com>
This CL adds the following helper options:
func FilterEnum(protoreflect.Enum, cmp.Option) cmp.Option
func FilterMessage(proto.Message, cmp.Option) cmp.Option
func FilterField(proto.Message, protoreflect.Name, cmp.Option) cmp.Option
func FilterOneof(proto.Message, protoreflect.Name, cmp.Option) cmp.Option
func FilterDescriptor(protoreflect.Descriptor, cmp.Option) cmp.Option
There is primarily exposing pre-existing functionality that the Ignore options
were already depending on to operate.
Change-Id: I44edf2ffa07de980a9ad3284525bfe3b45428d74
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/207177
Reviewed-by: Damien Neil <dneil@google.com>
For consistency with other options types in the protobuf module, make
the test function a method of the options.
Drop the ExtensionTypes option and just look up the extension types to
test with in the provided resolver.
Change-Id: I7918bd10b7c003e4af56d27521d30218653d5b4d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/219142
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Add methods to add a new, mutable message to a list or map, matching the
existing Message.Mutable.
These methods are purely a convenience, as each can be implemented in
terms of the existing interface.
Change-Id: I889c20fe37ea0f2a566555212e99e6378fb9fe1d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/220117
Reviewed-by: Joe Tsai <joetsai@google.com>
TODOs that we do not intend to address have been deleted.
Those that are blocking v2 release are marked with "blocks".
Change-Id: I7efa9e546d0637b562101d0edc7009893d762722
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/218878
Reviewed-by: Damien Neil <dneil@google.com>
The proto package tests often test several variations of messages with a
similar shape. For example, most tests are performed with a proto2
message with a regular field, a proto2 message with an extension field,
and a proto3 message.
Add a protobuild package which can initialize all these variations from
a single template. For example, these three messages:
&testpb.TestAllTypes{OptionalInt32: proto.Int32(1)}
&test3pb.TestAllTypes{OptionalInt32: 1}
m := &testpb.TestAllExtensions{}
proto.SetExtension(m, &testpb.E_OptionalInt32, 1)
can all be constructed from the template:
protobuild.Message{"optional_int32": 1}
This reduces redundancy in tests and will make it more practical to
test alternative code generators.
Change-Id: I3245a4bf74ee1bce957bc772fed513d427720677
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/217457
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The Format function and MarshalOptions.Format method are helper
functions for directly obtaining the formatted string for a message
without having to deal with errors or convert a []byte to string.
It is only intended for human consumption (e.g., debugging or logging).
We also add a MarshalOptions.Multiline option to specify that the output
should use some default indentation in a multiline output.
This assists in the v1 to v2 migration where:
protoV1.CompactTextString(m) => prototext.MarshalOptions{}.Format(m)
protoV1.MarshalTextString(m) => prototext.Format(m)
At Google, there are approximately 10x more usages of MarshalTextString than
CompactTextString, so it makes sense that the top-level Format function
does multiline expansion by default.
Fixes#850
Change-Id: I149c9e190a6d99b985d3884df675499a3313e9b3
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213460
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Herbie Ong <herbie@google.com>
Tweak the test message to allow creating messages with extensions that
contain extensions that contain extensions, etc.
Change-Id: I41844ae699c88ab96bf0d30db3a3fbaf09616161
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/216761
Reviewed-by: Joe Tsai <joetsai@google.com>
Avoid dots and dashes in the directory to avoid issues on
build systems that cannot support them well.
Change-Id: I7ea5e6ce0b16c7158c7e53bcf5c3c1a334fe4718
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/214342
Reviewed-by: Damien Neil <dneil@google.com>
We resisted adding Clone for a while since:
* It is a function that is perfectly suited for generics.
However, generics probably still won't be available in Go for some time
and it is impractical to block addition of this function when it is very
widely used and will be necessary for the v1 to v2 migration.
* In the past, there was no protoreflect.Message.IsValid, so there was
no proper API to detect invalid top-level messages and return them as such.
Since Clone relies on certain properties about proper round-tripping
of ProtoMessage.ProtoReflect <-> Message.Interface, we add a test
in testing/prototest to check for this.
Change-Id: Ic492b68f27b8b88322a6a3fa3a5e492228db79d9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213297
Reviewed-by: Damien Neil <dneil@google.com>
The v1 proto.Equal function treats (*Message)(nil) and new(Message)
as being different, while v2 proto.Equal treated them as equal since
a typed nil pointer is functionally an empty message since the
protobuf data model has no concept of presence as a first-class
property of messages.
Unfortunately, a significant amount of code depends on this distinction
that it would be difficult to migrate users from v1 to v2 unless we
preserved similar semantics in the v2 proto.Equal.
Also, double down on these semantics for protocmp.Transform.
Fixes#965
Change-Id: I21e78ba6251401a0ac0ccf495188093973cd7f3f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213238
Reviewed-by: Damien Neil <dneil@google.com>
The protoreflect documentation states that Get on an empty repeated or
map field returns an invalid (empty, read-only) List or Map. We weren't
doing this; fix it.
The documentation also states that an invalid List or Map may not be
assigned via Set. We were permitting Set with an invalid map; fix this.
Add tests for this behavior in prototest.
Change-Id: I4678af532e192210af0bde7c96a1439a4cd26efa
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/209019
Reviewed-by: Joe Tsai <joetsai@google.com>
Various protoreflect methods can return an "empty, read-only" message,
list, or map value. Provide a method to test if a value is one of these.
Fixesgolang/protobuf#966
Change-Id: I793d8426d6e2201755983c06f024412a7e09bc4c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/209018
Reviewed-by: Joe Tsai <joetsai@google.com>
This CL adds the following helper options:
func IgnoreEnums(...protoreflect.Enum) cmp.Option
func IgnoreMessages(...proto.Message) cmp.Option
func IgnoreFields(proto.Message, ...protoreflect.Name) cmp.Option
func IgnoreOneofs(proto.Message, ...protoreflect.Name) cmp.Option
func IgnoreDescriptors(...protoreflect.Descriptor) cmp.Option
func IgnoreDefaultScalars() cmp.Option
func IgnoreEmptyMessages() cmp.Option
func IgnoreUnknown() cmp.Option
It also augments transformMessage to unmarshal and expand Any messages
with the value of the underlying message. At this moment in time
we do not provide an API to provide a custom type resolver.
Change-Id: I51e1d9ff0d56d71161e510f366a7dcc32236d760
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/204577
Reviewed-by: Damien Neil <dneil@google.com>
Accessing Number as a method, rather than a field paves the
way to have Enum potentially implement protoreflect.Enum
in the future.
Change-Id: Iebe9c0ec12e067decf2121d12fe2fb1549477b32
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/207077
Reviewed-by: Damien Neil <dneil@google.com>
Remove repeated extension fields from the set of nullable fields,
so that Has reports false and Range does not visit a a zero-length
repeated extension field.
This corrects a fuzzer-detected case where unmarshaling and remarshaling
a wire-format message could result in a semantic change. For a repeated
extension field in non-packed encoding, unmarshaling a packed
representation of the field would result in a message which Has the
extension. Remarshaling it would discard the the field.
Fixesgolang.org/protobuf#975
Change-Id: Ie836559c93d218db5b5201742a3b8ebbaacf54ed
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/204897
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Joe Tsai <joetsai@google.com>
High-level API:
func Transform() cmp.Option
type Enum struct{ ... }
type Message map[string]interface{}
The Transform function transform messages into a Message type that
cmp.Equal and cmp.Diff then knows how to traverse and compare.
Change-Id: I445f3b5c69f054b6984f28c205cda69e44af3b89
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/164680
Reviewed-by: Damien Neil <dneil@google.com>
Add a dynamicpb.NewExtensionType function to permit creating extension
types from descriptors.
Also fix a some bugs around extension field handling:
When creating a new value for an extension field, use the
ExtensionType's Zero or New method to create the value.
Ensure that prototest exercises true zero-values of fields. (i.e.,
getting a list, map, or message from an empty message rather than
creating a new empty one with NewField.)
Change-Id: Idb8e87cdc92692610e12a4b8a68c34b129fae617
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186180
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Descriptor methods generally return a Descriptor with no Go type
information. ExtensionType's Descriptor is an exception, returning an
ExtensionTypeDescriptor containing both the proto descriptor and a
reference back to the ExtensionType. The pure descriptor is accessed
by xt.Descriptor().Descriptor().
Rename ExtensionType's Descriptor method to TypeDescriptor to make it
clear that it behaves a bit differently.
Change 1/2: Add the TypeDescriptor method and deprecate Descriptor.
Change-Id: I1806095044d35a474d60f94d2a28bdf528f12238
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/192139
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Rename build tag "proto1_legacy" -> "protolegacy"
to be consistent with the "protoreflect" tag.
Rename flag constant "Proto1Legacy" -> "ProtoLegacy" since
it covers more than simply proto1 legacy features.
For example, it covers alpha-features of proto3 that
were eventually removed from the final proto3 release.
Change-Id: I0f4fcbadd4b5a61c87645e2e5be11d187e59157c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189345
Reviewed-by: Damien Neil <dneil@google.com>
Change protoiface.ExtensionDescV1 to implement protoreflect.ExtensionType.
ExtensionDescV1's Name field conflicts with the Descriptor Name method,
so change the protoreflect.{Message,Enum,Extension}Type types to no
longer implement the corresponding Descriptor interface. This also leads
to a clearer distinction between the two types.
Introduce a protoreflect.ExtensionTypeDescriptor type which bridges
between ExtensionType and ExtensionDescriptor.
Add extension accessor functions to the proto package:
proto.{Has,Clear,Get,Set}Extension. These functions take a
protoreflect.ExtensionType parameter, which allows writing the
same function call using either the old or new API:
proto.GetExtension(message, somepb.E_ExtensionFoo)
Fixesgolang/protobuf#908
Change-Id: Ibc65d12a46666297849114fd3aefbc4a597d9f08
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189199
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Add methods to protoreflect.{Message,List,Map} to constrict values
assignable to a message field, list element, or map value. These
methods return the default value for scalar fields, the zero value for
scalar list elements and map values, and an empty, mutable value for
messages, lists, and maps.
Deprecate the NewMessage methods on these types, which are superseded.
Updates golang/protobuf#879
Change-Id: I0f064f60c89a239330ccea81523f559f14fd2c4f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/188997
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This implements generation of and reflection support for weak fields.
Weak fields are a proto1 feature where the "weak" option can be specified
on a singular message field. A weak reference results in generated code
that does not directly link in the dependency containing the weak message.
Weak field support is not added to any of the serialization logic.
Change-Id: I08ccfa72bc80b2ffb6af527a1677a0a81dcf33fb
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185399
Reviewed-by: Damien Neil <dneil@google.com>
The dynamicpb package permits creating Message values from a
MessageDescriptor.
Change-Id: Ice429ae45a0835dffb5a7ec8c0bd2c1df7aac8a2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/174960
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Added API:
Message.Len
Message.Range
Message.Has
Message.Clear
Message.Get
Message.Set
Message.Mutable
Message.NewMessage
Message.WhichOneof
Message.GetUnknown
Message.SetUnknown
Deprecated API (to be removed in subsequent CL):
Message.KnownFields
Message.UnknownFields
The primary difference with the new API is that the top-level
Message methods are keyed by FieldDescriptor rather than FieldNumber
with the following semantics:
* For known fields, the FieldDescriptor must exactly match the
field descriptor known by the message.
* For extension fields, the FieldDescriptor must implement ExtensionType,
where ContainingMessage.FullName matches the message name, and
the field number is within the message's extension range.
When setting an extension field, it automatically stores
the extension type information.
* Extension fields are always considered nullable,
implying that repeated extension fields are nullable.
That is, you can distinguish between a unpopulated list and an empty list.
* Message.Get always returns a valid Value even if unpopulated.
The behavior is already well-defined for scalars, but for unpopulated
composite types, it now returns an empty read-only version of it.
Change-Id: Ia120630b4db221aeaaf743d0f64160e1a61a0f61
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/175458
Reviewed-by: Damien Neil <dneil@google.com>
When encountering a type that does not have a MessageInfo, don't assume
that it's a legacy message that doesn't implement proto.Message. Add a
set of test messages exercising this case (panics prior to the
internal/impl change).
Change-Id: Ic1ec5ecfbe92278fbef44284ff52a0e0622a158c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/182477
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The prior code was comparing a to a, when it should compare a to b.
While we're at it, split the float32 and float64 cases apart to
ensure that the type provided by the implementation under test
really was the right float type. This matches the behavior of the
default case (which check that the types exactly match).
Change-Id: I01b1ff5b7f312a35cc8d1f012f3f747db26379fd
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/178377
Reviewed-by: Damien Neil <dneil@google.com>
Rename encoding/*pb to follow the convention of prefixing package names
with 'proto':
google.golang.org/protobuf/encoding/protojson
google.golang.org/protobuf/encoding/prototext
Move protogen under a compiler/ directory, just in case we ever do add
more compiler-related packages.
google.golang.org/protobuf/compiler/protogen
Change-Id: I31010cb5cabcea8274fffcac468477b58b56e8eb
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/177178
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Temporarily remove go.mod, since we can't generate an accurate one until
the corresponding v1 change is submitted.
Change-Id: I1e1ad97f2b455e33f61ffaeb8676289795e47e72
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/177000
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Added API:
FieldDescriptor.IsExtension
FieldDescriptor.IsList
FieldDescriptor.MapKey
FieldDescriptor.MapValue
FieldDescriptor.ContainingOneof
FieldDescriptor.ContainingMessage
Deprecated API (to be removed in subsequent CL):
FieldDescriptor.Oneof
FieldDescriptor.Extendee
These methods help cleanup several common usage patterns.
Change-Id: I9a3ffabc2edb2173c536509b22f330f98bba7cf3
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/176977
Reviewed-by: Damien Neil <dneil@google.com>
Added methods:
Enum.Descriptor
Message.Descriptor
EnumType.Descriptor
MessageType.Descriptor
ExtensionType.Descriptor
Message.New
All functionality is switched over to use those methods instead of
implicitly relying on the fact that {Enum,Message}Type implicitly
implement the associated descriptor interface.
This CL does not yet remove {Enum,Message}.Type or prevent
{Enum,Message,Extension}Type from implementating a descriptor.
That is a subsequent CL.
The Message.New method is also added to replace functionality
that will be lost when the Type methods are removed.
Change-Id: I7fefde1673bbd40bfdac489aca05cec9a6c98eb1
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/174918
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Herbie Ong <herbie@google.com>
Weak fields are an obsolete proto1 feature. They have been superseded
by extensions. However, some vestigial support for weak fields does
remain, mostly as Google-internal patches. (They aren't exciting;
extensions really do everything weak fields do in a cleaner and
more portable fashion.)
At the moment, the only visible impact of marking a field [weak=true]
is to exclude it from "internal/fileinit".FileBuilder.DependencyIndexes.
We want to preserve that behavior just in case we ever do add full weak
field support here.
Extend fileinit to look up message descriptors for weak fields in the
global registry. If the descriptor cannot be found, use a placeholder
instead.
Remove special-case handling of weak fields in the impl package. The
code generator doesn't do anything special for them, so they can be
treated as any other field.
Change-Id: Ifa2ee3d30d63680a0eeb59c66ebc9521f38fd660
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/175997
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The prototest package takes a message and exercises its implementation
of the protoreflect.Message interface in a variety of ways.
Change-Id: I4c150d1f1a6c41048f7e7f0d77efa4440ae8db0d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/174577
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
