Implement support for extension fields for messages that use the v1
data structures for extensions. The legacyExtensionFields type wraps a
v1 map to implement the v2 protoreflect.KnownFields interface.
Working on this change revealed a bug in the dynamic construction of
message types for protobuf messages that had cyclic dependencies (e.g.,
message Foo has a sub-field of message Bar, and Bar has a sub-field of Foo).
In such a situation, a deadlock occurs because initialization code depends on
the very initialization code that is currently running. To break these cycles,
we make some systematic changes listed in the following paragraphs.
Generally speaking, we separate the logic for construction and wrapping,
where constuction does not recursively rely on dependencies,
while wrapping may recursively inspect dependencies.
Promote the MessageType.MessageOf method as a standalone MessageOf function
that dynamically finds the proper *MessageType to use. We make it such that
MessageType only supports two forms of messages types:
* Those that fully implement the v2 API.
* Those that do not implement the v2 API at all.
This removes support for the hybrid form that was exploited by message_test.go
In impl/message_test.go, switch each message to look more like how future
generated messages will look like. This is done in reaction to the fact that
MessageType.MessageOf no longer exists.
In value/{map,vector}.go, fix Unwrap to return a pointer since the underlying
reflect.Value is addressable reference value, not a pointer value.
In value/convert.go, split the logic apart so that obtaining a v2 type and
wrapping a type as v2 are distinct operations. Wrapping requires further
initialization than simply creating the initial message type, and calling it
during initial construction would lead to a deadlock.
In protoreflect/go_type.go, we switch back to a lazy initialization of GoType
to avoid a deadlock since the user-provided fn may rely on the fact that
prototype.GoMessage returned.
Change-Id: I5dea00e36fe1a9899bd2ac0aed2c8e51d5d87420
Reviewed-on: https://go-review.googlesource.com/c/148826
Reviewed-by: Herbie Ong <herbie@google.com>
Rather than having the Converter carry a NewMessage method, have the struct
simply expose the MessageType or EnumType since they carry more information
and are retrieved anyways as part of the functionality of NewConverter.
While changing Converter, export the fields and remove all the methods.
Also, add an IsLegacy boolean, which is useful for the later implementation
of the extension fields.
Add a wrapLegacyEnum function which is used to wrap v1 enums as v2 enums.
We use this functionality in NewLegacyConverter to detrive the EnumType.
Additionally, modify wrapLegacyMessage to return a protoreflect.ProtoMessage
to be consistent with wrapLegacyEnum which must return a protoreflect.ProtoEnum.
Change-Id: Idc8989d07e4895d30de4ebc22c9ffa7357815cad
Reviewed-on: https://go-review.googlesource.com/c/148827
Reviewed-by: Herbie Ong <herbie@google.com>
The Go type descriptors protoreflect.{Enum,Message,Extension}Type are simple
wrappers over protoreflect.{Enum,Message,Extension}Descriptor with a small
number of additional methods. It is very unlikely that more will be added in
the near future.
For this reason, construct the types directly using arguments to the constructor
function, as opposed to taking in another struct (which was originally done
to provide flexibility in-case we needed more fields).
Furthmore, rename GoNew and New.
Change-Id: Ic7fb5bc250cdb2761ae03b388b5147ff50f37d15
Reviewed-on: https://go-review.googlesource.com/c/148822
Reviewed-by: Herbie Ong <herbie@google.com>
Follow the precedence of Go maps where deletion on a key without an entry in
the map is a noop. Similarly, document that the following methods are safe
to call with entries that do not exist:
* Map.Clear
* KnownFields.Clear
* ExtensionFieldTypes.Remove
Change the implementation for each of these to match the documented behavior.
Change-Id: Ifccff9b7b03baaeffdc366d05f6286ba60e14934
Reviewed-on: https://go-review.googlesource.com/c/148317
Reviewed-by: Herbie Ong <herbie@google.com>
The unknown fields in legacy messages is split across the XXX_unrecognized
field and also the XXX_InternalExtensions field. Implement support for
wrapping both fields and presenting it as if it were a unified set of
unknown fields.
Change-Id: If274fae2b48962520edd8a640080b6eced747684
Reviewed-on: https://go-review.googlesource.com/c/146517
Reviewed-by: Damien Neil <dneil@google.com>
Add wrapper data structures to get legacy XXX_unrecognized fields to support
the new protoreflect.UnknownFields interface. This is a challenge since the
field is a []byte, which does not give us much flexibility to work with
in terms of choice of data structures.
This implementation is relatively naive where every operation is O(n) since
it needs to strip through the entire []byte each time. The Range operation
operates slightly differently from ranging over Go maps since it presents a
stale version of RawFields should a mutation occur while ranging.
This distinction is unlikely to affect anyone in practice.
Change-Id: Ib3247cb827f9a0dd6c2192cd59830dca5eef8257
Reviewed-on: https://go-review.googlesource.com/c/144697
Reviewed-by: Damien Neil <dneil@google.com>
Setup scaffolding for implementing unknown and extension fields.
Add functions to MessageType to produce a protoreflect.KnownFields or
protoreflect.UnknownFields from a message pointer.
Within the implementation of known fields, delegate the logic to the underlying
extension fields (which also implements protoreflect.KnownFields) if the field
number is not found in the set of defined fields.
Change-Id: I2c35f4cdf1c7b58727ce6a582861ef18b8d69a61
Reviewed-on: https://go-review.googlesource.com/c/144280
Reviewed-by: Damien Neil <dneil@google.com>
In order for the v2 rollout to be as seamless as possible, we need to support
the situation where a v2 message depends on some other generated v1 message that
may be stale and does not support the v2 API. In such a situation, there needs
to be some way to wrap a legacy message or enum in such a way that it satisfies
the v2 API.
This wrapping is comprised of two parts:
1) Deriving an enum or message descriptor
2) Providing an reflection implementation for messages
This CL addresses part 1 (while part 2 has already been partially implemented,
since the implementation applies to both v1 and v2).
To derive the enum and message descriptor we rely on a mixture of parsing the
raw descriptor proto and also introspection on the fields in the message.
Methods for obtaining the raw descriptor protos were added in February, 2016,
and so has not always been available. For that reason, we attempt to derive
as much information from the Go type as possible.
As part of this change, we modify prototype to be able to create multiple
standalone messages as a set. This is needed since cyclic dependencies is allowed
between messages within a single proto file.
Change-Id: I71aaf5f977faf9fba03c370b1ee17b3758ce60a6
Reviewed-on: https://go-review.googlesource.com/c/143539
Reviewed-by: Damien Neil <dneil@google.com>
Dynamically generate functions for handling individual fields within an oneof.
This implementation uses Go reflection to interact with the currently generated
approach, which uses an interface that can only be set by a limited set of
wrapper structs.
Change-Id: Ic848df922d6547411a15c4a20bfbbcae362da5c0
Reviewed-on: https://go-review.googlesource.com/c/142895
Reviewed-by: Damien Neil <dneil@google.com>
Remove List from KnownFields, UnknownFields, ExtensionFieldTypes, and Map.
Rationale:
* Each of those interfaces already have a Range method, which provides
a superset of the functionality of List. Furthermore, Range is more expressive
as it allows you to terminate iteration and provides both keys and values.
* List must always allocate a slice and populates it.
* Range is allocation free in some cases. For example, if you simply wanted to
iterate over all the populated fields to clear them, there is no need for a
closure, so a static version of the function can be directly referenced
(i.e., there is no need to create a stub function header that references the
closed-over variables).
* In the cases where a closure is needed, the allocation cost is O(1) since
there are a finite number of variables being closed over.
* In highly performance sensitive cases, the closured function could close over
a struct, such that the function and struct are stored in a sync.Pool when not
in use. For example:
type MapLister struct {
Entries []struct{K MapKey; V Value}
f func(MapKey, Value) true
}
func (m *MapLister) Ranger() func(MapKey, Value) bool {
if m.f != nil {
m.f = func(k MapKey, v Value) bool {
m.Entries = append(m.Entries, ...)
return true
}
}
m.Entries = m.Entries[:0]
return m.f
}
The main benefit of List is the ease of use:
for _, num := range knownFields.List() {
...
}
as opposed to:
knownFields.Range(func(n FieldNumber, v Value) bool {
...
return true
})
However, this is a marginal benefit.
Thus, remove List as it mostly inferior to Range.
Change-Id: I25586c6ea07a4706072ba06b1cf25cb6efb5e8a7
Reviewed-on: https://go-review.googlesource.com/c/142888
Reviewed-by: Damien Neil <dneil@google.com>
Generate functions for wrapping []T to implement protoreflect.Vector.
This implementation uses Go reflection instead to provide a single implementation
that can handle all Go slice types.
The test harness was greatly expanded to be able to test vectors (in addition
to messages and maps in the near future).
Change-Id: I0106c175f84a1e7e0a0a5b0e02e2489b70b0d177
Reviewed-on: https://go-review.googlesource.com/c/135339
Reviewed-by: Damien Neil <dneil@google.com>
Given a pointer to a Go struct (that is well-formed according to the v1
struct field layout), wrap the type such that it implements the v2
protoreflect.Message interface.
Change-Id: I5987cad0d22e53970c613cdbbb1cfd4210897f69
Reviewed-on: https://go-review.googlesource.com/c/138897
Reviewed-by: Damien Neil <dneil@google.com>
This change was created by running:
git ls-files | xargs sed -i "s|google.golang.org/proto|github.com/golang/protobuf/v2|g"
This change is *not* an endorsement of "github.com/golang/protobuf/v2" as the
final import path when the v2 API is eventually released as stable.
We continue to reserve the right to make breaking changes as we see fit.
This change enables us to host the v2 API on a repository that is go-gettable
(since go.googlesource.com is not a known host by the "go get" tool;
and google.golang.org/proto was just a stub URL that is not currently served).
Thus, we can start work on a forked version of the v1 API that explores
what it would take to implement v1 in terms of v2 in a backwards compatible way.
Change-Id: Ia3ebc41ac4238af62ee140200d3158b53ac9ec48
Reviewed-on: https://go-review.googlesource.com/136736
Reviewed-by: Damien Neil <dneil@google.com>
This provides an implementation of the has, get, set, clear methods for each
field in a message. The approach taken here is similar to the table-driven
implementation in the current v1 proto package.
The pointer_reflect.go and pointer_unsafe.go files are a simplified version of
the same files in the v1 implementation. They provide a pointer abstraction
that enables a high-efficiency approach in a non-purego environment.
The unsafe fast-path is not implemented in this commit.
This commit only implements the accessor methods for scalars using pure
Go reflection.
Change-Id: Icdf707e9d4e3385e55434f93b30a341a7680ae11
Reviewed-on: https://go-review.googlesource.com/135136
Reviewed-by: Damien Neil <dneil@google.com>
