2018-10-31 18:23:42 -07:00
|
|
|
// Copyright 2018 The Go Authors. All rights reserved.
|
|
|
|
// Use of this source code is governed by a BSD-style
|
|
|
|
// license that can be found in the LICENSE file.
|
|
|
|
|
|
|
|
package impl
|
|
|
|
|
|
|
|
import (
|
|
|
|
"reflect"
|
|
|
|
"sync"
|
|
|
|
"unsafe"
|
|
|
|
|
|
|
|
protoV1 "github.com/golang/protobuf/proto"
|
|
|
|
pref "github.com/golang/protobuf/v2/reflect/protoreflect"
|
|
|
|
)
|
|
|
|
|
internal/impl: support legacy extension fields
Implement support for extension fields for messages that use the v1
data structures for extensions. The legacyExtensionFields type wraps a
v1 map to implement the v2 protoreflect.KnownFields interface.
Working on this change revealed a bug in the dynamic construction of
message types for protobuf messages that had cyclic dependencies (e.g.,
message Foo has a sub-field of message Bar, and Bar has a sub-field of Foo).
In such a situation, a deadlock occurs because initialization code depends on
the very initialization code that is currently running. To break these cycles,
we make some systematic changes listed in the following paragraphs.
Generally speaking, we separate the logic for construction and wrapping,
where constuction does not recursively rely on dependencies,
while wrapping may recursively inspect dependencies.
Promote the MessageType.MessageOf method as a standalone MessageOf function
that dynamically finds the proper *MessageType to use. We make it such that
MessageType only supports two forms of messages types:
* Those that fully implement the v2 API.
* Those that do not implement the v2 API at all.
This removes support for the hybrid form that was exploited by message_test.go
In impl/message_test.go, switch each message to look more like how future
generated messages will look like. This is done in reaction to the fact that
MessageType.MessageOf no longer exists.
In value/{map,vector}.go, fix Unwrap to return a pointer since the underlying
reflect.Value is addressable reference value, not a pointer value.
In value/convert.go, split the logic apart so that obtaining a v2 type and
wrapping a type as v2 are distinct operations. Wrapping requires further
initialization than simply creating the initial message type, and calling it
during initial construction would lead to a deadlock.
In protoreflect/go_type.go, we switch back to a lazy initialization of GoType
to avoid a deadlock since the user-provided fn may rely on the fact that
prototype.GoMessage returned.
Change-Id: I5dea00e36fe1a9899bd2ac0aed2c8e51d5d87420
Reviewed-on: https://go-review.googlesource.com/c/148826
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-06 13:05:20 -08:00
|
|
|
// TODO: The logic in the file is a hack and should be in the v1 repository.
|
|
|
|
// We need to break the dependency on proto v1 since it is v1 that will
|
|
|
|
// eventually need to depend on v2.
|
|
|
|
|
|
|
|
// TODO: The v1 API currently exposes no exported functionality for interacting
|
|
|
|
// with the extension data structures. We will need to make changes in v1 so
|
|
|
|
// that v2 can access these data structures without relying on unsafe.
|
2018-10-31 18:23:42 -07:00
|
|
|
|
|
|
|
var (
|
|
|
|
extTypeA = reflect.TypeOf(map[int32]protoV1.Extension(nil))
|
|
|
|
extTypeB = reflect.TypeOf(protoV1.XXX_InternalExtensions{})
|
|
|
|
)
|
|
|
|
|
|
|
|
type legacyExtensionIface interface {
|
|
|
|
Len() int
|
internal/impl: support legacy extension fields
Implement support for extension fields for messages that use the v1
data structures for extensions. The legacyExtensionFields type wraps a
v1 map to implement the v2 protoreflect.KnownFields interface.
Working on this change revealed a bug in the dynamic construction of
message types for protobuf messages that had cyclic dependencies (e.g.,
message Foo has a sub-field of message Bar, and Bar has a sub-field of Foo).
In such a situation, a deadlock occurs because initialization code depends on
the very initialization code that is currently running. To break these cycles,
we make some systematic changes listed in the following paragraphs.
Generally speaking, we separate the logic for construction and wrapping,
where constuction does not recursively rely on dependencies,
while wrapping may recursively inspect dependencies.
Promote the MessageType.MessageOf method as a standalone MessageOf function
that dynamically finds the proper *MessageType to use. We make it such that
MessageType only supports two forms of messages types:
* Those that fully implement the v2 API.
* Those that do not implement the v2 API at all.
This removes support for the hybrid form that was exploited by message_test.go
In impl/message_test.go, switch each message to look more like how future
generated messages will look like. This is done in reaction to the fact that
MessageType.MessageOf no longer exists.
In value/{map,vector}.go, fix Unwrap to return a pointer since the underlying
reflect.Value is addressable reference value, not a pointer value.
In value/convert.go, split the logic apart so that obtaining a v2 type and
wrapping a type as v2 are distinct operations. Wrapping requires further
initialization than simply creating the initial message type, and calling it
during initial construction would lead to a deadlock.
In protoreflect/go_type.go, we switch back to a lazy initialization of GoType
to avoid a deadlock since the user-provided fn may rely on the fact that
prototype.GoMessage returned.
Change-Id: I5dea00e36fe1a9899bd2ac0aed2c8e51d5d87420
Reviewed-on: https://go-review.googlesource.com/c/148826
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-06 13:05:20 -08:00
|
|
|
Has(pref.FieldNumber) bool
|
2018-10-31 18:23:42 -07:00
|
|
|
Get(pref.FieldNumber) legacyExtensionEntry
|
|
|
|
Set(pref.FieldNumber, legacyExtensionEntry)
|
internal/impl: support legacy extension fields
Implement support for extension fields for messages that use the v1
data structures for extensions. The legacyExtensionFields type wraps a
v1 map to implement the v2 protoreflect.KnownFields interface.
Working on this change revealed a bug in the dynamic construction of
message types for protobuf messages that had cyclic dependencies (e.g.,
message Foo has a sub-field of message Bar, and Bar has a sub-field of Foo).
In such a situation, a deadlock occurs because initialization code depends on
the very initialization code that is currently running. To break these cycles,
we make some systematic changes listed in the following paragraphs.
Generally speaking, we separate the logic for construction and wrapping,
where constuction does not recursively rely on dependencies,
while wrapping may recursively inspect dependencies.
Promote the MessageType.MessageOf method as a standalone MessageOf function
that dynamically finds the proper *MessageType to use. We make it such that
MessageType only supports two forms of messages types:
* Those that fully implement the v2 API.
* Those that do not implement the v2 API at all.
This removes support for the hybrid form that was exploited by message_test.go
In impl/message_test.go, switch each message to look more like how future
generated messages will look like. This is done in reaction to the fact that
MessageType.MessageOf no longer exists.
In value/{map,vector}.go, fix Unwrap to return a pointer since the underlying
reflect.Value is addressable reference value, not a pointer value.
In value/convert.go, split the logic apart so that obtaining a v2 type and
wrapping a type as v2 are distinct operations. Wrapping requires further
initialization than simply creating the initial message type, and calling it
during initial construction would lead to a deadlock.
In protoreflect/go_type.go, we switch back to a lazy initialization of GoType
to avoid a deadlock since the user-provided fn may rely on the fact that
prototype.GoMessage returned.
Change-Id: I5dea00e36fe1a9899bd2ac0aed2c8e51d5d87420
Reviewed-on: https://go-review.googlesource.com/c/148826
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-06 13:05:20 -08:00
|
|
|
Clear(pref.FieldNumber)
|
2018-10-31 18:23:42 -07:00
|
|
|
Range(f func(pref.FieldNumber, legacyExtensionEntry) bool)
|
|
|
|
}
|
|
|
|
|
|
|
|
func makeLegacyExtensionMapFunc(t reflect.Type) func(*messageDataType) legacyExtensionIface {
|
|
|
|
fx1, _ := t.FieldByName("XXX_extensions")
|
|
|
|
fx2, _ := t.FieldByName("XXX_InternalExtensions")
|
|
|
|
switch {
|
|
|
|
case fx1.Type == extTypeA:
|
|
|
|
return func(p *messageDataType) legacyExtensionIface {
|
|
|
|
rv := p.p.asType(t).Elem()
|
|
|
|
return (*legacyExtensionMap)(unsafe.Pointer(rv.UnsafeAddr() + fx1.Offset))
|
|
|
|
}
|
|
|
|
case fx2.Type == extTypeB:
|
|
|
|
return func(p *messageDataType) legacyExtensionIface {
|
|
|
|
rv := p.p.asType(t).Elem()
|
|
|
|
return (*legacyExtensionSyncMap)(unsafe.Pointer(rv.UnsafeAddr() + fx2.Offset))
|
|
|
|
}
|
|
|
|
default:
|
|
|
|
return nil
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
internal/impl: support legacy extension fields
Implement support for extension fields for messages that use the v1
data structures for extensions. The legacyExtensionFields type wraps a
v1 map to implement the v2 protoreflect.KnownFields interface.
Working on this change revealed a bug in the dynamic construction of
message types for protobuf messages that had cyclic dependencies (e.g.,
message Foo has a sub-field of message Bar, and Bar has a sub-field of Foo).
In such a situation, a deadlock occurs because initialization code depends on
the very initialization code that is currently running. To break these cycles,
we make some systematic changes listed in the following paragraphs.
Generally speaking, we separate the logic for construction and wrapping,
where constuction does not recursively rely on dependencies,
while wrapping may recursively inspect dependencies.
Promote the MessageType.MessageOf method as a standalone MessageOf function
that dynamically finds the proper *MessageType to use. We make it such that
MessageType only supports two forms of messages types:
* Those that fully implement the v2 API.
* Those that do not implement the v2 API at all.
This removes support for the hybrid form that was exploited by message_test.go
In impl/message_test.go, switch each message to look more like how future
generated messages will look like. This is done in reaction to the fact that
MessageType.MessageOf no longer exists.
In value/{map,vector}.go, fix Unwrap to return a pointer since the underlying
reflect.Value is addressable reference value, not a pointer value.
In value/convert.go, split the logic apart so that obtaining a v2 type and
wrapping a type as v2 are distinct operations. Wrapping requires further
initialization than simply creating the initial message type, and calling it
during initial construction would lead to a deadlock.
In protoreflect/go_type.go, we switch back to a lazy initialization of GoType
to avoid a deadlock since the user-provided fn may rely on the fact that
prototype.GoMessage returned.
Change-Id: I5dea00e36fe1a9899bd2ac0aed2c8e51d5d87420
Reviewed-on: https://go-review.googlesource.com/c/148826
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-06 13:05:20 -08:00
|
|
|
// TODO: We currently don't do locking with legacyExtensionSyncMap.p.mu.
|
|
|
|
// The locking behavior was already obscure "feature" beforehand,
|
|
|
|
// and it is not obvious how it translates to the v2 API.
|
|
|
|
// The v2 API presents a Range method, which calls a user provided function,
|
|
|
|
// which may in turn call other methods on the map. In such a use case,
|
|
|
|
// acquiring a lock within each method would result in a reentrant deadlock.
|
|
|
|
|
2018-10-31 18:23:42 -07:00
|
|
|
// legacyExtensionSyncMap is identical to protoV1.XXX_InternalExtensions.
|
|
|
|
// It implements legacyExtensionIface.
|
|
|
|
type legacyExtensionSyncMap struct {
|
|
|
|
p *struct {
|
|
|
|
mu sync.Mutex
|
|
|
|
m legacyExtensionMap
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
func (m legacyExtensionSyncMap) Len() int {
|
|
|
|
if m.p == nil {
|
|
|
|
return 0
|
|
|
|
}
|
|
|
|
return m.p.m.Len()
|
|
|
|
}
|
internal/impl: support legacy extension fields
Implement support for extension fields for messages that use the v1
data structures for extensions. The legacyExtensionFields type wraps a
v1 map to implement the v2 protoreflect.KnownFields interface.
Working on this change revealed a bug in the dynamic construction of
message types for protobuf messages that had cyclic dependencies (e.g.,
message Foo has a sub-field of message Bar, and Bar has a sub-field of Foo).
In such a situation, a deadlock occurs because initialization code depends on
the very initialization code that is currently running. To break these cycles,
we make some systematic changes listed in the following paragraphs.
Generally speaking, we separate the logic for construction and wrapping,
where constuction does not recursively rely on dependencies,
while wrapping may recursively inspect dependencies.
Promote the MessageType.MessageOf method as a standalone MessageOf function
that dynamically finds the proper *MessageType to use. We make it such that
MessageType only supports two forms of messages types:
* Those that fully implement the v2 API.
* Those that do not implement the v2 API at all.
This removes support for the hybrid form that was exploited by message_test.go
In impl/message_test.go, switch each message to look more like how future
generated messages will look like. This is done in reaction to the fact that
MessageType.MessageOf no longer exists.
In value/{map,vector}.go, fix Unwrap to return a pointer since the underlying
reflect.Value is addressable reference value, not a pointer value.
In value/convert.go, split the logic apart so that obtaining a v2 type and
wrapping a type as v2 are distinct operations. Wrapping requires further
initialization than simply creating the initial message type, and calling it
during initial construction would lead to a deadlock.
In protoreflect/go_type.go, we switch back to a lazy initialization of GoType
to avoid a deadlock since the user-provided fn may rely on the fact that
prototype.GoMessage returned.
Change-Id: I5dea00e36fe1a9899bd2ac0aed2c8e51d5d87420
Reviewed-on: https://go-review.googlesource.com/c/148826
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-06 13:05:20 -08:00
|
|
|
func (m legacyExtensionSyncMap) Has(n pref.FieldNumber) bool {
|
|
|
|
return m.p.m.Has(n)
|
|
|
|
}
|
2018-10-31 18:23:42 -07:00
|
|
|
func (m legacyExtensionSyncMap) Get(n pref.FieldNumber) legacyExtensionEntry {
|
|
|
|
if m.p == nil {
|
|
|
|
return legacyExtensionEntry{}
|
|
|
|
}
|
|
|
|
return m.p.m.Get(n)
|
|
|
|
}
|
|
|
|
func (m *legacyExtensionSyncMap) Set(n pref.FieldNumber, x legacyExtensionEntry) {
|
|
|
|
if m.p == nil {
|
|
|
|
m.p = new(struct {
|
|
|
|
mu sync.Mutex
|
|
|
|
m legacyExtensionMap
|
|
|
|
})
|
|
|
|
}
|
|
|
|
m.p.m.Set(n, x)
|
|
|
|
}
|
internal/impl: support legacy extension fields
Implement support for extension fields for messages that use the v1
data structures for extensions. The legacyExtensionFields type wraps a
v1 map to implement the v2 protoreflect.KnownFields interface.
Working on this change revealed a bug in the dynamic construction of
message types for protobuf messages that had cyclic dependencies (e.g.,
message Foo has a sub-field of message Bar, and Bar has a sub-field of Foo).
In such a situation, a deadlock occurs because initialization code depends on
the very initialization code that is currently running. To break these cycles,
we make some systematic changes listed in the following paragraphs.
Generally speaking, we separate the logic for construction and wrapping,
where constuction does not recursively rely on dependencies,
while wrapping may recursively inspect dependencies.
Promote the MessageType.MessageOf method as a standalone MessageOf function
that dynamically finds the proper *MessageType to use. We make it such that
MessageType only supports two forms of messages types:
* Those that fully implement the v2 API.
* Those that do not implement the v2 API at all.
This removes support for the hybrid form that was exploited by message_test.go
In impl/message_test.go, switch each message to look more like how future
generated messages will look like. This is done in reaction to the fact that
MessageType.MessageOf no longer exists.
In value/{map,vector}.go, fix Unwrap to return a pointer since the underlying
reflect.Value is addressable reference value, not a pointer value.
In value/convert.go, split the logic apart so that obtaining a v2 type and
wrapping a type as v2 are distinct operations. Wrapping requires further
initialization than simply creating the initial message type, and calling it
during initial construction would lead to a deadlock.
In protoreflect/go_type.go, we switch back to a lazy initialization of GoType
to avoid a deadlock since the user-provided fn may rely on the fact that
prototype.GoMessage returned.
Change-Id: I5dea00e36fe1a9899bd2ac0aed2c8e51d5d87420
Reviewed-on: https://go-review.googlesource.com/c/148826
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-06 13:05:20 -08:00
|
|
|
func (m legacyExtensionSyncMap) Clear(n pref.FieldNumber) {
|
|
|
|
m.p.m.Clear(n)
|
|
|
|
}
|
2018-10-31 18:23:42 -07:00
|
|
|
func (m legacyExtensionSyncMap) Range(f func(pref.FieldNumber, legacyExtensionEntry) bool) {
|
|
|
|
if m.p == nil {
|
|
|
|
return
|
|
|
|
}
|
|
|
|
m.p.m.Range(f)
|
|
|
|
}
|
|
|
|
|
|
|
|
// legacyExtensionMap is identical to map[int32]protoV1.Extension.
|
|
|
|
// It implements legacyExtensionIface.
|
|
|
|
type legacyExtensionMap map[pref.FieldNumber]legacyExtensionEntry
|
|
|
|
|
|
|
|
func (m legacyExtensionMap) Len() int {
|
|
|
|
return len(m)
|
|
|
|
}
|
internal/impl: support legacy extension fields
Implement support for extension fields for messages that use the v1
data structures for extensions. The legacyExtensionFields type wraps a
v1 map to implement the v2 protoreflect.KnownFields interface.
Working on this change revealed a bug in the dynamic construction of
message types for protobuf messages that had cyclic dependencies (e.g.,
message Foo has a sub-field of message Bar, and Bar has a sub-field of Foo).
In such a situation, a deadlock occurs because initialization code depends on
the very initialization code that is currently running. To break these cycles,
we make some systematic changes listed in the following paragraphs.
Generally speaking, we separate the logic for construction and wrapping,
where constuction does not recursively rely on dependencies,
while wrapping may recursively inspect dependencies.
Promote the MessageType.MessageOf method as a standalone MessageOf function
that dynamically finds the proper *MessageType to use. We make it such that
MessageType only supports two forms of messages types:
* Those that fully implement the v2 API.
* Those that do not implement the v2 API at all.
This removes support for the hybrid form that was exploited by message_test.go
In impl/message_test.go, switch each message to look more like how future
generated messages will look like. This is done in reaction to the fact that
MessageType.MessageOf no longer exists.
In value/{map,vector}.go, fix Unwrap to return a pointer since the underlying
reflect.Value is addressable reference value, not a pointer value.
In value/convert.go, split the logic apart so that obtaining a v2 type and
wrapping a type as v2 are distinct operations. Wrapping requires further
initialization than simply creating the initial message type, and calling it
during initial construction would lead to a deadlock.
In protoreflect/go_type.go, we switch back to a lazy initialization of GoType
to avoid a deadlock since the user-provided fn may rely on the fact that
prototype.GoMessage returned.
Change-Id: I5dea00e36fe1a9899bd2ac0aed2c8e51d5d87420
Reviewed-on: https://go-review.googlesource.com/c/148826
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-06 13:05:20 -08:00
|
|
|
func (m legacyExtensionMap) Has(n pref.FieldNumber) bool {
|
|
|
|
_, ok := m[n]
|
|
|
|
return ok
|
|
|
|
}
|
2018-10-31 18:23:42 -07:00
|
|
|
func (m legacyExtensionMap) Get(n pref.FieldNumber) legacyExtensionEntry {
|
|
|
|
return m[n]
|
|
|
|
}
|
|
|
|
func (m *legacyExtensionMap) Set(n pref.FieldNumber, x legacyExtensionEntry) {
|
|
|
|
if *m == nil {
|
|
|
|
*m = make(map[pref.FieldNumber]legacyExtensionEntry)
|
|
|
|
}
|
|
|
|
(*m)[n] = x
|
|
|
|
}
|
internal/impl: support legacy extension fields
Implement support for extension fields for messages that use the v1
data structures for extensions. The legacyExtensionFields type wraps a
v1 map to implement the v2 protoreflect.KnownFields interface.
Working on this change revealed a bug in the dynamic construction of
message types for protobuf messages that had cyclic dependencies (e.g.,
message Foo has a sub-field of message Bar, and Bar has a sub-field of Foo).
In such a situation, a deadlock occurs because initialization code depends on
the very initialization code that is currently running. To break these cycles,
we make some systematic changes listed in the following paragraphs.
Generally speaking, we separate the logic for construction and wrapping,
where constuction does not recursively rely on dependencies,
while wrapping may recursively inspect dependencies.
Promote the MessageType.MessageOf method as a standalone MessageOf function
that dynamically finds the proper *MessageType to use. We make it such that
MessageType only supports two forms of messages types:
* Those that fully implement the v2 API.
* Those that do not implement the v2 API at all.
This removes support for the hybrid form that was exploited by message_test.go
In impl/message_test.go, switch each message to look more like how future
generated messages will look like. This is done in reaction to the fact that
MessageType.MessageOf no longer exists.
In value/{map,vector}.go, fix Unwrap to return a pointer since the underlying
reflect.Value is addressable reference value, not a pointer value.
In value/convert.go, split the logic apart so that obtaining a v2 type and
wrapping a type as v2 are distinct operations. Wrapping requires further
initialization than simply creating the initial message type, and calling it
during initial construction would lead to a deadlock.
In protoreflect/go_type.go, we switch back to a lazy initialization of GoType
to avoid a deadlock since the user-provided fn may rely on the fact that
prototype.GoMessage returned.
Change-Id: I5dea00e36fe1a9899bd2ac0aed2c8e51d5d87420
Reviewed-on: https://go-review.googlesource.com/c/148826
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-06 13:05:20 -08:00
|
|
|
func (m *legacyExtensionMap) Clear(n pref.FieldNumber) {
|
|
|
|
delete(*m, n)
|
|
|
|
}
|
2018-10-31 18:23:42 -07:00
|
|
|
func (m legacyExtensionMap) Range(f func(pref.FieldNumber, legacyExtensionEntry) bool) {
|
|
|
|
for n, x := range m {
|
|
|
|
if !f(n, x) {
|
|
|
|
return
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
// legacyExtensionEntry is identical to protoV1.Extension.
|
|
|
|
type legacyExtensionEntry struct {
|
|
|
|
desc *protoV1.ExtensionDesc
|
|
|
|
val interface{}
|
|
|
|
raw []byte
|
|
|
|
}
|