mirror/protobuf-go
1
0
Fork 0
mirror of https://github.com/protocolbuffers/protobuf-go.git synced 2025-01-17 10:11:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
91b2604634
protobuf-go/reflect/protodesc/desc_init.go

248 lines
9.4 KiB
Go
Raw Normal View History

reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
// Copyright 2019 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package protodesc
import (
reflect/protodesc: enforce strict validation Hyrum's Law dictates that if we do not prevent naughty behavior, people will rely on it. If we do not validate that the provided file descriptor is correct today, it will be near impossible to add proper validation checks later on. The logic added validates that the provided file descriptor is correct according to the same semantics as protoc, which was reversed engineered to derive the set of rules implemented here. The rules are unfortunately complicated because protobuf is a language full of many non-orthogonal features. While our logic is complicated, it is still 1/7th the size of the equivalent C++ code! Change-Id: I6acc5dc3bd2e4c6bea6cd9e81214f8104402602a Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184837 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-02 22:19:08 +00:00
"google.golang.org/protobuf/internal/errors"
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
"google.golang.org/protobuf/internal/filedesc"
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
"google.golang.org/protobuf/internal/strs"
proto: add Clone function and MergeOptions.Clone method We resisted adding Clone for a while since: * It is a function that is perfectly suited for generics. However, generics probably still won't be available in Go for some time and it is impractical to block addition of this function when it is very widely used and will be necessary for the v1 to v2 migration. * In the past, there was no protoreflect.Message.IsValid, so there was no proper API to detect invalid top-level messages and return them as such. Since Clone relies on certain properties about proper round-tripping of ProtoMessage.ProtoReflect <-> Message.Interface, we add a test in testing/prototest to check for this. Change-Id: Ic492b68f27b8b88322a6a3fa3a5e492228db79d9 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213297 Reviewed-by: Damien Neil <dneil@google.com>
2020-01-04 17:09:04 +00:00
"google.golang.org/protobuf/proto"
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
"google.golang.org/protobuf/reflect/protoreflect"
"google.golang.org/protobuf/types/descriptorpb"
)
type descsByName map[protoreflect.FullName]protoreflect.Descriptor
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
func (r descsByName) initEnumDeclarations(eds []*descriptorpb.EnumDescriptorProto, parent protoreflect.Descriptor, sb *strs.Builder) (es []filedesc.Enum, err error) {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
es = make([]filedesc.Enum, len(eds)) // allocate up-front to ensure stable pointers
for i, ed := range eds {
e := &es[i]
e.L2 = new(filedesc.EnumL2)
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
if e.L0, err = r.makeBase(e, parent, ed.GetName(), i, sb); err != nil {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
return nil, err
}
if opts := ed.GetOptions(); opts != nil {
proto: add Clone function and MergeOptions.Clone method We resisted adding Clone for a while since: * It is a function that is perfectly suited for generics. However, generics probably still won't be available in Go for some time and it is impractical to block addition of this function when it is very widely used and will be necessary for the v1 to v2 migration. * In the past, there was no protoreflect.Message.IsValid, so there was no proper API to detect invalid top-level messages and return them as such. Since Clone relies on certain properties about proper round-tripping of ProtoMessage.ProtoReflect <-> Message.Interface, we add a test in testing/prototest to check for this. Change-Id: Ic492b68f27b8b88322a6a3fa3a5e492228db79d9 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213297 Reviewed-by: Damien Neil <dneil@google.com>
2020-01-04 17:09:04 +00:00
opts = proto.Clone(opts).(*descriptorpb.EnumOptions)
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
e.L2.Options = func() protoreflect.ProtoMessage { return opts }
}
for _, s := range ed.GetReservedName() {
e.L2.ReservedNames.List = append(e.L2.ReservedNames.List, protoreflect.Name(s))
}
for _, rr := range ed.GetReservedRange() {
e.L2.ReservedRanges.List = append(e.L2.ReservedRanges.List, [2]protoreflect.EnumNumber{
protoreflect.EnumNumber(rr.GetStart()),
protoreflect.EnumNumber(rr.GetEnd()),
})
}
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
if e.L2.Values.List, err = r.initEnumValuesFromDescriptorProto(ed.GetValue(), e, sb); err != nil {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
return nil, err
}
}
return es, nil
}
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
func (r descsByName) initEnumValuesFromDescriptorProto(vds []*descriptorpb.EnumValueDescriptorProto, parent protoreflect.Descriptor, sb *strs.Builder) (vs []filedesc.EnumValue, err error) {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
vs = make([]filedesc.EnumValue, len(vds)) // allocate up-front to ensure stable pointers
for i, vd := range vds {
v := &vs[i]
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
if v.L0, err = r.makeBase(v, parent, vd.GetName(), i, sb); err != nil {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
return nil, err
}
if opts := vd.GetOptions(); opts != nil {
proto: add Clone function and MergeOptions.Clone method We resisted adding Clone for a while since: * It is a function that is perfectly suited for generics. However, generics probably still won't be available in Go for some time and it is impractical to block addition of this function when it is very widely used and will be necessary for the v1 to v2 migration. * In the past, there was no protoreflect.Message.IsValid, so there was no proper API to detect invalid top-level messages and return them as such. Since Clone relies on certain properties about proper round-tripping of ProtoMessage.ProtoReflect <-> Message.Interface, we add a test in testing/prototest to check for this. Change-Id: Ic492b68f27b8b88322a6a3fa3a5e492228db79d9 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213297 Reviewed-by: Damien Neil <dneil@google.com>
2020-01-04 17:09:04 +00:00
opts = proto.Clone(opts).(*descriptorpb.EnumValueOptions)
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
v.L1.Options = func() protoreflect.ProtoMessage { return opts }
}
v.L1.Number = protoreflect.EnumNumber(vd.GetNumber())
}
return vs, nil
}
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
func (r descsByName) initMessagesDeclarations(mds []*descriptorpb.DescriptorProto, parent protoreflect.Descriptor, sb *strs.Builder) (ms []filedesc.Message, err error) {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
ms = make([]filedesc.Message, len(mds)) // allocate up-front to ensure stable pointers
for i, md := range mds {
m := &ms[i]
m.L2 = new(filedesc.MessageL2)
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
if m.L0, err = r.makeBase(m, parent, md.GetName(), i, sb); err != nil {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
return nil, err
}
if opts := md.GetOptions(); opts != nil {
proto: add Clone function and MergeOptions.Clone method We resisted adding Clone for a while since: * It is a function that is perfectly suited for generics. However, generics probably still won't be available in Go for some time and it is impractical to block addition of this function when it is very widely used and will be necessary for the v1 to v2 migration. * In the past, there was no protoreflect.Message.IsValid, so there was no proper API to detect invalid top-level messages and return them as such. Since Clone relies on certain properties about proper round-tripping of ProtoMessage.ProtoReflect <-> Message.Interface, we add a test in testing/prototest to check for this. Change-Id: Ic492b68f27b8b88322a6a3fa3a5e492228db79d9 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213297 Reviewed-by: Damien Neil <dneil@google.com>
2020-01-04 17:09:04 +00:00
opts = proto.Clone(opts).(*descriptorpb.MessageOptions)
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
m.L2.Options = func() protoreflect.ProtoMessage { return opts }
internal/filedesc: move message options to L1 initialization Avoid a deadlock when registering a legacy ExtensionType, caused by initialization of the "internal/impl".ExtensionInfo calling IsMessageSet on the MessageDescriptor of the type being extended. We can avoid this deadlock either by initializing the ExtensionType outside of the GlobalTypes mutex, or by moving IsMessageSet to L1 initialization of the MessageDescriptor so that it doesn't trigger lazy init. CL 204804 takes the former approach; this CL takes the latter. Change-Id: Idfc1ed36a23a139839290ea32492142a17f68cf5 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/205957 Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-11-07 23:30:44 +00:00
m.L1.IsMapEntry = opts.GetMapEntry()
m.L1.IsMessageSet = opts.GetMessageSetWireFormat()
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
}
for _, s := range md.GetReservedName() {
m.L2.ReservedNames.List = append(m.L2.ReservedNames.List, protoreflect.Name(s))
}
for _, rr := range md.GetReservedRange() {
m.L2.ReservedRanges.List = append(m.L2.ReservedRanges.List, [2]protoreflect.FieldNumber{
protoreflect.FieldNumber(rr.GetStart()),
protoreflect.FieldNumber(rr.GetEnd()),
})
}
for _, xr := range md.GetExtensionRange() {
m.L2.ExtensionRanges.List = append(m.L2.ExtensionRanges.List, [2]protoreflect.FieldNumber{
protoreflect.FieldNumber(xr.GetStart()),
protoreflect.FieldNumber(xr.GetEnd()),
})
var optsFunc func() protoreflect.ProtoMessage
if opts := xr.GetOptions(); opts != nil {
proto: add Clone function and MergeOptions.Clone method We resisted adding Clone for a while since: * It is a function that is perfectly suited for generics. However, generics probably still won't be available in Go for some time and it is impractical to block addition of this function when it is very widely used and will be necessary for the v1 to v2 migration. * In the past, there was no protoreflect.Message.IsValid, so there was no proper API to detect invalid top-level messages and return them as such. Since Clone relies on certain properties about proper round-tripping of ProtoMessage.ProtoReflect <-> Message.Interface, we add a test in testing/prototest to check for this. Change-Id: Ic492b68f27b8b88322a6a3fa3a5e492228db79d9 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213297 Reviewed-by: Damien Neil <dneil@google.com>
2020-01-04 17:09:04 +00:00
opts = proto.Clone(opts).(*descriptorpb.ExtensionRangeOptions)
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
optsFunc = func() protoreflect.ProtoMessage { return opts }
}
m.L2.ExtensionRangeOptions = append(m.L2.ExtensionRangeOptions, optsFunc)
}
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
if m.L2.Fields.List, err = r.initFieldsFromDescriptorProto(md.GetField(), m, sb); err != nil {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
return nil, err
}
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
if m.L2.Oneofs.List, err = r.initOneofsFromDescriptorProto(md.GetOneofDecl(), m, sb); err != nil {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
return nil, err
}
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
if m.L1.Enums.List, err = r.initEnumDeclarations(md.GetEnumType(), m, sb); err != nil {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
return nil, err
}
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
if m.L1.Messages.List, err = r.initMessagesDeclarations(md.GetNestedType(), m, sb); err != nil {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
return nil, err
}
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
if m.L1.Extensions.List, err = r.initExtensionDeclarations(md.GetExtension(), m, sb); err != nil {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
return nil, err
}
}
return ms, nil
}
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
func (r descsByName) initFieldsFromDescriptorProto(fds []*descriptorpb.FieldDescriptorProto, parent protoreflect.Descriptor, sb *strs.Builder) (fs []filedesc.Field, err error) {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
fs = make([]filedesc.Field, len(fds)) // allocate up-front to ensure stable pointers
for i, fd := range fds {
f := &fs[i]
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
if f.L0, err = r.makeBase(f, parent, fd.GetName(), i, sb); err != nil {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
return nil, err
}
if opts := fd.GetOptions(); opts != nil {
proto: add Clone function and MergeOptions.Clone method We resisted adding Clone for a while since: * It is a function that is perfectly suited for generics. However, generics probably still won't be available in Go for some time and it is impractical to block addition of this function when it is very widely used and will be necessary for the v1 to v2 migration. * In the past, there was no protoreflect.Message.IsValid, so there was no proper API to detect invalid top-level messages and return them as such. Since Clone relies on certain properties about proper round-tripping of ProtoMessage.ProtoReflect <-> Message.Interface, we add a test in testing/prototest to check for this. Change-Id: Ic492b68f27b8b88322a6a3fa3a5e492228db79d9 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213297 Reviewed-by: Damien Neil <dneil@google.com>
2020-01-04 17:09:04 +00:00
opts = proto.Clone(opts).(*descriptorpb.FieldOptions)
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
f.L1.Options = func() protoreflect.ProtoMessage { return opts }
f.L1.IsWeak = opts.GetWeak()
f.L1.HasPacked = opts.Packed != nil
f.L1.IsPacked = opts.GetPacked()
}
f.L1.Number = protoreflect.FieldNumber(fd.GetNumber())
f.L1.Cardinality = protoreflect.Cardinality(fd.GetLabel())
if fd.Type != nil {
f.L1.Kind = protoreflect.Kind(fd.GetType())
}
if fd.JsonName != nil {
internal/filedesc: use jsonName.Init method over JSONName constructor The JSONName constructor returns a struct value which shallow copies a sync.Once within it; this is a dubious pattern. Instead, add a jsonName.Init method to initialize the value. Change-Id: I190a7239b1b62a8041ee7e4e09c0fe37b64ff623 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213237 Reviewed-by: Damien Neil <dneil@google.com>
2020-01-03 23:53:47 +00:00
f.L1.JSONName.Init(fd.GetJsonName())
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
}
}
return fs, nil
}
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
func (r descsByName) initOneofsFromDescriptorProto(ods []*descriptorpb.OneofDescriptorProto, parent protoreflect.Descriptor, sb *strs.Builder) (os []filedesc.Oneof, err error) {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
os = make([]filedesc.Oneof, len(ods)) // allocate up-front to ensure stable pointers
for i, od := range ods {
o := &os[i]
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
if o.L0, err = r.makeBase(o, parent, od.GetName(), i, sb); err != nil {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
return nil, err
}
if opts := od.GetOptions(); opts != nil {
proto: add Clone function and MergeOptions.Clone method We resisted adding Clone for a while since: * It is a function that is perfectly suited for generics. However, generics probably still won't be available in Go for some time and it is impractical to block addition of this function when it is very widely used and will be necessary for the v1 to v2 migration. * In the past, there was no protoreflect.Message.IsValid, so there was no proper API to detect invalid top-level messages and return them as such. Since Clone relies on certain properties about proper round-tripping of ProtoMessage.ProtoReflect <-> Message.Interface, we add a test in testing/prototest to check for this. Change-Id: Ic492b68f27b8b88322a6a3fa3a5e492228db79d9 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213297 Reviewed-by: Damien Neil <dneil@google.com>
2020-01-04 17:09:04 +00:00
opts = proto.Clone(opts).(*descriptorpb.OneofOptions)
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
o.L1.Options = func() protoreflect.ProtoMessage { return opts }
}
}
return os, nil
}
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
func (r descsByName) initExtensionDeclarations(xds []*descriptorpb.FieldDescriptorProto, parent protoreflect.Descriptor, sb *strs.Builder) (xs []filedesc.Extension, err error) {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
xs = make([]filedesc.Extension, len(xds)) // allocate up-front to ensure stable pointers
for i, xd := range xds {
x := &xs[i]
x.L2 = new(filedesc.ExtensionL2)
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
if x.L0, err = r.makeBase(x, parent, xd.GetName(), i, sb); err != nil {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
return nil, err
}
if opts := xd.GetOptions(); opts != nil {
proto: add Clone function and MergeOptions.Clone method We resisted adding Clone for a while since: * It is a function that is perfectly suited for generics. However, generics probably still won't be available in Go for some time and it is impractical to block addition of this function when it is very widely used and will be necessary for the v1 to v2 migration. * In the past, there was no protoreflect.Message.IsValid, so there was no proper API to detect invalid top-level messages and return them as such. Since Clone relies on certain properties about proper round-tripping of ProtoMessage.ProtoReflect <-> Message.Interface, we add a test in testing/prototest to check for this. Change-Id: Ic492b68f27b8b88322a6a3fa3a5e492228db79d9 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213297 Reviewed-by: Damien Neil <dneil@google.com>
2020-01-04 17:09:04 +00:00
opts = proto.Clone(opts).(*descriptorpb.FieldOptions)
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
x.L2.Options = func() protoreflect.ProtoMessage { return opts }
x.L2.IsPacked = opts.GetPacked()
}
x.L1.Number = protoreflect.FieldNumber(xd.GetNumber())
internal: simplify ExtensionInfo initialization This CL: * Make the meaning of impl/ExtensionInfo.goType consistent. Before, it was sometimes a T and other times a []T depending on the current state of initialization. Change it so that it is the constructor's responsibility to pass in a []T if it is repeated. * Make internal/filetype responsible for constructing a []T for repeated extension fields. * Makes filedesc/Extension.Cardinality one of the eagerly initialized pieces of information since it is useful to internal/filetype. * Unify ExtensionInfo.desc and ExtensionInfo.tdesc.ExtensionField, which held the same information. * Remove the internal implementation for impl.X.ExtensionDescFromType since we are dropping support for this from v1. Change-Id: Ie95c4de66cd674c1d886da4f63b133b7d763c7ef Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/195777 Reviewed-by: Damien Neil <dneil@google.com>
2019-09-16 20:30:15 +00:00
x.L1.Cardinality = protoreflect.Cardinality(xd.GetLabel())
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
if xd.Type != nil {
x.L1.Kind = protoreflect.Kind(xd.GetType())
}
if xd.JsonName != nil {
internal/filedesc: use jsonName.Init method over JSONName constructor The JSONName constructor returns a struct value which shallow copies a sync.Once within it; this is a dubious pattern. Instead, add a jsonName.Init method to initialize the value. Change-Id: I190a7239b1b62a8041ee7e4e09c0fe37b64ff623 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213237 Reviewed-by: Damien Neil <dneil@google.com>
2020-01-03 23:53:47 +00:00
x.L2.JSONName.Init(xd.GetJsonName())
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
}
}
return xs, nil
}
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
func (r descsByName) initServiceDeclarations(sds []*descriptorpb.ServiceDescriptorProto, parent protoreflect.Descriptor, sb *strs.Builder) (ss []filedesc.Service, err error) {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
ss = make([]filedesc.Service, len(sds)) // allocate up-front to ensure stable pointers
for i, sd := range sds {
s := &ss[i]
s.L2 = new(filedesc.ServiceL2)
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
if s.L0, err = r.makeBase(s, parent, sd.GetName(), i, sb); err != nil {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
return nil, err
}
if opts := sd.GetOptions(); opts != nil {
proto: add Clone function and MergeOptions.Clone method We resisted adding Clone for a while since: * It is a function that is perfectly suited for generics. However, generics probably still won't be available in Go for some time and it is impractical to block addition of this function when it is very widely used and will be necessary for the v1 to v2 migration. * In the past, there was no protoreflect.Message.IsValid, so there was no proper API to detect invalid top-level messages and return them as such. Since Clone relies on certain properties about proper round-tripping of ProtoMessage.ProtoReflect <-> Message.Interface, we add a test in testing/prototest to check for this. Change-Id: Ic492b68f27b8b88322a6a3fa3a5e492228db79d9 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213297 Reviewed-by: Damien Neil <dneil@google.com>
2020-01-04 17:09:04 +00:00
opts = proto.Clone(opts).(*descriptorpb.ServiceOptions)
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
s.L2.Options = func() protoreflect.ProtoMessage { return opts }
}
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
if s.L2.Methods.List, err = r.initMethodsFromDescriptorProto(sd.GetMethod(), s, sb); err != nil {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
return nil, err
}
}
return ss, nil
}
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
func (r descsByName) initMethodsFromDescriptorProto(mds []*descriptorpb.MethodDescriptorProto, parent protoreflect.Descriptor, sb *strs.Builder) (ms []filedesc.Method, err error) {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
ms = make([]filedesc.Method, len(mds)) // allocate up-front to ensure stable pointers
for i, md := range mds {
m := &ms[i]
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
if m.L0, err = r.makeBase(m, parent, md.GetName(), i, sb); err != nil {
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
return nil, err
}
if opts := md.GetOptions(); opts != nil {
proto: add Clone function and MergeOptions.Clone method We resisted adding Clone for a while since: * It is a function that is perfectly suited for generics. However, generics probably still won't be available in Go for some time and it is impractical to block addition of this function when it is very widely used and will be necessary for the v1 to v2 migration. * In the past, there was no protoreflect.Message.IsValid, so there was no proper API to detect invalid top-level messages and return them as such. Since Clone relies on certain properties about proper round-tripping of ProtoMessage.ProtoReflect <-> Message.Interface, we add a test in testing/prototest to check for this. Change-Id: Ic492b68f27b8b88322a6a3fa3a5e492228db79d9 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213297 Reviewed-by: Damien Neil <dneil@google.com>
2020-01-04 17:09:04 +00:00
opts = proto.Clone(opts).(*descriptorpb.MethodOptions)
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
m.L1.Options = func() protoreflect.ProtoMessage { return opts }
}
m.L1.IsStreamingClient = md.GetClientStreaming()
m.L1.IsStreamingServer = md.GetServerStreaming()
}
return ms, nil
}
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
func (r descsByName) makeBase(child, parent protoreflect.Descriptor, name string, idx int, sb *strs.Builder) (filedesc.BaseL0, error) {
reflect/protodesc: enforce strict validation Hyrum's Law dictates that if we do not prevent naughty behavior, people will rely on it. If we do not validate that the provided file descriptor is correct today, it will be near impossible to add proper validation checks later on. The logic added validates that the provided file descriptor is correct according to the same semantics as protoc, which was reversed engineered to derive the set of rules implemented here. The rules are unfortunately complicated because protobuf is a language full of many non-orthogonal features. While our logic is complicated, it is still 1/7th the size of the equivalent C++ code! Change-Id: I6acc5dc3bd2e4c6bea6cd9e81214f8104402602a Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184837 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-02 22:19:08 +00:00
if !protoreflect.Name(name).IsValid() {
return filedesc.BaseL0{}, errors.New("descriptor %q has an invalid nested name: %q", parent.FullName(), name)
}
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
// Derive the full name of the child.
// Note that enum values are a sibling to the enum parent in the namespace.
var fullName protoreflect.FullName
if _, ok := parent.(protoreflect.EnumDescriptor); ok {
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
fullName = sb.AppendFullName(parent.FullName().Parent(), protoreflect.Name(name))
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
} else {
internal/strs: unify string manipulation functionality Create a new internal/strs package that unifies common functionality: * Since protobuf itself pseudo-specifies at least 4 different camel-case and snake-case conversion functions, we define all variants in one place. * We move the internal/filedesc.nameBuilder function to this package. We simplify its implementation to not depend on a strings.Builder fork under the hood since the semantics we desire is simpler than what strings.Builder provides. * We use strs.Builder in reflect/protodesc in its construction of all the full names. This is perfect use case of strs.Builder since all full names within a file descriptor share the same lifetime. * Add an UnsafeString and UnsafeBytes cast function that will be useful in the near future for optimizing encoding/prototext and encoding/protojson. Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-07 08:49:59 +00:00
fullName = sb.AppendFullName(parent.FullName(), protoreflect.Name(name))
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
}
reflect/protodesc: enforce strict validation Hyrum's Law dictates that if we do not prevent naughty behavior, people will rely on it. If we do not validate that the provided file descriptor is correct today, it will be near impossible to add proper validation checks later on. The logic added validates that the provided file descriptor is correct according to the same semantics as protoc, which was reversed engineered to derive the set of rules implemented here. The rules are unfortunately complicated because protobuf is a language full of many non-orthogonal features. While our logic is complicated, it is still 1/7th the size of the equivalent C++ code! Change-Id: I6acc5dc3bd2e4c6bea6cd9e81214f8104402602a Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184837 Reviewed-by: Damien Neil <dneil@google.com>
2019-07-02 22:19:08 +00:00
if _, ok := r[fullName]; ok {
return filedesc.BaseL0{}, errors.New("descriptor %q already declared", fullName)
}
reflect/protodesc: fix initialization bug This fixes a bug introduced by CL/182360. Overview of the problem: * CL/182360 removes the internal/prototype package, such that protodesc was re-implemented using internal/filedesc. * As a result of that change, resolving internal dependencies became the responsibility of protodesc. * Dependency resolution used the following two-pass algorithm: 1) first pass derives the full name of all declarations 2) second pass fully initializes each descriptor declaration, now being able to resolve local dependencies from the previous step. * When the second pass looks up a local dependency, it is guaranteed to find it, but it is not guaranteed that the dependency has been initialized (since it may appear later on). This is problematic for default enum values since it implies that the enum dependency may not be sufficiently initialized to be able to query its set of values, leading to panics. * CL/182360 recognized the problem and attempted to enforce an initialization ordering where nested enums were always initialized before the body of the message declaration itself. * However, that ordering fails to enforce that that enum declarations outside the parent tree are initialized beforehand. For example, referring to an enum value that is declared within a sibling of the parent message. * This CL fixes the problem with a three-pass algorithm: 1) first pass derives the full name *and* fully initialize the entire descriptor *except* for dependency references (i.e., type_name). 2) second pass only resolves dependency references, where we do not need to worry about initialization ordering. 3) third pass validates the descriptors are well-formed. This can now depend on all information being fully initialized. * While a lot of code moves, this change is actually very mechanical. Other than split things apart, no new logic is introduced nor removed. Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792 Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297 Reviewed-by: Damien Neil <dneil@google.com>
2019-06-29 07:05:22 +00:00
r[fullName] = child
// TODO: Verify that the full name does not already exist in the resolver?
// This is not as critical since most usages of NewFile will register
// the created file back into the registry, which will perform this check.
return filedesc.BaseL0{
FullName: fullName,
ParentFile: parent.ParentFile().(*filedesc.File),
Parent: parent,
Index: idx,
}, nil
}
Reference in New Issue Copy Permalink