11 Commits

Author SHA1 Message Date
Joe Tsai
6af2cee096 reflect/protodesc: improve documentation
Change-Id: Id1c0e0e702a7b0eb011c71de23cd5be76345d741
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/220341
Reviewed-by: Damien Neil <dneil@google.com>
2020-02-20 19:04:51 +00:00
Joe Tsai
25fc6fb4f2 reflect/protodesc: add FileOptions
The FileOptions type provides the ability to specify specialized options
for how a file descriptor is constructed. It follows the same optional
arguments pattern as used in the proto package.

The resolver is not an option since it almost always necessary
when constructing a file descriptor.

Change-Id: Ib98ac6289881ad8402dd615f6c895da5899cb8d9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/218940
Reviewed-by: Damien Neil <dneil@google.com>
2020-02-11 23:14:57 +00:00
Joe Tsai
93bccf763e all: scrub all TODOs
TODOs that we do not intend to address have been deleted.
Those that are blocking v2 release are marked with "blocks".

Change-Id: I7efa9e546d0637b562101d0edc7009893d762722
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/218878
Reviewed-by: Damien Neil <dneil@google.com>
2020-02-10 19:28:48 +00:00
Joe Tsai
b7695fab0d proto: add Clone function and MergeOptions.Clone method
We resisted adding Clone for a while since:
* It is a function that is perfectly suited for generics.
However, generics probably still won't be available in Go for some time
and it is impractical to block addition of this function when it is very
widely used and will be necessary for the v1 to v2 migration.
* In the past, there was no protoreflect.Message.IsValid, so there was
no proper API to detect invalid top-level messages and return them as such.

Since Clone relies on certain properties about proper round-tripping
of ProtoMessage.ProtoReflect <-> Message.Interface, we add a test
in testing/prototest to check for this.

Change-Id: Ic492b68f27b8b88322a6a3fa3a5e492228db79d9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213297
Reviewed-by: Damien Neil <dneil@google.com>
2020-01-06 21:07:28 +00:00
Joe Tsai
e815d6a43b all: remove dead code
Change-Id: I1344d6afca9d3348db849c2b5f387ac18b80d2ba
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189021
Reviewed-by: Damien Neil <dneil@google.com>
2019-08-06 21:16:48 +00:00
Joe Tsai
e182c917f0 reflect/protoreflect: add FileDescriptor.SourceLocations
This adds minimal support for preserving the source context information.

Change-Id: I4b3cac9690b7469ecb4e5434251a809be4d7894c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/183157
Reviewed-by: Damien Neil <dneil@google.com>
2019-07-13 00:15:59 +00:00
Joe Tsai
97a87391b1 internal/strs: unify string manipulation functionality
Create a new internal/strs package that unifies common functionality:
* Since protobuf itself pseudo-specifies at least 4 different camel-case
and snake-case conversion functions, we define all variants in one place.
* We move the internal/filedesc.nameBuilder function to this package.
We simplify its implementation to not depend on a strings.Builder fork
under the hood since the semantics we desire is simpler than what
strings.Builder provides.
* We use strs.Builder in reflect/protodesc in its construction of all
the full names. This is perfect use case of strs.Builder since all
full names within a file descriptor share the same lifetime.
* Add an UnsafeString and UnsafeBytes cast function that will be useful
in the near future for optimizing encoding/prototext and encoding/protojson.

Change-Id: I2cf07cbaf6f72e5f9fd6ae3d37b0d46f6af2ad59
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185198
Reviewed-by: Damien Neil <dneil@google.com>
2019-07-09 19:57:23 +00:00
Joe Tsai
15076350e8 reflect/protodesc: enforce strict validation
Hyrum's Law dictates that if we do not prevent naughty behavior,
people will rely on it. If we do not validate that the provided
file descriptor is correct today, it will be near impossible
to add proper validation checks later on.

The logic added validates that the provided file descriptor is
correct according to the same semantics as protoc,
which was reversed engineered to derive the set of rules implemented here.
The rules are unfortunately complicated because protobuf is a language
full of many non-orthogonal features. While our logic is complicated,
it is still 1/7th the size of the equivalent C++ code!

Change-Id: I6acc5dc3bd2e4c6bea6cd9e81214f8104402602a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184837
Reviewed-by: Damien Neil <dneil@google.com>
2019-07-03 20:46:51 +00:00
Joe Tsai
b2107fbd8d reflect/protodesc: robustify dependency resolution implementation
Overview of changes:
* Add an option that specifies whether to replace unresolvable references
with a placeholder instead of producing an error. Since the prior behavior
produced placeholders (not always), we default to that behavior for now,
but will enable strict resolving in a future CL.
* The option is not yet exported because there is concern about what the
public API should look like. This will be exposed in a future CL.
* Unlike before, we now permit placeholders for unresolvable enum values.
* We implement relative name resolution logic.
* We handle the case where the type is unknown, but type_name is specified.
In such a case, we populate both FieldDescriptor.{Enum,Message} and leave
the FieldDescriptor.Kind with the zero value. If the type_name happened
to resolve, we use that to determine the type.
* If a placeholder is used to represent a relative name,
the FullName reports an invalid full name with a "*." prefix.

Change-Id: Ifa8c750423c488fb9324eec4d033a2f251505fda
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184317
Reviewed-by: Damien Neil <dneil@google.com>
2019-07-03 19:17:36 +00:00
Joe Tsai
a691404b5d reflect/protodesc: return deep copies of the input
There is little performance benefit to aliasing the input since we copy
every field except the options. Thus, just go all the way and copy the
options as well and document this as such.

Change-Id: If6ca5ce0ee03c9f76e528023b6056ad99d3ca209
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184879
Reviewed-by: Damien Neil <dneil@google.com>
2019-07-03 16:36:09 +00:00
Joe Tsai
3274acc926 reflect/protodesc: fix initialization bug
This fixes a bug introduced by CL/182360.

Overview of the problem:
* CL/182360 removes the internal/prototype package, such that
protodesc was re-implemented using internal/filedesc.
* As a result of that change, resolving internal dependencies became
the responsibility of protodesc.
* Dependency resolution used the following two-pass algorithm:
	1) first pass derives the full name of all declarations
	2) second pass fully initializes each descriptor declaration,
	now being able to resolve local dependencies from the previous step.
* When the second pass looks up a local dependency, it is guaranteed to
find it, but it is not guaranteed that the dependency has been initialized
(since it may appear later on). This is problematic for default enum values
since it implies that the enum dependency may not be sufficiently
initialized to be able to query its set of values, leading to panics.
* CL/182360 recognized the problem and attempted to enforce an initialization
ordering where nested enums were always initialized before the body of the
message declaration itself.
* However, that ordering fails to enforce that that enum declarations outside
the parent tree are initialized beforehand. For example, referring to an
enum value that is declared within a sibling of the parent message.
* This CL fixes the problem with a three-pass algorithm:
	1) first pass derives the full name *and* fully initialize the
	entire descriptor *except* for dependency references (i.e., type_name).
	2) second pass only resolves dependency references,
	where we do not need to worry about initialization ordering.
	3) third pass validates the descriptors are well-formed.
	This can now depend on all information being fully initialized.
* While a lot of code moves, this change is actually very mechanical.
Other than split things apart, no new logic is introduced nor removed.

Change-Id: Ia91d4aade8f6187c19d704d43ae96b3b9d276792
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184297
Reviewed-by: Damien Neil <dneil@google.com>
2019-07-01 22:32:12 +00:00