Commit Graph

185 Commits

Author SHA1 Message Date
Joe Tsai
ccf645912d reflect/prototype: print {Enum,Message,Extension}Type.GoType
Add GoType to the list of accessors that can be pretty-printed.
Useful for debugging.

Change-Id: Ib2a240249ca5bf0176cb77fa5edc639d44ae0561
Reviewed-on: https://go-review.googlesource.com/c/150537
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-20 20:02:59 +00:00
Joe Tsai
4b7aff630a all: rename Vector as List
The terminology Vector does not occur in protobuf documentation at all,
so we should rename the Go use of the term to something more recognizable.
As such, all instances that match the regexp "[Vv]ect(or)?" were replaced.

The C++ documentation uses the term "Repeated", which is a reasonable name.
However, the term became overloaded in 2014, when maps were added as a feature
and implementated under the hood as repeated fields. This is confusing as it
means "repeated" could either refer to repeated fields proper (i.e., explicitly
marked with the "repeated" label in the proto file) or map fields. In the case
of the C++ reflective API, this is not a problem since repeated fields proper
and map fields are interacted with through the same RepeatedField type.

In Go, we do not use a single type to handle both types of repeated fields:
1) We are coming up with the Go protobuf reflection API for the first time
and so do not need to piggy-back on the repeated fields API to remain backwards
compatible since no former usages of Go protobuf reflection exists.
2) Map fields are commonly represented in Go as the Go map type, which do not
preserve ordering information. As such it is fundamentally impossible to present
an unordered map as a consistently ordered list. Thus, Go needs two different
interfaces for lists and maps.

Given the above situation, "Repeated" is not a great term to use since it
refers to two different things (when we only want one of the meanings).
To distinguish between the two, we'll use the terms "List" and "Map" instead.
There is some precedence for the term "List" in the protobuf codebase
(e.g., "getRepeatedInt32List").

Change-Id: Iddcdb6b78e1e60c14fa4ca213c15f45e214b967b
Reviewed-on: https://go-review.googlesource.com/c/149657
Reviewed-by: Damien Neil <dneil@google.com>
2018-11-14 23:03:53 +00:00
Joe Tsai
f0c01e459b internal/impl: support legacy extension fields
Implement support for extension fields for messages that use the v1
data structures for extensions. The legacyExtensionFields type wraps a
v1 map to implement the v2 protoreflect.KnownFields interface.

Working on this change revealed a bug in the dynamic construction of
message types for protobuf messages that had cyclic dependencies (e.g.,
message Foo has a sub-field of message Bar, and Bar has a sub-field of Foo).
In such a situation, a deadlock occurs because initialization code depends on
the very initialization code that is currently running. To break these cycles,
we make some systematic changes listed in the following paragraphs.
Generally speaking, we separate the logic for construction and wrapping,
where constuction does not recursively rely on dependencies,
while wrapping may recursively inspect dependencies.

Promote the MessageType.MessageOf method as a standalone MessageOf function
that dynamically finds the proper *MessageType to use. We make it such that
MessageType only supports two forms of messages types:
* Those that fully implement the v2 API.
* Those that do not implement the v2 API at all.
This removes support for the hybrid form that was exploited by message_test.go

In impl/message_test.go, switch each message to look more like how future
generated messages will look like. This is done in reaction to the fact that
MessageType.MessageOf no longer exists.

In value/{map,vector}.go, fix Unwrap to return a pointer since the underlying
reflect.Value is addressable reference value, not a pointer value.

In value/convert.go, split the logic apart so that obtaining a v2 type and
wrapping a type as v2 are distinct operations. Wrapping requires further
initialization than simply creating the initial message type, and calling it
during initial construction would lead to a deadlock.

In protoreflect/go_type.go, we switch back to a lazy initialization of GoType
to avoid a deadlock since the user-provided fn may rely on the fact that
prototype.GoMessage returned.

Change-Id: I5dea00e36fe1a9899bd2ac0aed2c8e51d5d87420
Reviewed-on: https://go-review.googlesource.com/c/148826
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-14 18:37:45 +00:00
Joe Tsai
1c40f4957d reflect/prototype: simplify Go type descriptor constructors
The Go type descriptors protoreflect.{Enum,Message,Extension}Type are simple
wrappers over protoreflect.{Enum,Message,Extension}Descriptor with a small
number of additional methods. It is very unlikely that more will be added in
the near future.

For this reason, construct the types directly using arguments to the constructor
function, as opposed to taking in another struct (which was originally done
to provide flexibility in-case we needed more fields).

Furthmore, rename GoNew and New.

Change-Id: Ic7fb5bc250cdb2761ae03b388b5147ff50f37d15
Reviewed-on: https://go-review.googlesource.com/c/148822
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-10 21:41:49 +00:00
Joe Tsai
2cdb23da01 reflect/prototype: remove TODO about custom extensions constructors
Most usages extensions are only for fields that have a singular message
as the type. In such a sitation, the Go type and constructors are determined
by the provided protoreflect.MessageType and is certainly what the user wanted.

Custom extensions constructors are mainly useful if the user wants to use their
own type to represent repeated fields. For example, we use *[]T to represent
repeated fields, while a user may want to use their own opaque type.

However, this is such a niche use-case that it is not worth supporting given
that there is a workaround:

	type MyExtensionType struct {
		protoreflect.ExtensionType
	}
	func (x MyExtensionType) GoNew() interface{} {
		... // return my custom value
	}
	func (x MyExtensionType) GoType() reflect.Type {
		... // return my custom type
	}
	func (x MyExtensionType) ValueOf(v interface{}) protoreflect.Value {
		... // convert my custom type to a protoreflect.Value
	}
	func (x MyExtensionType) InterfaceOf(v protoreflect.Value) interface{} {
		... // convert a protoreflect.Value to my custom type
	}

Change-Id: Iafc838b1c95b0ffc8583461ee9eb327bd3ce9c3c
Reviewed-on: https://go-review.googlesource.com/c/148820
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-10 00:59:49 +00:00
Joe Tsai
34eb7ef6d5 reflect/protoreflect: clarify Clear and Remove operation on missing entries
Follow the precedence of Go maps where deletion on a key without an entry in
the map is a noop. Similarly, document that the following methods are safe
to call with entries that do not exist:
* Map.Clear
* KnownFields.Clear
* ExtensionFieldTypes.Remove

Change the implementation for each of these to match the documented behavior.

Change-Id: Ifccff9b7b03baaeffdc366d05f6286ba60e14934
Reviewed-on: https://go-review.googlesource.com/c/148317
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-08 02:34:04 +00:00
Joe Tsai
f0ad886d73 reflect/prototype: implement NewGoExtension
Implement the constructor for protoreflect.ExtensionType.
The constructor is more complicated than NewGoEnum and NewGoMessage because
it is responsible for providing the wrappers to present *[]T as a
protoreflect.Vector.

There are no tests since we need the follow-up logic in internal/impl to actually
make use of extensions. A subsequent CL will add that logic and comprehensively
test extensions.

Change-Id: I2d7893de299fe40be2ccedd8f39a92c40c41e59a
Reviewed-on: https://go-review.googlesource.com/c/147578
Reviewed-by: Herbie Ong <herbie@google.com>
2018-11-06 20:55:45 +00:00
Damien Neil
a485fbd353 reflect/protoreflect: add FieldDescriptor.DefaultEnumValue method
Default values for enums are specified by name, not number. An enum
may contain multiple values with different names but the same number.
Representing the default as a protoreflect.Value containing an EnumNumber
can discard information.

Add a method returning the EnumValueDescriptor.

Change-Id: If8beee3f81d41c4f9af45423252603b86949c7a5
Reviewed-on: https://go-review.googlesource.com/c/145158
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-29 15:59:23 +00:00
Joe Tsai
e2afdc27e7 internal/impl: support legacy unknown fields
Add wrapper data structures to get legacy XXX_unrecognized fields to support
the new protoreflect.UnknownFields interface. This is a challenge since the
field is a []byte, which does not give us much flexibility to work with
in terms of choice of data structures.

This implementation is relatively naive where every operation is O(n) since
it needs to strip through the entire []byte each time. The Range operation
operates slightly differently from ranging over Go maps since it presents a
stale version of RawFields should a mutation occur while ranging.
This distinction is unlikely to affect anyone in practice.

Change-Id: Ib3247cb827f9a0dd6c2192cd59830dca5eef8257
Reviewed-on: https://go-review.googlesource.com/c/144697
Reviewed-by: Damien Neil <dneil@google.com>
2018-10-26 19:55:21 +00:00
Joe Tsai
78f02a161f reflect/protoreflect: clarify Mutable semantics
The Mutable semantics are simpler than currently documented. In an older design
of the protobuf reflection API, it was intended that there could be a mutable
version of the scalar kinds. However, support for this has been dropped,
simplifying the documented semantics.

Change-Id: If66740ffa5da65a3f2c8cf46700e0c23393a16d7
Reviewed-on: https://go-review.googlesource.com/c/144857
Reviewed-by: Damien Neil <dneil@google.com>
2018-10-26 00:35:03 +00:00
Joe Tsai
620bc28bae reflect/protoreflect: clarify Range semantics
Restrict mutation operations during Range to only the current iteration key.
This simplifies the guarantees that any given implementation may need to provide.

While none of the range operations have a defined order, UnknownFields.Range
is special in that the iteration order is at least deterministic.
This exception occurs because there is no obviously correct way to re-order them
(since order of unknown fields can have semantic significance).

Change-Id: I35145e500dbc7c87be7270f0d886ef52e13d07d8
Reviewed-on: https://go-review.googlesource.com/c/144700
Reviewed-by: Damien Neil <dneil@google.com>
2018-10-25 23:23:31 +00:00
Joe Tsai
ad8ee05d0e reflect/prototype: make extensions use proto2 semantics for IsPacked
Extensions are normally thought of as a proto2 feature.
However, proto3 actually allows declarations of field extensions *only*
if they extend messages declared in descriptor.proto.

In such a situation it is not well defined whether the extension field
operates under proto2 or proto3 properties.

In some ways, such declarations are proto3-like.
For example, the declaration in the proto file still follows
proto3's syntactical grammar (e.g., no "optional" label required)
and proto3 semantics (e.g., default values are not allowed).

However, in other ways, such declarations are proto2-like.
For example, a repeated field of numeric scalars is not automatically packed.
This property is determined by the following test.

Compile the following proto file:

	syntax = "proto3";
	import "google/protobuf/descriptor.proto";
	extend google.protobuf.FieldOptions {
		repeated sint32 test_field = 20181023;
	}

then compile and run a C++ program with the following snippet:

	google::protobuf::FieldOptions m;
	m.AddExtension(test_field, 1);
	m.AddExtension(test_field, 2);
	m.AddExtension(test_field, 3);
	fstream output("out", ios::out | ios::trunc | ios::binary);
	m.SerializeToOstream(&output);

which produces an "out" file with the following contents:

	$ pbdump -sints 20181023 out
	Message{
		Tag{20181023, Varint}, Svarint(1),
		Tag{20181023, Varint}, Svarint(2),
		Tag{20181023, Varint}, Svarint(3),
	}

which is indicative that packed encoding was not used by default (proto2-like).
If we repeat the above experiment and explicit set "[packed = true]",
then packed encoding is used:

	$ pbdump -sints 20181023 out
	Message{
		Tag{20181023, Bytes}, LengthPrefix{Svarint(1), Svarint(2), Svarint(3)},
	}

Note that this change does not mean that field extensions are always proto2
since the experiment above does not conclusively prove that. Thus, the Syntax
on extensions derived from the descriptor protos may still report proto3.

The behavior of packed encoding with regards to proto3 says:

	In proto3, repeated fields of scalar numeric types use packed encoding by default.

One way to interpret this is:

	In proto3 (messages), repeated fields of scalar numeric types use packed encoding by default.

In which case packedness is a property of the message's syntax
rather than the field's syntax (if such a distinction exists).
Since only proto2 messages can be extended, we can safely assume that all
extension fields use proto2 semantics for IsPacked.

Change-Id: Iae595c6d88c6e252cae7552cae083bad42f2494a
Reviewed-on: https://go-review.googlesource.com/c/144278
Reviewed-by: Damien Neil <dneil@google.com>
2018-10-24 17:43:20 +00:00
Joe Tsai
2f68590cf0 reflect/prototype: use proto2 syntax for standalone extension fields
Without resolving the question of whether an extension field is always proto2
or sometimes proto3, just make it impossible to construct a standalone extension
field of the proto3 syntax.

By dropping the syntax field from the constructor, we can just assume that the
syntax is always proto2. There is no benefit for creating a proto3 standalone
extension, so there is no loss of functionality.

In the unlikely future where there needs to be distinction, we can add the field
back into the descriptor. The zero value of protoreflect.Syntax is invalid,
so we can determine if the user did not set that field.

Change-Id: Ie93fa55de96f29553fb04ff6649bbea79a144407
Reviewed-on: https://go-review.googlesource.com/c/144279
Reviewed-by: Damien Neil <dneil@google.com>
2018-10-24 17:35:49 +00:00
Damien Neil
204f1c0ad8 reflect/protoreflect: add Descriptor.Options method
Add a method to fetch descriptor options. Since options are proto
messages (e.g., google.protobuf.FieldOptions), and proto message
packages depend on the protoreflect package, returning the actual option
type would cause a dependency cycle. Instead, we return an interface
value which can be type asserted to the appropriate concrete type.

Add options support to the prototype package.

Some of the prototype constructors included fields (such as
Field.IsPacked) which represent information from the options
(such as google.protobuf.FieldOptions.packed). To avoid confusion about
the canonical source of information, drop these fields in favor of the
options.

Drop the unimplemented Descriptor.DescriptorOptionsProto.

Change-Id: I66579b6a7d10d99eb6977402a247306a78913e74
Reviewed-on: https://go-review.googlesource.com/c/144277
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-23 23:44:11 +00:00
Joe Tsai
90fe996bd1 internal/impl: derive descriptors for legacy enums and messages
In order for the v2 rollout to be as seamless as possible, we need to support
the situation where a v2 message depends on some other generated v1 message that
may be stale and does not support the v2 API. In such a situation, there needs
to be some way to wrap a legacy message or enum in such a way that it satisfies
the v2 API.

This wrapping is comprised of two parts:
1) Deriving an enum or message descriptor
2) Providing an reflection implementation for messages

This CL addresses part 1 (while part 2 has already been partially implemented,
since the implementation applies to both v1 and v2).

To derive the enum and message descriptor we rely on a mixture of parsing the
raw descriptor proto and also introspection on the fields in the message.
Methods for obtaining the raw descriptor protos were added in February, 2016,
and so has not always been available. For that reason, we attempt to derive
as much information from the Go type as possible.

As part of this change, we modify prototype to be able to create multiple
standalone messages as a set. This is needed since cyclic dependencies is allowed
between messages within a single proto file.

Change-Id: I71aaf5f977faf9fba03c370b1ee17b3758ce60a6
Reviewed-on: https://go-review.googlesource.com/c/143539
Reviewed-by: Damien Neil <dneil@google.com>
2018-10-22 17:18:39 +00:00
Joe Tsai
3903b218df reflect/protoreflect: remove List method
Remove List from KnownFields, UnknownFields, ExtensionFieldTypes, and Map.

Rationale:
* Each of those interfaces already have a Range method, which provides
a superset of the functionality of List. Furthermore, Range is more expressive
as it allows you to terminate iteration and provides both keys and values.
* List must always allocate a slice and populates it.
* Range is allocation free in some cases. For example, if you simply wanted to
iterate over all the populated fields to clear them, there is no need for a
closure, so a static version of the function can be directly referenced
(i.e., there is no need to create a stub function header that references the
closed-over variables).
* In the cases where a closure is needed, the allocation cost is O(1) since
there are a finite number of variables being closed over.
* In highly performance sensitive cases, the closured function could close over
a struct, such that the function and struct are stored in a sync.Pool when not
in use. For example:

	type MapLister struct {
		Entries []struct{K MapKey; V Value}
		f       func(MapKey, Value) true
	}
	func (m *MapLister) Ranger() func(MapKey, Value) bool {
		if m.f != nil {
			m.f = func(k MapKey, v Value) bool {
				m.Entries = append(m.Entries, ...)
				return true
			}
		}
		m.Entries = m.Entries[:0]
		return m.f
	}

The main benefit of List is the ease of use:

	for _, num := range knownFields.List() {
		...
	}

as opposed to:

	knownFields.Range(func(n FieldNumber, v Value) bool {
		...
		return true
	})

However, this is a marginal benefit.
Thus, remove List as it mostly inferior to Range.

Change-Id: I25586c6ea07a4706072ba06b1cf25cb6efb5e8a7
Reviewed-on: https://go-review.googlesource.com/c/142888
Reviewed-by: Damien Neil <dneil@google.com>
2018-10-17 19:30:39 +00:00
Damien Neil
21f62f4d53 reflect/prototype: fix name resolution when file has no package name
A .proto source file with no 'package' statement may still contain
references to descriptors within the file.

Change-Id: I86e942c9c06e5a2915e9722162e0455ffa9ba2ab
Reviewed-on: https://go-review.googlesource.com/c/140899
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-16 16:16:46 +00:00
Joe Tsai
1dab2cb0ad reflect/prototype: avoid unescaping string defaults
According to google.protobuf.FieldDescriptorProto.default_value documentation:
>>>
For strings, contains the default text contents (not escaped in any way).
For bytes, contains the C escaped value.  All bytes >= 128 are escaped.
<<<

Thus, use the default value ad-verbatim if the kind is a string.
For the bytes kind, we use the text format hack to unescape the string,
since the text format uses the same unescape syntax as C due to its heritage.

Change-Id: I79e6d4d7e8f0d1ebedccecb9424a49384485589d
Reviewed-on: https://go-review.googlesource.com/138515
Reviewed-by: Damien Neil <dneil@google.com>
2018-09-28 20:50:27 +00:00
Joe Tsai
01ab29648e go.mod: rename google.golang.org/proto as github.com/golang/protobuf/v2
This change was created by running:
	git ls-files | xargs sed -i "s|google.golang.org/proto|github.com/golang/protobuf/v2|g"

This change is *not* an endorsement of "github.com/golang/protobuf/v2" as the
final import path when the v2 API is eventually released as stable.
We continue to reserve the right to make breaking changes as we see fit.

This change enables us to host the v2 API on a repository that is go-gettable
(since go.googlesource.com is not a known host by the "go get" tool;
and google.golang.org/proto was just a stub URL that is not currently served).
Thus, we can start work on a forked version of the v1 API that explores
what it would take to implement v1 in terms of v2 in a backwards compatible way.

Change-Id: Ia3ebc41ac4238af62ee140200d3158b53ac9ec48
Reviewed-on: https://go-review.googlesource.com/136736
Reviewed-by: Damien Neil <dneil@google.com>
2018-09-24 16:11:50 +00:00
Joe Tsai
ded92f8033 reflect/prototype: lower minimum support to Go1.9
The minimum supported version is currently Go1.10 since we use strings.Builder
for a fairly significant optimization when constructing all of the descriptor
full names.

The strings.Builder implementation is not particularly complicated,
so just fork it into our code. The golang/protobuf and the golang/go
projects share the same authors and copyright license.

Change-Id: Ibb9519dbe756327a07369f10f80c15761002b5e7
Reviewed-on: https://go-review.googlesource.com/136735
Reviewed-by: Damien Neil <dneil@google.com>
2018-09-21 19:30:05 +00:00
Damien Neil
d4803f5a4f cmd/protoc-gen-go: add "packed" encoding tag to fields
Change-Id: I423039d6c9edb50a07781a6e76336ed364f21866
Reviewed-on: https://go-review.googlesource.com/136359
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-20 17:08:44 +00:00
Joe Tsai
dd00a27837 reflect/prototype: simplify Format test
Derive the compact string from the multi string output.
This makes is such that we don't have to update two sets of golden data.

Change-Id: I94b94f1094b448539f8bee234be83ac7186fdd30
Reviewed-on: https://go-review.googlesource.com/135335
Reviewed-by: Herbie Ong <herbie@google.com>
2018-09-13 23:01:44 +00:00
Joe Tsai
d0f8aa294e reflect/prototype: add NewGoEnum and NewGoMessage constructors
Add constructors for creating protoreflect.{Enum,Message}Type,
which extend the protobuf descriptor with Go type information.

Rationale for API:
* NewGoEnum and NewGoMessage take in a struct instead of multiple
function parameters to provide future flexibility in adding more parameters.
* NewGoEnum and NewGoMessage take in a pointer to a struct so that generated
code can batch allocate these structs and convert them to them to EnumTypes
or MessageTypes efficiently. In other words, allocations are entirely in the
control of the caller.
* The user provides a New function as opposed to a reflect.Type or pointer to
the concrete type to support dynamic protobufs, where a single Go type may be
used to represent multiple protobuf types.
* The New functions take in an EnumType or MessageType since an implementation
for dynamic protobufs would need to close over these types. All statically
generated messages can ignore that argument.

There are no tests added since these are just wrappers;
they will be heavily exercised by other code.

Change-Id: I9fca44e06f3388375be7eaca94f8fd75c17233a3
Reviewed-on: https://go-review.googlesource.com/135356
Reviewed-by: Damien Neil <dneil@google.com>
2018-09-13 23:01:27 +00:00
Joe Tsai
757806cdda reflect/prototype: fix build due to protoreflect.Value method change
protoreflect.Value.IsNull was inverted to become IsValid in CL 134998.
However, that CL was submitted without a rebase,
which broke the build since CL 135255 added a few more usages of IsNull.

Change-Id: I5e52fb880ffc92e9d8f6cab30848e65b2d6dcbdc
Reviewed-on: https://go-review.googlesource.com/135337
Reviewed-by: Damien Neil <dneil@google.com>
2018-09-13 20:00:42 +00:00
Joe Tsai
812d9137df reflect/protoreflect: add Has and Clear to KnownFields and Map
Change the API to use explicit Has/Clear methods instead of relying on the
"invalid" form of Value to represent nullability.

There are several reasons for this change:

* Much of the ecosystem around protobufs do not care about nullability alone.
For example, for the encoder to determine whether to emit a field:
it needs to first check if a field is nulled, and if not, it still needs to go
through a series of type-assertion to check whether the value is the zero value.
It is much easier to be able to just call Has.

* It enables representing the default value as part of the value API, rather
than only as part of the descriptor API.

* The C++ API also uses explicit has and clear methods. However, we differ from
them by defining Has for proto3 scalars (while C++ panics instead).

For internal consistency, we also use a Has/Clear API for Maps.

Change-Id: I30eda482c959d3e1454d72d9fc761c761ace27a6
Reviewed-on: https://go-review.googlesource.com/134998
Reviewed-by: Damien Neil <dneil@google.com>
2018-09-13 19:52:11 +00:00
Damien Neil
e6fa22ab33 reflect/prototype: add HasDefault to FieldDescriptor
Provide a way to distinguish between a field with a zero-value
default and one with no default.

Change-Id: I4b1231de2d1bb141099cb1a415b35184dd198f93
Reviewed-on: https://go-review.googlesource.com/135255
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-13 18:13:20 +00:00
Damien Neil
2c8b670c6c reflect/protoreflect: put Value and MapKey types next to their methods
The Value and MayKey types are defined in value.go, but the methods on
these types are defined in value_union.go. It's not immediately clear
to the reader of the code that these types have methods, or where they
are defined.

Move the type definitions to value_union.go to keep the entire type in a
single place.

Change-Id: I7b3f3fc219a24a3b0236c2c3335e5d46f9086d25
Reviewed-on: https://go-review.googlesource.com/134997
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-12 22:16:46 +00:00
Joe Tsai
17764983d4 reflect/prototype: implement Format methods
Implement Format methods to provide the ability to pretty-print the descriptor
in a humanly readable fashion. While this functionality is not strictly necessary
for the operation of Go protobuf, it is a useful aid for humans.

Change-Id: I88807b38b5be713867f2f2aab5a0843fc147dc35
Reviewed-on: https://go-review.googlesource.com/131255
Reviewed-by: Herbie Ong <herbie@google.com>
2018-09-11 23:25:30 +00:00
Damien Neil
3de06c0868 reflect: add Index method to descriptor
Useful for dealing with SourceCodeInfo location paths, which identify
entities by their index.

Change-Id: I2034fc06b14c9b29b26e356fad21e106f63fbd14
Reviewed-on: https://go-review.googlesource.com/134115
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-07 17:57:37 +00:00
Damien Neil
ef18063941 reflect/prototype: default proto file syntax is "proto2"
protoc leaves FileDescriptorProto.Syntax unset when the syntax is
"proto2".

Change-Id: Id1370145fc9c5c67344f5a998d118fd0f9f65f24
Reviewed-on: https://go-review.googlesource.com/133636
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-06 15:52:05 +00:00
Joe Tsai
23ddbd1430 reflect/prototype: add NewFileFromDescriptorProto constructor
Implement NewFileFromDescriptorProto, which constructs a
protoreflect.FileDescriptor from a provided descriptor.FileDescriptorProto.

Some other minor changes:
* Allow calling Find and Range methods on nil *protoregistry.Files to match
the behavior of maps where index operations are permitted, but store panics.
* Switch protoregistry test to be protoregistry_test to avoid cyclic dependency.

Change-Id: I5536901ef5096014a3421e63bc4a9dfcad335ea4
Reviewed-on: https://go-review.googlesource.com/132455
Reviewed-by: Damien Neil <dneil@google.com>
2018-09-05 21:25:53 +00:00
Joe Tsai
095e462e1c reflect/protoregistry: initial commit
Package protoregistry provides data structures to register and lookup
protobuf descriptor types.

High-level API:
	var GlobalFiles = new(Files)
	var NotFound = errors.New("not found")
	type Files struct{ ... }
		func NewFiles(...pref.FileDescriptor) *Files
		func (*Files) Register(...pref.FileDescriptor) error
		func (*Files) FindDescriptorByName(pref.FullName) (pref.Descriptor, error)
		func (*Files) RangeFiles(func(pref.FileDescriptor) bool)
		func (*Files) RangeFilesByPackage(pref.FullName, func(pref.FileDescriptor) bool)
		func (*Files) RangeFilesByPath(string, func(pref.FileDescriptor) bool)

To support the FindDescriptorByName method, we add a DescriptorByName to
protoreflect.FileDescriptor and associated logic to prototype.

Change-Id: I14d65f74d2bd9f4f48641da9dfa70190310e5878
Reviewed-on: https://go-review.googlesource.com/129499
Reviewed-by: Damien Neil <dneil@google.com>
2018-08-24 23:46:08 +00:00
Joe Tsai
bfda014ecd reflect/prototype: initial commit
The prototype package provides constructors to create protobuf types that
implement the interfaces defined in the protoreflect package.

High-level API:
	func NewFile(t *File) (protoreflect.FileDescriptor, error)
	type File struct{ ... }
	type Message struct{ ... }
	type Field struct{ ... }
	type Oneof struct{ ... }
	type Enum struct{ ... }
	type EnumValue struct{ ... }
	type Extension struct{ ... }
	type Service struct{ ... }
	type Method struct{ ... }

	func NewEnum(t *StandaloneEnum) (protoreflect.EnumDescriptor, error)
	func NewMessage(t *StandaloneMessage) (protoreflect.MessageDescriptor, error)
	func NewExtension(t *StandaloneExtension) (protoreflect.ExtensionDescriptor, error)
	type StandaloneEnum struct{ ... }
	type StandaloneMessage struct{ ... }
	type StandaloneExtension struct{ ... }

	func PlaceholderFile(path string, pkg protoreflect.FullName) protoreflect.FileDescriptor
	func PlaceholderEnum(name protoreflect.FullName) protoreflect.EnumDescriptor
	func PlaceholderMessage(name protoreflect.FullName) protoreflect.MessageDescriptor

This CL is missing some features that are to be added later:
* The stringer methods are not implemented, providing no way to print the
descriptors in a humanly readable manner.
* There is no support for proto options or retrieving the raw descriptor.
* There are constructors for Go specific types (e.g., protoreflect.MessageType).

We drop go1.9 support since we use strings.Builder.
We switch to go.11rc1 to obtain some bug fixes for modules.

Change-Id: Ieac9a2530afc81e5a5bb9ab5816804372f652b18
Reviewed-on: https://go-review.googlesource.com/129057
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Herbie Ong <herbie@google.com>
2018-08-15 18:44:04 +00:00
Joe Tsai
dbc9a12dbd reflect/protoreflect: initial commit
Package protoreflect provides APIs for programatically interacting with messages
according to the protobuf type system.

Change-Id: I11fa3874e4b3f94771514c69efb2ae8bb812d7fa
Reviewed-on: https://go-review.googlesource.com/127823
Reviewed-by: Damien Neil <dneil@google.com>
2018-08-10 19:59:50 +00:00
Joe Tsai
27c2a76c85 internal/encoding/text: initial commit of proto text format parser/serializer
Package text provides a parser and serializer for the proto text format.
This focuses on the grammar of the format and is agnostic towards specific
semantics of protobuf types.

High-level API:
	func Marshal(v Value, indent string, delims [2]byte, outputASCII bool) ([]byte, error)
	func Unmarshal(b []byte) (Value, error)
	type Type uint8
		const Bool Type ...
	type Value struct{ ... }
		func ValueOf(v interface{}) Value
		func (v Value) Type() Type
		func (v Value) Bool() (x bool, ok bool)
		func (v Value) Int(b64 bool) (x int64, ok bool)
		func (v Value) Uint(b64 bool) (x uint64, ok bool)
		func (v Value) Float(b64 bool) (x float64, ok bool)
		func (v Value) Name() (protoreflect.Name, bool)
		func (v Value) String() string
		func (v Value) List() []Value
		func (v Value) Message() [][2]Value
		func (v Value) Raw() []byte

Change-Id: I4a78ec4474c160d0de4d32120651edd931ea2c1e
Reviewed-on: https://go-review.googlesource.com/127455
Reviewed-by: Herbie Ong <herbie@google.com>
2018-08-07 22:44:06 +00:00