14 Commits

Author SHA1 Message Date
Damien Neil
a485fbd353 reflect/protoreflect: add FieldDescriptor.DefaultEnumValue method
Default values for enums are specified by name, not number. An enum
may contain multiple values with different names but the same number.
Representing the default as a protoreflect.Value containing an EnumNumber
can discard information.

Add a method returning the EnumValueDescriptor.

Change-Id: If8beee3f81d41c4f9af45423252603b86949c7a5
Reviewed-on: https://go-review.googlesource.com/c/145158
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-29 15:59:23 +00:00
Joe Tsai
e2afdc27e7 internal/impl: support legacy unknown fields
Add wrapper data structures to get legacy XXX_unrecognized fields to support
the new protoreflect.UnknownFields interface. This is a challenge since the
field is a []byte, which does not give us much flexibility to work with
in terms of choice of data structures.

This implementation is relatively naive where every operation is O(n) since
it needs to strip through the entire []byte each time. The Range operation
operates slightly differently from ranging over Go maps since it presents a
stale version of RawFields should a mutation occur while ranging.
This distinction is unlikely to affect anyone in practice.

Change-Id: Ib3247cb827f9a0dd6c2192cd59830dca5eef8257
Reviewed-on: https://go-review.googlesource.com/c/144697
Reviewed-by: Damien Neil <dneil@google.com>
2018-10-26 19:55:21 +00:00
Joe Tsai
78f02a161f reflect/protoreflect: clarify Mutable semantics
The Mutable semantics are simpler than currently documented. In an older design
of the protobuf reflection API, it was intended that there could be a mutable
version of the scalar kinds. However, support for this has been dropped,
simplifying the documented semantics.

Change-Id: If66740ffa5da65a3f2c8cf46700e0c23393a16d7
Reviewed-on: https://go-review.googlesource.com/c/144857
Reviewed-by: Damien Neil <dneil@google.com>
2018-10-26 00:35:03 +00:00
Joe Tsai
620bc28bae reflect/protoreflect: clarify Range semantics
Restrict mutation operations during Range to only the current iteration key.
This simplifies the guarantees that any given implementation may need to provide.

While none of the range operations have a defined order, UnknownFields.Range
is special in that the iteration order is at least deterministic.
This exception occurs because there is no obviously correct way to re-order them
(since order of unknown fields can have semantic significance).

Change-Id: I35145e500dbc7c87be7270f0d886ef52e13d07d8
Reviewed-on: https://go-review.googlesource.com/c/144700
Reviewed-by: Damien Neil <dneil@google.com>
2018-10-25 23:23:31 +00:00
Damien Neil
204f1c0ad8 reflect/protoreflect: add Descriptor.Options method
Add a method to fetch descriptor options. Since options are proto
messages (e.g., google.protobuf.FieldOptions), and proto message
packages depend on the protoreflect package, returning the actual option
type would cause a dependency cycle. Instead, we return an interface
value which can be type asserted to the appropriate concrete type.

Add options support to the prototype package.

Some of the prototype constructors included fields (such as
Field.IsPacked) which represent information from the options
(such as google.protobuf.FieldOptions.packed). To avoid confusion about
the canonical source of information, drop these fields in favor of the
options.

Drop the unimplemented Descriptor.DescriptorOptionsProto.

Change-Id: I66579b6a7d10d99eb6977402a247306a78913e74
Reviewed-on: https://go-review.googlesource.com/c/144277
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-10-23 23:44:11 +00:00
Joe Tsai
3903b218df reflect/protoreflect: remove List method
Remove List from KnownFields, UnknownFields, ExtensionFieldTypes, and Map.

Rationale:
* Each of those interfaces already have a Range method, which provides
a superset of the functionality of List. Furthermore, Range is more expressive
as it allows you to terminate iteration and provides both keys and values.
* List must always allocate a slice and populates it.
* Range is allocation free in some cases. For example, if you simply wanted to
iterate over all the populated fields to clear them, there is no need for a
closure, so a static version of the function can be directly referenced
(i.e., there is no need to create a stub function header that references the
closed-over variables).
* In the cases where a closure is needed, the allocation cost is O(1) since
there are a finite number of variables being closed over.
* In highly performance sensitive cases, the closured function could close over
a struct, such that the function and struct are stored in a sync.Pool when not
in use. For example:

	type MapLister struct {
		Entries []struct{K MapKey; V Value}
		f       func(MapKey, Value) true
	}
	func (m *MapLister) Ranger() func(MapKey, Value) bool {
		if m.f != nil {
			m.f = func(k MapKey, v Value) bool {
				m.Entries = append(m.Entries, ...)
				return true
			}
		}
		m.Entries = m.Entries[:0]
		return m.f
	}

The main benefit of List is the ease of use:

	for _, num := range knownFields.List() {
		...
	}

as opposed to:

	knownFields.Range(func(n FieldNumber, v Value) bool {
		...
		return true
	})

However, this is a marginal benefit.
Thus, remove List as it mostly inferior to Range.

Change-Id: I25586c6ea07a4706072ba06b1cf25cb6efb5e8a7
Reviewed-on: https://go-review.googlesource.com/c/142888
Reviewed-by: Damien Neil <dneil@google.com>
2018-10-17 19:30:39 +00:00
Joe Tsai
01ab29648e go.mod: rename google.golang.org/proto as github.com/golang/protobuf/v2
This change was created by running:
	git ls-files | xargs sed -i "s|google.golang.org/proto|github.com/golang/protobuf/v2|g"

This change is *not* an endorsement of "github.com/golang/protobuf/v2" as the
final import path when the v2 API is eventually released as stable.
We continue to reserve the right to make breaking changes as we see fit.

This change enables us to host the v2 API on a repository that is go-gettable
(since go.googlesource.com is not a known host by the "go get" tool;
and google.golang.org/proto was just a stub URL that is not currently served).
Thus, we can start work on a forked version of the v1 API that explores
what it would take to implement v1 in terms of v2 in a backwards compatible way.

Change-Id: Ia3ebc41ac4238af62ee140200d3158b53ac9ec48
Reviewed-on: https://go-review.googlesource.com/136736
Reviewed-by: Damien Neil <dneil@google.com>
2018-09-24 16:11:50 +00:00
Joe Tsai
812d9137df reflect/protoreflect: add Has and Clear to KnownFields and Map
Change the API to use explicit Has/Clear methods instead of relying on the
"invalid" form of Value to represent nullability.

There are several reasons for this change:

* Much of the ecosystem around protobufs do not care about nullability alone.
For example, for the encoder to determine whether to emit a field:
it needs to first check if a field is nulled, and if not, it still needs to go
through a series of type-assertion to check whether the value is the zero value.
It is much easier to be able to just call Has.

* It enables representing the default value as part of the value API, rather
than only as part of the descriptor API.

* The C++ API also uses explicit has and clear methods. However, we differ from
them by defining Has for proto3 scalars (while C++ panics instead).

For internal consistency, we also use a Has/Clear API for Maps.

Change-Id: I30eda482c959d3e1454d72d9fc761c761ace27a6
Reviewed-on: https://go-review.googlesource.com/134998
Reviewed-by: Damien Neil <dneil@google.com>
2018-09-13 19:52:11 +00:00
Damien Neil
e6fa22ab33 reflect/prototype: add HasDefault to FieldDescriptor
Provide a way to distinguish between a field with a zero-value
default and one with no default.

Change-Id: I4b1231de2d1bb141099cb1a415b35184dd198f93
Reviewed-on: https://go-review.googlesource.com/135255
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-13 18:13:20 +00:00
Damien Neil
2c8b670c6c reflect/protoreflect: put Value and MapKey types next to their methods
The Value and MayKey types are defined in value.go, but the methods on
these types are defined in value_union.go. It's not immediately clear
to the reader of the code that these types have methods, or where they
are defined.

Move the type definitions to value_union.go to keep the entire type in a
single place.

Change-Id: I7b3f3fc219a24a3b0236c2c3335e5d46f9086d25
Reviewed-on: https://go-review.googlesource.com/134997
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-12 22:16:46 +00:00
Damien Neil
3de06c0868 reflect: add Index method to descriptor
Useful for dealing with SourceCodeInfo location paths, which identify
entities by their index.

Change-Id: I2034fc06b14c9b29b26e356fad21e106f63fbd14
Reviewed-on: https://go-review.googlesource.com/134115
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-07 17:57:37 +00:00
Joe Tsai
095e462e1c reflect/protoregistry: initial commit
Package protoregistry provides data structures to register and lookup
protobuf descriptor types.

High-level API:
	var GlobalFiles = new(Files)
	var NotFound = errors.New("not found")
	type Files struct{ ... }
		func NewFiles(...pref.FileDescriptor) *Files
		func (*Files) Register(...pref.FileDescriptor) error
		func (*Files) FindDescriptorByName(pref.FullName) (pref.Descriptor, error)
		func (*Files) RangeFiles(func(pref.FileDescriptor) bool)
		func (*Files) RangeFilesByPackage(pref.FullName, func(pref.FileDescriptor) bool)
		func (*Files) RangeFilesByPath(string, func(pref.FileDescriptor) bool)

To support the FindDescriptorByName method, we add a DescriptorByName to
protoreflect.FileDescriptor and associated logic to prototype.

Change-Id: I14d65f74d2bd9f4f48641da9dfa70190310e5878
Reviewed-on: https://go-review.googlesource.com/129499
Reviewed-by: Damien Neil <dneil@google.com>
2018-08-24 23:46:08 +00:00
Joe Tsai
dbc9a12dbd reflect/protoreflect: initial commit
Package protoreflect provides APIs for programatically interacting with messages
according to the protobuf type system.

Change-Id: I11fa3874e4b3f94771514c69efb2ae8bb812d7fa
Reviewed-on: https://go-review.googlesource.com/127823
Reviewed-by: Damien Neil <dneil@google.com>
2018-08-10 19:59:50 +00:00
Joe Tsai
27c2a76c85 internal/encoding/text: initial commit of proto text format parser/serializer
Package text provides a parser and serializer for the proto text format.
This focuses on the grammar of the format and is agnostic towards specific
semantics of protobuf types.

High-level API:
	func Marshal(v Value, indent string, delims [2]byte, outputASCII bool) ([]byte, error)
	func Unmarshal(b []byte) (Value, error)
	type Type uint8
		const Bool Type ...
	type Value struct{ ... }
		func ValueOf(v interface{}) Value
		func (v Value) Type() Type
		func (v Value) Bool() (x bool, ok bool)
		func (v Value) Int(b64 bool) (x int64, ok bool)
		func (v Value) Uint(b64 bool) (x uint64, ok bool)
		func (v Value) Float(b64 bool) (x float64, ok bool)
		func (v Value) Name() (protoreflect.Name, bool)
		func (v Value) String() string
		func (v Value) List() []Value
		func (v Value) Message() [][2]Value
		func (v Value) Raw() []byte

Change-Id: I4a78ec4474c160d0de4d32120651edd931ea2c1e
Reviewed-on: https://go-review.googlesource.com/127455
Reviewed-by: Herbie Ong <herbie@google.com>
2018-08-07 22:44:06 +00:00