This change removes most usages of Syntax() from the repository and uses
edition features for instead. The appropriate edition feature defaults are
loaded for proto2/proto3 when the initialization of the descriptors
start.
All of these changes were tested on the Google corpus.
Change-Id: Ieca076a2b38ca8e50e084cd32e725b7b3dcb4171
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/572435
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Add a new TextName accessor that returns the field name that should
be used for the text format. It is usually just the field name, except:
1) it uses the inlined message name for groups,
2) uses the full name surrounded by brackets for extensions, and
3) strips the "message_set_extension" for well-formed extensions
to the proto1 MessageSet.
We make similar adjustments to the JSONName accessor so that it applies
similar semantics for extensions.
The two changes simplifies all logic that wants the humanly readable
name for a field.
Change-Id: I524b6e017fb955146db81819270fe197f8f97980
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/239838
Reviewed-by: Herbie Ong <herbie@google.com>
In the upcoming 3.12.x release of protoc, the proto3 language will be
amended to support true presence for scalars. This CL adds support
to both the generator and runtime to support these semantics.
Newly added public API:
protogen.Plugin.SupportedFeatures
protoreflect.FieldDescriptor.HasPresence
protoreflect.FieldDescriptor.HasOptionalKeyword
protoreflect.OneofDescriptor.IsSynthetic
Change-Id: I7c86bf66d0ae56642109beb5f2132184593747ad
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/230698
Reviewed-by: Damien Neil <dneil@google.com>
Add support for an ExtensionDesc with only Field populated as returned by
protoV1.ExtensionDescs. Rather than panicking when TypeDescriptor is called,
return a placeholder that preserves the name and/or number.
Change-Id: I60352a7aec8ccd8a0c1fb08db5891043a441695f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/193725
Reviewed-by: Damien Neil <dneil@google.com>
This CL:
* Make the meaning of impl/ExtensionInfo.goType consistent. Before,
it was sometimes a T and other times a []T depending on the current
state of initialization. Change it so that it is the constructor's
responsibility to pass in a []T if it is repeated.
* Make internal/filetype responsible for constructing a []T for
repeated extension fields.
* Makes filedesc/Extension.Cardinality one of the eagerly initialized
pieces of information since it is useful to internal/filetype.
* Unify ExtensionInfo.desc and ExtensionInfo.tdesc.ExtensionField,
which held the same information.
* Remove the internal implementation for impl.X.ExtensionDescFromType
since we are dropping support for this from v1.
Change-Id: Ie95c4de66cd674c1d886da4f63b133b7d763c7ef
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/195777
Reviewed-by: Damien Neil <dneil@google.com>
This interface is only ever used within the impl package.
Unexport it to avoid potential abuse by users.
Change-Id: I58d0cc2e8cb01c8c9d36cdec469c8d2196c4f836
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/193197
Reviewed-by: Damien Neil <dneil@google.com>
Descriptor methods generally return a Descriptor with no Go type
information. ExtensionType's Descriptor is an exception, returning an
ExtensionTypeDescriptor containing both the proto descriptor and a
reference back to the ExtensionType. The pure descriptor is accessed
by xt.Descriptor().Descriptor().
Rename ExtensionType's Descriptor method to TypeDescriptor to make it
clear that it behaves a bit differently.
Change 1/2: Add the TypeDescriptor method and deprecate Descriptor.
Change-Id: I1806095044d35a474d60f94d2a28bdf528f12238
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/192139
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
In order to migrate v1 to wrap v2, we need a way to reproduce
the awful enum "names" that v1 used, which was the concatenation of
the proto package with the Go identifier used for the enum.
To support this:
* Move the camel case logic from compiler/protogen to internal/strs
* Add a small stub in internal/impl to expose this functionality
Change-Id: I8ff31daa9ae541e5788dc04d2e89eae1574877e4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/191637
Reviewed-by: Damien Neil <dneil@google.com>
Too much code inside Google rely on ExtensionDescV1.Name being the
target message name rather than the real name of the extension.
Double down on this assumption consistently across our codebase.
Change-Id: If0b7cd16dd1f2f7acfb2e562dbf1a2b5487162b5
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186980
Reviewed-by: Herbie Ong <herbie@google.com>
Change protoV1.ExtensionDesc to directly implement ExtensionType
rather than delegating to one.
Unify the previous types protoiface.ExtensionDescV1 and
filetype.Extension in impl.ExtensionInfo. The protoV1.ExtensionDesc
type becomes an alias to ExtensionInfo.
This gives us:
- Just one implementation of ExtensionType.
- Generated foopb.E_Ext vars are canonical ExtensionTypes.
- Generated foopb.E_Ext vars are also v1.ExtensionDescs for backwards
compatibility.
- Conversion between legacy and modern representations happens
transparently when lazily initializing an ExtensionInfo.
Overall, a simplification for users of generated code, since they can
mostly ignore the ExtensionDesc/ExtentionType distinction and use the
same value in either the old or new API.
This is change 3/5 in a series of commits changing protoV1.ExtensionDesc
to directly implement protoreflect.ExtensionType.
1. [v2] Add protoimpl.ExtensionInfo as an alias for
protoiface.ExtensionDescV1.
2. [v1] Update references to protoimpl.ExtensionInfo to use
protoiface.ExtensionInfo.
3. [v2] Create protoimpl.ExtensionInfo (an alias to a new type in
the impl package) and remove protoiface.ExtensionDescV1.
4. [v1] Remove unneeded explicit conversions between ExtensionDesc and
ExtensionType (since the former now directly implements the latter).
5. [v2] Remove stub conversion functions.
Change-Id: I96ee890541ec11b2412e1a72c9d7b96e4d7f66b4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189563
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Change protoiface.ExtensionDescV1 to implement protoreflect.ExtensionType.
ExtensionDescV1's Name field conflicts with the Descriptor Name method,
so change the protoreflect.{Message,Enum,Extension}Type types to no
longer implement the corresponding Descriptor interface. This also leads
to a clearer distinction between the two types.
Introduce a protoreflect.ExtensionTypeDescriptor type which bridges
between ExtensionType and ExtensionDescriptor.
Add extension accessor functions to the proto package:
proto.{Has,Clear,Get,Set}Extension. These functions take a
protoreflect.ExtensionType parameter, which allows writing the
same function call using either the old or new API:
proto.GetExtension(message, somepb.E_ExtensionFoo)
Fixesgolang/protobuf#908
Change-Id: Ibc65d12a46666297849114fd3aefbc4a597d9f08
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189199
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
We occasionally need to work with immutable, empty lists, maps, and
messages. Notably, Message.Get on an empty repeated field will return a
"frozen" empty value.
Move handling of these immutable, zero-length composites into Converter,
to unify the behavior of regular and extension fields.
Add a Zero method to Converter, MessageType, and ExtensionType, to
provide a consistent way to get an empty, frozen value of a composite
type. Adding this method to the public {Message,Extension}Type
interfaces does increase our API surface, but lets us (for example)
cleanly represent an empty map as a nil map rather than a non-nil
one wrapped in a frozenMap type.
Drop the frozen{List,Map,Message} types as no longer necessary.
(These types did have support for creating a read-only view of a
non-empty value, but we are not currently using that feature.)
Change-Id: Ia76f149d591da07b40ce75b7404a7ab8a60cb9d8
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189339
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
A Converter converts between reflect.Values and protoreflect.Values.
The existing usage of Converter is somewhat confusing: The
internal/value package creates Converters for scalar types only, the
internal/impl package creates Converters for legacy messages and enums,
and the reflect/prototype package creates Converters for repeated fields.
Change the Converter type to an interface. The constructor for
Converter takes a FieldDescriptor and reflect.Type, and directly
handles conversions for all field types: Scalars, lists, maps, and
legacy types.
Move Converter into the internal/impl package, since that package
contains the necessary support for dealing with legacy messages and
enums. Drop the internal/value package.
Replace two uses of prototype.Extension with more focused
implementations, since the implementation is trivial with the
refactored Converter. Drop prototype.Extension for the moment since
it is now unused.
Change-Id: If0c570fefac002cc5925b3d56281b6eb17e90d5f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/187857
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
CL/174938 removed these methods in favor of a method that returned
only the descriptors. This CL adds back in the Type methods alongside
the Descriptor methods.
In a vast majority of protobuf usages, only the descriptor information
is needed. However, there is a small percentage that legitimately needs
the Go type information. We should provide both, but document that the
descriptor-only information is preferred.
Change-Id: Ia0a098997fb1bd009994940ae8ea5257ccd87cae
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184578
Reviewed-by: Damien Neil <dneil@google.com>
The internal/fileinit package is split apart into two packages:
* internal/filedesc constructs descriptors from the raw proto.
It is very similar to the previous internal/fileinit package.
* internal/filetype wraps descriptors with Go type information
Overview:
* The internal/fileinit package will be deleted in a future CL.
It is kept around since the v1 repo currently depends on it.
* The internal/prototype package is deleted. All former usages of it
are now using internal/filedesc instead. Most significantly,
the reflect/protodesc package was almost entirely re-written.
* The internal/impl package drops support for messages that do not
have a Descriptor method (pre-2016). This removes a significant amount
of technical debt.
filedesc.Builder to parse raw descriptors.
* The internal/encoding/defval package now handles enum values by name.
Change-Id: I3957bcc8588a70470fd6c7de1122216b80615ab7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/182360
Reviewed-by: Damien Neil <dneil@google.com>
The internal/legacy package was originally separated out from internal/impl
to avoid a cyclic dependency on descriptor proto. However, the dependency
that legacy has on descriptor has long been dropped such that we can
now merge the two packages together again.
All legacy related logic are in a file with a legacy prefix.
Change-Id: I2424fc0f50721696ad06fa7cebb9bdd0babea13c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/178542
Reviewed-by: Damien Neil <dneil@google.com>
Both legacy_unknown.go and legacy_extension.go deal with the
current data structure for unknown fields and extensions.
While the current data structure is sub-optimial,
strictly speaking, they are not "legacy" since they are very much
used by the v2 API as well.
Change-Id: I3852dba0dcebaee00bbeb94d9936cd759f774f0a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/178477
Reviewed-by: Damien Neil <dneil@google.com>
CL/177620 modifies v1 to stop touching the Value field directly,
such that it is now unexport the Value field.
Change-Id: I0a05bbe59146862fc77c261349d7d90d6fa312e0
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/177621
Reviewed-by: Damien Neil <dneil@google.com>
Once the ExtensionFieldV1.Value field is unexported, it forces all
read operations to go through GetValue, which can perform some form
of lazy retrieval.
The SetEagerValue method is equivalent to setting the Value directly.
The SetLazyValue method permits setting a lazy equivalent of the Value
by simply storing a function closure returning the value.
Change-Id: I410f74924f0957f5eadc82eea3669ca335a02701
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/177619
Reviewed-by: Damien Neil <dneil@google.com>
Temporarily remove go.mod, since we can't generate an accurate one until
the corresponding v1 change is submitted.
Change-Id: I1e1ad97f2b455e33f61ffaeb8676289795e47e72
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/177000
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Added API:
FieldDescriptor.IsExtension
FieldDescriptor.IsList
FieldDescriptor.MapKey
FieldDescriptor.MapValue
FieldDescriptor.ContainingOneof
FieldDescriptor.ContainingMessage
Deprecated API (to be removed in subsequent CL):
FieldDescriptor.Oneof
FieldDescriptor.Extendee
These methods help cleanup several common usage patterns.
Change-Id: I9a3ffabc2edb2173c536509b22f330f98bba7cf3
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/176977
Reviewed-by: Damien Neil <dneil@google.com>
Added methods:
Enum.Descriptor
Message.Descriptor
EnumType.Descriptor
MessageType.Descriptor
ExtensionType.Descriptor
Message.New
All functionality is switched over to use those methods instead of
implicitly relying on the fact that {Enum,Message}Type implicitly
implement the associated descriptor interface.
This CL does not yet remove {Enum,Message}.Type or prevent
{Enum,Message,Extension}Type from implementating a descriptor.
That is a subsequent CL.
The Message.New method is also added to replace functionality
that will be lost when the Type methods are removed.
Change-Id: I7fefde1673bbd40bfdac489aca05cec9a6c98eb1
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/174918
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Herbie Ong <herbie@google.com>
CL/175579 switches v2 to use XXX_unrecognized exclusively.
CL/175838 switches v1 to use XXX_unrecognized exclusively.
This means that it is now safe to delete the Raw field.
Change-Id: Ic60c58147c2df4078e72c34d1202d226a7ea4bed
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/175839
Reviewed-by: Damien Neil <dneil@google.com>
The protobuf data model makes no distinction between unknown fields
that are within the extension field ranges or not. Now that we eagerly
unmarshal extensions, there is even less need for storing unknown
fields in the extension map. Instead, use the XXX_unrecognized field
exclusively for this purpose.
Change-Id: I673a7d6259fe9fdbdc295bcfa8252ef4da415343
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/175579
Reviewed-by: Damien Neil <dneil@google.com>
The protobuf type system uses the word "descriptor" instead of "type".
We should avoid the "type" verbage when we aren't talking about Go types.
The old names are temporarily kept around for compatibility reasons.
Change-Id: Icc99c913528ead011f7a74aa8399d9c5ec6dc56e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172238
Reviewed-by: Herbie Ong <herbie@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
The v2 implementation will eagerly unmarshal extension fields,
which means that the logic to synchronize the extensions fields
is no longer necessary. Using type aliases, we can ensure
impl.ExtensionsV1 is identical to the extensions map prior to
a4ab9ec5 from May 2016. This makes it such that we only have one
legacy implementation that we have to deal with.
Change-Id: I41b977e912fbb899c5bc4f055890d474b419429c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172240
Reviewed-by: Damien Neil <dneil@google.com>
Add a method that provides efficiently querying for which member field
in a oneof is actually set. This is useful when dealing with oneofs
with many member fields.
Change-Id: I918b566c432f8bdd24dcecbb5501d231ffefef29
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170580
Reviewed-by: Damien Neil <dneil@google.com>
As a goal, v2 should not depend on v1. As another step towards that end,
we move all the types that used to be in the v1 protoapi package over to v2.
For now, we place MessageV1, ExtensionRangeV1, and ExtensionDescV1
in runtime/protoiface since these are types that generated messages will
probably have to reference forever. An alternative location could be
reflect/protoreflect, but it seems unfortunate to have to dirty the
namespace of that package with these types.
We move ExtensionFieldV1, ExtensionFieldsV1, and ExtensionFieldsOf
to internal/impl, since these are related to the implementation of a
generated message.
Since moving these types from v1 to v2 implies that the v1 protoapi
package is useless, we update all usages of v1 protoapi in the v2
repository to point to the relevant v2 type or functionality.
CL/168538 is the corresponding change to alter v1.
There will be a temporary build failure as it is not possible
to submit CL/168519 and CL/168538 atomically.
Change-Id: Ide4025c1b6af5b7f0696f4b65b988b4d10a50f0b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168519
Reviewed-by: Herbie Ong <herbie@google.com>
Unfortunately a good amount of code uses pointer comparisons on the
v1 ExtensionDesc to determine exactly which extension field is set,
rather than checking whether the extension descriptor semantically
describes the field that they are interested in.
To preserve this behavior in v1, we need a 1:1 mapping between
a v2 ExtensionType and a specific v1 ExtensionDesc.
Change-Id: I852b3cefb4585bd656e48e5adad6cc28795d02df
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/167759
Reviewed-by: Damien Neil <dneil@google.com>
Don't rely on legacyWrapper.ExtensionTypeFromDesc when desc.Type
is populated. Avoids panics when working with extensions when
internal/legacy has not been imported.
Change-Id: Id41ffa3c60b6e5503d0a3e29ef48e4f523fad588
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/167465
Reviewed-by: Joe Tsai <joetsai@google.com>
This CL takes a significantly different approach to generating support
for protobuf reflection. The previous approach involved generating a
large number of Go literals to represent the reflection information.
While that approach was correct, it resulted in too much binary bloat.
The approach taken here initializes the reflection information from
the raw descriptor proto, which is a relatively dense representation
of the protobuf reflection information. In order to keep initialization
cost low, several measures were taken:
* At program init, the bare minimum is parsed in order to initialize
naming information for enums, messages, extensions, and services declared
in the file. This is done because those top-level declarations are often
relevant for registration.
* Only upon first are most of the other data structures for protobuf
reflection actually initialized.
* Instead of using proto.Unmarshal, a hand-written unmarshaler is used.
This allows us to avoid a dependendency on the descriptor proto and also
because the API for the descriptor proto is fundamentally non-performant
since it requires an allocation for every primitive field.
At a high-level, the new implementation lives in internal/fileinit.
Several changes were made to other parts of the repository:
* cmd/protoc-gen-go:
* Stop compressing the raw descriptors. While compression does reduce
the size of the descriptors by approximately 2x, it is a pre-mature
optimization since the descriptors themselves are around 1% of the total
binary bloat that is due to generated protobufs.
* Seeding protobuf reflection from the raw descriptor significantly
simplifies the generator implementation since it is no longer responsible
for constructing a tree of Go literals to represent the same information.
* We remove the generation of the shadow types and instead call
protoimpl.MessageType.MessageOf. Unfortunately, this incurs an allocation
for every call to ProtoReflect since we need to allocate a tuple that wraps
a pointer to the message value, and a pointer to message type.
* internal/impl:
* We add a MessageType.GoType field and make it required that it is
set prior to first use. This is done so that we can avoid calling
MessageType.init except for when it is actually needed. The allows code
to call (*FooMessage)(nil).ProtoReflect().Type() without fearing that the
init code will run, possibly triggering a recursive deadlock (where the
init code depends on getting the Type of some dependency which may be
declared within the same file).
* internal/cmd/generate-types:
* The code to generate reflect/prototype/protofile_list_gen.go was copied
and altered to generated internal/fileinit.desc_list_gen.go.
At a high-level this CL adds significant technical complexity.
However, this is offset by several possible future changes:
* The prototype package can be drastically simplified. We can probably
reimplement internal/legacy to use internal/fileinit instead, allowing us
to drop another dependency on the prototype package. As a result, we can
probably delete most of the constructor types in that package.
* With the prototype package significantly pruned, and the fact that generated
code no longer depend on depends on that package, we can consider merging
what's left of prototype into protodesc.
Change-Id: I6090f023f2e1b6afaf62bd3ae883566242e30715
Reviewed-on: https://go-review.googlesource.com/c/158539
Reviewed-by: Herbie Ong <herbie@google.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This change preserves consistency with CL/157077,
where New returns a value closer to the reflective type.
Change-Id: I85bfdae24e1ce1a10c3c7b939420fa1043bff743
Reviewed-on: https://go-review.googlesource.com/c/157078
Reviewed-by: Damien Neil <dneil@google.com>
Most usages of New actually prefer to interact with the reflective view
rather than the native Go type. Thus, change New to return that instead.
This parallels reflect.New, which returns the reflective view
(i.e., reflect.Value) instead of native type (i.e., interface{}).
We make the equivalent change to KnownFields.NewMessage, List.NewMessage,
and Map.NewMessage for consistency.
Since this is a subtle change where the type system will not always
catch the changed type, this change was made by both changing the type
and renaming the function to NewXXX and manually looking at every usage
of the the function to ensure that the usage correctly operates
on either the native Go type or the reflective view of the type.
After the entire codebase was cleaned up, a rename was performed to convert
NewXXX back to New.
Change-Id: I153fef627b4bf0a427e4039ce0aaec52e20c7950
Reviewed-on: https://go-review.googlesource.com/c/157077
Reviewed-by: Damien Neil <dneil@google.com>
Remove the Mutable methods from KnownFields, List, and Map, replacing
them with methods which return a new, empty message value without adding
that value to the collection.
The new API is simpler, since it clearly applies only to message values,
and more orthogonal, since it provides a way to create a value without
mutating the collection. This latter point is particularly useful in
map deserialization, where the key may be unknown at the time the value
is deserialized.
Drop the Mutable interface, since it is no longer necessary.
Change-Id: Ic5f3d06a2aa331a5d5cd2b4e670a3dba4a74f77c
Reviewed-on: https://go-review.googlesource.com/c/153278
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Similar to how generated messages allow you to call Get methods on a
nil pointer, we permit similar functionality when protobuf reflection
is used on a nil pointer.
Change-Id: Ie2f596d39105c191073b42d7d689525c3b715240
Reviewed-on: https://go-review.googlesource.com/c/152021
Reviewed-by: Damien Neil <dneil@google.com>
The impl package currently supports wrapping legacy v1 enums and messages
so that they implement the v2 reflective APIs. This functionality is necessary
for v1 and v2 to interoperate. However, the existence of this functionality
presents several problems:
* A significant portion of the complexity in impl is for legacy wrapping.
* This complexity is linked into a Go binary even if all the other messages
in the binary natively support v2 reflection.
* It presents a cyclic dependency when trying to generate descriptor proto.
Suppose you are generating descriptor.proto. The generated code would want to
depend on the impl package because impl is the runtime implementation for
protobuf messages. However, impl currently depends depends on descriptor in
order to wrap legacy enum and messages since it needs the ability to dynamically
create new protobuf descriptor types. In the case of descriptor.proto, it would
presumably be generated with native reflection support, so the legacy wrapping
logic is unneccessary.
To break the dependency of impl on descriptor, we move the legacy support logic
to a different package and instead add hooks in impl so that legacy support could
be dynamically registered at runtime. This is dependency injection.
Change-Id: I01a582908ed5629993f6699e9bf2f4bee93857a4
Reviewed-on: https://go-review.googlesource.com/c/151877
Reviewed-by: Herbie Ong <herbie@google.com>
Add a Proto prefix before the Unwrap method to reduce the probability that
it would ever conflict with a method of the same name that a
custom implementation of Enum, Message, List, or Map may have.
Change-Id: I628bf8335583f2747ab4589f3e6ff82e4501ce98
Reviewed-on: https://go-review.googlesource.com/c/151817
Reviewed-by: Herbie Ong <herbie@google.com>
The new v1 protoapi package enables:
* Referencing types in the protoapi package instead of protoV1, which further
reduces the number of situations where we need to depend on protoV1.
This is for the goal of eventually breaking all cases where the v2 implementation
relies on v1, so that in the near future, proto v1 can rely on proto v2 instead.
* Removes the need for legacy_extension_hack.go since that functionality has now
been exported into the protoapi package.
Change-Id: If71002d9ec711bfabfe494636829df9abf19e23e
Reviewed-on: https://go-review.googlesource.com/c/151403
Reviewed-by: Herbie Ong <herbie@google.com>
Clearly specify that Get on an unpopulated field:
* returns the default value for scalars
* returns a mutable (but empty) List for repeated fields
* returns a mutable (but empty) Map for map fields
* returns an invalid value for message fields
The difference in semantics between List+Maps and Messages is because
protobuf semantics provide no distinction between an unpopulated and empty list
or map. On the other hand, there is a semantic difference between an unpopulated
message and an empty message.
Default values for scalars is trivial to implement with FieldDescriptor.Default.
A mutable, but empty List and Map is easy to implement for known fields since
known fields are generated as a slice or map field in a struct.
Since struct fields are addressable, the implementation can just return a
reference to the slice or map.
Repeated, extension fields are a little more tricky since extension fields
are implemented under the hood as a map[FieldNumber]Extension.
Rather than allocating an empty list in KnownFields.Get upon first retrieval
(which presents a race), delegate the work to ExtensionFieldTypes.Register,
which must occur before any Get operation. Register is not a concurrent-safe
operation, so that is an excellent time to initilize empty lists.
The implementation of extensions will need to be careful that Clear on a repeated
field simply truncates it zero instead of deleting the object.
For unpopulated messages, we return an invalid value, instead of the prior
behavior of returning a typed nil-pointer to the Go type for the message.
The approach is problematic because it assumes that
1) all messages are always implemented on a pointer reciever
2) a typed nil-pointer is an appropriate "read-only, but empty" message
These assumptions are not true of all message types (e.g., dynamic messages).
Change-Id: Ie96e6744c890308d9de738b6cf01d3b19e7e7c6a
Reviewed-on: https://go-review.googlesource.com/c/150319
Reviewed-by: Damien Neil <dneil@google.com>
The terminology Vector does not occur in protobuf documentation at all,
so we should rename the Go use of the term to something more recognizable.
As such, all instances that match the regexp "[Vv]ect(or)?" were replaced.
The C++ documentation uses the term "Repeated", which is a reasonable name.
However, the term became overloaded in 2014, when maps were added as a feature
and implementated under the hood as repeated fields. This is confusing as it
means "repeated" could either refer to repeated fields proper (i.e., explicitly
marked with the "repeated" label in the proto file) or map fields. In the case
of the C++ reflective API, this is not a problem since repeated fields proper
and map fields are interacted with through the same RepeatedField type.
In Go, we do not use a single type to handle both types of repeated fields:
1) We are coming up with the Go protobuf reflection API for the first time
and so do not need to piggy-back on the repeated fields API to remain backwards
compatible since no former usages of Go protobuf reflection exists.
2) Map fields are commonly represented in Go as the Go map type, which do not
preserve ordering information. As such it is fundamentally impossible to present
an unordered map as a consistently ordered list. Thus, Go needs two different
interfaces for lists and maps.
Given the above situation, "Repeated" is not a great term to use since it
refers to two different things (when we only want one of the meanings).
To distinguish between the two, we'll use the terms "List" and "Map" instead.
There is some precedence for the term "List" in the protobuf codebase
(e.g., "getRepeatedInt32List").
Change-Id: Iddcdb6b78e1e60c14fa4ca213c15f45e214b967b
Reviewed-on: https://go-review.googlesource.com/c/149657
Reviewed-by: Damien Neil <dneil@google.com>
Implement support for extension fields for messages that use the v1
data structures for extensions. The legacyExtensionFields type wraps a
v1 map to implement the v2 protoreflect.KnownFields interface.
Working on this change revealed a bug in the dynamic construction of
message types for protobuf messages that had cyclic dependencies (e.g.,
message Foo has a sub-field of message Bar, and Bar has a sub-field of Foo).
In such a situation, a deadlock occurs because initialization code depends on
the very initialization code that is currently running. To break these cycles,
we make some systematic changes listed in the following paragraphs.
Generally speaking, we separate the logic for construction and wrapping,
where constuction does not recursively rely on dependencies,
while wrapping may recursively inspect dependencies.
Promote the MessageType.MessageOf method as a standalone MessageOf function
that dynamically finds the proper *MessageType to use. We make it such that
MessageType only supports two forms of messages types:
* Those that fully implement the v2 API.
* Those that do not implement the v2 API at all.
This removes support for the hybrid form that was exploited by message_test.go
In impl/message_test.go, switch each message to look more like how future
generated messages will look like. This is done in reaction to the fact that
MessageType.MessageOf no longer exists.
In value/{map,vector}.go, fix Unwrap to return a pointer since the underlying
reflect.Value is addressable reference value, not a pointer value.
In value/convert.go, split the logic apart so that obtaining a v2 type and
wrapping a type as v2 are distinct operations. Wrapping requires further
initialization than simply creating the initial message type, and calling it
during initial construction would lead to a deadlock.
In protoreflect/go_type.go, we switch back to a lazy initialization of GoType
to avoid a deadlock since the user-provided fn may rely on the fact that
prototype.GoMessage returned.
Change-Id: I5dea00e36fe1a9899bd2ac0aed2c8e51d5d87420
Reviewed-on: https://go-review.googlesource.com/c/148826
Reviewed-by: Herbie Ong <herbie@google.com>
The current logic in this file is temporary and will be removed.
Rename the file as legacy_extension_hack.go so that the future
legacy_extension.go file can contain the actual legacy code that cannot be
removed anytime in the near future.
Change-Id: I7edd9d682836379c4c97d18ea2c2a2e3b15db5a2
Reviewed-on: https://go-review.googlesource.com/c/147918
Reviewed-by: Herbie Ong <herbie@google.com>
The unknown fields in legacy messages is split across the XXX_unrecognized
field and also the XXX_InternalExtensions field. Implement support for
wrapping both fields and presenting it as if it were a unified set of
unknown fields.
Change-Id: If274fae2b48962520edd8a640080b6eced747684
Reviewed-on: https://go-review.googlesource.com/c/146517
Reviewed-by: Damien Neil <dneil@google.com>
