The genid package unifies the genname, fieldnum, and detectknown
packages into a single package.
Whenever possible use the generated constants rather than
hard-coded literals. This makes it easier to search the entire
module for special logic that deal with well-known types.
Change-Id: I13beff1f4149444a0c0b9e607ebf759657f000f4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/235301
Reviewed-by: Herbie Ong <herbie@google.com>
A common mistake is to pass proto.Message to protoreflect.ValueOf
instead of a protoreflect.Message. Specialize for this case
and provide a panic message that is more helpful.
Change-Id: I5def43341aa9607182edde309dea2823e61c1fdb
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/234117
Reviewed-by: Damien Neil <dneil@google.com>
This is step 4 of 6 in a multi-stage migration
to move the well-known types
from the google.golang.org/genproto module
to the google.golang.org/protobuf module.
The generated Go packages for field_mask.proto, api.proto,
type.proto, and source_context.proto are being moved over
to this module alongside all the other well-known types.
In order to move these types between two modules,
there needs to be a sequence of changes submitted in
decently rapid succession. It is impossible to atomically
make these changes, so a brief breakage is inevitable.
The steps are as follows:
Step 1: Submit a change to cloud.google.com/go/internal/gapicgen
to avoid generating the well-known types. Otherwise, the tool
will undo the changes made in step 3.
See https://code-review.googlesource.com/c/gocloud/+/56810
Step 2: Submit a change to google.golang.org/protobuf that
adds the generated well-known types being migrated to that module.
In order to prevent the situation where a user links in
too old a version of the genproto module such that
duplicate registration occurs for the well-known types,
the registry is specially modified to provide an error
message that instructs users to upgrade the genproto module.
See https://golang.org/cl/234937
Step 3: Submit a change to google.golang.org/genproto that
switches all generated well-known types to be aliases to the
ones declared in google.golang.org/protobuf from the previous step.
This will cause the genproto module to incur an dependency
on an unreleased version of the protobuf module.
See https://github.com/googleapis/go-genproto/pull/372
Step 4: Submit a change to google.golang.org/protobuf that
adds a weak module depdency on the genproto module at the
revision from the previous step.
See https://golang.org/cl/235297
Step 5: Release google.golang.org/protobuf@v1.24.0.
See https://golang.org/cl/235299
Step 6: Submit a change to google.golang.org/genproto that
updates the protobuf module dependency to v1.24.0.
Change-Id: I22d496e7496446becad742eab05b169abb735ae2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/235297
Reviewed-by: Damien Neil <dneil@google.com>
This is step 2 of 6 in a multi-stage migration
to move the well-known types
from the google.golang.org/genproto module
to the google.golang.org/protobuf module.
The generated Go packages for field_mask.proto, api.proto,
type.proto, and source_context.proto are being moved over
to this module alongside all the other well-known types.
In order to move these types between two modules,
there needs to be a sequence of changes submitted in
decently rapid succession. It is impossible to atomically
make these changes, so a brief breakage is inevitable.
The steps are as follows:
Step 1: Submit a change to cloud.google.com/go/internal/gapicgen
to avoid generating the well-known types. Otherwise, the tool
will undo the changes made in step 3.
See https://code-review.googlesource.com/c/gocloud/+/56810
Step 2: Submit a change to google.golang.org/protobuf that
adds the generated well-known types being migrated to that module.
In order to prevent the situation where a user links in
too old a version of the genproto module such that
duplicate registration occurs for the well-known types,
the registry is specially modified to provide an error
message that instructs users to upgrade the genproto module.
See https://golang.org/cl/234937
Step 3: Submit a change to google.golang.org/genproto that
switches all generated well-known types to be aliases to the
ones declared in google.golang.org/protobuf from the previous step.
This will cause the genproto module to incur an dependency
on an unreleased version of the protobuf module.
See https://github.com/googleapis/go-genproto/pull/372
Step 4: Submit a change to google.golang.org/protobuf that
adds a weak module depdency on the genproto module at the
revision from the previous step.
Step 5: Release google.golang.org/protobuf@v1.24.0.
Step 6: Submit a change to google.golang.org/genproto that
updates the protobuf module dependency to v1.24.0.
Change-Id: I36a19049d2240b67a37dfad20e154505aee7c784
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/234937
Reviewed-by: Damien Neil <dneil@google.com>
In the upcoming 3.12.x release of protoc, the proto3 language will be
amended to support true presence for scalars. This CL adds support
to both the generator and runtime to support these semantics.
Newly added public API:
protogen.Plugin.SupportedFeatures
protoreflect.FieldDescriptor.HasPresence
protoreflect.FieldDescriptor.HasOptionalKeyword
protoreflect.OneofDescriptor.IsSynthetic
Change-Id: I7c86bf66d0ae56642109beb5f2132184593747ad
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/230698
Reviewed-by: Damien Neil <dneil@google.com>
Expand the mention of the protodesc package to indicate that it can
construct protoreflect.Descriptors from the descriptor messages, but
also convert a protoreflect.Descriptor back to the message form.
Change-Id: I9f3d75c6050fff603655ee0a4e8e29d0a2c0cf84
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/221611
Reviewed-by: Damien Neil <dneil@google.com>
Add a function that takes a protoreflect.FileDescriptorSet and returns
a protoregistry.Files.
Updates golang/protobuf##1065.
Change-Id: I2715d042053ef7d3f1bfcee1866c20cac423e327
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/226237
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Under some rare circumstances registration can deadlock
when lazy descriptor initialization consults the registry.
Move the call triggering the lazy init out of the critical section.
Fixesgolang/protobuf#1052.
Change-Id: Ic266e06b0db99fea65e797b879ce53e5342fff95
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/204804
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
An Is prefix implies it returns a boolean.
A Check prefix better suggests that it could return an error.
Change-Id: I6ffcb32099a944c656c07654c294a0980efb2d0e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/220338
Reviewed-by: Damien Neil <dneil@google.com>
The IsInitialized flag is unused and unnecessary as the error returned
inherently reports whether a message is uninitialized if it is non-nil.
Also cleanup the documentation in protoiface of stale references.
Change-Id: Id75772f4f75f8d7a2a9fbe772d57f7d69ae9fb9d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/220337
Reviewed-by: Damien Neil <dneil@google.com>
Move all fast-path inputs and outputs into the Input/Output structs.
Collapse all booleans into bitfields.
Change-Id: I79ebfbac9cd1d8ef5ec17c4f955311db007391ca
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/219505
Reviewed-by: Joe Tsai <joetsai@google.com>
Add methods to add a new, mutable message to a list or map, matching the
existing Message.Mutable.
These methods are purely a convenience, as each can be implemented in
terms of the existing interface.
Change-Id: I889c20fe37ea0f2a566555212e99e6378fb9fe1d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/220117
Reviewed-by: Joe Tsai <joetsai@google.com>
There were a few stale references due to renamed identifiers.
Change-Id: I0ac7eda93c46143292799c3806214dc89f1e80cd
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/219504
Reviewed-by: Damien Neil <dneil@google.com>
Clarify what the return bool of the Range callback does.
Change-Id: Ib584e48df9fa1c8053652cd4105b343316729719
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/219503
Reviewed-by: Damien Neil <dneil@google.com>
After a header, there needs to be a paragraph in order for godoc
to recognize the header as a header.
Change-Id: I6c814b24dbb5eb2d5efcb0040556521c05587393
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/219140
Reviewed-by: Damien Neil <dneil@google.com>
The FileOptions type provides the ability to specify specialized options
for how a file descriptor is constructed. It follows the same optional
arguments pattern as used in the proto package.
The resolver is not an option since it almost always necessary
when constructing a file descriptor.
Change-Id: Ib98ac6289881ad8402dd615f6c895da5899cb8d9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/218940
Reviewed-by: Damien Neil <dneil@google.com>
TODOs that we do not intend to address have been deleted.
Those that are blocking v2 release are marked with "blocks".
Change-Id: I7efa9e546d0637b562101d0edc7009893d762722
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/218878
Reviewed-by: Damien Neil <dneil@google.com>
Add a sentinel proto.Error error which matches all errors returned by
packages in this module.
Document that protoregistry.NotFound is an exact sentinel value for
performance reasons.
Add a Wrap function to the internal/errors package and use it to wrap
errors from outside sources (resolvers). Wrapped errors match
proto.Error.
Fixesgolang/protobuf#1021.
Change-Id: I45567df3fd6c8dc9a5caafdb55654827f6fb1941
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/215338
Reviewed-by: Joe Tsai <joetsai@google.com>
Change the representation of option flags in protoiface from bools to a
bitfield. This brings the representation of options in protoiface in
sync with that in internal/impl.
This change has several benefits:
1. We will probably find that we need to add more option flags over time.
Converting to the more efficient representation of these flags as high
in the call stack as possible minimizes the performance implication of
the struct growing.
2. On a similar note, this avoids the need to convert from the compact
representation to the larger one when passing from internal/impl to
proto, since the {Marshal,Unmarshal}State methods take the compact form.
3. This removes unused options from protoiface. Instead of documenting
that AllowPartial is always set, we can just not include an AllowPartial
flag in the protoiface options.
4. Conversely, this provides a way to add option flags to protoiface
that we don't want to expose in the proto package.
name old time/op new time/op delta
EmptyMessage/Wire/Marshal-12 11.1ns ± 7% 10.1ns ± 1% -9.35% (p=0.000 n=8+8)
EmptyMessage/Wire/Unmarshal-12 7.07ns ± 0% 6.74ns ± 1% -4.58% (p=0.000 n=8+8)
EmptyMessage/Wire/Validate-12 4.30ns ± 1% 3.80ns ± 8% -11.45% (p=0.000 n=7+8)
RepeatedInt32/Wire/Marshal-12 1.17µs ± 1% 1.21µs ± 7% +4.09% (p=0.000 n=8+8)
RepeatedInt32/Wire/Unmarshal-12 938ns ± 0% 942ns ± 3% ~ (p=0.178 n=7+8)
RepeatedInt32/Wire/Validate-12 521ns ± 4% 543ns ± 7% ~ (p=0.157 n=7+8)
Required/Wire/Marshal-12 97.2ns ± 1% 95.3ns ± 1% -1.98% (p=0.001 n=7+7)
Required/Wire/Unmarshal-12 41.0ns ± 9% 38.6ns ± 3% -5.73% (p=0.048 n=8+8)
Required/Wire/Validate-12 25.4ns ±11% 21.4ns ± 3% -15.62% (p=0.000 n=8+7)
Change-Id: I3ac1b00ab36cfdf61316ec087a5dd20d9248e4f6
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/216760
Reviewed-by: Joe Tsai <joetsai@google.com>
Add a fast check for required fields to the fast path unmarshal.
This is best-effort and will fail to detect some initialized
messages: Messages with more than 64 required fields, messages
split across multiple tags, possibly other cases.
In the cases where it works (which is most of them in practice),
this permits us to skip the IsInitialized check.
Change-Id: I6b70953a333033a5e64fb7ca37a59786cb0f75a0
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/215878
Reviewed-by: Joe Tsai <joetsai@google.com>
Add functions to the proto package which plumb through the fast-path state.
As a sample use case: A followup CL adds an Initialized field to
protoiface.UnmarshalOutput, permitting the unmarshaller to report back
when it can confirm that a message is fully initialized. We want to
preserve that information when an unmarshal operation threads through
the proto package (such as when unmarshaling extensions).
To allow these functions to be added as methods of MarshalOptions and
UnmarshalOptions rather than top-level functions, separate the options
from the input structs.
Also update options passed to fast-path methods to set AllowPartial and
Merge to reflect the expected behavior of those methods. (Always allow
partial, never merge.)
Change-Id: I482477b0c9340793be533e75a86d0bb88708716a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/215877
Reviewed-by: Joe Tsai <joetsai@google.com>
We may want to make changes to the inputs and outputs of the fast-path
functions in the future. For example, we likely want to add the ability
for the fast-path unmarshal to report back whether the unmarshaled
message is known to be initialized.
Change the signatures of these functions to take in and return struct
types which can be extended with whatever fields we want in the future.
Change-Id: Idead360785df730283a4630ea405265b72482e62
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/215719
Reviewed-by: Joe Tsai <joetsai@google.com>
Promote the fast-path magic ProtoMethods method to first-class citizen
of the protoreflect.Message interface.
To avoid polluting the protoreflect package with the various types
required by this method, make the necessary protoiface types unnamed and
duplicate them in protoreflect.
Updates golang/protobuf#1022.
Change-Id: I9595bae40b3bc7536d727fb6f99b3bce8f73da87
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/215718
Reviewed-by: Joe Tsai <joetsai@google.com>
The encoding/testprotos and reflect/protoregistry/testprotos are
accessible by other modules. Move them under internal/testprotos
to dissuade programmers who are too lazy to use their own test protos
when they need one.
Change-Id: I3dbfbce74e68ef033ec252bed076861cb47dd21e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/214341
Reviewed-by: Damien Neil <dneil@google.com>
We resisted adding Clone for a while since:
* It is a function that is perfectly suited for generics.
However, generics probably still won't be available in Go for some time
and it is impractical to block addition of this function when it is very
widely used and will be necessary for the v1 to v2 migration.
* In the past, there was no protoreflect.Message.IsValid, so there was
no proper API to detect invalid top-level messages and return them as such.
Since Clone relies on certain properties about proper round-tripping
of ProtoMessage.ProtoReflect <-> Message.Interface, we add a test
in testing/prototest to check for this.
Change-Id: Ic492b68f27b8b88322a6a3fa3a5e492228db79d9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213297
Reviewed-by: Damien Neil <dneil@google.com>
The JSONName constructor returns a struct value which shallow copies
a sync.Once within it; this is a dubious pattern.
Instead, add a jsonName.Init method to initialize the value.
Change-Id: I190a7239b1b62a8041ee7e4e09c0fe37b64ff623
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213237
Reviewed-by: Damien Neil <dneil@google.com>
Various protoreflect methods can return an "empty, read-only" message,
list, or map value. Provide a method to test if a value is one of these.
Fixesgolang/protobuf#966
Change-Id: I793d8426d6e2201755983c06f024412a7e09bc4c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/209018
Reviewed-by: Joe Tsai <joetsai@google.com>
Remove previously deprecated types, functions, and methods.
Update github.com/golang/protobuf module version to one which does not
depend on any deprecated APIs.
Fixexs golang/protobuf#963
Change-Id: Ida451ef5ef3f34830808f737cc0d1c98f32ce76a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/206017
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Avoid a deadlock when registering a legacy ExtensionType, caused by
initialization of the "internal/impl".ExtensionInfo calling IsMessageSet
on the MessageDescriptor of the type being extended.
We can avoid this deadlock either by initializing the ExtensionType
outside of the GlobalTypes mutex, or by moving IsMessageSet to L1
initialization of the MessageDescriptor so that it doesn't trigger lazy
init.
CL 204804 takes the former approach; this CL takes the latter.
Change-Id: Idfc1ed36a23a139839290ea32492142a17f68cf5
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/205957
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Remove repeated extension fields from the set of nullable fields,
so that Has reports false and Range does not visit a a zero-length
repeated extension field.
This corrects a fuzzer-detected case where unmarshaling and remarshaling
a wire-format message could result in a semantic change. For a repeated
extension field in non-packed encoding, unmarshaling a packed
representation of the field would result in a message which Has the
extension. Remarshaling it would discard the the field.
Fixesgolang.org/protobuf#975
Change-Id: Ie836559c93d218db5b5201742a3b8ebbaacf54ed
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/204897
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Joe Tsai <joetsai@google.com>
Add type-safe methods to register message, enum, and extension types.
Deprecate the NewTypes function and the (*Types).Register method.
Add (*File).RegisterFile and deprecate the NewFiles function and
the (*File).Register method.
Updates golang/protobuf#963
Change-Id: Ie89e77526e0874539e9bd929ca0ba8d758e65a6e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/199898
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The MessageInfo cache, once set, must not be cleared, otherwise
there exists a *messageState value where the MessageInfo value is nil.
Fix the generation of the Reset method to avoid clearing this value.
Change-Id: Ic84ca8b2640a43e967c36993da1ccd3f2b7096c4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/201478
Reviewed-by: Damien Neil <dneil@google.com>
These methods are difficult or impossible to use correctly; types
created by the dynamicpb package, for example, all have the same GoType.
Fixesgolang/protobuf#938
Change-Id: I33d4ef381579ff18569b11df501d0ba7f38a6b5c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/199060
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The global registry is initialized via generated code.
The Go language guarantees that these are serialized (non concurrently).
The main concern is when a concurrent read operation occurs while
registration is still ongoing. In such a case, we do need a lock to
serialize the read with regard to the writes (i.e. registrations).
Change-Id: Ied35d6f8d2620f448cb281c3ec46d8de893b5671
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/199217
Reviewed-by: Damien Neil <dneil@google.com>
This CL:
* Make the meaning of impl/ExtensionInfo.goType consistent. Before,
it was sometimes a T and other times a []T depending on the current
state of initialization. Change it so that it is the constructor's
responsibility to pass in a []T if it is repeated.
* Make internal/filetype responsible for constructing a []T for
repeated extension fields.
* Makes filedesc/Extension.Cardinality one of the eagerly initialized
pieces of information since it is useful to internal/filetype.
* Unify ExtensionInfo.desc and ExtensionInfo.tdesc.ExtensionField,
which held the same information.
* Remove the internal implementation for impl.X.ExtensionDescFromType
since we are dropping support for this from v1.
Change-Id: Ie95c4de66cd674c1d886da4f63b133b7d763c7ef
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/195777
Reviewed-by: Damien Neil <dneil@google.com>
Use internal/detrand in the construction of our error messages.
This alters whether there is one or two spaces following the "proto:" prefix.
While it is easy for users to still work around this mutation,
sit at least forces them to write test infrastructure to more fuzzily
match on error strings.
Change-Id: I4ddca717526ee3fc4dbb1e0b36cfca8c6e0df36d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/194038
Reviewed-by: Herbie Ong <herbie@google.com>
Rather than using the <unknown:%d> syntax for GoString,
emit something similar to TypeName(%d),
which is functionally equivalent to casting the integer
to the specified type.
Change-Id: Ibf4bd680d1672fcaba9022fb6bd03bbfe249b8b7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/191580
Reviewed-by: Herbie Ong <herbie@google.com>
The Num methods provide an O(1) lookup for the number of entries that Range
would return. This is needed to implement efficient cache invalidation logic
for caches that wrap the global registry.
Change-Id: I7c4ff97f674c4e9e4caae291f017cfad7294856c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/193599
Reviewed-by: Damien Neil <dneil@google.com>
Add a way to typecheck a Value or interface{} without converting it to
the other form. This permits implementations which store field values as
a Value (such as dynamicpb, or (soon) extensions in generated messages)
to validate inputs without an unnecessary conversion.
Fixesgolang/protobuf#905
Change-Id: I1b78612b22ae832efbb55f81ae420871729e3a02
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/192457
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
