Generate the needed infrastructure to ensure that we can statically
enforce minimum and maximum versions. This enables us to have a policy
when we release v2 where it fails to build for:
* new generated code with really old runtimes
* new runtimes with really old generated code
Change-Id: Ib699ad62c06dff8f9285806394a741c18db00288
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/178546
Reviewed-by: Damien Neil <dneil@google.com>
Instead of accepting a concrete protoregistry.Types type,
accept an interface that provides the necessary functionality
to perform the serialization.
The advantages of this approach:
* There is no need for complex logic to allow a Parent or custom
Resolver on the protoregistry.Types type.
* Users can pass their own custom resolver implementations directly
to the serialization functions.
* This is a more principled approach to plumbing custom resolvers
than the previous approach of overloading behavior on the concrete
Types type.
The disadvantages of this approach:
* A pointer to a concrete type is 8B, while an interface is 16B.
However, the expansion of the {Marshal,Unmarshal}Options structs
should be a concern solved separately from how to plumb custom resolvers.
* The resolver interfaces as defined today may be insufficient to
provide functionality needed in the future if protobuf expands its
feature set. For example, let's suppose the Any message permits
directly representing a enum by name. This would require the ability
to lookup an enum by name. To support that hypothetical need,
we can document that the serializers type-assert the provided Resolver
to a EnumTypeResolver and use that if possible. There is some loss
of type safety with this approach, but provides a clear path forward.
Change-Id: I81ca80e59335d36be6b43d57ec8e17abfdfa3bad
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/177044
Reviewed-by: Damien Neil <dneil@google.com>
The name MessageType is easily confused with protoreflect.MessageType.
Rename it as MessageInfo, which follows the pattern set by v1,
where the equivalent data structure is called InternalMessageInfo.
Change-Id: I535956e1f7c6e9b07e9585e889d5e93388d0d2ce
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/178478
Reviewed-by: Damien Neil <dneil@google.com>
A proto2 message can reference a proto3 enum. It is currently cumbersome
to set proto3 enums in proto2 messages. Add the Enum method in all situations
to support this situation. It also removes yet another point of divergence
between proto2 and proto3.
We could consider removing this method if Go ever gets generics,
but that hypothetical future is long ways away.
Change-Id: Ib83bc87e46b49f3271b90bacb2893bb8e278e4f2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/177257
Reviewed-by: Damien Neil <dneil@google.com>
Rename each generated protobuf package such that the base of the
Go package path is always equal to the Go package name to follow
proper Go package naming conventions.
The Go package name is derived from the .proto source file name by
replacing ".proto" with "pb" and stripping all underscores.
Change-Id: Iea05d1b5d94b1b2821ae10276ab771bb2df93c0e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/177380
Reviewed-by: Damien Neil <dneil@google.com>
While it is general convention that the receiver being mutated
is the first argument, the standard library specifically goes against
this convention when it comes to the Unmarshal function.
Switch the ordering of the Unmarshal function to match the Go stdlib.
Change-Id: I893346680233ef9fec7104415a54a0a7ae353378
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/177258
Reviewed-by: Damien Neil <dneil@google.com>
Rename encoding/*pb to follow the convention of prefixing package names
with 'proto':
google.golang.org/protobuf/encoding/protojson
google.golang.org/protobuf/encoding/prototext
Move protogen under a compiler/ directory, just in case we ever do add
more compiler-related packages.
google.golang.org/protobuf/compiler/protogen
Change-Id: I31010cb5cabcea8274fffcac468477b58b56e8eb
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/177178
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Temporarily remove go.mod, since we can't generate an accurate one until
the corresponding v1 change is submitted.
Change-Id: I1e1ad97f2b455e33f61ffaeb8676289795e47e72
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/177000
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Added API:
FieldDescriptor.IsExtension
FieldDescriptor.IsList
FieldDescriptor.MapKey
FieldDescriptor.MapValue
FieldDescriptor.ContainingOneof
FieldDescriptor.ContainingMessage
Deprecated API (to be removed in subsequent CL):
FieldDescriptor.Oneof
FieldDescriptor.Extendee
These methods help cleanup several common usage patterns.
Change-Id: I9a3ffabc2edb2173c536509b22f330f98bba7cf3
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/176977
Reviewed-by: Damien Neil <dneil@google.com>
Added methods:
Enum.Descriptor
Message.Descriptor
EnumType.Descriptor
MessageType.Descriptor
ExtensionType.Descriptor
Message.New
All functionality is switched over to use those methods instead of
implicitly relying on the fact that {Enum,Message}Type implicitly
implement the associated descriptor interface.
This CL does not yet remove {Enum,Message}.Type or prevent
{Enum,Message,Extension}Type from implementating a descriptor.
That is a subsequent CL.
The Message.New method is also added to replace functionality
that will be lost when the Type methods are removed.
Change-Id: I7fefde1673bbd40bfdac489aca05cec9a6c98eb1
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/174918
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Herbie Ong <herbie@google.com>
This feature seems to be used quite a bit, and the conformance tests
treat this as required, perhaps as a "required option" since the
developer guide states:
"Proto3 JSON parser should reject unknown fields by default but may
provide an option to ignore unknown fields in parsing."
Also, all invalid UTF-8 errors in skipped values are also returned as it
is similar to a parse error, except it is a non-fatal one.
Change-Id: Ia26e9a355daecdbf99af23f3061353fffa32d47d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/174017
Reviewed-by: Damien Neil <dneil@google.com>
The protobuf type system uses the word "descriptor" instead of "type".
We should avoid the "type" verbage when we aren't talking about Go types.
The old names are temporarily kept around for compatibility reasons.
Change-Id: Icc99c913528ead011f7a74aa8399d9c5ec6dc56e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172238
Reviewed-by: Herbie Ong <herbie@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Using a named fields gives us the flexibility to change the underlying
representation of special fields without needing to regenerate user code.
We add a named type for ExtensionFields, UnknownFields, and SizeCache.
Change-Id: I107cf82899850ea76665310ce79def60f0f7ab97
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172402
Reviewed-by: Damien Neil <dneil@google.com>
Lower-casing the FileDescriptor variable provides for shorter and
more readable global variable names. This has a very minor benefit
that the read-only data section of a binary is slightly smaller
since function names do end up in the final binary.
Change-Id: I077364905e5c9adea69873b3ea580fddc68b9eb8
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172119
Reviewed-by: Herbie Ong <herbie@google.com>
Keeping this consistent with textpb - golang.org/cl/168439.
Change-Id: Ie4c57615f4ac5eaf6b3836edaf7ae2082ebaae64
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171240
Reviewed-by: Damien Neil <dneil@google.com>
Go1.12 on GoLLVM with LLVM 9.0.0 has a bug where the simpler but
equivalent expression results in memory corruption.
Revert to the more complex expression to avoid breaking GoLLVM and
it is not worth waiting for a GoLLVM fix.
Change-Id: I8b42965ca9bd333deb8693021e7b22f966e13521
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171463
Reviewed-by: Damien Neil <dneil@google.com>
Seperate out the generation of the oneof wrapper types from the
code block that is about getter methods.
Change-Id: Ief44ef953d0b5ad8c998a8542c830ca70468a3bf
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171029
Reviewed-by: Damien Neil <dneil@google.com>
Move all checks for required fields into a proto.IsInitialized function.
Initial testing makes me confident that we can provide a fast-path
implementation of IsInitialized which will perform more than
acceptably. (In the degenerate-but-common case where a message
transitively contains no required fields, this check can be nearly
zero cost.)
Unifying checks into a single function provides consistent behavior
between the wire, text, and json codecs.
Performing the check after decoding eliminates the wire decoder bug
where a split message is incorrectly seen as missing required fields.
Performing the check after decoding also provides consistent and
arguably more correct behavior when the target message was partially
prepopulated.
Change-Id: I9478b7bebb263af00c0d9f66a1f26e31ff553522
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170787
Reviewed-by: Herbie Ong <herbie@google.com>
In marshalMap, replace Map.Range and sortMap with
internal/mapsort.Range. Also, make sure to capture fatal error properly
for return.
In appendExtensions, capture fatal error properly for return. Added a
testcase that shows this was a bug.
In unmarshalAny, remove unnecessary checks and use internal/fieldnum for
Any's field numbers.
Change-Id: Id8574d5b4eb820ad961f6ad5e886f8ae2e9b90f0
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170627
Reviewed-by: Damien Neil <dneil@google.com>
Unmarshaling should fail if multiple fields in the same oneof exists in
the input.
Change-Id: I76efd88681a50c18f3eaf770c9eb48727efb412b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170517
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Allow message implementations to provide optimized versions of standard
operations. Generated messages now include a ProtoReflectMethods method,
returning a protoiface.Methods struct containing pointers to assorted
optional functions.
The Methods struct also includes a Flags field indicating support for
optional features such as deterministic marshaling.
Implementation of the fast paths (and tests) will come in later CLs.
Change-Id: Idd1beed0ecf43ec5e5e7b8da2ee1e08d3ce32213
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170340
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Change use of regular expression to manually parsing the value.
Allow value with + symbol in front, e.g. "+3s". Previous regex missed
this.
Do not allow values without numbers, e.g. "-s". Previous regex missed
this as well.
name old time/op new time/op delta
Unmarshal_Duration-4 1.96µs ± 0% 1.24µs ± 0% ~ (p=1.000 n=1+1)
name old alloc/op new alloc/op delta
Unmarshal_Duration-4 703B ± 0% 512B ± 0% ~ (p=1.000 n=1+1)
name old allocs/op new allocs/op delta
Unmarshal_Duration-4 20.0 ± 0% 17.0 ± 0% ~ (p=1.000 n=1+1)
Change-Id: I4db58d70f55607213631c49d698ee6a048b5e094
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170012
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
A JSON "null" field should set the NullValue enum field because
NullValue has the custom encoding format of "null".
Change-Id: I2bfa0900de64d7e2874f7c6db04b1cbc0b61b904
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170107
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Also added json.Decoder.Clone API for unmarshaling Any to look
ahead remaining bytes for @type field.
Change-Id: I2f803743534dfb64f9092d716805b115faa5975a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170102
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Minor changes:
* Use x as the receiver since "e" and "m" are meaningless in the presence
of user-defined enum and message names.
* Consistently keep enum methods together, rather awkwardly split apart
by the value maps.
Change-Id: I68e5666efb56ac7a4d062fb223b9f826dc72aba9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170357
Reviewed-by: Herbie Ong <herbie@google.com>
This reduces the init-time cost slightly since the GZIP'd
raw descriptor is constructed lazily on demand.
Change-Id: I482c6a2201b8786e425d7dee5612fdfd60ab1500
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169917
Reviewed-by: Herbie Ong <herbie@google.com>
The "custom" JSON representation for Empty is {}. And hence, when
embedded in Any, it needs to be represented as a field value in the
"value" field.
Discussion at https://github.com/protocolbuffers/protobuf/issues/5390.
As of this CL, code currently handles marshaling of Empty embedded
in Any, but not unmarshaling of Any yet. I'd like to get this corrected
first in order to proceed with unmarshaling of Any.
Change-Id: Ic0c321be188a979909b99d6b467aabc57c48e95c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169702
Reviewed-by: Damien Neil <dneil@google.com>
Added tests related to required fields and AllowPartial. I somehow
missed this before.
Change-Id: I3eb17347b1f3a99be3d65af06c4549abcc87ae39
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169701
Reviewed-by: Damien Neil <dneil@google.com>
I had defined these types at some point in order to reuse another
instance of these within, but it's not needed anymore and hence removing
them.
Change-Id: I8aa127326a5926c6a8688d83cce7ca83c160b39b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169700
Reviewed-by: Damien Neil <dneil@google.com>
Provide AllowPartial option to accept messages with missing required
field during marshaling and unmarshaling.
Change-Id: Ia23783870a8125633f8ddc0b686984b4c5ca15ba
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169500
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Rearranged well_known_types.go such that marshaling and unmarshaling
funcs for a particular type are closer, and added description for its
JSON representation.
Change-Id: Id9f5e4060158f66392a351debaffe1d7260919e4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169497
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Also, fixed unmarshaling of map messages where non-fatal errors were not
propagated up.
Change-Id: I06415b4a4ccd12135f0fdfaa38ccda54866139e7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168997
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This removes yet another set of dependencies of v2 on v1.
The only remaining dependency are in the _test.go files,
primarily for proto.Equal.
Changes made:
* cmd/protoc-gen-go no longer generates any functionality that depends
on the v1 package, and instead only depends on v2.
* internal/fileinit.FileBuilder.MessageOutputTypes is switched from
protoreflect.MessageType to protoimpl.MessageType since the
implementation must be fully inialized before registration occurs.
* The test for internal/legacy/file_test.go is switched to a legacy_test
package to avoid a cyclic dependency.
This requires Load{Enum,Message,File}Desc to be exported.
Change-Id: I43e2fe64cff4eea204258ce11e791aca5eb6e569
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/169298
Reviewed-by: Damien Neil <dneil@google.com>
Description of message Value states:
`Value` represents a dynamically typed value which can be either null, a
number, a string, a boolean, a recursive struct value, or a list of values. A
producer of value is expected to set one of that variants, absence of any
variant indicates an error.
https://github.com/protocolbuffers/protobuf/blob/3.7.x/src/google/protobuf/struct.proto#L57-L60
Previous implementation was following C++ lib behavior.
Change-Id: Id51792e2fc8cc465a05a978e63410d3b6802b522
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168901
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
As a goal, v2 should not depend on v1. As another step towards that end,
we move all the types that used to be in the v1 protoapi package over to v2.
For now, we place MessageV1, ExtensionRangeV1, and ExtensionDescV1
in runtime/protoiface since these are types that generated messages will
probably have to reference forever. An alternative location could be
reflect/protoreflect, but it seems unfortunate to have to dirty the
namespace of that package with these types.
We move ExtensionFieldV1, ExtensionFieldsV1, and ExtensionFieldsOf
to internal/impl, since these are related to the implementation of a
generated message.
Since moving these types from v1 to v2 implies that the v1 protoapi
package is useless, we update all usages of v1 protoapi in the v2
repository to point to the relevant v2 type or functionality.
CL/168538 is the corresponding change to alter v1.
There will be a temporary build failure as it is not possible
to submit CL/168519 and CL/168538 atomically.
Change-Id: Ide4025c1b6af5b7f0696f4b65b988b4d10a50f0b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168519
Reviewed-by: Herbie Ong <herbie@google.com>
Remove unnecessary v1 wrapping. Change most tests to operate on message
Any directly.
Change-Id: I19bbca6c1af72894f6a292aab8ebd09e0301a260
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168517
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Generate field numbers for the well-known types,
so that encoding/jsonpb can benefit from them as well.
This CL fixes internal/cmd/generate-protos, which was silently failing
because the modulePath was not properly initialized. We fix this by
moving it to the start of the init function.
Change-Id: I87637176f29218cffa512b4baa49f39dae924061
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168497
Reviewed-by: Herbie Ong <herbie@google.com>
Also, changed MarshalOptions.Compact to Indent for consistency with v1
and to make compact as the default.
Change-Id: Id08aaa5ca5656f18e7925d2eabc0b6b055b1cebb
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168352
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
