An Is prefix implies it returns a boolean.
A Check prefix better suggests that it could return an error.
Change-Id: I6ffcb32099a944c656c07654c294a0980efb2d0e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/220338
Reviewed-by: Damien Neil <dneil@google.com>
Remove the generated proto packages that already exist in
google.golang.org/genproto. We want to eventually move these
packages here, but it doesn't need to happen yet.
Add a local copy of fieldmaskpb for use in tests.
Refactor proto generation to override import paths using the
M<source>=<import_path> compiler option instead of by patching the
source files.
Change-Id: I8d31f67e931d70140182f19f3e0106111f71c4b4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/219598
Reviewed-by: Joe Tsai <joetsai@google.com>
Move Multiline and Indent to the top so that there is a separation
between options with semantic significance and those that are merely
for aesthetic purposes.
Change-Id: Icd5ee94ec010db8139a5e720f5b9842274fb3755
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/219500
Reviewed-by: Damien Neil <dneil@google.com>
All unmarshaling error messages now contain line number and column
information, except for the following errors:
- `unexpected EOF`
- `no support for proto1 MessageSets`
- `required fields X not set`
Changes to internal/encoding/json:
- Moved encoding funcs in string.go and number.go into encode.go.
- Separated out encoding kind constants from decoding ones.
- Renamed file string.go to decode_string.go.
- Renamed file number.go to decode_number.go.
- Renamed Type struct to Kind.
- Renamed Value struct to Token.
- Token accessor methods no longer return error.
Name, Bool, ParsedString will panic if called on the wrong kind.
Float, Int, Uint has ok bool result to check against.
- Changed Peek to return Token and error.
Changes to encoding/protojson:
- Updated internal/encoding/json API calls.
- Added line info on most unmarshaling error messages and kept
description simple and consistent.
Change-Id: Ie50456694f2214c5c4fafd2c9b9239680da0deec
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/218978
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The Format function and MarshalOptions.Format method are helper
functions for directly obtaining the formatted string for a message
without having to deal with errors or convert a []byte to string.
It is only intended for human consumption (e.g., debugging or logging).
We also add a MarshalOptions.Multiline option to specify that the output
should use some default indentation in a multiline output.
This assists in the v1 to v2 migration where:
protoV1.CompactTextString(m) => prototext.MarshalOptions{}.Format(m)
protoV1.MarshalTextString(m) => prototext.Format(m)
At Google, there are approximately 10x more usages of MarshalTextString than
CompactTextString, so it makes sense that the top-level Format function
does multiline expansion by default.
Fixes#850
Change-Id: I149c9e190a6d99b985d3884df675499a3313e9b3
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/213460
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Herbie Ong <herbie@google.com>
The encoding/testprotos and reflect/protoregistry/testprotos are
accessible by other modules. Move them under internal/testprotos
to dissuade programmers who are too lazy to use their own test protos
when they need one.
Change-Id: I3dbfbce74e68ef033ec252bed076861cb47dd21e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/214341
Reviewed-by: Damien Neil <dneil@google.com>
Remove support for running benchmarks with APIv1.
The comparisons have served their purpose, and this removes the last
dependency on the github.com/golang/protobuf module.
Fixesgolang/protobuf#962.
Change-Id: I55758e19451fcd16ab1a5d66244eb8214ceb9fa7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/214040
Reviewed-by: Joe Tsai <joetsai@google.com>
The MessageInfo cache, once set, must not be cleared, otherwise
there exists a *messageState value where the MessageInfo value is nil.
Fix the generation of the Reset method to avoid clearing this value.
Change-Id: Ic84ca8b2640a43e967c36993da1ccd3f2b7096c4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/201478
Reviewed-by: Damien Neil <dneil@google.com>
Change tests which use private types registries to use the global one.
Except in cases where we want to explicitly test that the private
registry is used, it's simpler to use the global registry.
Change-Id: I998fb463b6beef91c7f5ce2ca2083251ae24d1db
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/199897
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Herbie Ong <herbie@google.com>
Suppose a oneof has N fields, the previous marshaling logic
would traverse every field checking for presence. This is O(N).
Using the protoreflect.Message.WhichOneof method, we can reduce
this to O(1). This optimization is exceptionally useful for oneofs
with a large number of fields.
Change-Id: I5f4aa8b1a899930f5c95e9cf1d68bac4b0b7884d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/196121
Reviewed-by: Herbie Ong <herbie@google.com>
Ironically, the "real" protobuf name of a group is not the name
of the field descriptor, but the message descriptor.
Change-Id: I26ab546a94e934766fa6af6252cacd294442a221
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/195780
Reviewed-by: Herbie Ong <herbie@google.com>
We change Unmarshal to reset a message by default.
* We add a Merge option to UnmarshalOptions for explicit merging.
* We speed up Reset by checking for the Reset method.
* Remove TODOs in prototext and protojson about reset behavior.
Fixesgolang/protobuf#890
Change-Id: Ibd8963c741053f564acf061fbdb846699942109c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/195457
Reviewed-by: Damien Neil <dneil@google.com>
This CL adds support for discarding unknown fields from the input.
We add support for parsing and resolving field numbers, so that
the DiscardUnknown option can ignore all unresolvable fields.
We continue to reject known fields identified by field number
since there are a number of edge cases that a difficult to resolve.
Change-Id: I5c88b7bae8656ce20e85e4b5c92d8564a5ff8bb6
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/195779
Reviewed-by: Herbie Ong <herbie@google.com>
This changes text marshaling to avoid unknown fields by default
and instead adds an option so that unknown fields be emitted.
This ensures that the default marshal/unknown can round-trip.
Change-Id: I85c84ba6ab7916d538ec6bfd4e9d399a8fcba14e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/195778
Reviewed-by: Herbie Ong <herbie@google.com>
This is more consistent with the indent documentation:
If indent is a non-empty string, it causes every entry in a List or Message
to be preceded by the indent and trailed by a newline.
Since an empty message has no entries, there should be no newlines.
Change-Id: I5d57165aaf94ca6b184bb35bf05d5d68f5ee9dd5
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/194877
Reviewed-by: Herbie Ong <herbie@google.com>
This is meant to deter users from doing byte for byte comparison.
Change-Id: If005d2dc1eba45eaa4254171d2f247820db109e4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/194037
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
UseProtoNames=true uses proto field name in JSON field names.
Change-Id: I23249dc1787d9735bef780b1ef8d294a9c55c043
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/193998
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Add option to marshal out all fields including unset ones except for
unset oneof fields and extension fields.
This is to make V2 compatible with V1's EmitDefaults option.
Change-Id: Ifa7bae48e82740b623c74f936bcbe9e66b11344a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/193759
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
After resolving a FieldDescriptor from the resolver, verify that the
returned descriptor truly does extend the target message.
This will never happen with the global registry, since it checks this
but can happen with poorly written custom resolvers.
Change-Id: I2de5b9d2fb9d32e076ec4e82adda98d672891444
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/193737
Reviewed-by: Herbie Ong <herbie@google.com>
Descriptor methods generally return a Descriptor with no Go type
information. ExtensionType's Descriptor is an exception, returning an
ExtensionTypeDescriptor containing both the proto descriptor and a
reference back to the ExtensionType. The pure descriptor is accessed
by xt.Descriptor().Descriptor().
Rename ExtensionType's Descriptor method to TypeDescriptor to make it
clear that it behaves a bit differently.
Change 1/2: Add the TypeDescriptor method and deprecate Descriptor.
Change-Id: I1806095044d35a474d60f94d2a28bdf528f12238
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/192139
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Too much code inside Google rely on ExtensionDescV1.Name being the
target message name rather than the real name of the extension.
Double down on this assumption consistently across our codebase.
Change-Id: If0b7cd16dd1f2f7acfb2e562dbf1a2b5487162b5
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186980
Reviewed-by: Herbie Ong <herbie@google.com>
Change protoV1.ExtensionDesc to directly implement ExtensionType
rather than delegating to one.
Unify the previous types protoiface.ExtensionDescV1 and
filetype.Extension in impl.ExtensionInfo. The protoV1.ExtensionDesc
type becomes an alias to ExtensionInfo.
This gives us:
- Just one implementation of ExtensionType.
- Generated foopb.E_Ext vars are canonical ExtensionTypes.
- Generated foopb.E_Ext vars are also v1.ExtensionDescs for backwards
compatibility.
- Conversion between legacy and modern representations happens
transparently when lazily initializing an ExtensionInfo.
Overall, a simplification for users of generated code, since they can
mostly ignore the ExtensionDesc/ExtentionType distinction and use the
same value in either the old or new API.
This is change 3/5 in a series of commits changing protoV1.ExtensionDesc
to directly implement protoreflect.ExtensionType.
1. [v2] Add protoimpl.ExtensionInfo as an alias for
protoiface.ExtensionDescV1.
2. [v1] Update references to protoimpl.ExtensionInfo to use
protoiface.ExtensionInfo.
3. [v2] Create protoimpl.ExtensionInfo (an alias to a new type in
the impl package) and remove protoiface.ExtensionDescV1.
4. [v1] Remove unneeded explicit conversions between ExtensionDesc and
ExtensionType (since the former now directly implements the latter).
5. [v2] Remove stub conversion functions.
Change-Id: I96ee890541ec11b2412e1a72c9d7b96e4d7f66b4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189563
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Generate the current index into depIdxs for easier human debugging.
Change-Id: Ida42aa95137b2044a4dc267c31cebec5023bdfb1
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/190278
Reviewed-by: Herbie Ong <herbie@google.com>
Drop the dependency from generated files on prototype.Enum: Generated
code should only depend on runtime/proto{iface,impl}.
Drop the Enums, Messages, and Extensions returns form
filetype.Builder.Build. Of these, only Enums was used by generated code.
Change the generated init function to pass the builder a slice of values
to fill in (as is done for messages and extensions).
Remove the filetype dependency on prototype in preparation for
eventually dropping the prototype package entirely.
Change-Id: I28a3420f5dfcc13fed531a64ef07b9afddfd9d55
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189200
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Rename build tag "proto1_legacy" -> "protolegacy"
to be consistent with the "protoreflect" tag.
Rename flag constant "Proto1Legacy" -> "ProtoLegacy" since
it covers more than simply proto1 legacy features.
For example, it covers alpha-features of proto3 that
were eventually removed from the final proto3 release.
Change-Id: I0f4fcbadd4b5a61c87645e2e5be11d187e59157c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189345
Reviewed-by: Damien Neil <dneil@google.com>
Change protoiface.ExtensionDescV1 to implement protoreflect.ExtensionType.
ExtensionDescV1's Name field conflicts with the Descriptor Name method,
so change the protoreflect.{Message,Enum,Extension}Type types to no
longer implement the corresponding Descriptor interface. This also leads
to a clearer distinction between the two types.
Introduce a protoreflect.ExtensionTypeDescriptor type which bridges
between ExtensionType and ExtensionDescriptor.
Add extension accessor functions to the proto package:
proto.{Has,Clear,Get,Set}Extension. These functions take a
protoreflect.ExtensionType parameter, which allows writing the
same function call using either the old or new API:
proto.GetExtension(message, somepb.E_ExtensionFoo)
Fixesgolang/protobuf#908
Change-Id: Ibc65d12a46666297849114fd3aefbc4a597d9f08
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189199
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
In order for protoc-gen-go to output the current version,
it needs to know what version it is currently running as.
However, we cannot rely on the git tags since the tags are not
made until *after* the commit has been submitted.
Instead, we manually encode the version into the code and
make sure that git tags match up with the version in the code.
The version.go file in runtime/protoimpl contains instructions
for how to make a release. Essentially:
* Every non-release commit has a version string with "devel" in it.
* Every release commit must not have "devel" in it and must be unique.
* The "release process" involves submitting two CLs.
The first CL creates a version string without "devel",
which is the commit that a git tag will actually reference.
The second CL follows immediately and re-introduces "devel"
into the version string.
The following example shows a possible sequence of VersionStrings
for git commits in time-ascending order:
v1.19.0-devel (this CL)
v1.19.0-devel
v1.19.0-devel
v1.19.0-devel
v1.20.0-rc.1 <- tagged
v1.20.0-rc.1.devel
v1.20.0-rc.1.devel
v1.20.0-rc.1.devel
v1.20.0-rc.2 <- tagged
v1.20.0-rc.2.devel
v1.20.0 <- tagged (future public release)
v1.20.0-devel
v1.20.0-devel
v1.20.0-devel
v1.20.0-devel
v1.20.1 <- tagged
v1.20.1-devel
v1.20.1-devel
v1.21.0 <- tagged
v1.21.0-devel
Note that we start today with v1.19.0-devel, which means that our initial
release will be v1.20.0. This number was intentionally chosen since
1) the number 20 has some correlation to the fact that we keep calling
the new implementation the "v2" implementation, and
2) the set of tagged versions for github.com/golang/protobuf
and google.golang.org/protobuf are unlikely to ever overlap.
This way, the version of protoc-gen-go is never ambiguous which module
it was built from.
Now that we have version information, we add support for generating .pb.go
files with the version information recorded. However, we do not emit
these for .pb.go files in our own repository since they are always guaranteed
to be at the right version (enforced by integration_test.go).
Updates golang/protobuf#524
Change-Id: I25495a45042c2aa39a39cb7e7738ae8e831a9d26
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186117
Reviewed-by: Damien Neil <dneil@google.com>
The following improvements were made:
* All standalone comments above the "syntax" marker are preserved
similar to Java and some other generators.
* All standalone comments above the "package" marker are preserved
to be consistent with our former behavior.
* Leading comments are now generated for enums and extension fields.
* Single-line trailing comments are now generated for
enum values, message fields, and extension fields.
* The leading comments for each field that is part of a oneof are now
generated with the wrapper types rather than being shoved into the
comment for the oneof itself in an unreadable way.
* The deprecation marker is always generated as being above the declaration
rather than sometimes being an inlined comment.
* The deprecation marker is now properly generated for weak field setters.
Updates golang/protobuf#666
Change-Id: I7fd832dd4f86d15bfff70d7c22c6ba4934c05fcf
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189238
Reviewed-by: Damien Neil <dneil@google.com>
For better readability in godoc, group extension fields by the
target message that they are extending.
Change-Id: Icc0a247b37639e3dbf7a107810923b8ca8294724
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189257
Reviewed-by: Damien Neil <dneil@google.com>
Since the enum maps are here to stay, group the declarations together
in a var block for better readability in godoc.
Change-Id: I9a313266539e9a60781f98b80a5293379f82607b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189077
Reviewed-by: Damien Neil <dneil@google.com>
These were originally kept separate to assist Google-internal patches,
but it turns out that Google-internal patches do not use the
genMessageInternalFields function.
Change-Id: Idfa962b943d3bede9982b5b0875ba90c86c6d181
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/188979
Reviewed-by: Damien Neil <dneil@google.com>
This CL unifies common MessageSet logic in prototext and protojson
into the messageset package. While we are at it, also enable
MessageSet support only if the proto1_legacy build flag is enabled.
Change-Id: I1a7d475e8bb1dad61ecd286df45e4239e5bef072
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185898
Reviewed-by: Damien Neil <dneil@google.com>
The v2 implementation strictly enforces that there are no conflicts at
all in the protobuf namespace unlike the prior v1 implementation.
This change is almost certainly going to cause loud failures for users
that were unknowingly tolerating registration conflicts.
We modify internal/filedesc to be able to record the Go package path
that the file descriptor is declared within. This information is used
by reflect/protoregistry to print both the previous Go package that
registered some declaration, and current Go package that is attempting
to register some declaration.
Change-Id: Ib5eb21c1c98495afc51aa08bd4404bd9d64b5b57
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186177
Reviewed-by: Damien Neil <dneil@google.com>
If the message for a weak field is linked in,
we treat it as if it were identical to a normal known field.
However, if the weak field is not linked in,
we treat it as if the field were not known.
Change-Id: I576d911deec98e13211304024a6353734d055465
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185457
Reviewed-by: Herbie Ong <herbie@google.com>
CL/174938 removed these methods in favor of a method that returned
only the descriptors. This CL adds back in the Type methods alongside
the Descriptor methods.
In a vast majority of protobuf usages, only the descriptor information
is needed. However, there is a small percentage that legitimately needs
the Go type information. We should provide both, but document that the
descriptor-only information is preferred.
Change-Id: Ia0a098997fb1bd009994940ae8ea5257ccd87cae
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184578
Reviewed-by: Damien Neil <dneil@google.com>
Usage of these is pervasive in code which works with proto2, and proto2
will be with us for a long, long time to come. Move them to the proto
package.
Change-Id: I1b2e57429fd5a8f107a848a4492d20c27f304bd7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185543
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
We define MessageState, which is essentially an atomically set *MessageInfo.
By nesting this as the first field in every generated message, we can
implement the reflective methods on a *MessageState when obtained by
unsafe casting a concrete message pointer as a *MessageState.
The MessageInfo held by MessageState provides additional Go type information
to interpret the memory that comes after the contents of the MessageState.
Since we are nesting a MessageState in every message,
the memory use of every message instance grows by 8B.
On average, the body of ProtoReflect grows from 133B to 202B (+50%).
However, this is offset by XXX_Methods, which is 108B and
will be removed in a future CL. Taking into account the eventual removal
of XXX_Methods, this is a net reduction of 25%.
name old time/op new time/op delta
Name/Value-4 70.3ns ± 2% 17.5ns ± 6% -75.08% (p=0.000 n=10+10)
Name/Nil-4 70.6ns ± 3% 33.4ns ± 2% -52.66% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
Name/Value-4 16.0B ± 0% 0.0B -100.00% (p=0.000 n=10+10)
Name/Nil-4 16.0B ± 0% 0.0B -100.00% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Name/Value-4 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10)
Name/Nil-4 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10)
Change-Id: I92bd58dc681c57c92612fd5ba7fc066aea34e95a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185460
Reviewed-by: Damien Neil <dneil@google.com>
