Change the storage type of ExtensionField from interface{} to
protoreflect.Value.
Replace the codec functions operating on interface{}s with ones
operating on Values.
Values are potentially more efficient, since they can represent
non-pointer types without allocation. This also reduces the number of
types used to represent field values.
Additionally, this change lays groundwork for changing the
user-visible representation of repeated extension fields from
*[]T to []T. The storage type for extension fields must support mutation
(thus *[]T currently); changing the storage type to a Value permits this
without the need to introduce yet another view on field values.
Change-Id: Ida336be14112bb940f655236eb58df21bf312525
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/192218
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Passing a non-pointer type to protoreflect.NewValue causes an
unnecessary allocation in order to store the value in an interface{}.
While this allocation could be avoided by a smarter compiler, no such
compiler exists today.
Add functions for creating new values of a specific type, avoiding the
allocation. (And also adding a small amount of type safety, although
this is unlikely to be important.)
Update the proto and internal/impl packages to use these functions.
Change-Id: Ic733de22ddf19c530189166c853348e1b54b7391
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/191457
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The use of internal/cmd/conformance/conformance.sh means that the test
is not hermetic since the script uses the system version of go,
rather than the specific versions chosen by integration_test.go.
Change the way the conformance test is invoked by turning it
into a Go unit test that integration_test.go can directly call.
Change-Id: I37d23341e1eda984f23f78757a38e862e5fac3d4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/190518
Reviewed-by: Herbie Ong <herbie@google.com>
A conformance test has been added for the text format.
Update our conformance runner to handle that format and
update with the list of current failures.
Change-Id: I36a271fcfbe328c4ee8c870caff4661659ad27ef
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/190517
Reviewed-by: Herbie Ong <herbie@google.com>
Remove the remaining uses of the prototype package.
The most significant change is to impl.MessageInfo, which now directly
implements the MessageType interface. This involves two notable changes
to exported fields of MessageInfo:
- PBType is now Desc.
- GoType is now GoReflectType. (Name changed to avoid a conflict with
the GoType method of the MessageType interface.)
Fixesgolang/protobuf#911
Change-Id: Ie2aa4766d6887ceaa9cf06b1f109aa6e6e2a208f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189340
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
In order for protoc-gen-go to output the current version,
it needs to know what version it is currently running as.
However, we cannot rely on the git tags since the tags are not
made until *after* the commit has been submitted.
Instead, we manually encode the version into the code and
make sure that git tags match up with the version in the code.
The version.go file in runtime/protoimpl contains instructions
for how to make a release. Essentially:
* Every non-release commit has a version string with "devel" in it.
* Every release commit must not have "devel" in it and must be unique.
* The "release process" involves submitting two CLs.
The first CL creates a version string without "devel",
which is the commit that a git tag will actually reference.
The second CL follows immediately and re-introduces "devel"
into the version string.
The following example shows a possible sequence of VersionStrings
for git commits in time-ascending order:
v1.19.0-devel (this CL)
v1.19.0-devel
v1.19.0-devel
v1.19.0-devel
v1.20.0-rc.1 <- tagged
v1.20.0-rc.1.devel
v1.20.0-rc.1.devel
v1.20.0-rc.1.devel
v1.20.0-rc.2 <- tagged
v1.20.0-rc.2.devel
v1.20.0 <- tagged (future public release)
v1.20.0-devel
v1.20.0-devel
v1.20.0-devel
v1.20.0-devel
v1.20.1 <- tagged
v1.20.1-devel
v1.20.1-devel
v1.21.0 <- tagged
v1.21.0-devel
Note that we start today with v1.19.0-devel, which means that our initial
release will be v1.20.0. This number was intentionally chosen since
1) the number 20 has some correlation to the fact that we keep calling
the new implementation the "v2" implementation, and
2) the set of tagged versions for github.com/golang/protobuf
and google.golang.org/protobuf are unlikely to ever overlap.
This way, the version of protoc-gen-go is never ambiguous which module
it was built from.
Now that we have version information, we add support for generating .pb.go
files with the version information recorded. However, we do not emit
these for .pb.go files in our own repository since they are always guaranteed
to be at the right version (enforced by integration_test.go).
Updates golang/protobuf#524
Change-Id: I25495a45042c2aa39a39cb7e7738ae8e831a9d26
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186117
Reviewed-by: Damien Neil <dneil@google.com>
Add methods to protoreflect.{Message,List,Map} to constrict values
assignable to a message field, list element, or map value. These
methods return the default value for scalar fields, the zero value for
scalar list elements and map values, and an empty, mutable value for
messages, lists, and maps.
Deprecate the NewMessage methods on these types, which are superseded.
Updates golang/protobuf#879
Change-Id: I0f064f60c89a239330ccea81523f559f14fd2c4f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/188997
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Fix decoding of zero-length bytes fields to produce a non-nil []byte.
Change-Id: Ifb7791a47df81091700f7226523371d1386fb1ad
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/188765
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The messageState.mi field is atomically checked and set
in generated code to the *MessageInfo associated with that message.
However, the messageState type accesses the mi field without
any atomic loads, thus being a potential race.
We fix this by always calling a messageInfo method that performs
a atomic.LoadPointer on the *MessageInfo.
There is no performance effect from this change on x86 since
an atomic.LoadPointer is identical to a MOV instruction.
From an assembly perspective, there was no memory race previously.
However, the lack of an atomic.LoadPointer meant that the compiler
could in theory reorder the "normal" load to produce truly racy code.
Change-Id: I8afefaf35c1916872781abc0239cbb63d62edf16
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189017
Reviewed-by: Damien Neil <dneil@google.com>
The v2 MessageInfo is needed by v1 to be able to access the OneofWrappers
and Exporter function. The ProtoMessageInfo method can be deleted
once v1 is entirely implemented in terms of v2.
Change-Id: Iabb1b429af5210faffc6477f52b5020b3aa1fb50
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186577
Reviewed-by: Damien Neil <dneil@google.com>
In 2014, when proto3 was being developed, there were a number of early
adopters of the new syntax. Before the finalization of proto3 when
it was released in open-source in July 2016, a decision was made to
strictly validate strings in proto3. However, some of the early adopters
were already using invalid UTF-8 with string fields.
The google.protobuf.FieldOptions.enforce_utf8 option only exists to support
those grandfathered users where they can opt-out of the validation logic.
Practical use of that option in open source is impossible even if a user
specifies the proto1_legacy build tag since it requires a hacked
variant of descriptor.proto that is not externally available.
This CL supports enforce_utf8 by modifiyng internal/filedesc to
expose the flag if it detects it in the raw descriptor.
We add an strs.EnforceUTF8 function as a centralized place to determine
whether to perform validation. Validation opt-out is supported
only in builds with legacy support.
We implement support for validating UTF-8 in all proto3 string fields,
even if they are backed by a Go []byte.
Change-Id: I9c0628b84909bc7181125f09db730c80d490e485
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186002
Reviewed-by: Damien Neil <dneil@google.com>
CL/174938 removed these methods in favor of a method that returned
only the descriptors. This CL adds back in the Type methods alongside
the Descriptor methods.
In a vast majority of protobuf usages, only the descriptor information
is needed. However, there is a small percentage that legitimately needs
the Go type information. We should provide both, but document that the
descriptor-only information is preferred.
Change-Id: Ia0a098997fb1bd009994940ae8ea5257ccd87cae
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184578
Reviewed-by: Damien Neil <dneil@google.com>
Usage of these is pervasive in code which works with proto2, and proto2
will be with us for a long, long time to come. Move them to the proto
package.
Change-Id: I1b2e57429fd5a8f107a848a4492d20c27f304bd7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185543
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
We define MessageState, which is essentially an atomically set *MessageInfo.
By nesting this as the first field in every generated message, we can
implement the reflective methods on a *MessageState when obtained by
unsafe casting a concrete message pointer as a *MessageState.
The MessageInfo held by MessageState provides additional Go type information
to interpret the memory that comes after the contents of the MessageState.
Since we are nesting a MessageState in every message,
the memory use of every message instance grows by 8B.
On average, the body of ProtoReflect grows from 133B to 202B (+50%).
However, this is offset by XXX_Methods, which is 108B and
will be removed in a future CL. Taking into account the eventual removal
of XXX_Methods, this is a net reduction of 25%.
name old time/op new time/op delta
Name/Value-4 70.3ns ± 2% 17.5ns ± 6% -75.08% (p=0.000 n=10+10)
Name/Nil-4 70.6ns ± 3% 33.4ns ± 2% -52.66% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
Name/Value-4 16.0B ± 0% 0.0B -100.00% (p=0.000 n=10+10)
Name/Nil-4 16.0B ± 0% 0.0B -100.00% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Name/Value-4 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10)
Name/Nil-4 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10)
Change-Id: I92bd58dc681c57c92612fd5ba7fc066aea34e95a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185460
Reviewed-by: Damien Neil <dneil@google.com>
The code organization is simpler if we keep the functions encoding and
decoding a particular type (e.g., maps) together rather than split
across files. Rename various "encode" files to "codec" in preparation
for adding fast-path decoding.
Change-Id: If1e271da99d31533ffefc19b1fc847936fa9484a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185241
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Hyrum's Law dictates that if we do not prevent naughty behavior,
people will rely on it. If we do not validate that the provided
file descriptor is correct today, it will be near impossible
to add proper validation checks later on.
The logic added validates that the provided file descriptor is
correct according to the same semantics as protoc,
which was reversed engineered to derive the set of rules implemented here.
The rules are unfortunately complicated because protobuf is a language
full of many non-orthogonal features. While our logic is complicated,
it is still 1/7th the size of the equivalent C++ code!
Change-Id: I6acc5dc3bd2e4c6bea6cd9e81214f8104402602a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/184837
Reviewed-by: Damien Neil <dneil@google.com>
Immediately abort (un)marshal operations when encountering invalid UTF-8
data in proto3 strings. No other proto implementation supports non-UTF-8
data in proto3 strings (and many reject it in proto2 strings as well).
Producing invalid output is an interoperability threat (other
implementations won't be able to read it).
The case where existing string data is found to contain non-UTF8 data is
better handled by changing the field to the `bytes` type, which (aside
from UTF-8 validation) is wire-compatible with `string`.
Remove the errors.NonFatal type, since there are no remaining cases
where it is needed. "Non-fatal" errors which produce results and a
non-nil error are problematic because they compose poorly; the better
approach is to take an option like AllowPartial indicating which
conditions to check for.
Change-Id: I9d189ec6ffda7b5d96d094aa1b290af2e3f23736
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/183098
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The primary (cross-language) protobuf repository contains benchmark data
sets. Add benchmarks using this data. (A version of this benchmark exists
in the protobuf repository, but it uses the v1 API and isn't trivial to
get working.)
Fetch the small benchmark datasets from the
github.com/protocolbuffers/protobuf repo by default. Add a
download_benchdata.bash script which fetches the larger datasets as
well.
Generate necessary packages under internal/testprotos/benchmarks.
To run:
go run ./proto -bench=BenchmarkData
Usual caveats about benchmarking apply: While these benchmarks use
realistic data, isolated microbenchmarking of proto operations is not
necessarily representitive of performance in production systems.
Change-Id: I58d107554baf104568c86997b5ad50be8b2a5790
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/183297
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The internal/fileinit package is split apart into two packages:
* internal/filedesc constructs descriptors from the raw proto.
It is very similar to the previous internal/fileinit package.
* internal/filetype wraps descriptors with Go type information
Overview:
* The internal/fileinit package will be deleted in a future CL.
It is kept around since the v1 repo currently depends on it.
* The internal/prototype package is deleted. All former usages of it
are now using internal/filedesc instead. Most significantly,
the reflect/protodesc package was almost entirely re-written.
* The internal/impl package drops support for messages that do not
have a Descriptor method (pre-2016). This removes a significant amount
of technical debt.
filedesc.Builder to parse raw descriptors.
* The internal/encoding/defval package now handles enum values by name.
Change-Id: I3957bcc8588a70470fd6c7de1122216b80615ab7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/182360
Reviewed-by: Damien Neil <dneil@google.com>
Added API:
Message.Len
Message.Range
Message.Has
Message.Clear
Message.Get
Message.Set
Message.Mutable
Message.NewMessage
Message.WhichOneof
Message.GetUnknown
Message.SetUnknown
Deprecated API (to be removed in subsequent CL):
Message.KnownFields
Message.UnknownFields
The primary difference with the new API is that the top-level
Message methods are keyed by FieldDescriptor rather than FieldNumber
with the following semantics:
* For known fields, the FieldDescriptor must exactly match the
field descriptor known by the message.
* For extension fields, the FieldDescriptor must implement ExtensionType,
where ContainingMessage.FullName matches the message name, and
the field number is within the message's extension range.
When setting an extension field, it automatically stores
the extension type information.
* Extension fields are always considered nullable,
implying that repeated extension fields are nullable.
That is, you can distinguish between a unpopulated list and an empty list.
* Message.Get always returns a valid Value even if unpopulated.
The behavior is already well-defined for scalars, but for unpopulated
composite types, it now returns an empty read-only version of it.
Change-Id: Ia120630b4db221aeaaf743d0f64160e1a61a0f61
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/175458
Reviewed-by: Damien Neil <dneil@google.com>
When encountering a type that does not have a MessageInfo, don't assume
that it's a legacy message that doesn't implement proto.Message. Add a
set of test messages exercising this case (panics prior to the
internal/impl change).
Change-Id: Ic1ec5ecfbe92278fbef44284ff52a0e0622a158c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/182477
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The name descfmt more closely matches the fact that this package
provides pretty-printing for descriptors.
Change-Id: I11b0e4a0302962ee7bbf315fb259d050847cadbb
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/178479
Reviewed-by: Damien Neil <dneil@google.com>
Rename each generated protobuf package such that the base of the
Go package path is always equal to the Go package name to follow
proper Go package naming conventions.
The Go package name is derived from the .proto source file name by
replacing ".proto" with "pb" and stripping all underscores.
Change-Id: Iea05d1b5d94b1b2821ae10276ab771bb2df93c0e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/177380
Reviewed-by: Damien Neil <dneil@google.com>
While it is general convention that the receiver being mutated
is the first argument, the standard library specifically goes against
this convention when it comes to the Unmarshal function.
Switch the ordering of the Unmarshal function to match the Go stdlib.
Change-Id: I893346680233ef9fec7104415a54a0a7ae353378
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/177258
Reviewed-by: Damien Neil <dneil@google.com>
Rename encoding/*pb to follow the convention of prefixing package names
with 'proto':
google.golang.org/protobuf/encoding/protojson
google.golang.org/protobuf/encoding/prototext
Move protogen under a compiler/ directory, just in case we ever do add
more compiler-related packages.
google.golang.org/protobuf/compiler/protogen
Change-Id: I31010cb5cabcea8274fffcac468477b58b56e8eb
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/177178
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Temporarily remove go.mod, since we can't generate an accurate one until
the corresponding v1 change is submitted.
Change-Id: I1e1ad97f2b455e33f61ffaeb8676289795e47e72
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/177000
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This feature seems to be used quite a bit, and the conformance tests
treat this as required, perhaps as a "required option" since the
developer guide states:
"Proto3 JSON parser should reject unknown fields by default but may
provide an option to ignore unknown fields in parsing."
Also, all invalid UTF-8 errors in skipped values are also returned as it
is similar to a parse error, except it is a non-fatal one.
Change-Id: Ia26e9a355daecdbf99af23f3061353fffa32d47d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/174017
Reviewed-by: Damien Neil <dneil@google.com>
The protobuf type system uses the word "descriptor" instead of "type".
We should avoid the "type" verbage when we aren't talking about Go types.
The old names are temporarily kept around for compatibility reasons.
Change-Id: Icc99c913528ead011f7a74aa8399d9c5ec6dc56e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/172238
Reviewed-by: Herbie Ong <herbie@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
The protobuf type system hacks the representation of map entries into that
of a pseudo-message descriptor.
Previously, we made all message descriptors implement MessageType
where type descriptors had a GoType method that simply returned nil.
Unfortunately, this violates a nice property in the Go type system
where being able to assert to a MessageType guarantees that Go type
information is truly associated with that descriptor.
This CL makes it such that message descriptors for map entries
do not implement MessageType.
Change-Id: I23873cb71fe0ab3c0befd8052830ea6e53c97ca9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168399
Reviewed-by: Damien Neil <dneil@google.com>
Generate field numbers for the well-known types,
so that encoding/jsonpb can benefit from them as well.
This CL fixes internal/cmd/generate-protos, which was silently failing
because the modulePath was not properly initialized. We fix this by
moving it to the start of the init function.
Change-Id: I87637176f29218cffa512b4baa49f39dae924061
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168497
Reviewed-by: Herbie Ong <herbie@google.com>
The .pb.go.meta file is checked into the repository as testdata.
The deliberate instability of outputs breaks golden tests within
our own repository. It is reasonable for us to depend on stability
since we control the output.
Change-Id: I1f73027a08a0757732d46610f334d40840cc4cfd
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/168001
Reviewed-by: Herbie Ong <herbie@google.com>
The prototype package was initially used by generated reflection support,
but has now been replaced by internal/fileinit.
Eventually, this functionality should be deleted and re-written in terms
of other components in the repo.
Usages that prototype currently provides (but should be moved) are:
* Constructing standalone messages and enums, which is behavior we should
provide in reflect/protodesc. The google.protobuf.{Enum,Type} are well-known
proto messages designed for this purpose.
* Constructing placeholder files, enums, and messages.
* Consructing protoreflect.{Message,Enum,Extension}Types, which are protobuf
descriptors with associated Go type information.
Change-Id: Id7dbefff952682781b439aa555508c59b2629f9e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/167383
Reviewed-by: Damien Neil <dneil@google.com>
Now that binary bloat concerns have been addressed, remove the flag
to control whether to generate support for reflection.
Change-Id: Ia0d11183707572caaf91d2f01dfa77e3aac0a417
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/167140
Reviewed-by: Damien Neil <dneil@google.com>
Running "go build ./..." does not descend into testdata directories.
However, the testdata in this repository is source code that is
intended to build properly. We could rename the directory, but that does
not test whether the generated packages can initialize properly.
Thus, we generate a trivial test that simply blank imports all packages.
Doing this reveals that some of the generated files have incorrect imports,
leading to registration conflicts.
To avoid introducing a dependency on gRPC from our go.mod file, we put
the testdata directories in their own module. Also, we avoid running
internal/testprotos through the grpc plugin because the servie and method
definitions in that directory are more for testing proto file initialization
rather than testing grpc generation.
Change-Id: Iaa6a06449787a085200e31bc7606e3ac904d3180
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/164917
Reviewed-by: Damien Neil <dneil@google.com>
Generate a list of descriptor fields from the descriptor.proto itself
as an internal package. Use these fields for the internal implementation.
Change-Id: Ib1ab0c5c6deb332ba6c8018ef55136b7e5974944
Reviewed-on: https://go-review.googlesource.com/c/164864
Reviewed-by: Herbie Ong <herbie@google.com>
Create a single binary for handling generation of protos.
This replaces previous logic spread throughout the repo in:
* regenerate.bash
* cmd/protoc-gen-go/golden_test.go
* cmd/protoc-gen-go-grpc/golden_test.go
* (indirectly) internal/protogen/goldentest
One of the problems with the former approaches is that they relied on
a version of protoc that was specific to a developer's workstation.
This meant that the result of generation was not hermetic.
To address this, we rely on the hard-coded version of protobuf specified
in the test.bash script.
A summary of changes in this CL are:
* The internal_gengo.GenerateFile and internal_gengogrpc.GenerateFile
functions are unified to have consistent signatures. It seems that the
former accepted a *protogen.GeneratedFile to support v1 where gRPC code
was generated into the same file as the base .pb.go file. However, the
same functionality can be achieved by having the function return
the generated file object.
* The test.bash script patches the protobuf toolchain to have properly
specified go_package options in each proto source file.
* The test.bash script accepts a "-regenerate" argument.
* Add generation for the well-known types. Contrary to how these were
laid out in the v1 repo, all the well-known types are placed in the
same Go package.
* Add generation for the conformance proto.
* Remove regenerate.bash
* Remove internal/protogen
* Remove cmd/protoc-gen-go/golden_test.go
* Remove cmd/protoc-gen-go-grpc/golden_test.go
* Add cmd/protoc-gen-go/annotation_test.go
Change-Id: I4a1a97ae6f66e2fabcf4e4d292c95ab2a2db0248
Reviewed-on: https://go-review.googlesource.com/c/164477
Reviewed-by: Damien Neil <dneil@google.com>
This CL takes a significantly different approach to generating support
for protobuf reflection. The previous approach involved generating a
large number of Go literals to represent the reflection information.
While that approach was correct, it resulted in too much binary bloat.
The approach taken here initializes the reflection information from
the raw descriptor proto, which is a relatively dense representation
of the protobuf reflection information. In order to keep initialization
cost low, several measures were taken:
* At program init, the bare minimum is parsed in order to initialize
naming information for enums, messages, extensions, and services declared
in the file. This is done because those top-level declarations are often
relevant for registration.
* Only upon first are most of the other data structures for protobuf
reflection actually initialized.
* Instead of using proto.Unmarshal, a hand-written unmarshaler is used.
This allows us to avoid a dependendency on the descriptor proto and also
because the API for the descriptor proto is fundamentally non-performant
since it requires an allocation for every primitive field.
At a high-level, the new implementation lives in internal/fileinit.
Several changes were made to other parts of the repository:
* cmd/protoc-gen-go:
* Stop compressing the raw descriptors. While compression does reduce
the size of the descriptors by approximately 2x, it is a pre-mature
optimization since the descriptors themselves are around 1% of the total
binary bloat that is due to generated protobufs.
* Seeding protobuf reflection from the raw descriptor significantly
simplifies the generator implementation since it is no longer responsible
for constructing a tree of Go literals to represent the same information.
* We remove the generation of the shadow types and instead call
protoimpl.MessageType.MessageOf. Unfortunately, this incurs an allocation
for every call to ProtoReflect since we need to allocate a tuple that wraps
a pointer to the message value, and a pointer to message type.
* internal/impl:
* We add a MessageType.GoType field and make it required that it is
set prior to first use. This is done so that we can avoid calling
MessageType.init except for when it is actually needed. The allows code
to call (*FooMessage)(nil).ProtoReflect().Type() without fearing that the
init code will run, possibly triggering a recursive deadlock (where the
init code depends on getting the Type of some dependency which may be
declared within the same file).
* internal/cmd/generate-types:
* The code to generate reflect/prototype/protofile_list_gen.go was copied
and altered to generated internal/fileinit.desc_list_gen.go.
At a high-level this CL adds significant technical complexity.
However, this is offset by several possible future changes:
* The prototype package can be drastically simplified. We can probably
reimplement internal/legacy to use internal/fileinit instead, allowing us
to drop another dependency on the prototype package. As a result, we can
probably delete most of the constructor types in that package.
* With the prototype package significantly pruned, and the fact that generated
code no longer depend on depends on that package, we can consider merging
what's left of prototype into protodesc.
Change-Id: I6090f023f2e1b6afaf62bd3ae883566242e30715
Reviewed-on: https://go-review.googlesource.com/c/158539
Reviewed-by: Herbie Ong <herbie@google.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Most usages of New actually prefer to interact with the reflective view
rather than the native Go type. Thus, change New to return that instead.
This parallels reflect.New, which returns the reflective view
(i.e., reflect.Value) instead of native type (i.e., interface{}).
We make the equivalent change to KnownFields.NewMessage, List.NewMessage,
and Map.NewMessage for consistency.
Since this is a subtle change where the type system will not always
catch the changed type, this change was made by both changing the type
and renaming the function to NewXXX and manually looking at every usage
of the the function to ensure that the usage correctly operates
on either the native Go type or the reflective view of the type.
After the entire codebase was cleaned up, a rename was performed to convert
NewXXX back to New.
Change-Id: I153fef627b4bf0a427e4039ce0aaec52e20c7950
Reviewed-on: https://go-review.googlesource.com/c/157077
Reviewed-by: Damien Neil <dneil@google.com>
Add proto.Unmarshal.
Test cases all produce identical results to the v1 unmarshaller.
Change-Id: I42259266018a14e88a650c5d83a043cb17a3a15d
Reviewed-on: https://go-review.googlesource.com/c/153918
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Custom descriptor types would want to benefit from descriptor formatting.
As such, move the logic out from prototype into an internal package for
the benefit of usages outside the prototype package.
Change-Id: I4bb2144221e656aa36909d33a77189fe084f700b
Reviewed-on: https://go-review.googlesource.com/c/152777
Reviewed-by: Herbie Ong <herbie@google.com>
