10 Commits

Author SHA1 Message Date
Damien Neil
c600d6c086 all: do best-effort initialization check on fast path unmarshal
Add a fast check for required fields to the fast path unmarshal.
This is best-effort and will fail to detect some initialized
messages: Messages with more than 64 required fields, messages
split across multiple tags, possibly other cases.

In the cases where it works (which is most of them in practice),
this permits us to skip the IsInitialized check.

Change-Id: I6b70953a333033a5e64fb7ca37a59786cb0f75a0
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/215878
Reviewed-by: Joe Tsai <joetsai@google.com>
2020-01-22 20:57:14 +00:00
Damien Neil
f0831e87e2 internal/impl: change unmarshal func return to unmarshalOptions
The fast-path unmarshal funcs return the number of bytes consumed.

Change these functions to return an unmarshalOutput struct instead, to
make it easier to add to the results. This is groundwork for allowing
the fast-path unmarshaler to indicate when the unmarshaled message is
known to be initialized.

Change-Id: Ia8c44731a88f5be969a55cd98ea26282f412c7ae
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/215720
Reviewed-by: Joe Tsai <joetsai@google.com>
2020-01-22 00:22:58 +00:00
Damien Neil
f2427c09d6 proto, internal/impl: reject invalid field numbers in map items
Change-Id: I44a44a36538f6f8b94078b43711d865edb6244f5
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/212257
Reviewed-by: Herbie Ong <herbie@google.com>
2019-12-21 00:16:12 +00:00
Damien Neil
7e690b5b4c internal/impl: fix map decode when value is before key
Fix a bug in handling the case where the encoding for a map item places
the value field (2) before the key field (1).

Change-Id: I2e6ad9af729a199e960e566ed7ef96bba3726990
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/211804
Reviewed-by: Joe Tsai <joetsai@google.com>
2019-12-18 17:42:10 +00:00
Damien Neil
3e42b667d2 internal/impl: faster map fast path
Avoid using protobuf reflection on map values in the fast path. Range
operations in particular are expensive in protoreflect, because the
closure passed to Map.Range escapes.

Iterate maps using a reflect.MapIter when available.

When operating on maps of messages where we have a *MessageInfo for the
message type, directly jump to the fast-path *MessageInfo methods rather
than passing through the proto package.

Benchmarks deltas for a google.protobuf.Struct with JSON represention:
  {"parameters":{"a":{"b":{"c":{"d":{"e":{"f":{"g":{}}}}}}}}}

Compared to previous revision:

  name                      old time/op  new time/op  delta
  NestedStruct/Size         7.22µs ± 2%  4.84µs ± 2%  -32.96%  (p=0.000 n=8+8)
  NestedStruct/Size-8       9.30µs ± 2%  5.89µs ± 2%  -36.60%  (p=0.000 n=8+8)
  NestedStruct/Marshal      77.6µs ±12%   9.8µs ± 4%  -87.33%  (p=0.000 n=8+8)
  NestedStruct/Marshal-8    91.6µs ± 2%  11.9µs ± 2%  -86.99%  (p=0.000 n=8+8)
  NestedStruct/Unmarshal    11.5µs ± 4%   8.7µs ± 2%  -24.76%  (p=0.000 n=8+8)
  NestedStruct/Unmarshal-8  15.4µs ± 4%  11.9µs ± 2%  -22.41%  (p=0.000 n=8+8)

Compared to github.com/golang/protobuf:

  name                      old time/op  new time/op  delta
  NestedStruct/Size         5.42µs ± 1%  4.84µs ± 2%  -10.61%  (p=0.000 n=8+8)
  NestedStruct/Size-8       6.34µs ± 2%  5.89µs ± 2%   -7.10%  (p=0.000 n=8+8)
  NestedStruct/Marshal      12.5µs ± 2%   9.8µs ± 4%  -21.41%  (p=0.000 n=7+8)
  NestedStruct/Marshal-8    14.1µs ± 3%  11.9µs ± 2%  -15.52%  (p=0.000 n=8+8)
  NestedStruct/Unmarshal    9.66µs ± 1%  8.65µs ± 2%  -10.40%  (p=0.000 n=7+8)
  NestedStruct/Unmarshal-8  11.7µs ± 3%  11.9µs ± 2%   +1.95%  (p=0.038 n=8+8)

Change-Id: I0effe6491f30d41f31904777f74eca3ac3694db3
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/211737
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-12-17 22:14:17 +00:00
Damien Neil
4b3a82f6b1 internal/impl: clean up Value codecs
Remove the Go type from consideration when creating Value codecs, as it
is unnecessary. Value codecs convert between wire form and Values,
while Converters convert between Values and the Go type.

Change-Id: Iaa4bc7db81ad0a29dabd42c2229e6f33a0c91c67
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/193457
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-09-05 16:31:05 +00:00
Damien Neil
68b81c3117 internal/impl: store extension values as Values
Change the storage type of ExtensionField from interface{} to
protoreflect.Value.

Replace the codec functions operating on interface{}s with ones
operating on Values.

Values are potentially more efficient, since they can represent
non-pointer types without allocation. This also reduces the number of
types used to represent field values.

Additionally, this change lays groundwork for changing the
user-visible representation of repeated extension fields from
*[]T to []T. The storage type for extension fields must support mutation
(thus *[]T currently); changing the storage type to a Value permits this
without the need to introduce yet another view on field values.

Change-Id: Ida336be14112bb940f655236eb58df21bf312525
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/192218
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-09-03 20:58:28 +00:00
Joe Tsai
e815d6a43b all: remove dead code
Change-Id: I1344d6afca9d3348db849c2b5f387ac18b80d2ba
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/189021
Reviewed-by: Damien Neil <dneil@google.com>
2019-08-06 21:16:48 +00:00
Damien Neil
e91877de26 internal/impl: add fast-path unmarshal
Benchmarks run with:
  go test ./benchmarks/ -bench=Wire  -benchtime=500ms -benchmem -count=8

Fast-path vs. parent commit:

  name                                      old time/op    new time/op    delta
  Wire/Unmarshal/google_message1_proto2-12    1.35µs ± 2%    0.45µs ± 4%  -67.01%  (p=0.000 n=8+8)
  Wire/Unmarshal/google_message1_proto3-12    1.07µs ± 1%    0.31µs ± 1%  -71.04%  (p=0.000 n=8+8)
  Wire/Unmarshal/google_message2-12            691µs ± 2%     188µs ± 2%  -72.78%  (p=0.000 n=7+8)

  name                                      old allocs/op  new allocs/op  delta
  Wire/Unmarshal/google_message1_proto2-12      60.0 ± 0%      25.0 ± 0%  -58.33%  (p=0.000 n=8+8)
  Wire/Unmarshal/google_message1_proto3-12      42.0 ± 0%       7.0 ± 0%  -83.33%  (p=0.000 n=8+8)
  Wire/Unmarshal/google_message2-12            28.6k ± 0%      8.5k ± 0%  -70.34%  (p=0.000 n=8+8)

Fast-path vs. -v1:

  name                                      old time/op    new time/op    delta
  Wire/Unmarshal/google_message1_proto2-12     702ns ± 1%     445ns ± 4%   -36.58%  (p=0.000 n=8+8)
  Wire/Unmarshal/google_message1_proto3-12     604ns ± 1%     311ns ± 1%   -48.54%  (p=0.000 n=8+8)
  Wire/Unmarshal/google_message2-12            179µs ± 3%     188µs ± 2%    +5.30%  (p=0.000 n=7+8)

  name                                      old allocs/op  new allocs/op  delta
  Wire/Unmarshal/google_message1_proto2-12      26.0 ± 0%      25.0 ± 0%    -3.85%  (p=0.000 n=8+8)
  Wire/Unmarshal/google_message1_proto3-12      8.00 ± 0%      7.00 ± 0%   -12.50%  (p=0.000 n=8+8)
  Wire/Unmarshal/google_message2-12            8.49k ± 0%     8.49k ± 0%    -0.01%  (p=0.000 n=8+8)

Change-Id: I6247ac3fd66a63d9acb902cbd192094ee3d151c3
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185147
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-07-09 19:56:42 +00:00
Damien Neil
edf7bdda31 internal/impl: rename encode->codec in various places
The code organization is simpler if we keep the functions encoding and
decoding a particular type (e.g., maps) together rather than split
across files. Rename various "encode" files to "codec" in preparation
for adding fast-path decoding.

Change-Id: If1e271da99d31533ffefc19b1fc847936fa9484a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/185241
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-07-08 20:12:42 +00:00