7 Commits

Author SHA1 Message Date
Damien Neil
cedb595154 proto, internal/impl: avoid string->[]byte conversions
We trust the compiler to optimize away the string->[]byte conversion in
code like:

	b = wire.AppendBytes(b, []byte(s))

In testing (go 1.12.5 linux/amd64), this optimization is not happening.
Perhaps newer versions of the compiler will optimize this, but we
shouldn't rely on it; avoid unnecessary conversions.

Benchmark differences vs https://golang.org/cl/171462:

  name                                   old time/op    new time/op    delta
  Wire/Marshal/google_message1_proto2-6     310ns ± 2%     189ns ± 3%  -39.20%  (p=0.000 n=8+8)
  Wire/Marshal/google_message1_proto3-6     389ns ± 8%     261ns ± 2%  -33.03%  (p=0.000 n=8+8)
  Wire/Marshal/google_message2-6            103µs ±11%      59µs ± 4%  -42.17%  (p=0.000 n=8+8)

  name                                   old alloc/op   new alloc/op   delta
  Wire/Marshal/google_message1_proto2-6      592B ± 0%      240B ± 0%  -59.46%  (p=0.000 n=8+8)
  Wire/Marshal/google_message1_proto3-6      576B ± 0%      224B ± 0%  -61.11%  (p=0.000 n=8+8)
  Wire/Marshal/google_message2-6            196kB ± 0%      90kB ± 0%  -54.05%  (p=0.000 n=8+8)

  name                                   old allocs/op  new allocs/op  delta
  Wire/Marshal/google_message1_proto2-6      5.00 ± 0%      1.00 ± 0%  -80.00%  (p=0.000 n=8+8)
  Wire/Marshal/google_message1_proto3-6      5.00 ± 0%      1.00 ± 0%  -80.00%  (p=0.000 n=8+8)
  Wire/Marshal/google_message2-6            1.66k ± 0%     0.00k ± 0%  -99.94%  (p=0.000 n=8+8)

Change-Id: Idab7634b8c86604dffa46895ba2e61be38c9bd9c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/183380
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-06-24 17:41:45 +00:00
Damien Neil
8c86fc5e7d all: remove non-fatal UTF-8 validation errors (and non-fatal in general)
Immediately abort (un)marshal operations when encountering invalid UTF-8
data in proto3 strings. No other proto implementation supports non-UTF-8
data in proto3 strings (and many reject it in proto2 strings as well).
Producing invalid output is an interoperability threat (other
implementations won't be able to read it).

The case where existing string data is found to contain non-UTF8 data is
better handled by changing the field to the `bytes` type, which (aside
from UTF-8 validation) is wire-compatible with `string`.

Remove the errors.NonFatal type, since there are no remaining cases
where it is needed. "Non-fatal" errors which produce results and a
non-nil error are problematic because they compose poorly; the better
approach is to take an option like AllowPartial indicating which
conditions to check for.

Change-Id: I9d189ec6ffda7b5d96d094aa1b290af2e3f23736
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/183098
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-06-20 20:55:13 +00:00
Joe Tsai
378c1329de reflect/protoreflect: add alternative message reflection API
Added API:
	Message.Len
	Message.Range
	Message.Has
	Message.Clear
	Message.Get
	Message.Set
	Message.Mutable
	Message.NewMessage
	Message.WhichOneof
	Message.GetUnknown
	Message.SetUnknown

Deprecated API (to be removed in subsequent CL):
	Message.KnownFields
	Message.UnknownFields

The primary difference with the new API is that the top-level
Message methods are keyed by FieldDescriptor rather than FieldNumber
with the following semantics:
* For known fields, the FieldDescriptor must exactly match the
field descriptor known by the message.
* For extension fields, the FieldDescriptor must implement ExtensionType,
where ContainingMessage.FullName matches the message name, and
the field number is within the message's extension range.
When setting an extension field, it automatically stores
the extension type information.
* Extension fields are always considered nullable,
implying that repeated extension fields are nullable.
That is, you can distinguish between a unpopulated list and an empty list.
* Message.Get always returns a valid Value even if unpopulated.
The behavior is already well-defined for scalars, but for unpopulated
composite types, it now returns an empty read-only version of it.

Change-Id: Ia120630b4db221aeaaf743d0f64160e1a61a0f61
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/175458
Reviewed-by: Damien Neil <dneil@google.com>
2019-06-17 17:33:24 +00:00
Damien Neil
e89e6244e0 all: change module to google.golang.org/protobuf
Temporarily remove go.mod, since we can't generate an accurate one until
the corresponding v1 change is submitted.

Change-Id: I1e1ad97f2b455e33f61ffaeb8676289795e47e72
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/177000
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-05-14 17:28:29 +00:00
Damien Neil
bc310b58c6 proto: validate UTF-8 in proto3 strings
Change-Id: I6a495730c3f438e7b2c4ca86edade7d6f25aa47d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/171700
Reviewed-by: Herbie Ong <herbie@google.com>
2019-04-12 17:43:10 +00:00
Damien Neil
96c229ab14 proto: check for required fields in encoding/decoding
Change-Id: I0555a92e0399782f075b1dcd248e880dd48c7d6d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/170579
Reviewed-by: Herbie Ong <herbie@google.com>
2019-04-04 17:56:20 +00:00
Damien Neil
99f24c33b0 proto: wire encoding support
Add proto.Marshal.

Change-Id: If7254bb4c4cbbee782a2a163762f9fcf98e7ab08
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/167388
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-03-14 19:12:27 +00:00