898 Commits

Author SHA1 Message Date
Damien Neil
711224230b internal/encoding/text: escape Unicode control characters in strings
Escape not only ASCII control characters, but Unicode as well.

Change-Id: I5f5791ae51fc5624599f66ce012ecef364e7ea97
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/262682
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Joe Tsai <joetsai@google.com>
2020-10-15 23:10:26 +00:00
Damien Neil
b1cd8e38bc internal/encoding/text: escape \x7f (DEL) in strings
We already escape ASCII control characters in the range [0x00,0x20).
Escape the one control character outside this range as well.

Change-Id: If954da0d4a178b36128d1a53d25397d1b3fd2e17
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/262681
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2020-10-15 21:51:19 +00:00
Joe Tsai
9712361eda all: rely on dynamic dependency check for genproto
The genproto module incurs an unfortunate amount of dependency bloat
since it has a relatively large set of transitive dependencies.
Even though these are brought in as a weak dependency
(i.e., there is no hard code dependency on genproto),
it complicates dependency inspection since users need to vet
modules that they may not actually be using.

Avoid this weak dependency and instead rely on a dynamic check
to ensure that a sufficiently new version of genproto is used.
While it can't statically prevent old versions of genproto from being
linked into a binary, it guarantees that it will panic at init
with a helpful message on what's wrong.

Change-Id: I2b5a9759a5cd4c4c57aced54278502b1b6352ba7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/262679
Trust: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
2020-10-15 21:46:17 +00:00
Tzu-Chiao Yeh
f62736dede proto/equal: reduce equal scalar value allocation
Instead of lifting values to interface and then do equality test,
compare scalar values with defined types to reduce allocation
from interface.

Change-Id: Ieb777fbd1a48c83d896d0ebe6ad605433f44c16c
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/253100
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Trust: Damien Neil <dneil@google.com>
Trust: Joe Tsai <thebrokentoaster@gmail.com>
2020-10-08 23:42:17 +00:00
Joe Tsai
53a58a916f cmd/protoc-gen-go: remove unused protoV1Package variable
The logic that formerly depended on this was deleted in
https://golang.org/cl/259901. This is dead code now.

Updates 

Change-Id: Ieb1b14243d9503b1baac7026c6ce226923873405
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/260817
Trust: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
2020-10-08 21:11:27 +00:00
Joe Tsai
687df2333b reflect/protodesc: fix round-tripping for package field
If a .proto file does not have a package statement,
then protoc will not populate the package field.
Emulate the same behavior for ToFileDescriptorProto.

Fixes 

Change-Id: I617102e50c96f0fa1bac6a6bc4de1cc485750bb9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/259899
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Joe Tsai <thebrokentoaster@gmail.com>
2020-10-07 22:44:53 +00:00
Joe Tsai
4394568c4b cmd/protoc-gen-go: remove reference to legacy ProtoPackageIsVersion4
A reference to this constant was originally generated to ensure that users
were using a sufficiently new version of the github.com/golnag/protobuf
module to avoid subtle runtime incompatibility issues.
Such a technique is not needed if users were using modules,
which would enforce this constraint based on our go.mod requirements.

When we first released the google.golang.org/protobuf module,
we suggested that we would consider removing it after 6 months.
It has now been 7 months since the release.

Fixes 

Change-Id: I9f2010da27b27fe6b12541a15833721441df8409
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/259901
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Joe Tsai <thebrokentoaster@gmail.com>
2020-10-07 22:34:28 +00:00
Joe Tsai
ac374a2335 internal/encoding/messageset: fix spelling typo
Change-Id: Ic397fba6b382fbbe72779aa919f9109c23c8f209
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/259898
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Joe Tsai <thebrokentoaster@gmail.com>
2020-10-07 22:15:00 +00:00
Joe Tsai
db5c900f0c all: fix touching of dependency directories
Adjust the setup script in integration_test.go to better keep
track of what directories are in current use.
Previously, the staticcheck directory was not tracked as used,
causing it to be deleted unnecessarily.

We ensure consistent tracking, but having each depedency setup step
start with a startWork call, and end with a finishWork call.

Change-Id: I4f5d929f323509237f4f44ed8f57d71b8756f031
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/253617
Reviewed-by: Damien Neil <dneil@google.com>
2020-09-08 20:20:17 +00:00
Joe Tsai
5ebc0b43aa all: drop 386 support for darwin
Darwin 1.15 no longer supports i386 code execution.
Stop supporting it.

See https://support.apple.com/en-us/HT208436
See http://golang.org/issue/34749

Change-Id: I2d1f3cb933ee2555aa95c72b1cba7ca3847e0146
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/253618
Reviewed-by: Damien Neil <dneil@google.com>
2020-09-08 20:15:21 +00:00
Joe Tsai
0fd4f3a506 encoding/protojson: restrict valid values for google.protobuf.Value.number_value
The purpose of struct.proto is to be an exact mapping of JSON in protobufs.
Since JSON doesn't support NaN and Inf, we should reject serialization
of such values. Prior to this CL, they would be serialzed as a JSON string,
which would change the interpretation of the value when round-tripped.

Fixes 

Change-Id: I6dba9973b1c24d99e5688b509611c0a952c00022
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/247737
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Herbie Ong <herbie@google.com>
2020-09-04 00:11:24 +00:00
Joe Tsai
16365ed3d8 all: update to go1.13.15, go1.14.7, and go1.15
Change-Id: I9675b6e1196afb185fab1c2d650894cf68b325e5
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/248277
Reviewed-by: Damien Neil <dneil@google.com>
2020-08-17 16:53:14 +00:00
Joe Tsai
1a3946737f internal/encoding/text: remove dead code
The newline field is unused. Remove it.

Change-Id: I8a6c6ea64eff718920102b87aa1fc66d8e736ac7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/248297
Reviewed-by: Herbie Ong <herbie@google.com>
2020-08-12 21:05:01 +00:00
Joe Tsai
a180de9f97 testing/protocmp: fix representation of empty bytes
The protoreflect API does not use the distinction between nil and non-nil
empty []byte to preserve presence. As such, avoid accidentally presenting
such information through the transformation.

Change-Id: Ia46b5e2d3fb534d7cd29f478ad47722bfaa06810
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/247458
Reviewed-by: Damien Neil <dneil@google.com>
2020-08-08 01:16:14 +00:00
Joe Tsai
e96d591a68 internal/msgfmt: fix formatting of negative Duration
If the seconds is zero, but the nanoseconds are negative,
there should still be a leading negative sign.

Change-Id: Ia72a26bc3455a80e572b6bf2ff41395381f811c7
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/247457
Reviewed-by: Herbie Ong <herbie@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2020-08-08 01:07:43 +00:00
Joe Tsai
f2423bb714 encoding/protojson: simplify Duration formatting
Simplify the implementation by reducing the number of branches.

Change-Id: I6e2ffee0fc2d77f7e2a70f76e03d081f4fc0e99d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/247459
Reviewed-by: Herbie Ong <herbie@google.com>
2020-08-08 00:59:21 +00:00
cybrcodr
a709e31e5d types/known/fieldmaskpb: repeated and map fields are only valid in the last position of a path
Fixes .

Change-Id: Ia40c287036889c798305cdaf7c8930811d25f957
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/246097
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2020-08-05 23:11:51 +00:00
Joe Tsai
5d63473da8 internal/impl: add runtime support for *[]byte unknown representation
This CL adds runtime support for unknown fields to be represented
as *[]byte in addition to the current representation as []byte.
This CL does not change generated code to use *[]byte.

Comparison between using *[]byte and []byte:

• Every message supports unknown fields, so use of []byte
  expands a message size by 24B (for 64-bit systems).
  In contrast, *[]byte only expands a message by 8B.
  This has significant memory implications for small messages.

• If unknown fields are encountered, *[]byte has extra overhead
  allocating the 24B slice header. However, it is assumed
  that messages rarely see any unknown fields at runtime,
  or generally do so for a temporary period of time.

Change-Id: I81935e4ea7394166e61ff4579f76f59fa792dfc9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/244937
Reviewed-by: Damien Neil <dneil@google.com>
2020-07-28 16:36:39 +00:00
Joe Tsai
4194195e12 internal/impl: rename legacyProtoMethods as aberrantProtoMethods
This variable is only used by the aberrantMessage type,
rename it to properly reflect what it is used for.

Change-Id: I081e8954eb1b282f01f6d2d764bffaf31c6677e9
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/245077
Reviewed-by: Damien Neil <dneil@google.com>
2020-07-27 20:52:20 +00:00
Damien Neil
a0f95d5b14 internal/impl: make errInvalidUTF8 be a proto.Error
Fixes .

Change-Id: If95f9f79fff76dc0c44f7ee96e484f59fa287e58
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/244718
Reviewed-by: Joe Tsai <joetsai@google.com>
2020-07-24 21:43:34 +00:00
Joe Tsai
1ecb1f411c proto: add MessageName helper
The MessageName returns the full name of a message.
It is a shorthand for:

	var name protoreflect.FullName
	if m != nil {
		name = m.ProtoReflect().Descriptor().FullName()
	}

Generally, we avoid the addition of helper functions unless
their use is justified. Arguments for this helper:

• The legacy proto.MessageName is widely used.
  For example, inside Google, there are thousands of usages of it.
  It is commonly used in logging and error construction to
  report the name of a protobuf message.

• The fact that a nil-check may be neccessary means that users
  either forget the nil-check and risk a nil-panic in production code,
  or users have to write cumbersome logic to check for it.

  For example, compare use with the helper:
      return fmt.Errorf("invalid message type: %v", proto.MessageName(m))
  to use without the helper:
      var messageName protoreflect.FullName
      if m != nil {
          messageName = m.ProtoReflect().Descriptor().FullName()
      }
      return fmt.Errorf("invalid message type: %v", messageName)

A point of consideration is whether this should return a string
or a protoreflect.FullName. This CL returns the latter type:

• Most uses of it are for logging via log.Printf or error construction
  via fmt.Errorf where it does not matter what the exact string type is
  since its formatted the same way.

• Another use of it is to check for message type equality by doing
  something similar to:
      if proto.MessageName(got) != proto.MessageName(want) {
          ...
      }
  in which case it still does not matter what the exact string type is.

• The other major use of proto.MessageName is to call proto APIs
  that actually do expect a protoreflect.FullName
  (e.g., protoregistry.GlobalTypes.FindExtensionByNumber) where
  the user currently type-casts the legacy proto.MessageName
  to a protoreflect.FullName anyways.

As such, there does not seem to be much justified use for MessageName
as a string. The rare cases that need a string can trivially type cast
it to a string.

Change-Id: I840758df828eef72e7e63620569363a496366afa
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/242377
Reviewed-by: Damien Neil <dneil@google.com>
2020-07-23 23:43:59 +00:00
Joe Tsai
42cc4c592f reflect/protoreflect: improve source information usability
Added API:
  SourceLocations.ByPath
  SourceLocations.ByDescriptor
  SourceLocation.Next
  SourcePath.String
  SourcePath.Equal

We modify compiler/protogen to use SourceLocations.ByDescriptor.
In retrospect, if this had existed during the development of protogen,
we would not have exposed protogen.Location and related fields.

Change-Id: I58f17e59f90b9ba16f0982c4b71c2542e4ff6e75
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/238000
Reviewed-by: Damien Neil <dneil@google.com>
2020-07-23 21:50:46 +00:00
Joe Tsai
1a290e9a0e reflect/protoreflect: add MessageFieldTypes
The MessageFieldTypes interface (if implemented by a MessageType)
provides Go type information about the fields if they are
an enum or message type.

Change-Id: I68b20f5726377f6b0f2c20a8b6e45f9802b43f67
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/236777
Reviewed-by: Damien Neil <dneil@google.com>
2020-07-23 21:24:58 +00:00
Damien Neil
b5523e32b3 all: return less-specific, but more informative wire unmarshal errors
When proto.Unmarshal fails, it is almost always due to being passed
input which is not a wire-format message. (A text-format message, a
message with framing left intact, and so forth.) An error like
"variable length integer overflow" can be confusing in this case, since
it implies the problem is the varint rather than the input being
entirely wrong.

Replace all Unmarshal parse errors with "cannot parse invalid
wire-format data".

Change-Id: Id97253bd39ac604e569df71778194f37b3c86c28
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/244297
Reviewed-by: Joe Tsai <joetsai@google.com>
2020-07-23 20:09:00 +00:00
Joe Tsai
28b807b56e encoding/protojson: use synthetic @type field for Any messages
In order for the synthetic @type field to potentially get reordered,
we implement insertion of that synthetic field by adding it
as a synthetic field that Range may iterate over.

This change sets up this code to more readily support a
hypothetical serialization mode for canonical serialization.

Change-Id: Ia0015a1a0804c15805dc5f3a3511fcf0f8513418
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/243817
Reviewed-by: Herbie Ong <herbie@google.com>
2020-07-22 17:37:26 +00:00
Joe Tsai
e7c3f89377 cmd/protoc-gen-go: remove name mangling for MessageSet extensions
MessageSet are a proto1 feature only used by Google.
We no longer rely on the name mangling logic performed here,
so remove it.

Change-Id: I5d66ebd86875894632f0d0c1e9816ae47ee0d5f4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/242657
Reviewed-by: Herbie Ong <herbie@google.com>
2020-07-14 22:22:58 +00:00
Joe Tsai
aabe152426 testing/prototest: fix suggestion for field name
We should use TextName instead of Name,
since it is the name that actually appears in the .proto file.

Change-Id: Ie526b200d18b0994d93dbb347c266fa5c50cddfe
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/241658
Reviewed-by: Herbie Ong <herbie@google.com>
2020-07-13 19:57:56 +00:00
Joe Tsai
e14d6b3cdc reflect/protoreflect: add FieldDescriptor.TextName
Add a new TextName accessor that returns the field name that should
be used for the text format. It is usually just the field name, except:
1) it uses the inlined message name for groups,
2) uses the full name surrounded by brackets for extensions, and
3) strips the "message_set_extension" for well-formed extensions
to the proto1 MessageSet.

We make similar adjustments to the JSONName accessor so that it applies
similar semantics for extensions.

The two changes simplifies all logic that wants the humanly readable
name for a field.

Change-Id: I524b6e017fb955146db81819270fe197f8f97980
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/239838
Reviewed-by: Herbie Ong <herbie@google.com>
2020-07-08 23:23:57 +00:00
Joe Tsai
467a9cdc10 all: fix tests for 32-bit architectures
Fix the tests to work on GOARCH=386.
Also add another test suite that tests on GOARCH=386.

Change-Id: I41290998053dc059347adb8af70012dc5cc31811
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/241537
Reviewed-by: Herbie Ong <herbie@google.com>
2020-07-08 22:37:09 +00:00
Martin Möhrmann
91eb7d3a63 protoregistry: avoid checking for '/' in FindMessageByName
Profiling shows FindMessageByName spending considerable time
in strings.LastIndexByte in FindMessageByURL.

This overhead can be avoided by giving FindMessageByName
its own implementation not pruning the name string up
to the last '/' which can not appear in a protobuf name.

Change-Id: Ifec07689f72790ead8b7552cd04dd99fa67b61e4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/240378
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2020-07-01 19:55:25 +00:00
Joe Tsai
92679665d7 internal/order: add a package for ordered iteration over messages and maps
The order package replaces the mapsort and fieldsort packages.
It presents a common API for ordered iteration over message fields
and map fields.

It has a number of pre-defined orderings.

Change-Id: Ie6cd423da30b4757864c352cb04454f21fe07ee2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/239837
Reviewed-by: Herbie Ong <herbie@google.com>
2020-07-01 18:57:38 +00:00
Joe Tsai
b78321453d reflect/protoregistry: centralize MessageSet extension resolution logic
Centralize the MessageSet extension resolution logic in the registry.
This avoids needless replication of this exact logic in multiple places
(for JSON and text) and elsewhere.

Change-Id: I70bfea899e295e8c589f418965bf0dd099f93628
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/240077
Reviewed-by: Herbie Ong <herbie@google.com>
2020-07-01 17:56:02 +00:00
Joe Tsai
1726b83dc4 internal/impl: introduce instability to protoreflect.Message.Range order
Change-Id: I6f53aaf4071b4cbf596f3438a83cf3e5c93f1712
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/240017
Reviewed-by: Herbie Ong <herbie@google.com>
2020-07-01 17:34:30 +00:00
Joe Tsai
a9513ebdb8 all: start v1.25.0-devel
Change-Id: Iab5d460374c67a995a5d3058cfbc55e6aaa21b38
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/239378
Reviewed-by: Damien Neil <dneil@google.com>
2020-06-22 20:32:22 +00:00
Joe Tsai
3f7a61f89b all: release v1.25.0
Change-Id: I1f2ad6da83ea488c814ad10d6890d188ab6c2a27
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/239377
Reviewed-by: Damien Neil <dneil@google.com>
v1.25.0
2020-06-22 20:32:15 +00:00
Kevin Burke
95ba8ac240 cmd/protoc-gen-go: --version should exit 0
Fixes .

Change-Id: Id2ada2353413c3371e677dcf3fdfa6fdc24a3c56
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/239339
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2020-06-22 19:34:47 +00:00
Joe Tsai
5a96da2901 cmd/protoc-gen-go: generate package documentation for well-known types
Change-Id: Ia079805e24d3490301355e09693255b157cabaed
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/239167
Reviewed-by: Damien Neil <dneil@google.com>
2020-06-22 19:32:21 +00:00
Joe Tsai
b0c4001c72 all: update to cmp@v0.5.0 release
Change-Id: If2f6cd6b4ad63a60ad19df8602cc97cca92e5a10
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/238757
Reviewed-by: Herbie Ong <herbie@google.com>
2020-06-19 21:50:03 +00:00
Joe Tsai
c0ad199d14 all: update to go1.13.12 and go1.14.4
Change-Id: I38b18cf8124fcc52d56741e0cefb68fe973ab087
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/238758
Reviewed-by: Herbie Ong <herbie@google.com>
2020-06-19 21:39:59 +00:00
Joe Tsai
c9c87ce6a4 internal/filedesc: use genid constants for map entry fields
Change-Id: If551fc4f3e8da6325af7a0125f0e96fb014d3636
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/238998
Reviewed-by: Herbie Ong <herbie@google.com>
2020-06-19 21:29:54 +00:00
Joe Tsai
8de6675fa8 test.bash: add staticcheck to our integration tests
This adds the use of the staticcheck tool to check our code.
The use of staticcheck would have caught a bug in CL/211737.
Also, we adjust one minor code patterns to avoid triggering
one of the staticcheck checks.

This CL also modifies the integration script to check SHA256
checksums for archives downloaded from a third-party source.

Change-Id: Ieb69ae79464de62970448195c77cdd88e21fc48f
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/212220
Reviewed-by: Damien Neil <dneil@google.com>
2020-06-17 19:14:03 +00:00
Joe Tsai
5f5c066b7f types/known: minor adjustments to error handling
Trivial adjustments to error handling to reduce differences between
the generated helpers and the legacy ptypes package.

Change-Id: Ib3022eb50d9a9c0906b7809fe7e8011ee9399b0a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/238009
Reviewed-by: Damien Neil <dneil@google.com>
2020-06-17 18:52:16 +00:00
Joe Tsai
a0d1c75183 types/known: handle nil in IsValid and CheckValid methods
Many usages of the legacy ptypes.Timestamp and ptypes.Duration rely
on the functions implicitly returning an error for a nil message.
In order to migrate previous usages of the ptypes constructors to
the new API, we modify CheckValid to preserve similar semantics.

There is some consistency with this change in that:
	(*M)(nil).IsValid() == (*M)(nil).ProtoReflect().IsValid()
where M is Duration, Timestamp, and FieldMask.

Change-Id: I05ae152511f9875ea034e78aefb7aab18d17a318
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/238007
Reviewed-by: Damien Neil <dneil@google.com>
2020-06-16 21:18:46 +00:00
Joe Tsai
44e4150b30 reflect/protoreflect: optimize Name.IsValid and FullName.IsValid
For simplicity, IsValid was implemented in terms of regular expressions,
which are useful for verifying a string according to a strict grammar,
but performs poorly. Implement the check in terms of hand-written code,
which provides a 20x improvement to validate the name "google.protobuf.Any".

name               old time/op  new time/op  delta
FullNameIsValid-8   683ns ± 2%    35ns ± 1%  -94.86%  (p=0.000 n=10+10)

Change-Id: I980403befca0b72cea22acd274064a46cb02644b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/238002
Reviewed-by: Damien Neil <dneil@google.com>
2020-06-16 19:13:20 +00:00
Joe Tsai
0084168af8 types/known/anypb: add New constructor
While the MarshalFrom method and function are  more flexible since
it permits the reuse of an existing Any, the most common use case
by far is to construct a new Any. Optimize for that use case
by providing a constructor that directly returns a new Any instance.
We do not provide a variant that accepts a MarshalOptions since
the more flexible MarshalFrom function can be used for that.

Change-Id: Iab0219229c7d3aaa7390a2340a4f248002995293
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/237997
Reviewed-by: Damien Neil <dneil@google.com>
2020-06-16 00:35:17 +00:00
Joe Tsai
bfc31022a5 types/known/anypb: gracefully handle nil sources
Add checks so that MarshalFrom and UnmarshalTo consistently matches
the behavior of the proto package with regard to nil messages.
In particular, using a nil message as the destination results in
a panic (which is already the current behavior), while using a
nil message as the source does not panic (it is usually treated as
an untyped empty message). Since an untyped message has no
sensible meaning in the context of Any, return an error.

Change-Id: I99e86c2cdfbd8be57cc039efd550458dc56aadbc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/237920
Reviewed-by: Damien Neil <dneil@google.com>
2020-06-15 18:37:58 +00:00
cybrcodr
beaa55256c encoding/protojson: add better validation to FieldMask serialization
For marshaling, apart from already existing check that each item in
paths field is reversible, also make sure that string is a valid
protobuf name.

For unmarshaling, make sure that each resulting item in paths field is
a valid protobuf name and input path item does not contain _. The latter
check satisfies the conformance test
Recommended.Proto3.JsonInput.FieldMaskInvalidCharacter.

Fixes .

Change-Id: Iffc278089b20e496b7216d5b8c966b21b70e782d
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/236998
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2020-06-12 06:33:55 +00:00
Joe Tsai
596bfbf068 testing/protocmp: surface comment about unused Transform options
Change-Id: I83bdb0f3f937f745b5758c282e1bd58203a8c784
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/237221
Reviewed-by: Damien Neil <dneil@google.com>
2020-06-10 19:24:07 +00:00
Damien Neil
7e8b902d03 types/dynamicpb: add NewEnumType
Add support for creating dynamic EnumTypes.

Change-Id: Ic9f5b73630734848b29cc436f6c179549a8ea74a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/237219
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Joe Tsai <joetsai@google.com>
2020-06-10 19:22:58 +00:00
Joe Tsai
f49fd502d3 all: implement first-class WKT support
This CL introduces generation of specialized APIs directly into the
generated packages for certain well-known types. This follows the pattern
set forth by the other language implementations that have specialized
generated support for certain well-known types.

Overview of new API:
  package anypb
    func MarshalFrom(*Any, proto.Message, proto.MarshalOptions) error
    func UnmarshalTo(*Any, proto.Message, proto.UnmarshalOptions) error
    func UnmarshalNew(*Any, proto.UnmarshalOptions) (proto.Message, error)
    func (*Any) MessageIs(proto.Message) bool
    func (*Any) MessageName() protoreflect.FullName
    func (*Any) MarshalFrom(proto.Message) error
    func (*Any) UnmarshalTo(proto.Message) error
    func (*Any) UnmarshalNew() (proto.Message, error)

  package timestamppb
    func Now() *Timestamp
    func New(time.Time) *Timestamp
    func (*Timestamp) AsTime() time.Time
    func (*Timestamp) IsValid() bool
    func (*Timestamp) CheckValid() error

  package durationpb
    func New(time.Duration) *Duration
    func (*Duration) AsDuration() time.Duration
    func (*Duration) IsValid() bool
    func (*Duration) CheckValid() error

  package structpb
    func NewStruct(map[string]interface{}) (*Struct, error)
    func (*Struct) AsMap() map[string]interface{}
    func (*Struct) MarshalJSON() ([]byte, error)
    func (*Struct) UnmarshalJSON(b []byte) error

    func NewList([]interface{}) (*ListValue, error)
    func (*ListValue) AsSlice() []interface{}
    func (*ListValue) MarshalJSON() ([]byte, error)
    func (*ListValue) UnmarshalJSON(b []byte) error

    func NewValue(interface{}) (*Value, error)
    func NewNullValue() *Value
    func NewBoolValue(bool) *Value
    func NewNumberValue(float64) *Value
    func NewStringValue(string) *Value
    func NewStructValue(*Struct) *Value
    func NewListValue(*ListValue) *Value
    func (*Value) AsInterface() interface{}
    func (*Value) MarshalJSON() ([]byte, error)
    func (*Value) UnmarshalJSON(b []byte) error

  package fieldmaskpb
    func New(proto.Message, ...string) (*FieldMask, error)
    func Union(*FieldMask, *FieldMask, ...*FieldMask) *FieldMask
    func Intersect(*FieldMask, *FieldMask, ...*FieldMask) *FieldMask
    func (*FieldMask) IsValid(proto.Message) bool
    func (*FieldMask) Append(proto.Message, ...string) error
    func (*FieldMask) Normalize()

  package wrapperspb
    func Bool(bool) *BoolValue
    func Int32(int32) *Int32Value
    func Int64(int64) *Int64Value
    func UInt32(uint32) *UInt32Value
    func UInt64(uint64) *UInt64Value
    func Float(float32) *FloatValue
    func Double(float64) *DoubleValue
    func String(string) *StringValue
    func Bytes([]byte) *BytesValue

This functionality expands upon and supersedes the
older github.com/golang/protobuf/ptypes package,
which provided helpers for Any, Timestamp, and Duration.

Comparison with older ptypes package:

* ptypes.AnyMessageName is replaced by anypb.Any.MessageName.
  The former returned an error for malformed type URLs,
  while the latter simply returns an empty string.

* ptypes.Is is replaced by anypb.Any.MessageIs.

* ptypes.Empty has no direct replacement as it is equivalent to:
	mt, err := protoregistry.GlobalTypes.FindMessageByURL(any.GetTypeUrl())
	if err != nil {
		return nil, err
	}
	return mt.New().Interface(), nil
  Analysis of user code revealed that this function is seldom used.

* ptypes.MarshalAny is replaced by anypb.Any.MarshalFrom.
  The former creates a new Any message and returns it,
  while the latter is a method that modifies the receiver.

* ptypes.UnmarshalAny is replaced by anypb.Any.UnmarshalTo.

* ptypes.DynamicAny is loosely replaced by anypb.Any.UnmarshalNew.
  The DynamicAny type is a custom proto.Message that is special
  to ptypes.UnmarshalAny where it would allocate a new message
  and store it into the DynamicAny instance. The UnmarshalNew method
  accomplishes the equivalent functionality in a more direct fashion.

* ptypes.TimestampNow is replaced by timestamppb.Now.

* ptypes.TimestampProto is replaced by timestamppb.New.
  The former returned an error if the timestamp was outside the
  10000-year range recommended by timestamp.proto,
  while the latter always succeeded. To preserve the behavior of
  the former validation check, the replacement can additionally
  call the timestamppb.Timestamp.CheckValid method.

* ptypes.Timestamp is replaced by timestamppb.Timestamp.AsTime.
  The former returned an error if the timestamp was outside the
  10000-year range recommended by timestamp.proto,
  while the latter always succeeded. To preserve the behavior of
  the former validation check, the replacement can additionally
  call the timestamppb.Timestamp.CheckValid method.

* ptypes.TimestampString has no direct replacement as it is equivalent to:
	ts.AsTime().Format(time.RFC3339Nano)

* ptypes.DurationProto is replaced by durationpb.New.

* ptypes.Duration is replaced by durationpb.Duration.AsDuration.
  The former returned an error if the duration would overflow
  when converting to a time.Duration, while the latter uses
  saturation arithmetic (similiar to the time package itself).
  Underflow resulted in time.Duration(math.MinInt64), while
  overflow resulted in time.Duration(math.MaxInt64).
  To preserve the behavior of former validation checks,
  the replacement can call the durationpb.Duration.CheckValid method
  and check whether the duration is fixed to one of the overflow values.

Change-Id: Ia996b1037a1fcafced7c7e10e9408ef7fa22863a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/225298
Reviewed-by: Damien Neil <dneil@google.com>
2020-06-09 21:21:54 +00:00