The staticcheck version being used (2020.1.4) can no longer
work with latest Go, spitting out errors like this one:
> could not load export data: cannot import
> "unicode" (unknown iexport format version 2), export data is newer
> version - update tool (compile)
Update to latest version, 2022.1.2, to fix this.
Change-Id: I745482561a46baebc70931443008b95bec2fa377
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/406842
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Lasse Folger <lassefolger@google.com>
Correctly distinguish between 0 and -0 in tests.
Change-Id: I41f6702a903ca0cf9edf9e95be812a3a29930978
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/406917
Reviewed-by: Lasse Folger <lassefolger@google.com>
Avoid very long errors returned by limiting the length of what errId
returns to 32 bytes (the value is chosen so that the error will not
be too long yet useful).
Append ellipsis to the returned value to denote that it was truncated.
Change-Id: I232d5192a2d9ad675daa0be0fe0c8518489c2953
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/406694
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
This eliminates the last user of the regexp package, which should save
about 130K from the resulting stripped binary importing this package
(unless, of course, regexp is brought in directly of via another
dependency).
Added some new cases to TestDecoder to test the new function.
Benchmark (not included) shows the following results, comparing to
old implementation using regexp.Find:
name old time/op new time/op delta
ErrId-4 1.93µs ± 1% 0.21µs ± 1% -89.20% (p=0.002 n=6+6)
name old alloc/op new alloc/op delta
ErrId-4 128B ± 0% 0B -100.00% (p=0.002 n=6+6)
name old allocs/op new allocs/op delta
ErrId-4 13.0 ± 0% 0.0 -100.00% (p=0.002 n=6+6)
Change-Id: I5569a47580f41cc60f92c444e8d43bb3f26faa4e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/402774
Reviewed-by: Cassondra Foesch <cfoesch@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
There are a number of message types, which can be confusing.
Expand the documentation. In particular, use Go's reflection model
as an analogy, which helps users if they are at least familiar
with how Go reflection works. After all, protobuf reflection
was modeled after Go reflection to a large degree.
Change-Id: I388ae41c792905145ccc86dc938b901a2b79f408
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/399654
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Lasse Folger <lassefolger@google.com>
+ This change introduce a default and configurable depth limit for
proto.Unmarshal. If a message is nested deeper than the limit,
unmarshaling will fail. There are two ways to nest messages. Either by
having fields which are message types itself or by using groups.
+ The default limit is 10,000 for now. This might change in the future
to align it with other language implementation (C++ and Java use 100
as limit).
+ If pure groups (groups that don't contain message fields) are nested
deeper than the default limit the unmarshaling fails with:
proto: cannot parse invalid wire-format data
+ Note: the configured limit does not apply to pure groups.
+ This change is introduced to improve security and robustness. Because
unmarshaling is implemented using recursion it can lead to stack overflows
for certain inputs. The introduced limit protects against this.
+ A secondary motivation for this limit is the alignment with other
languages. Protocol buffers are a language interoperability mechanism
and thus either all implementations should accept the input or all
implementation should reject the input.
Change-Id: I14bdb44d06e4bd1aa90d6336c2cf6446003b2037
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/385854
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Damien Neil <dneil@google.com>
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
Go 1.9 and 1.10 produce binaries that don't run on the most recent version
of macOS. Drop these versions from the tested set.
Update minor revision where a newer one is available:
1.15.9 -> 1.15.15
1.16.1 -> 1.16.10
Add 1.17.3.
Bump go.mod version to go1.11.
Fixesgolang/protobuf#1394.
Change-Id: I7850b86819d356891917d2987f080d3e7dbd7cc5
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/370055
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Florian Zenker <floriank@google.com>
The Unwrap method returns the original concrete message value.
In theory this allows users to mutate the original message when the
cmp documentation says that all options should be mutation free.
If users want to disregard this documented restriction, they can
already do so in a number of different ways.
Updates #1347
Change-Id: I65225681ab5dbce0763a140fd02666a4ab542a04
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/340489
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
Previously, this call strconv.FormatInt with base 32,
when the intention is to call it with base 10.
Change-Id: I31cdd2415b3a80936cdcdeb5612a486204404ecb
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/331149
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
There are inconsistencies between implementations on what happens when
a single program contains generated code for multiple files with the
same source path. At least one canonical implementation (C++) will fail
at link time. Others print warnings. Some silently resolve the registry
conflict in favor of one file or the other. The protobuf maintainers
agree, however, that the desired behavior is for this condition to be an
error.
Updates golang/protobuf#1122
Change-Id: I716708f16ef90210bdfceb0888691e47783df172
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/322729
Trust: Damien Neil <dneil@google.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
macOS's sed always considers the parameter following -i to be an
extension, so "-i -e" means "edit in place saving backup with a -e
extension".
macOS's sed does not support \s in regexps.
Change-Id: Ic9b57baa5fc88fff2251c640d11ed417a7257f13
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/322449
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Remove the requirement that all registered files have unique names.
Files.FindFileByPath now returns an error if multiple files match
the path. Files.RangeFiles iterates over all files, including ones with
the same name.
Unlike descriptor names, file names are not part of any of the wire
encodings. Conflicts in file names only affect file lookups by path,
which is an uncommon operation.
The Files.RegisterFile documentation already states (inaccurately),
"It is permitted for multiple files to have the same file path."
This change brings the behavior of RegisterFile in line with the
documentation.
Fixesgolang/protobuf#1122
Change-Id: I2750e35736fd34def23fa6ab6d6a829095a65a26
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/321531
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The path "sub.example.com" is a valid Go import path and
should not be rejected. Relax the check to require at least one
dot or slash. Either way, it still prevents the situation where
a user erroneously treats this option as just the package name.
Change-Id: I3df811e1eae61c9d2a0b81c001a27cc7c08c3838
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/316949
Trust: Joe Tsai <joetsai@digital-static.net>
Trust: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
The Go generator has historically always prefixed an enum value
with the name of the enum type, when it was unnecessary to do so.
For example:
enum Status {
STATUS_FAILED = 0;
STATUS_PASSED = 1;
}
would be generated as:
type Status int32
const (
Status_STATUS_FAILED Status = 0
Status_STATUS_PASSED Status = 1
)
It is common for the enum values to be manually prefixed by the
enum type since protobuf enums use C++ namespace rules where
enum types and enum values are in the same namespace scope.
Thus, having the Go generator add a prefix is redundant.
See https://github.com/golang/protobuf/issues/513.
Some custom Go generators like protoc-gen-gogo allow removing
the prefix with the gogoproto.goproto_enum_prefix feature.
However, this leads to interoperability issues between
protoc-gen-go and protoc-gen-gogo, where the enum value names
cannot be accurately inferred.
Avoid this problem by just hard-coding the enum value number
for values declared in other packages. This provides benefits
in interoperability at the small cost of enum values possibly
being stale if their value were ever changed in a remote package.
However, this would only occur with use of proto2 enums and
default values, which seems to be an exceptionally rare situation.
Before:
Default_MyMessage_MyField = remotepb.FooEnum_FOO_ENUM
After:
Default_MyMessage_MyField = remotepb.FooEnum(4) // remotepb.FooEnum_FOO_ENUM
Before:
func (x *MyMessage) GetField() remotepb.FooEnum {
...
return remotepb.FooEnum_FOO_ZERO
}
After:
func (x *MyMessage) GetField() remotepb.FooEnum {
...
return remotepb.FooEnum(0) // always 0 for proto3 and often 0 for proto2
}
Change-Id: I3a06cd553f2eaf6124666f6c36c196d500d35718
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/319649
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
Add a .exe suffix to the protoc-gen-go Windows binary.
Use zip instead of tar for the Windows release archive.
Fixesgolang/protobuf#1323
Change-Id: If42fa6dfd8b5496148310fa2a3359161f1124229
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/321530
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
When merging aberrant messages with legacy Marshal and Unmarshal
methods, check for a typed nil source before calling Marshal.
Add an aberrant message with Marshal/Unmarshal methods to
internal/testprotos/nullable and use it to test the internal/impl
support for these methods.
Fixesgolang/protobuf#1324
Change-Id: Ib6ce85b30b46e3392a226ca6abe411932a371f02
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/321529
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Inside decoder.skipValue we should not be calling skipValue again
since we had already read the value earlier. The only possible
composite type in the context of a list is another message,
which is already handled in the case above.
Change-Id: If40da2d369e0a64a64ba9b961377331231158fe2
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/317430
Trust: Joe Tsai <joetsai@digital-static.net>
Trust: Herbie Ong <herbie@google.com>
Reviewed-by: Herbie Ong <herbie@google.com>
The protobuf wire format is insufficiently self-decribing such that
it is impossible to know whether for sure whether an unknown bytes value
is a sub-message or not. However, protopack is primarily used for debugging
where a best-effort guess is still very useful.
The Message.UnmarshalAbductive unmarshals an unknown bytes value as a message
if it is syntactically well-formed. Otherwise, it is left as is.
Change-Id: I5e2b4b995e2b5eb60942a242558bf4cea1da9891
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/309669
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
Change-Id: Ib09bd3cbd15088d534de34b08d01a4d7a4880c0a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/306209
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Joe Tsai <thebrokentoaster@gmail.com>
Trust: Damien Neil <dneil@google.com>
The protopath package provides a means to programmatically represent
a sequence of protobuf reflection operations.
The protorange package traverses through a message and
calls a user-provided function as it iterates.
This feature sets the groundwork for the often requested feature
of being able to exclude certain fields when merging or serializing.
package protopath
type Path []Step
type Step struct{ ... }
func Root(protoreflect.MessageDescriptor) Step
func FieldAccess(protoreflect.FieldDescriptor) Step
func UnknownAccess() Step
func ListIndex(int) Step
func MapIndex(protoreflect.MapKey) Step
func AnyExpand(protoreflect.MessageDescriptor) Step
func (Step) Kind() StepKind
func (Step) FieldDescriptor() protoreflect.FieldDescriptor
func (Step) MessageDescriptor() protoreflect.MessageDescriptor
func (Step) ListIndex() int
func (Step) MapIndex() protoreflect.MapKey
func (Step) String() string
type StepKind int
const RootStep StepKind
const FieldAccessStep StepKind
const UnknownAccessStep StepKind
const ListIndexStep StepKind
const MapIndexStep StepKind
const AnyExpandStep StepKind
type Values struct {
Path Path
Values []protoreflect.Value
}
func (Values) Index(int) (out struct{ ... })
func (Values) Len() int
func (Values) String() string
package protorange
var Break error
var Terminate error
func Range(protoreflect.Message, func(protopath.Values) error) error
type Options struct {
Stable bool
Resolver interface { ... }
}
func (Options) Range(m protoreflect.Message, push, pop func(protopath.Values) error) error
Change-Id: I29cbd5142fe169d78367d54a95d37801888b64f4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/236540
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
The ExtensionRangeArray was a pseudo-internal API that was used by
the protobuf runtime implementation to know what the extension ranges are.
For some time now, the runtime implementation does not make use of this.
According to all the latest modules known by the module proxy,
no code depends directly on this method being present.
In the extremely rare case where some user code is depending on this method,
the user can place a .go file next to the generated .pb.go file that
injects this method.
Change-Id: Iae40a9a33b8c9b5a243d48db14f25b05ca24e3dc
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/305574
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
Add back the weak dependency on github.com/golang/protobuf.
Change-Id: I2e6fabbe2cac3b694fb56a86bb3aa85cc8615b42
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/262684
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
Temporarily remove the dependency on github.com/golang/protobuf
and add it back in a subsequent CL. This allows us to break some of
the infinite cycles of old versions in the go.sum file.
Change-Id: Ifb5c2c160713d47a43ba2bcedc18ebe2ce33dcea
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/262683
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
When someone has no idea what a binary does, it is convention to
pass the "--help" flag. When done, we should point the user
to the devsite documentation on protoc with protoc-gen-go.
Change-Id: I36289a1ae025b9e12521b34362370aba5235a44b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/302330
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
Since the release of v1.20, we have warned that it be required
that the import path for every generated package be specified.
This CL simplifies the mechanism for specifying such information
to just the "M" command-line flags and the "go_package" options
in the .proto source files, where the former takes precedence
over the latter.
Changes made:
* Remove "import_prefix" and "import_path" flags.
* Make the Go import path and package name derivation logic simpler
where both "M" flags and "go_package" options are parsed the same way
by calling the common splitImportPathAndPackageName function, and
where "M" flags take precedence over "go_package" options.
The exception to this occurs when neither "M" nor "go_package" specify
an explicit Go package name, where the derivation may be non-intuitive.
See the "NOTE" comment for the rationale for this behavior,
which actually matches what was already the case.
* Remove the pathTypeLegacy output mode and make pathTypeImport
the default. The pathTypeLegacy mode becomes even more non-sensible
now that we require that the import path always be provided.
* Remove the baseName function. After deleting unsupported functionality,
this seems to only be used by GeneratedFile.QualifiedGoIdent.
However, that method does not need to create stable package names
since they are only used locally within the generated file.
They only need to guarantee the property of validity and uniqueness.
* Remove the warn function since there are no more warnings.
Change-Id: Ic95fb3cde5ffcb71bbcc829fcff34369758cebef
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/301953
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
Implement support in the protobuf runtime to better understand
message types that are not generated by the official generator.
In particular:
* Add a best-effort implementation of protobuf reflection for
"non-nullable" fields which are supposed to be represented by *T,
but are instead represented by a T. "Non-nullable" message fields
report presence based on whether the message is the zero Go value.
* We do NOT implement support for "non-nullable" fields in the
table-driven implementation since we assume that the aberrant messages
that we care about have a Marshal and Unmarshal method.
* We better handle custom messages that implement Marshal and Unmarshal,
but do NOT implement Merge. In that case, we implement merge in terms of
a back-to-back marshal and unmarshal.
* We better tolerate the situations where a protobuf message field
cannot be mapped to a Go struct field since the latter is missing.
In such cases, reflection treats the field as if it were unpopulated.
Setting such fields will panic.
This change allows the runtime to handle all message types declared
in the "go.etcd.io/etcd" and "k8s.io" modules where protobuf reflection,
Marshal, Unmarshal, Reset, Merge, and Equal all work.
The only types that still do not fully work are:
* "k8s.io/api/authentication/v1".ExtraValue
* "k8s.io/api/authentication/v1beta1".ExtraValue
* "k8s.io/api/authorization/v1".ExtraValue
* "k8s.io/api/authorization/v1beta1".ExtraValue
* "k8s.io/api/certificates/v1".ExtraValue
* "k8s.io/api/certificates/v1beta1".ExtraValue
* "k8s.io/apimachinery/pkg/apis/meta/v1".MicroTime
* "k8s.io/apimachinery/pkg/apis/meta/v1".Time
* "k8s.io/apimachinery/pkg/apis/meta/v1".Verbs
While Marshal, Unmarshal, Reset, and Merge continue to work,
protobuf reflection and any functionality that depends on it
(e.g., prototext, protojson, Equal, etc.) will not work.
Change-Id: I67a9d2f1bec35248045ad0c16220d02fc2e0e172
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/300869
Trust: Joe Tsai <joetsai@digital-static.net>
Trust: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Travis-CI is dead. The new hotness if GitHub actions.
For now we only test on Linux since testing on macOS and Windows
takes more effort to figure out.
Change-Id: I464df75b9142dafcdf07281f70c1bcf52514f71a
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/298089
Trust: Joe Tsai <joetsai@digital-static.net>
Reviewed-by: Damien Neil <dneil@google.com>
