+ This change introduce a default and configurable depth limit for
proto.Unmarshal. If a message is nested deeper than the limit,
unmarshaling will fail. There are two ways to nest messages. Either by
having fields which are message types itself or by using groups.
+ The default limit is 10,000 for now. This might change in the future
to align it with other language implementation (C++ and Java use 100
as limit).
+ If pure groups (groups that don't contain message fields) are nested
deeper than the default limit the unmarshaling fails with:
proto: cannot parse invalid wire-format data
+ Note: the configured limit does not apply to pure groups.
+ This change is introduced to improve security and robustness. Because
unmarshaling is implemented using recursion it can lead to stack overflows
for certain inputs. The introduced limit protects against this.
+ A secondary motivation for this limit is the alignment with other
languages. Protocol buffers are a language interoperability mechanism
and thus either all implementations should accept the input or all
implementation should reject the input.
Change-Id: I14bdb44d06e4bd1aa90d6336c2cf6446003b2037
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/385854
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Damien Neil <dneil@google.com>
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
Reviewed-by: Chressie Himpel <chressie@google.com>
The UnmarshalInitialized flag produced by Unmarshal and Validate are
filters such that must never have false positives (i.e., report a
partial message as initialized) otherwise it is incorrect.
It can tolerate some degree of false negatives (i.e., report an
initialized message as partial), but that leads to significant
performance degradation needing to do the full initialization check.
These should be the exception, not the norm.
Adjust the fuzzer to search for false-negative cases. For now, we only
require that the Unmarshal and Validate report initialized for any
"normalized" messages which we produce by marshaling intermediate
message again. This is to work around a known case where they cannot
determine initialization status if the wire data relies on protobuf's
merge functionality (where two or more partial messages merge
together to form an initialized message).
Change-Id: I6bb6c6594981ca08a92583bae77e5a2d44924af6
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/231577
Reviewed-by: Damien Neil <dneil@google.com>
An Is prefix implies it returns a boolean.
A Check prefix better suggests that it could return an error.
Change-Id: I6ffcb32099a944c656c07654c294a0980efb2d0e
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/220338
Reviewed-by: Damien Neil <dneil@google.com>
Move all fast-path inputs and outputs into the Input/Output structs.
Collapse all booleans into bitfields.
Change-Id: I79ebfbac9cd1d8ef5ec17c4f955311db007391ca
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/219505
Reviewed-by: Joe Tsai <joetsai@google.com>
Return the size of the field read from the validator, permitting us to
avoid an extra parse when skipping over groups.
Return an UnmarshalOutput from the validator, since it already combines
two of the validator outputs: bytes read and initialization status.
Remove initialization status from the ValidationStatus enum, since it's
covered by the UnmarshalOutput.
Change-Id: I3e684c45d15aa1992d8dc3bde0f608880d34a94b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/217763
Reviewed-by: Joe Tsai <joetsai@google.com>
This adds a experimental function to the internal/impl package which
validates a wire-format message against a message type. The validator
reports whether the message can be successfully unmarshaled, and whether
the result is initialized (all required fields are set). In some cases,
the validator returns ambiguous results when full validation would be
expensive.
The validator is unused outside of tests. In the future, it may be used
to permit lazy unmarshaling of some data. It is being added now for
testing; in particular, the wire fuzzer now checks the validator output
for consistency with the unmarshaler.
The validator adds a small amount of unused per-MessageType state. If
this becomes a concern, we could conditionalize it with a build tag.
Change-Id: I4216ef81d6a9ed975302eed189b02d08608858b4
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/212302
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
All the fuzzers have the same test, which runs the fuzzer against every
entry in the corpus. Move the test logic into a separate package.
Change-Id: I3a7e2ca75d20a5ff6d51ed9e6151629e6667684b
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/212258
Reviewed-by: Joe Tsai <joetsai@google.com>
Add a new Fuzz message containing all the message types we want to make
available to fuzzers. Previously, testing (for example) required fields
would require modifying the fuzzer; now, it's just a matter of adding a
message with required fields as a field of the top-level Fuzz message.
Add internal/cmd/generate-corpus to codify where the fuzz seed corpus
comes from. This will simplify adding text and json fuzzers.
Rename internal/fuzz/wire to internal/fuzz/wirefuzz to minimize package
name ambiguity. Also, the addition of the Fuzz container message
invalidates the existing corpus, so using a new name seems like a good
idea.
Change-Id: I94f8f64ba93596c8e8cecb4d42bcc5b98c17d838
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/212218
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
