The driver-vs-reference checks compare test results in different
configurations. Ignore the test results that report differences in
configurations that were the point of the comparison. Do compare other
configuration reports: this will let us know if the configurations diverge
in an unexpected way.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Generate option-on and option-off cases for test_suite_config, for all
boolean options (MBEDTLS_xxx and PSA_WANT_xxx, collected from the mbedtls
and PSA config files).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Add a test suite intended to report configuration options in the outcome
file: we're only interested in SKIP vs PASS.
Add a few test cases for some interesting combinations of options. The
selection here is just for illustration purposes, more will be added later.
A subsequent commit will automatically generate test cases for single options.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Alert if all tests are filtered out or skipped: that probably indicates a
test script that set up an unintended configuration or an overly strict
filter. You can pass `--min 0` to bypass this check. You can pass `--min`
with a larger value to require that many test cases to run.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
On the CI, the git version when running on
Ubuntu 16.04 is 2.7 and it does not support
the "--recurse-submodules" option of
"git ls-files" thus do not use it.
Another argument to not use it is that
when TF-PSA-Crypto will be a submodule of
mbedtls we will not want check_files.py to
check the TF-PSA-Crypto files as well.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Change the way we decide if for a given
configuration we need to run tests with
and without MBEDTLS_USE_PSA_CRYPTO enabled.
That makes the script suitable for 3.6 and
development branch.
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
Replace more sample PSK by longer (GnuTLS-compatible) strings, taking care
of keeping distinct PSK distinct for wrong-PSK tests.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
This allows many tests to pass with the system openssl and gnutls-*. As
before, not all test cases will pass due to differences between versions and
build options.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Some OpenSSL or GnuTLS interoperability test cases fail if the other
implementation is recent enough to support TLS 1.3. Force those test cases
to use TLS 1.2 so that the script works with more recent $OPENSSL or
$GNUTLS_CLI or $GNUTLS_SERV than our official CI versions.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
ssl-opt.sh uses a 3-byte PSK in many test cases. Unfortunately GnuTLS >=3.4.0
rejects a PSK that is less than 4 bytes long:
> Error setting the PSK credentials: The request is invalid.
Use a longer PSK throughout ssl-opt. Only the test cases involving GnuTLS
need to change, but it's easier to do a global search-and-replace, and it's
easier to not have to worry about mismatches in constructed test cases
later, so replace everything.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When given a PSK key but no username, gnutls-cli prompts for a password.
Prevent that by passing --pskusername with the same identity that
ssl_server2 uses by default.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Only s_server has a -nocert option, s_client doesn't. Fixes OpenSSL client
test cases in PSK-only builds.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Tighten the matching when detecting which certificates are in use to
determine algorithm requirements. This fixes a bug whereby all tests were
skipped in configurations without RSA except for an Mbed TLS client against
a GnuTLS or OpenSSL server, due to *server2* matching ssl_server2.
Fixes#8366.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
When trying to decrypt data with an invalid key, we found that `mbedtls`
returned `0x6200` (`-25088`), which means "_CIPHER - Input data contains
invalid padding and is rejected_" from `mbedtls_cipher_finish`, but it also
set the output len as `18446744073709551516`.
In case we detect an error with padding, we leave the output len zero'ed
and return `MBEDTLS_ERR_CIPHER_INVALID_PADDING`.
Here's a reference for the way `openssl` checks the padding length:
- 1848c561ec/crypto/evp/evp_enc.c (L1023)
- b554eef43b
So add a check ensuring output is set to the least-harmful value in the
error cases.
With the robustness fix:
`PASSED (125 suites, 26644 tests run)`
Without the robustness fix:
`FAILED (125 suites, 26644 tests run)`
Signed-off-by: Andre Goddard Rosa <andre.goddard@gmail.com>
Signed-off-by: Andre Goddard Rosa <agoddardrosa@roku.com>
Since this python module resides in the framework submodule we must
add an extra explicit path to it for mypy.
Signed-off-by: David Horstmann <david.horstmann@arm.com>
