166 Commits

Author SHA1 Message Date
Gilles Peskine
9b7cdd91dc Move test case ignore list to the master Task class
No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-01 19:32:55 +02:00
Gilles Peskine
0a7d96d266 Remove now-useless level of method call indirection
No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-01 19:32:55 +02:00
Gilles Peskine
95b2b0c160 Move analysis functions into their respective classes
No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-01 19:32:55 +02:00
Gilles Peskine
0e5e57b34c Remove dead code that was handling stringly typed data
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-01 19:32:55 +02:00
Gilles Peskine
92cc8dba27 Replace stringly typed data by class: driver vs reference (data)
Work on the stringly typed KNOWN_TASKS by classes for each category
of tasks, with a structure that matches the behavior.

This commit migrates the data for driver-vs-reference analysis and gets rid
of the transitional code that was using the old form of the data.

No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-01 19:32:55 +02:00
Gilles Peskine
17e071b0ab Replace stringly typed data by class: driver vs reference (code)
Work on the stringly typed KNOWN_TASKS by classes for each category
of tasks, with a structure that matches the behavior.

This commit migrates the code for driver-vs-reference analysis. To
facilitate review, this commit preserves the layout of the data that
parametrizes each task. The next commit will migrate the data.

No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-01 19:32:55 +02:00
Gilles Peskine
0316f1006c Replace stringly typed data by class: coverage
Work on replacing the stringly typed KNOWN_TASKS by classes for each category
of tasks, with a structure that matches the behavior.

This commit migrates test coverage analysis.

No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-01 19:32:55 +02:00
Gilles Peskine
00ed057738 Replace stringly typed data by class: prepare
Start replacing the stringly typed KNOWN_TASKS by classes for each category
of tasks, with a structure that matches the behavior.

This commit introduces some transition code.

No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-01 19:32:55 +02:00
Gilles Peskine
ced0edc43e Move config-tfm.h testing to separate all.sh component
Rename the existing component_test_tfm_config which tests a modified version
of config-tfm.h for the sake of driver-vs-reference comparison.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-09-19 18:57:11 +02:00
Gilles Peskine
d210bf73b2
Merge pull request #9338 from sezrab/analyze_driver_vs_reference_header_correction-3.6
Backport 3.6:  Fix inconsistent ordering of driver vs reference in analyze_outcomes
2024-09-05 16:36:02 +00:00
Elena Uziunaite
6496d56329 Make error line consistent with the header
Signed-off-by: Elena Uziunaite <elena.uziunaite@arm.com>
2024-09-02 15:34:02 +01:00
Gilles Peskine
472c10f4a5 key_custom: update analyze_outcomes.py
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-08-06 13:13:05 +02:00
Gilles Peskine
33218d2b30
Merge pull request #9329 from sezrab/replace_MBEDTLS_MD_CAN_MD5_with_PSA_WANT-3.6
Backport 3.6: Add MD5 support for requires_hash_alg
2024-07-18 09:41:05 +00:00
Sam Berry
602550162c Add TLS: password protected... to ignored_tests list
Signed-off-by: Sam Berry <sam.berry@arm.com>
2024-07-17 10:03:40 +01:00
Sam Berry
f231386f7f Corrected header line of analyze_driver_vs_reference
The header line said “driver vs reference” whereas the error line said
“reference -> driver”. Updated the header line to be consistent with the
error line.

Signed-off-by: Sam Berry <sam.berry@arm.com>
2024-07-02 15:10:58 +01:00
Gilles Peskine
331d7d053d Driver vs referenee: ignore relevant configuration differences
The driver-vs-reference checks compare test results in different
configurations. Ignore the test results that report differences in
configurations that were the point of the comparison. Do compare other
configuration reports: this will let us know if the configurations diverge
in an unexpected way.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-06-21 00:01:20 +02:00
Gilles Peskine
7fbbb43a57 Pacify pylint
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-05-22 18:09:39 +02:00
Gilles Peskine
8941033a40 Generate test data before coverage analysis
Fixes #8300.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-05-22 18:09:39 +02:00
Gilles Peskine
89ef2fabb5 Driver-only FFDH is not good enough for DHE support in TLS 1.2
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-05-14 12:06:20 +02:00
Gilles Peskine
0f63028809
Merge pull request #8815 from gilles-peskine-arm/psa_generate_key_ext-prototype
Introduce psa_generate_key_ext
2024-02-26 07:16:49 +00:00
Manuel Pégourié-Gonnard
a7f651cf16
Merge pull request #8804 from valeriosetti/issue8799
mbedtls_rsa_parse_key and mbedtls_rsa_parse_pubkey accept trailing garbage
2024-02-20 11:58:52 +00:00
Valerio Setti
eba4ca19c6 test_suite_pem: solve driver test disparities
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2024-02-19 07:42:18 +01:00
Gilles Peskine
63072b1f94 Only test custom-e RSA key generation when built in
Custom-e RSA key generation is not yet supported in the test driver, and we
don't support fallback from the test driver to the built-in
implementation (even though we're testing with MBEDTLS_RSA_C). So for the
time being, only run psa_generate_key_ext test cases for RSA with a custom
public exponent when using the built-in implementation.

Add a test case to validate that psa_generate_key_ext test cases for RSA
with a custom public exponent returns NOT_SUPPORTED (rather than silently
doing the wrong thing) when not using built-in RSA (which is subtly
different from when having accelerated RSA: if both are enabled, which we
currently don't do in all.sh, then this should be supported and this is
validated by the test cases above).

This wart will be resolved when we add support for drivers with a
generate_key_ext entry point.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-02-15 12:02:04 +01:00
Gilles Peskine
3ea9450463
Merge pull request #8734 from valeriosetti/issue8564
Add test for driver-only HMAC
2024-02-14 13:43:40 +00:00
Ryan Everett
67f3568895 Reduce analyze_block_cipher_dispatch exceptions
Signed-off-by: Ryan Everett <ryan.everett@arm.com>
2024-02-09 13:02:23 +00:00
Ryan Everett
afb2eee263 Add PKCS5/12 exceptions to analyze_block_cipher_dispatch
Signed-off-by: Ryan Everett <ryan.everett@arm.com>
2024-02-08 14:31:54 +00:00
Valerio Setti
89d8a12e9c analyze_outcomes: fix typo
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2024-01-26 15:04:05 +01:00
Valerio Setti
cd89b0b536 all.sh: disable legacy hash support in test_psa_crypto_config_accel_hmac()
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2024-01-24 14:24:55 +01:00
Valerio Setti
20cea94fd4 analyze_outcomes: add task for HMAC coverage
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2024-01-22 16:23:25 +01:00
Gilles Peskine
b1f96c0354
Merge pull request #7815 from gilles-peskine-arm/ecp-export-partial
ECP keypair utility functions
2024-01-18 10:29:05 +00:00
Manuel Pégourié-Gonnard
7f48d5e203 Rename test components to better reflect content
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-01-08 11:35:01 +01:00
Valerio Setti
a0c9c6684d analyze_outcomes: ignore only test concerning AES/ARIA/Camellia in CMAC
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2023-12-29 14:14:11 +01:00
Valerio Setti
0635cca7d1 analyze_outcomes: update skipped tests following latest changes to all.sh
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2023-12-28 18:33:17 +01:00
Valerio Setti
ab0494f193 analyze_outcomes: update comments of skipped tests
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2023-12-28 13:56:52 +01:00
Gilles Peskine
3b17ae78d2 Add ECP-heavy-only test cases to the driver parity analysis ignore list
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2023-12-24 19:51:57 +01:00
Valerio Setti
5f665c3a0d analyze_outcomes: add exceptions to disparities for block_cipher dispatch
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2023-12-20 09:56:05 +01:00
Valerio Setti
9afa329b80 analyze_outcomes: allow ignored test suites to have a dot in the name
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2023-12-20 09:55:28 +01:00
Valerio Setti
4a8ef7cd9b all.sh: disable legacy AES/ARIA/CAMELLIA in test_full_block_cipher_psa_dispatch
This commit also:
- rename the reference component as component_test_full_block_cipher_legacy_dispatch()
- add a common configuration function, named common_block_cipher_dispatch() that
  is used from both accelerated and reference components

Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2023-12-20 07:35:41 +01:00
Valerio Setti
52ab8fa565 analyze_outcomes/all.sh: add reference component and entry for coverage comparison
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2023-12-14 18:09:26 +01:00
Pengyu Lv
98a90c6542 Fix various issue
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
2023-12-07 17:23:25 +08:00
Pengyu Lv
3cd16c47bd Add analyze_driver_vs_reference_rsa for analyze_outcomes
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
2023-12-07 10:24:29 +08:00
Gilles Peskine
18eab984c7
Merge pull request #8560 from lpy4105/issue/8423/optimize-analyze_outcomes_py
Optimize analyze_outcomes.py
2023-11-29 14:51:41 +00:00
Pengyu Lv
5dcfd0c613 Some improvements
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
2023-11-29 18:03:28 +08:00
Pengyu Lv
550cd6f9b2 Use boolean hit instead of int hits
Also fix a typo in the comments.

Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
2023-11-29 09:17:59 +08:00
Dave Rodgman
c3cd410acf
Merge pull request #8286 from gilles-peskine-arm/check_mbedtls_calloc_overallocation-disable_with_asan
Fix test_suite_platform failure with Asan on modern Clang
2023-11-28 16:48:31 +00:00
Pengyu Lv
451ec8a4bc Add comment to read_outcome_file in analyze_outcomes.py
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
2023-11-28 17:59:05 +08:00
Pengyu Lv
c2e8f3a080 Add type annotations to analyze_outcomes.py
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
2023-11-28 17:22:04 +08:00
Pengyu Lv
20e3ca391e Run tests for ref_vs_driver outside task function
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
2023-11-28 15:30:03 +08:00
Pengyu Lv
18908ec276 Define named tuple for component outcomes
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
2023-11-28 13:04:11 +08:00
Pengyu Lv
28ae4648a6 Use mutable set all the time
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
2023-11-28 11:35:19 +08:00