169 Commits

Author SHA1 Message Date
Gilles Peskine
0930b331c0 Don't use the "allow list" terminology any longer
What was formerly called an allow list is now an ignore table.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-09-27 18:28:29 +02:00
Gilles Peskine
54cfe77951 Switch coverage analysis to IGNORE_TESTS for its allowlist
No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-09-24 22:11:50 +02:00
Gilles Peskine
dba8010384 Simplify sub-test-suite handling in is_test_case_ignored
No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-09-24 22:11:50 +02:00
Gilles Peskine
0297605641 Move test case ignore list to the master Task class
No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-09-24 22:11:50 +02:00
Gilles Peskine
b4daeb4fd2 Remove now-useless level of method call indirection
No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-09-24 22:11:50 +02:00
Gilles Peskine
3f5022e66d Move analysis functions into their respective classes
No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-09-24 22:11:50 +02:00
Gilles Peskine
0f31f76f83 Remove dead code that was handling stringly typed data
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-09-24 22:11:50 +02:00
Gilles Peskine
9df375b018 Replace stringly typed data by class: driver vs reference (data)
Work on the stringly typed KNOWN_TASKS by classes for each category
of tasks, with a structure that matches the behavior.

This commit migrates the data for driver-vs-reference analysis and gets rid
of the transitional code that was using the old form of the data.

No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-09-24 22:11:48 +02:00
Gilles Peskine
82b16721bd Replace stringly typed data by class: driver vs reference (code)
Work on the stringly typed KNOWN_TASKS by classes for each category
of tasks, with a structure that matches the behavior.

This commit migrates the code for driver-vs-reference analysis. To
facilitate review, this commit preserves the layout of the data that
parametrizes each task. The next commit will migrate the data.

No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-09-24 22:10:20 +02:00
Gilles Peskine
f646dbf71d Replace stringly typed data by class: coverage
Work on replacing the stringly typed KNOWN_TASKS by classes for each category
of tasks, with a structure that matches the behavior.

This commit migrates test coverage analysis.

No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-09-24 22:07:11 +02:00
Gilles Peskine
19ef1ae72e Replace stringly typed data by class: prepare
Start replacing the stringly typed KNOWN_TASKS by classes for each category
of tasks, with a structure that matches the behavior.

This commit introduces some transition code.

No intended behavior change.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-09-24 22:07:11 +02:00
Gilles Peskine
effa6a0769 Move config-tfm.h testing to separate all.sh component
Rename the existing component_test_tfm_config which tests a modified version
of config-tfm.h for the sake of driver-vs-reference comparison.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-09-14 18:20:57 +02:00
Gilles Peskine
1ad1954e8f
Merge pull request #9304 from sezrab/analyze_driver_vs_reference_header_correction
Fix inconsistent ordering of driver vs reference in analyze_outcomes
2024-09-05 16:36:06 +00:00
Elena Uziunaite
c21675e297 Make error line consistent with the header
Signed-off-by: Elena Uziunaite <elena.uziunaite@arm.com>
2024-09-02 15:32:07 +01:00
Gilles Peskine
1084e8eb4f key_custom: update analyze_outcomes.py
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-08-05 15:52:37 +02:00
Gilles Peskine
7ab9867fa7
Merge pull request #9290 from sezrab/replace_MBEDTLS_MD_CAN_MD5_with_PSA_WANT
Add MD5 support for `requires_hash_alg`
2024-07-18 09:41:17 +00:00
Sam Berry
4beeb0ce66 Add TLS: password protected... to ignored_tests list
Signed-off-by: Sam Berry <sam.berry@arm.com>
2024-06-28 14:46:35 +01:00
Sam Berry
e262c237f8 Corrected header line of analyze_driver_vs_reference
The header line said “driver vs reference” whereas the error line said
“reference -> driver”. Updated the header line to be consistent with the
error line.

Signed-off-by: Sam Berry <sam.berry@arm.com>
2024-06-21 10:03:37 +01:00
Gilles Peskine
a7469d3e8c Driver vs referenee: ignore relevant configuration differences
The driver-vs-reference checks compare test results in different
configurations. Ignore the test results that report differences in
configurations that were the point of the comparison. Do compare other
configuration reports: this will let us know if the configurations diverge
in an unexpected way.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-06-20 17:12:51 +02:00
Gilles Peskine
2ad2f3207e Pacify pylint
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-05-22 09:35:11 +02:00
Gilles Peskine
78ae4f6fe1 Generate test data before coverage analysis
Fixes #8300.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-05-21 20:26:18 +02:00
Gilles Peskine
ff3b8211ff Driver-only FFDH is not good enough for DHE support in TLS 1.2
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-04-30 16:20:20 +02:00
Gilles Peskine
0f63028809
Merge pull request #8815 from gilles-peskine-arm/psa_generate_key_ext-prototype
Introduce psa_generate_key_ext
2024-02-26 07:16:49 +00:00
Manuel Pégourié-Gonnard
a7f651cf16
Merge pull request #8804 from valeriosetti/issue8799
mbedtls_rsa_parse_key and mbedtls_rsa_parse_pubkey accept trailing garbage
2024-02-20 11:58:52 +00:00
Valerio Setti
eba4ca19c6 test_suite_pem: solve driver test disparities
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2024-02-19 07:42:18 +01:00
Gilles Peskine
63072b1f94 Only test custom-e RSA key generation when built in
Custom-e RSA key generation is not yet supported in the test driver, and we
don't support fallback from the test driver to the built-in
implementation (even though we're testing with MBEDTLS_RSA_C). So for the
time being, only run psa_generate_key_ext test cases for RSA with a custom
public exponent when using the built-in implementation.

Add a test case to validate that psa_generate_key_ext test cases for RSA
with a custom public exponent returns NOT_SUPPORTED (rather than silently
doing the wrong thing) when not using built-in RSA (which is subtly
different from when having accelerated RSA: if both are enabled, which we
currently don't do in all.sh, then this should be supported and this is
validated by the test cases above).

This wart will be resolved when we add support for drivers with a
generate_key_ext entry point.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-02-15 12:02:04 +01:00
Gilles Peskine
3ea9450463
Merge pull request #8734 from valeriosetti/issue8564
Add test for driver-only HMAC
2024-02-14 13:43:40 +00:00
Ryan Everett
67f3568895 Reduce analyze_block_cipher_dispatch exceptions
Signed-off-by: Ryan Everett <ryan.everett@arm.com>
2024-02-09 13:02:23 +00:00
Ryan Everett
afb2eee263 Add PKCS5/12 exceptions to analyze_block_cipher_dispatch
Signed-off-by: Ryan Everett <ryan.everett@arm.com>
2024-02-08 14:31:54 +00:00
Valerio Setti
89d8a12e9c analyze_outcomes: fix typo
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2024-01-26 15:04:05 +01:00
Valerio Setti
cd89b0b536 all.sh: disable legacy hash support in test_psa_crypto_config_accel_hmac()
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2024-01-24 14:24:55 +01:00
Valerio Setti
20cea94fd4 analyze_outcomes: add task for HMAC coverage
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2024-01-22 16:23:25 +01:00
Gilles Peskine
b1f96c0354
Merge pull request #7815 from gilles-peskine-arm/ecp-export-partial
ECP keypair utility functions
2024-01-18 10:29:05 +00:00
Manuel Pégourié-Gonnard
7f48d5e203 Rename test components to better reflect content
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-01-08 11:35:01 +01:00
Valerio Setti
a0c9c6684d analyze_outcomes: ignore only test concerning AES/ARIA/Camellia in CMAC
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2023-12-29 14:14:11 +01:00
Valerio Setti
0635cca7d1 analyze_outcomes: update skipped tests following latest changes to all.sh
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2023-12-28 18:33:17 +01:00
Valerio Setti
ab0494f193 analyze_outcomes: update comments of skipped tests
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2023-12-28 13:56:52 +01:00
Gilles Peskine
3b17ae78d2 Add ECP-heavy-only test cases to the driver parity analysis ignore list
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2023-12-24 19:51:57 +01:00
Valerio Setti
5f665c3a0d analyze_outcomes: add exceptions to disparities for block_cipher dispatch
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2023-12-20 09:56:05 +01:00
Valerio Setti
9afa329b80 analyze_outcomes: allow ignored test suites to have a dot in the name
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2023-12-20 09:55:28 +01:00
Valerio Setti
4a8ef7cd9b all.sh: disable legacy AES/ARIA/CAMELLIA in test_full_block_cipher_psa_dispatch
This commit also:
- rename the reference component as component_test_full_block_cipher_legacy_dispatch()
- add a common configuration function, named common_block_cipher_dispatch() that
  is used from both accelerated and reference components

Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2023-12-20 07:35:41 +01:00
Valerio Setti
52ab8fa565 analyze_outcomes/all.sh: add reference component and entry for coverage comparison
Signed-off-by: Valerio Setti <valerio.setti@nordicsemi.no>
2023-12-14 18:09:26 +01:00
Pengyu Lv
98a90c6542 Fix various issue
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
2023-12-07 17:23:25 +08:00
Pengyu Lv
3cd16c47bd Add analyze_driver_vs_reference_rsa for analyze_outcomes
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
2023-12-07 10:24:29 +08:00
Gilles Peskine
18eab984c7
Merge pull request #8560 from lpy4105/issue/8423/optimize-analyze_outcomes_py
Optimize analyze_outcomes.py
2023-11-29 14:51:41 +00:00
Pengyu Lv
5dcfd0c613 Some improvements
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
2023-11-29 18:03:28 +08:00
Pengyu Lv
550cd6f9b2 Use boolean hit instead of int hits
Also fix a typo in the comments.

Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
2023-11-29 09:17:59 +08:00
Dave Rodgman
c3cd410acf
Merge pull request #8286 from gilles-peskine-arm/check_mbedtls_calloc_overallocation-disable_with_asan
Fix test_suite_platform failure with Asan on modern Clang
2023-11-28 16:48:31 +00:00
Pengyu Lv
451ec8a4bc Add comment to read_outcome_file in analyze_outcomes.py
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
2023-11-28 17:59:05 +08:00
Pengyu Lv
c2e8f3a080 Add type annotations to analyze_outcomes.py
Signed-off-by: Pengyu Lv <pengyu.lv@arm.com>
2023-11-28 17:22:04 +08:00