12027 Commits

Author SHA1 Message Date
Gilles Peskine
f5025e2680 Merge remote-tracking branch 'development' into test-cases-not-executed-enforce-development
Conflicts:
* framework: update to the head of 'main'.
2024-10-17 21:20:01 +02:00
David Horstmann
13d2939563
Merge pull request from eleuzi01/issue-54-fw
Move TLS auxiliary test scripts to the framework
2024-10-17 17:15:46 +00:00
Manuel Pégourié-Gonnard
dea880f035 all.sh: fix missing quotes
Without quotes, when the variable is empty, the shell will see three
tokens: [, -n, ]. After skipping ] as usual, it will see a single token,
so it will consider it not as command, but a string to be tested for "is
it empty", and since "-n" is not empty, the command will return true.

With quotes it see 4 tokens: [, -n, <empty string>, ] and interprets -n
as desired.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-10-17 09:21:19 +02:00
Gilles Peskine
e1f37c58a2
Merge pull request from gilles-peskine-arm/error-codes-4.0-enforce-add-macro
Error codes: enforce add macro
2024-10-16 17:28:04 +00:00
Gilles Peskine
8085f51108 Use MBEDTLS_ERROR_ADD instead of explicit addition: enforcement
Reject direct additions of error constants (regex-based approximation).

Fix the lone straggler.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-16 11:52:27 +02:00
Gilles Peskine
ed6feae513 Use MBEDTLS_ERROR_ADD instead of explicit addition: simple cases
Replace obvious additions of an `MBEDTLS_ERR_xxx` constant by a call to
`MBEDTLS_ERROR_ADD`.

Skip `case` statements since `MBEDTLS_ERROR_ADD(pp_constant)` is not a
preprocessor constant.

This commit does not replace additions split over lines. Those will be
handled in a subsequent commit.

```
git ls-files '*.h' '*.c' '*.function' '*.data' |
xargs perl -i -pe '
    next if /\bcase\b/;
    s/\b(MBEDTLS_ERR_\w+)\s*\+\s*(\w+)\b/MBEDTLS_ERROR_ADD($1, $2)/g;
    s/\b(\w+)\s*\+\s*(MBEDTLS_ERR_\w+)\b/MBEDTLS_ERROR_ADD($1, $2)/g'
```

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-16 11:52:24 +02:00
Manuel Pégourié-Gonnard
6c0b4e73c0 all.sh: adjust for when tf-psa-crypto is absent
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-10-16 10:50:43 +02:00
Manuel Pégourié-Gonnard
f48d4edd0c all.sh: re-instate 3.6-specific code
This aligns development and 3.6 in preparation for moving to the
framework repo.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-10-16 10:50:15 +02:00
Ronald Cron
ede04b34d1 Remove unnecessary -O2 compilation flag
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2024-10-16 10:47:15 +02:00
Ronald Cron
a0afbfb2a4 Fix/Add documentation
Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2024-10-15 17:09:36 +02:00
Ronald Cron
67cc6a73aa Use config_adjust_test_accelerators.h
Use config_adjust_test_accelerators.h
to adjust the configuration needed for
test_psa_crypto_drivers all.sh component
with PSA_CRYPTO_CONFIG enabled.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2024-10-15 17:09:36 +02:00
Ronald Cron
db11561352 Introduce config_adjust_test_accelerators.h
As we move to PSA_CRYPTO_CONFIG always on,
the way to configure the build with both
the builtin drivers and the transparent test
drivers (that are wrappers around the
builtin drivers) cannot be done through the
MBEDTLS_USER_CONFIG_FILE mechanism anymore.

With this mechanism and PSA_CRYPTO_CONFIG
enabled, the PSA_ACCEL_ macros are defined
before including config_adjust_legacy_from_psa.h
and the builtin implementations are removed.

Thus, we will rather define the PSA_ACCEL_
just after the inclusion of
config_adjust_legacy_from_psa.h through the
dedicated config_adjust_test_accelerators.h
header introduced by this commit.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2024-10-15 17:09:36 +02:00
Elena Uziunaite
9669eeabcf Adjust file path for translate_ciphers.py
Signed-off-by: Elena Uziunaite <elena.uziunaite@arm.com>
2024-10-15 11:45:15 +01:00
Elena Uziunaite
b74c3eaf87 Adjust file path for generate_tls13_compat_tests.py
Signed-off-by: Elena Uziunaite <elena.uziunaite@arm.com>
2024-10-15 11:45:05 +01:00
Elena Uziunaite
09fee364a3 Adjust file path for generate_ssl_debug_helpers.py
Signed-off-by: Elena Uziunaite <elena.uziunaite@arm.com>
2024-10-15 11:43:51 +01:00
Elena Uziunaite
2df289290f Move some files to framework repository
Signed-off-by: Elena Uziunaite <elena.uziunaite@arm.com>
2024-10-15 11:41:39 +01:00
Gilles Peskine
5e3ed3f8a0 Minor readability improvements
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-11 12:00:44 +02:00
Gilles Peskine
095561cca1 Switch outcome analysis to enforcing that all tests are executed
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-10 17:26:11 +02:00
Gilles Peskine
d9c40f538d We never test without the PSA client code.
Should we? To be decided later.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-10 17:26:11 +02:00
Gilles Peskine
24b03d8b72 Add ignore list for restartable ECDH in TLS
TLS only supports actual restartable ECDH with the legacy code that's going
away, not with the MBEDTLS_USE_PSA_CRYPTO code that's becoming the only
variant. This leaves a few test cases that validate restartable ECDH in TLS
as desirable, but not currently able to pass.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-10 17:26:11 +02:00
Gilles Peskine
2fd25bb2a5 Add ignore list entries for configurations that are not tested
For each ignore list entry, link to a GitHub issue for its resolution,
except for a few configurations which there is a good reason to leave
uncovered.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-10 17:26:11 +02:00
Gilles Peskine
b0ec85dcc6 Add ignore list entries for crypto tests that are not executed
For each ignore list entry, link to a GitHub issue for its resolution.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-10 17:26:11 +02:00
Gilles Peskine
de2316b8ea Add ignore list entries for TLS tests that are not executed
For each ignore list entry, link to a GitHub issue for its resolution,
except for ssl-opt Valgrind tests which we never intend to run on the CI.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-10 17:26:11 +02:00
Gilles Peskine
e3c64c3f26 Fix typo in dependency
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-10 17:26:11 +02:00
Gilles Peskine
ab5cc9bd30 In PSA generated tests, ignore patterns for which an issue has been raised
Ignore certain test cases which either should not be generated or should be
executed. For each ignore list entry, link to a GitHub issue whose
definition of done includes removing the entry.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-10 17:26:11 +02:00
Gilles Peskine
5872c0d4c0 In PSA generated tests, don't ignore not-implemented in some negative tests
Some negative tests involving unsupported mechanisms are executed, because
they're testing what happens if the mechanism is unsupported. Refine the
ignore list for `test_suite_psa_crypto_generate_key.generated` and
`test_suite_psa_crypto_op_fail.generated` accordingly.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-10 17:26:11 +02:00
Gilles Peskine
2a71fac470 In PSA generated tests, ignore mechanisms that are not implemented
This clears more than half of the test cases that are not executed.
This also captures a few negative test cases that are executed.
Subsequent commits will refine the filtering.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-10 17:26:11 +02:00
Gilles Peskine
80352ac58c
Merge pull request from gilles-peskine-arm/analyze_outcome-split-dev
Split check_test_cases.py and outcome_analysis.py
2024-10-10 13:45:49 +00:00
Manuel Pégourié-Gonnard
58c09bd642 all.sh: update documentation
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-10-10 10:13:19 +02:00
Manuel Pégourié-Gonnard
e4e65aa10a all.sh: move top-level code to a function
Now the output of
    grep '^[^ {}#]' tests/scripts/all-*.sh | grep -v '()'
is clean, with the only results being:
- shopt -s extglob which is needing for parsing the rest
- usage message

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-10-10 10:13:19 +02:00
Manuel Pégourié-Gonnard
7b556951f9 all.sh: document new file structure
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-10-10 10:13:19 +02:00
Manuel Pégourié-Gonnard
1cb8ee81f6 all.sh: move definitions to all-core.sh
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-10-10 10:13:18 +02:00
Manuel Pégourié-Gonnard
731128ce50 all.sh: wrap main code into main() function
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-10-10 10:13:17 +02:00
Manuel Pégourié-Gonnard
e953a7adac all.sh: move clang_version() to helpers file
This function is only used in (a few) components, it's not part of the core.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-10-10 10:13:17 +02:00
Manuel Pégourié-Gonnard
e6df2e4dbc all.sh: move component helpers to separate file
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-10-10 10:13:16 +02:00
Gilles Peskine
96db2ccedd Default to allowing partial test coverage
Currently, many test cases are not executed. A follow-up pull request will
take care of that. In the meantime, continue allowing partial test coverage.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-09 14:12:59 +02:00
Gilles Peskine
9f85399e7b Move test case analysis modules to framework repository
Move `collect_test_cases.py` (split from `check_test_cases.py`),
`check_test_cases.py`, and `outcome_analysis.py` (split from
`analyze_outcomes.py`) to the framework repository.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-09 14:12:05 +02:00
Manuel Pégourié-Gonnard
84649feef5 all.sh: Move some functions to the right section
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-10-09 12:54:05 +02:00
Manuel Pégourié-Gonnard
8f08bcd603 all.sh: rename a helper function
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-10-09 12:54:05 +02:00
Manuel Pégourié-Gonnard
73e2b056ca all.sh: extract repeated code to a function
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-10-09 12:54:05 +02:00
Manuel Pégourié-Gonnard
78c7592755 all.sh: group psasim helpers
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-10-09 12:54:04 +02:00
Manuel Pégourié-Gonnard
9896e08f5e all.sh: rm obsolete functions
The only occurrences found by git grep were the definitions and the
comment saying these are kept for backwards compatibility.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-10-09 12:54:04 +02:00
Manuel Pégourié-Gonnard
f7325865bb all.sh: first define functions, then call them.
Don't interleave defining functions with running some code.

The only exception is calling shopt, which needs to come first as it
affects how the following function definitions are parsed.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-10-09 12:54:04 +02:00
Gilles Peskine
3146772575 Adjust paths for impending moves to the framework
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-08 19:37:23 +02:00
Harry Ramsey
d617283187 Fix failing psasim tests
This commit fixes failing psasim tests by ensuring that
MBEDTLS_VERSION_C is defined before attempting to access version
information.

Signed-off-by: Harry Ramsey <harry.ramsey@arm.com>
2024-10-07 14:50:00 +01:00
Harry Ramsey
f7e7acc9c1 Move version tests from tf-psa-crypto/test/suite to test/suites
This commit reverts the moves the test for version from
tf-psa-crypto/test/suite to test/suites.

Signed-off-by: Harry Ramsey <harry.ramsey@arm.com>
2024-10-04 13:37:42 +01:00
Harry Ramsey
468c0aede9 Revert move of version.h to tf-psa-crypto libray
This commit moves version.h back into /include/mbedtls from
tf-psa-crypto/drivers/builtin/include. This commit also changes the
necessary build files and scripts to generate version_features.c

Signed-off-by: Harry Ramsey <harry.ramsey@arm.com>
2024-10-04 13:37:42 +01:00
Ronald Cron
467edcd64a
Merge pull request from ronald-cron-arm/tf-psa-crypto-cmake-build
Add crypto only CMake build system
2024-10-03 20:56:00 +00:00
Gilles Peskine
082eadef4e Separate code and data of outcome analysis
Place the code of outcome analysis (auxiliary functions, tasks, command line
entry point) into a separate module, which will be moved to the
version-independent framework repository so that it can be shared between
maintained branches. Keep the branch-specific list of driver components and
ignore lists in the per-repository script.

We keep the executable script at `tests/scripts/analyze_outcomes.py`. It's
simpler that way, because that path is hard-coded in CI scripts.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-03 18:42:37 +02:00
Gilles Peskine
e41cde57c3 Pass KNOWN_TASKS as an argument to main
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-03 18:36:09 +02:00