57 Commits

Author SHA1 Message Date
David Horstmann
5b93d97b95 Update references to test helpers
Replace:
* tests/src -> framework/tests/src
* tests/include -> framework/tests/include

Except for occurrences of:
* tests/src/test_helpers (since this only contains ssl_helpers.c)
* tests/src/test_certs.h
* tests/include/alt_dummy

Signed-off-by: David Horstmann <david.horstmann@arm.com>
2024-11-18 15:50:44 +00:00
Gilles Peskine
3146772575 Adjust paths for impending moves to the framework
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2024-10-08 19:37:23 +02:00
Manuel Pégourié-Gonnard
4575d230bf Add a note on hits usefulness
And fix a typo while at it.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-04-15 10:54:49 +02:00
Manuel Pégourié-Gonnard
432e3b4198 Misc fixes & improvements to driver testing doc
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-04-12 12:44:41 +02:00
Manuel Pégourié-Gonnard
a47a3c4e13 Rephrase description of the KDF situation
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-04-12 12:44:41 +02:00
Manuel Pégourié-Gonnard
ae22f04769 Refine paragraphs about incomplete entry points
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-04-12 12:44:41 +02:00
Manuel Pégourié-Gonnard
0ca2fd0e2b Update libtestdriver1 vs internal
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-04-12 12:44:40 +02:00
Manuel Pégourié-Gonnard
dde1abd572 Update of opaque asymmetric encrypt/decrypt
https://github.com/Mbed-TLS/mbedtls/pull/8700 merged in the meantime.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-04-12 12:44:40 +02:00
Manuel Pégourié-Gonnard
6c45361a9c Update for HMAC testing
Been merged in the meantime.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-04-12 12:44:40 +02:00
Manuel Pégourié-Gonnard
98f8da1b1a Update names of components renamed in the meantime
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-04-12 12:44:40 +02:00
Manuel Pégourié-Gonnard
f2089dab5e Update status of RSA testing
Improved by https://github.com/Mbed-TLS/mbedtls/pull/8616/ - closing
8553.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-04-12 12:40:01 +02:00
Manuel Pégourié-Gonnard
b18bc80133 Add note about fallback to other entry points
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-04-12 12:40:01 +02:00
Manuel Pégourié-Gonnard
6a96f42051 Document driver wrapper suite & tested configs
The coverage data for the test drivers was generated using the following
patch:

diff --git a/scripts/lcov.sh b/scripts/lcov.sh
index 9258ba788874..1ef071a65c06 100755
--- a/scripts/lcov.sh
+++ b/scripts/lcov.sh
@@ -63,8 +63,8 @@ if [ $# -gt 0 ] && [ "$1" = "--help" ]; then
 fi

 if in_mbedtls_build_dir; then
-    library_dir='library'
-    title='Mbed TLS'
+    library_dir='tests/src/drivers'
+    title='Mbed TLS test drivers'
 else
     library_dir='core'
     title='TF-PSA-Crypto'
diff --git a/tests/scripts/all.sh b/tests/scripts/all.sh
index 734d8323ca73..f6b17ca5692b 100755
--- a/tests/scripts/all.sh
+++ b/tests/scripts/all.sh
@@ -4795,14 +4795,17 @@ component_test_psa_crypto_drivers () {
     msg "build: full + test drivers dispatching to builtins"
     scripts/config.py full
     scripts/config.py unset MBEDTLS_PSA_CRYPTO_CONFIG
-    loc_cflags="$ASAN_CFLAGS -DPSA_CRYPTO_DRIVER_TEST_ALL"
+    loc_cflags="--coverage -DPSA_CRYPTO_DRIVER_TEST_ALL"
     loc_cflags="${loc_cflags} '-DMBEDTLS_USER_CONFIG_FILE=\"../tests/configs/user-config-for-test.h\"'"
-    loc_cflags="${loc_cflags} -I../tests/include -O2"
+    loc_cflags="${loc_cflags} -I../tests/include -Og -g3"

-    make CC=gcc CFLAGS="${loc_cflags}" LDFLAGS="$ASAN_CFLAGS"
+    make CC=gcc CFLAGS="${loc_cflags}" LDFLAGS="--coverage" -C tests test_suite_psa_crypto_driver_wrappers

     msg "test: full + test drivers dispatching to builtins"
-    make test
+    (cd tests && ./test_suite_psa_crypto_driver_wrappers --verbose)
+    #make test
+
+    scripts/lcov.sh
 }

 component_test_make_shared () {

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-04-12 12:40:01 +02:00
Manuel Pégourié-Gonnard
b66f9dba11 Document test-driver status per family
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-04-12 12:40:00 +02:00
Manuel Pégourié-Gonnard
1a827a3422 Start documenting test-driver framework.
Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2024-04-12 12:40:00 +02:00
Dave Rodgman
e183ecef3d
Merge pull request #7136 from yanrayw/5692-record-compatsh-test-cases
Record the outcome of each test case in compat.sh
2023-07-10 12:08:32 +01:00
Fredrik Hesse
cc207bc379 Replace references to Mbed Crypto with Mbed TLS through-out documentation and comments.
Signed-off-by: Fredrik Hesse <fredrik@hesse.se>
2023-05-12 14:59:01 +01:00
Yanray Wang
7fc349e903 test-framework.md: document compat.sh
Signed-off-by: Yanray Wang <yanray.wang@arm.com>
2023-02-28 14:30:26 +08:00
Dave Rodgman
b319684bca Additional updates to docs links
Signed-off-by: Dave Rodgman <dave.rodgman@arm.com>
2022-10-12 16:47:08 +01:00
bootstrap-prime
6dbbf44d78
Fix typos in documentation and constants with typo finding tool
Signed-off-by: bootstrap-prime <bootstrap.prime@gmail.com>
2022-05-18 14:15:33 -04:00
Shaun Case
8b0ecbccf4 Redo of PR#5345. Fixed spelling and typographical errors found by CodeSpell.
Signed-off-by: Shaun Case <warmsocks@gmail.com>
Signed-off-by: Dave Rodgman <dave.rodgman@arm.com>
2022-05-11 21:25:51 +01:00
Gilles Peskine
790f7428d2 Storage format test regressions are now checked mechanically
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2022-02-22 19:16:42 +01:00
Gilles Peskine
8ec3c8f015 Do not require test data to be in the repository
What matters is that we validate that test data is not removed. Keeping the
test data is the most obvious way, but not the only way.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-15 16:22:14 +02:00
Gilles Peskine
b91f81a55b Discuss lifetimes, in particular persistence levels
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-15 16:22:14 +02:00
Gilles Peskine
77f8e5cb59 Add considerations on key material representations
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-15 16:22:14 +02:00
Gilles Peskine
d131e400f0 Clarification: forward and backward compatibility
The import-and-save and load-and-check parts of the tests don't have to be
actually the same test cases.

Introduce the terms “forward compatibility” and “backward compatibility” and
relate them to import-and-save and load-and-check actions.

These are clarifications of intent that do not represent an intended change
in the strategy or intended coverage.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-15 16:22:14 +02:00
Gilles Peskine
f31c6c111e Typo
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-15 16:22:14 +02:00
Bence Szépkúti
bb0cfeb2d4 Rename config.h to mbedtls_config.h
This commit was generated using the following script:

# ========================
#!/bin/sh
git ls-files | grep -v '^ChangeLog' | xargs sed -b -E -i '
s/((check|crypto|full|mbedtls|query)_config)\.h/\1\nh/g
s/config\.h/mbedtls_config.h/g
y/\n/./
'
mv include/mbedtls/config.h include/mbedtls/mbedtls_config.h
# ========================

Signed-off-by: Bence Szépkúti <bence.szepkuti@arm.com>
2021-06-28 09:28:33 +01:00
Chris Jones
6f554e388e Remove reference to include/mbedtls/*_internal.h files
Signed-off-by: Chris Jones <christopher.jones@arm.com>
2021-04-01 09:52:37 +01:00
Ronald Cron
a120146afe
Merge pull request #3962 from gilles-peskine-arm/psa-storage-format-test-strategy
Keystore format stability test strategy
2021-01-25 16:44:23 +01:00
Gilles Peskine
ff457506d3 Remind the reader of what is done about old formats
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-01-19 14:25:20 +01:00
Gilles Peskine
697ee190b5 Add a section about non-default lifetimes
Alternative locations should be covered. We don't yet support
alternative persistence levels.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-01-18 23:38:21 +01:00
Gilles Peskine
528144f523 Clarify the methods of key storage testing
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-01-18 23:36:18 +01:00
Gilles Peskine
cf62f10d3f Clarify interoperability non-requirement
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-01-05 11:49:18 +01:00
Gilles Peskine
739e08a68d Keystore format stability test strategy
Initial revision.

Save-compare-load approach: the test case data contains attributes of
the object under test and the expected file content. Create the
object, save it, check that the file has the expected content, load
the file and check that the new object has the expected attributes.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-12-14 18:51:47 +01:00
Gilles Peskine
24cebf6671 Add a section for transparent drivers
No strategy yet, just state some basic requirements.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-12-08 15:19:04 +01:00
Gilles Peskine
f0e2853d46 Minimal update to mention unified-interface opaque drivers
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-12-08 15:19:04 +01:00
Gilles Peskine
db6b03b4f2 Remove old-style accelerator and entropy driver interfaces
The driver interfaces described in crypto_accel_driver.h and
crypto_entropy_driver.h are no longer being worked on. We do not
intend to finish the design of these interfaces or to implement them
in Mbed TLS. They have been superseded by the unified driver
interface (docs/proposed/psa-driver-interface.md), which is being
actively worked on both to finalize the specification and to implement
it in Mbed TLS.

The partially implemented dynamic secure element interface is staying
for now.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-11-30 17:37:14 +01:00
Ronald Cron
cf56a0a320 psa: Move from key handle to key identifier
Move all the PSA crypto APIs using key handles
to use key identifiers but psa_key_open() and
psa_key_close(). This is done without modifying
any test as key handles and key identifiers are
now the same.

Update the library modules using PSA crypto APIs
to get rid of key handles.

Programs and unit tests are updated to not use
key handles in subsequent commits, not in this
one.

Signed-off-by: Ronald Cron <ronald.cron@arm.com>
2020-11-10 16:00:41 +01:00
Gilles Peskine
fb4f933f8e Rename Python scripts to use '_' and not '-'
You can't import a Python script whose name includes '-'.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-06-25 14:22:06 +02:00
Gilles Peskine
24ba42cef7 Fix explanation of rules for function substitution
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-11 23:07:42 +02:00
Gilles Peskine
688f6cc591 There are test programs, not just unit tests
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-11 23:07:40 +02:00
Gilles Peskine
5925183b8a Fix explanation of PSA_DONE
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-11 23:06:37 +02:00
Gilles Peskine
2182585776 Introduction: present the top-level sections
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-11 23:03:33 +02:00
Gilles Peskine
d04b9ed7dd Spelling
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-05-11 23:03:24 +02:00
Gilles Peskine
fa51820e39 Expand the rule for internal functions exposed for tests only
Clarify that using a header in library/ rather than include/ for
internal functions is a rule, not just a possibility.

As suggested by Manuel, state a rule for functions that need to be
static for best optimization but that we want to unit-test.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-24 22:56:38 +01:00
Gilles Peskine
4b7279e5d9 Write up some solutions
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-24 22:56:31 +01:00
Gilles Peskine
dff10c773b Add a note that TLS requires further consideration
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-24 22:56:22 +01:00
Gilles Peskine
ab4b9b4165 New test strategy document: invasive testing
Evaluate possible approaches for invasive testing.

State some rules.

This commit was originally written for Mbed Crypto only.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2020-03-24 22:56:15 +01:00
Gilles Peskine
9afbfdc833 Merge remote-tracking branch 'myfork-public/development' into merge-crypto-development-20191115
First deal with deleted files.

* Files deleted by us: keep them deleted.
* Files deleted by them, whether modified by us or not: keep our version.

```
git rm $(git status -s | sed -n 's/^DU //p')
git reset -- $(git status -s | sed -n 's/^D  //p')
git checkout -- $(git status -s | sed -n 's/^ D //p')
git add -- $(git status -s | sed -n 's/^UD //p')
```

Individual files with conflicts:

* `3rdparty/everest/library/Hacl_Curve25519_joined.c`: spurious conflict because git mistakenly identified this file as a rename. Keep our version.
* `README.md`: conflict due to their change in a paragraph that doesn't exist in our version. Keep our version of this paragraph.
* `docs/architecture/Makefile`: near-identical additions. Adapt the definition of `all_markdown` and include the clean target.
* `doxygen/input/docs_mainpage.h`: conflict in the version number. Keep our version number.
* `include/mbedtls/config.h`: two delete/modify conflicts. Keep the removed chunks out.
* `library/CMakeLists.txt`: discard all their changes as they are not relevant.
* `library/Makefile`:
    * Discard the added chunk about the crypto submodule starting with `INCLUDING_FROM_MBEDTLS:=1`.
    * delete/modify: keep the removed chunk out.
    * library build: This is almost delete/modify. Their changes are mostly not applicable. Do keep the `libmbedcrypto.$(DLEXT): | libmbedcrypto.a` order dependency.
    * `.c.o`: `-o` was added on both sides but in a different place. Change to their place.
* `library/error.c`: to be regenerated.
* `library/version_features.c`: to be regenerated.
* `programs/Makefile`: Most of the changes are not relevant. The one relevant change is in the `clean` target for Windows; adapt it by removing `/S` from our version.
* `programs/test/query_config.c`: to be regenerated.
* `scripts/config.py`: added in parallel on both sides. Keep our version.
* `scripts/footprint.sh`: parallel changes. Keep our version.
* `scripts/generate_visualc_files.pl`: one delete/modify conflict. Keep the removed chunks out.
* `tests/Makefile`: discard all of their changes.
* `tests/scripts/all.sh`:
    * `pre_initialize_variables` add `append_outcome`: add it.
    * `pre_initialize_variables` add `ASAN_CFLAGS`: already there, keep our version.
    * `pre_parse_command_line` add `--no-append-outcome`: add it.
    * `pre_parse_command_line` add `--outcome-file`: add it.
    * `pre_print_configuration`: add `MBEDTLS_TEST_OUTCOME_FILE`.
    * Several changes in SSL-specific components: keep our version without them.
    * Several changes where `config.pl` was changed to `config.py` and there was an adjacent difference: keep our version.
    * Changes regarding the inclusion of `MBEDTLS_MEMORY_xxx`: ignore them here, they will be normalized in a subsequent commit.
    * `component_test_full_cmake_gcc_asan`: add it without the TLS tests.
    * `component_test_no_use_psa_crypto_full_cmake_asan`: keep the fixed `msg`, discard other changes.
    * `component_test_memory_buffer_allocator_backtrace`, `component_test_memory_buffer_allocator`: add them without the TLS tests.
    * `component_test_m32_everest`: added in parallel on both sides. Keep our version.
* `tests/scripts/check-names.sh`, `tests/scripts/list-enum-consts.pl`, `tests/scripts/list-identifiers.sh`, ``tests/scripts/list-macros.sh`: discard all of their changes.
* `tests/scripts/test-ref-configs.pl`: the change in the conflict is not relevant, so keep our version there.
* `visualc/VS2010/*.vcxproj`: to be regenerated.

Regenerate files:

```
scripts/generate_visualc_files.pl
git add visualc/VS2010/*.vcxproj
scripts/generate_errors.pl
git add library/error.c
scripts/generate_features.pl
git add library/version_features.c
scripts/generate_query_config.pl
git add programs/test/query_config.c
```

Rejected changes in non-conflicting files:

* `CMakeLists.txt`: discard their addition which has already been side-ported.
* `doxygen/mbedtls.doxyfile`: keep the version number change. Discard the changes related to `../crypto` paths.

Keep the following changes after examination:

* `.travis.yml`: all of their changes are relevant.
* `include/mbedtls/error.h`: do keep their changes. Even though Crypto doesn't use TLS errors, it must not encroach on TLS's allocated numbers.
* `tests/scripts/check-test-cases.py`: keep the code dealing with `ssl-opt.sh`. It works correctly when the file is not present.
2019-11-15 11:47:14 +01:00