Always have tasks_list be a list, not potentially some fancier iterable.
Bypass mypy's somewhat legitimate complaint about REFERENCE and DRIVER in
task_class: they could potentially be instance attributes, but we rely on
them being class attributes. Python does normally guarantee their existence
as class attributes (unless a derived class explicitly deletes them), but
they could be overridden by an instance attribute; that's just something
we don't do, so the class attribute's value is legitimate. We can't
expect mypy to know that, so work around its complaint.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Use different names for task name, a task class and a task instance. The
interpreter doesn't care, but it's less confusing for both humans and type
checkers.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The ignore list for coverage only has two test cases out of ~10000 that are
currently reported as not executed. This is a drop in the sea and not
useful. Remove them so that the class can be used generically. A follow-up
will construct a comprehensive ignore list.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Move the test case collection code out of check_test_cases.py and into its
own module. This allows outcome analysis to depend only on the new module
and not on check_test_cases.py.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Stop testing configurations without PSA (MBEDTLS_PSA_CRYPTO_C or at least
MBEDTLS_PSA_CRYPTO_CLIENT). No future release from this branch will support
such configurations, and we can no longer build the SSL sample programs
without psa_crypto_init.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Work on the stringly typed KNOWN_TASKS by classes for each category
of tasks, with a structure that matches the behavior.
This commit migrates the data for driver-vs-reference analysis and gets rid
of the transitional code that was using the old form of the data.
No intended behavior change.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Work on the stringly typed KNOWN_TASKS by classes for each category
of tasks, with a structure that matches the behavior.
This commit migrates the code for driver-vs-reference analysis. To
facilitate review, this commit preserves the layout of the data that
parametrizes each task. The next commit will migrate the data.
No intended behavior change.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Work on replacing the stringly typed KNOWN_TASKS by classes for each category
of tasks, with a structure that matches the behavior.
This commit migrates test coverage analysis.
No intended behavior change.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Start replacing the stringly typed KNOWN_TASKS by classes for each category
of tasks, with a structure that matches the behavior.
This commit introduces some transition code.
No intended behavior change.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Fix `tls13-compat.sh` changing based on exactly how
`generate_tls13_compat_tests.py` was run (e.g. from which directory). This
made `check-generated-files.sh` behave differently from `make`. The script
has no official variations of the content of its output file, so we don't
need to record the full command line.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
`tests/opt-testcases/tls13-compat.sh` is supposed to be automatically
generated by `tests/scripts/generate_tls13_compat_tests.py`. So far, the
output has been updated by running the script manually and committing the
output. Switch to using our framework for generated files.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
With no options, update the output file (former behavior with -a).
Pass -1 to generate a single test case.
Also have the intended output file location as the default.
This way, you can just run the script after updating it, without having to
know the details of the directory structure.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
MBEDTLS_SSL_TLS1_3_COMPATIBILITY_MODE is no longer required, except in test
cases that are specifically about it. This commit removes the requirement in
tls13-compat.sh (which does not have test cases that actually depend on the
feature).
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
The *.sh files in opt-testcases cannot be executed directly: they can only
be sourced by ssl-opt.sh. So don't make them executable and don't give them
a shebang line.
Also make sure that the first paragraph of each file is a short description.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
In the components migrated from test-ref-configs.pl, we don't need to
activate PSA: it's always on. Also, since there is no "_legacy" component to
contrast with, drop "_psa" from the component names.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Remove the components migrated from test-ref-configs.pl that use legacy
crypto (no enabling of MBEDTLS_USE_PSA_CRYPTO). In the 4.0 preparation
branch, we are no longer interested in such configurations.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Rename the existing component_test_tfm_config which tests a modified version
of config-tfm.h for the sake of driver-vs-reference comparison.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
Add a test that does some basic validation of the pkg-config files.
Example run:
./tests/scripts/all.sh test_cmake_as_package
<snip>
******************************************************************
* test_cmake_as_package: build: cmake 'as-package' build
* Wed Sep 11 16:17:41 UTC 2024
******************************************************************
cmake .
make
Built against Mbed TLS 3.6.0
testing package config file: mbedtls ... passed
testing package config file: mbedx509 ... passed
testing package config file: mbedcrypto ... passed
make clean
Signed-off-by: Bill Roberts <bill.roberts@arm.com>
Running test suites from another directory only works when the auto-chdir
code in host_test.function is enabled, which is platform-dependent and
configuration-dependent.
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
