1313 Commits

Author SHA1 Message Date
Yuto Takano
5c1acf2735 Match macros with spaces between # and define
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
d24e037dd1 Warn user if files are excluded from search
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
5939a2a4de Check environment by verifying other dirs' existence
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
201f9e85ca Make use of -v to log some debug information
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
81528c058a Add documentation, fix identifier parsing
- Add documentation to all classes and functions that were
  not self-explanatory.
- Fix the parsing of identifiers, so it now behaves identically
  to the original shell script. Detects the same amount of identifiers.
- Fix macro parsing so MBEDTLS_PSA_ACCEL didn't error out
- Reformat output to be comprehensible

Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
6f38ab3bca Fix legacy troublesome regex
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
fe02684049 Python port of f6643ccd90694ae99d05541990b78738a8444ab0
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
0fd48f7939 Python port of 7cc4c68eb63a24f9cbf814254cd537df819958e5
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
17220988dc Parse identifiers from library headers: Python port of d9eee3b417c2e8f63dd10d835ab9a9472242c2ed
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
56e3a5caa6 Add test driver symbols: Python port of 7f13fa2454282b21930045a3f4f9a2835d80425e
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
e77f699ed5 Exclude FStar and Hacl: Python port of 9b33e7d7d7426e3d7f27cd7d206765ae33e3e61f
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
062289c657 Invoke config.py instead of pl: Python port of 5d46f6a89b25603f0a77466c618213200c328510
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
c62b4084a2 Per-line opt-out of typo check: Python port of b6837761815e1a8f6f475be4575824fc386a08dd
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
e503d61b99 Remove 1.3 to 2.0 helpers: Python port of 7d48b2821808e964ab594462e419fbed0e015729
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
157444c24d Add library header files: Python port of 65a6fa3e2669cb02af5399d0f60b5bed3e62a9be
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
c763cc368f Check for double underscores: Python port of 712f7a804e391737b0e9d2593abe291f4ccb0303
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
fa950ae344 Look in 3rdparty: Python port of 8a0f5bb3c11196a5bc0df6393a47e56c40adb7ac
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
bb7dca495f Work with PSA constants: Python port of 03091d1114450dd19a10215094682f14761540d9
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
ed91cf003a Remove Yotta: Python port of 3ad2efdc82a3d15f373b9d12e6764efec3577b55
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
c1838937f1 Also check PSA: Python port of 2d9d6db60f5fd0a4993d90e47f39462647624ad6
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
3963967ebc Restructure check-names.py with more verbose error messages
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:39 +01:00
Yuto Takano
b61f0e1151 Merge upstream 3.0 from 'development' in ARMmbed/mbedtls
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:07:34 +01:00
Darryl Green
6c79b5dce7 Keep compatibility with python versions prior to 3.5
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:04:36 +01:00
Darryl Green
a783d9c5ef Remove check-names.sh and sub-scripts it used
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:04:34 +01:00
Darryl Green
4e9b51bc18 Update scripts to use check-names.py
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:04:32 +01:00
Darryl Green
d5802926d9 Rewrite check-names.sh in python
Signed-off-by: Yuto Takano <yuto.takano@arm.com>
2021-08-06 21:04:26 +01:00
Gilles Peskine
80ddb991c2 Add --restore option to clean up but not necessarily run components
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-06 11:51:59 +02:00
Gilles Peskine
03af678911 Documentation improvements
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-06 11:35:17 +02:00
Gilles Peskine
86f6129067 Documentation improvement
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-05 15:11:33 +02:00
Gilles Peskine
7530163f3b Make --quiet more effective when running make generated_files
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-05 15:10:47 +02:00
Gilles Peskine
bf66e2cc8f Documentation improvements
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-03 13:44:28 +02:00
Gilles Peskine
1d475b6398 Disable wildcards when checking for unsupported components
Otherwise $COMMAND_LINE_COMPONENTS would try to expand wildcard patterns
based on files in the current directory.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-03 13:43:36 +02:00
Manuel Pégourié-Gonnard
ae505eeeed Fix missing dependency on Travis
Was getting errors like:

In file included from /usr/include/limits.h:25:0,
                 from /usr/lib/gcc-cross/arm-linux-gnueabi/5/include-fixed/limits.h:168,
                 from /usr/lib/gcc-cross/arm-linux-gnueabi/5/include-fixed/syslimits.h:7,
                 from /usr/lib/gcc-cross/arm-linux-gnueabi/5/include-fixed/limits.h:34,
                 from ../include/mbedtls/check_config.h:30,
                 from ../include/mbedtls/build_info.h:81,
                 from common.h:26,
                 from asn1write.c:20:
/usr/include/features.h:367:25: fatal error: sys/cdefs.h: No such file or directory

There are two packages to choose from: armhf or armel. Since the comment
in all.sh says we're trying to be close to Debian's "armel"
architecture, choose that, and fix a comment that was mentioning
gnueabihf for no apparent reason.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2021-08-03 11:19:59 +02:00
Manuel Pégourié-Gonnard
3a6c76937a Add arm-linux-gnueabi-gcc build to all.sh
Currently it can't be mandatory, since we can't install the required toolchain
on Jenkins right away.

Also, while at it, remove `SHELL='sh -x'` from the other arm5vte component; it
was a leftover from debugging.

Signed-off-by: Manuel Pégourié-Gonnard <manuel.pegourie-gonnard@arm.com>
2021-08-03 11:19:59 +02:00
Gilles Peskine
c111e24292 Improve the detection of keep-going commands
Have simpler patterns related to 'test' (the central objective being to keep
going if 'make test' or 'tests/...' fails, but not if 'make tests' fails).

Add 'cd' as a can't-keep-going command.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-02 23:29:53 +02:00
Gilles Peskine
88a7c2b32e Improve --error-test reporting
Count invocations from 1 to n instead of n to 1.

Explain how changing the loop variable would cause an error if the function
was not executed in a subshell.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-02 23:28:00 +02:00
Gilles Peskine
ec135544c8 Clarify some comments
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-08-02 23:27:42 +02:00
Hanno Becker
6c53ecc01d all.sh: Run basic TLS 1.3 with and without record padding
Signed-off-by: Hanno Becker <hanno.becker@arm.com>
2021-08-02 04:54:03 +01:00
Gilles Peskine
d8d19327d5 Fix mixup about the directory containing the success indicator file
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-22 12:30:14 +02:00
Gilles Peskine
6ee3b7ed1f More robust failure detection for the coverage report generation
The previous implementation was hard to understand and could in principle
fail to notice if there was a test case failure and the writing of the
line "Note: $TOTAL_FAIL failures." failed. KISS.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-22 11:08:30 +02:00
Gilles Peskine
b969570650 Explain the final error checking
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-22 10:56:39 +02:00
Gilles Peskine
fc70b5252b Save the coverage report to a file
Save the "Test Report Summary" to a file. This can help both CI scripts and
human readers who want the summary after the fact without having to copy the
console output.

Take care to exit with a nonzero status if there is a failure while
generating the test report summary.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-13 23:27:01 +02:00
Gilles Peskine
bbced87390 Show the udp_proxy seed in the console log
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-13 23:26:00 +02:00
Gilles Peskine
a3d2264b88 Only run an unbridled parallel make (make -j) if MAKEFLAGS is unset
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-13 23:23:23 +02:00
Gilles Peskine
568f53a9d8 Don't unconditionally restore **/Makefile
all.sh restores **/Makefile from git in case the version in the worktree was
from doing a cmake in-tree build. Instead of doing this unconditionally, do
it only if the toplevel Makefile seems to have been automatically
generated (by cmake or otherwise, e.g. by mbedtls-prepare-build). This way
all.sh no longer silently wipes changes made to Makefile but not committed yet.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-12 18:19:39 +02:00
Gilles Peskine
f83eb82a4d Don't restore *config.h before backing it up
Back up the config files at the beginning of all.sh, rather than before each
component. In particular, create the backup before running cleanup for the
first time. This fixes #3139 (all.sh using a config.h.bak from a previous
job), and makes all.sh more robust against accidentally using a modified
config.h midway through because a component messed with the backup.

Use a different extension (*.all.bak rather than *.bak) for the backups.
This is necessary to ensure that auxiliary scripts such as depends*.pl that
make their own backup don't remove all.sh's backup, which the code from this
commit does not support.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-12 18:19:31 +02:00
Gilles Peskine
03ab544832 Generate cpp_cummy_build.cpp dynamically
Generate programs/test/cpp_dummy_build.cpp dynamically instead of
maintaining it manually. This removes the need to update it when the list of
headers changes.

Include all the headers unconditionally except for the ones that cannot be
included directly.

Support this dynamic generation both with make and with cmake.

Adapt all.sh accordingly. Remove the redundant C build from
component_build_default_make_gcc_and_cxx (it was also done in
component_test_default_out_of_box), leaving a component_test_make_cxx. Also
run the C++ program, because why not. Do this in the full configuration
which may catch a bit more problems in headers.

Fixes #2570 for good.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-12 18:19:31 +02:00
Gilles Peskine
7238503642 Heed --quiet when running make generated_files
Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-12 18:19:31 +02:00
Gilles Peskine
88a07457c7 Remove barely-used redirect functions
redirect_out was no longer used and redirect_err was only used to
quiet dd. Change the dd invocation to only print diagnostics on
error (on platforms where this is possible).

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-12 18:19:31 +02:00
Gilles Peskine
aca0b32132 Keep going after a shell "[" a.k.a. "test" fails
This is necessary to actually keep going and finish the
component-specific cleanup in component_test_cmake_out_of_source if
ssl-opt.err is non-empty.

Signed-off-by: Gilles Peskine <Gilles.Peskine@arm.com>
2021-07-12 18:19:31 +02:00