579 Commits

Author SHA1 Message Date
Gilles Peskine
314bc89b36 Revert "scripts: Remove dependency on TLS"
This reverts commit b58ff9541ba6ce14d34215f8e40d3c0d90ade268.
2020-03-19 13:36:33 +01:00
Gilles Peskine
1bf45e1980 Revert "scripts: Remove dependency on X.509"
This reverts commit 43a450c858c4b4d681fc3cb695622fe8fd05c66a.
2020-03-19 13:36:33 +01:00
Gilles Peskine
1fcf721e11 Revert "scripts: Remove dependency on NET"
This reverts commit 356acc82ad413dfec8d49745793e94a2e2f4c69e.

Conflicts:
* scripts/generate_errors.pl: a line adjacent to a changed line has
  independently changed in the meantime. Just revert the change done
  in the commit that's being reverted.
2020-03-19 13:36:12 +01:00
Simon Butcher
e30d03e4f4 Change the use of pylint to optionally use pylint3
Pylint when installed as a distro package can be installed as pylint3, whilst as
a PEP egg, it can be installed as pylint.

This commit changes the scripts to first use pylint if installed, and optionally
look for pylint3 if not installed. This is to allow a preference for the PEP
version over the distro version, assuming the PEP one is more likely to be
the correct one.

Signed-off-by: Simon Butcher <simon.butcher@arm.com>
2020-03-16 11:38:02 +00:00
Simon Butcher
9693ea2490 Fix output_env.sh for varying pylint3 output
`pylint3 --version` will output to stderr the status of the config file it's
using. This can be "No config file found" or "Using config file" or nothing.
This means the pylint version may or may not be on the first line.

Therefore this commit changes the filters on the pylint3 version output to first
strip out the config line, and then to select only the pylint line.

Signed-off-by: Simon Butcher <simon.butcher@arm.com>
2020-03-06 15:06:14 +00:00
Simon Butcher
6910122469 Add Python, Perl and Pylint to output_env.sh
Add the versions of Python, Perl, and Pylint to the version dump provided by
the output_env.sh script.

Signed-off-by: Simon Butcher <simon.butcher@arm.com>
2020-03-06 15:06:14 +00:00
Gilles Peskine
302e43f122 Revert "scripts: Remove unneeded scripts"
This reverts commit ed05b29ea335dd12415b40570b31b08fa8c8bd09.
2020-03-04 15:39:14 +01:00
Gilles Peskine
40f17dc803 Revert "Remove certs.h"
This reverts commit b8e4ae18cf24644fa8daea6add26ad33aa1e52a7.
2020-03-04 15:39:14 +01:00
Simon Butcher
7bfeb66d1a Make minor fixes to output_env.sh after review
A number of clean-up improvements following review.
 * removal of redundant `` quotes
 * removal of non-portable echo "\n", in favour of additional echo commands
 * change to use of uname to detemine if the platform is Linux or not
 * revised formatting of output
 * change to dpkg-query from dpkg to find installed libasan variants

Co-Authored-By: Gilles Peskine <gilles.peskine@arm.com>
Signed-off-by: Simon Butcher <simon.butcher@arm.com>
2020-02-27 15:19:58 +00:00
Simon Butcher
07d5a47bd8 Enhance the output_env.sh script
This commit adds additional information to the output_env.sh script of:
  * Linux distribution version (if available)
  * GDB version (if available)

It also makes some information clearer:
  * the type of OpenSSL/GNUTLS version (legacy/default/next)
  * and whether certain versions are not installed, or not configured

And it simplifies the error messages for absent tools.

Signed-off-by: Simon Butcher <simon.butcher@arm.com>
2020-02-26 15:58:52 +00:00
Gilles Peskine
d362d0bf86 Tweak spacing in Visual Studio files
Tweak the code to be slightly simpler and closer to mbedtls. This
changes non-significant whitespace in the generated files.
2020-02-26 14:37:17 +01:00
Gilles Peskine
7156d8cda9 Don't hard-code include paths in templates
generate_visualc_files.pl has a list of directories that it pulls
headers from, so it knows what directories to put on the include path.
Make it inject the include path into the output files, rather than
hard-coding the include paths in template files.

A similar change (but with different code) was made in Mbed TLS in
commit b78cf2b261f5549184c7a2e6aea472da879606af
"Adjust visual studio file generation to always use the crypto submodule".

No semantic change: this commit does not change the generated files.
2020-02-26 14:37:07 +01:00
Gilles Peskine
b41f88f47a Simplify the code around the call to gen_main_file
There's no need to keep the directory lists separated until the last
minute.

No semantic change. The generated files change slightly because there
was one directory list where slashes were not changed to backslashes
like in the other five. This does not affect their semantics.
2020-02-26 14:36:50 +01:00
Gilles Peskine
13fac98aca Generalize everest support to generic 3rdparty support
Other third-party components can now be added by just adding lines to
the definitions of @thirdparty_header_dirs and
@thirdparty_source_dirs.

No semantic change. The output does not change at all.
2020-02-19 20:13:29 +01:00
Jaeden Amero
7cb47de12a query_config: Move to programs/test
As the SSL programs, like ssl_client2 and ssl_server2, are dependent on
SSL and therefore about to be removed, the only consumer of query_config
is the query_compile_time_config test. As such, it makes sense to move
query_config to be next to what uses it.
2020-02-11 19:26:27 +01:00
Gilles Peskine
5da20cc569
Merge pull request #3023 from gilles-peskine-arm/config-crypto
Add crypto-only preset configurations
2020-02-05 11:17:56 +01:00
Gilles Peskine
2579675935 Merge remote-tracking branch 'upstream-public/development' into development
Files deleted by us: keep them deleted.

```
git rm $(git status -s | sed -n 's/^DU //p')
```

Individual files with conflicts:

* `README.md`: keep the crypto version.
* `doxygen/input/doc_mainpage.h`: keep the crypto version (with an obsolete Mbed Crypto version number).
* `include/mbedtls/error.h`:
    * `ERROR`: similar additions made through parallel commits, with only whitespace differences. Align with the tls version.
* `library/CMakeLists.txt`: keep the crypto version.
* `library/Makefile`: keep the crypto version.
* `scripts/generate_errors.pl`: keep the crypto version (the relevant changes were made through parallel commits).
* `tests/scripts/check-test-cases.py`:
    * `Results`: keep the crypto version, which has both the new argument to the constructor (added in crypto only) and the class docstring (added through parallel commits).
* `tests/suites/helpers.function`:
    * `ARRAY_LENGTH`, `ASSERT_ALLOC`: additions in the same location. Keep both, in indifferent order.
* `tests/suites/target_test.function`:
    * `receive_uint32`: keep the crypto version which has an additional bug fix. The tls changes made in tls are irrelevant after this bug fix.
* `visualc/VS2010/mbedTLS.vcxproj`: run `scripts/generate_visualc_files.pl`.

Review of non-conflicting changes:

* `all.sh`: 1 change.
    * zlib test components: don't add them.
* `include/CMakeLists.txt`: 1 change.
    * `target_include_directories`: doesn't work as is (different target name). Don't take the change.
* All other non-conflicting changes: take them.
2020-02-03 18:52:36 +01:00
Gilles Peskine
31987c6b88 Add config presets with only crypto
Add config presets with only the crypto parts of the default
configuration, of "full" and of "baremetal".
2020-01-31 15:14:18 +01:00
Gilles Peskine
f65ed6f254 Change key types to a 16-bit encoding
All key types now have an encoding on 32 bits where the bottom 16 bits
are zero. Change to using 16 bits only.

Keep 32 bits for key types in storage, but move the significant
half-word from the top to the bottom.

Likewise, change EC curve and DH group families from 32 bits out of
which the top 8 and bottom 16 bits are zero, to 8 bits only.

Reorder psa_core_key_attributes_t to avoid padding.
2020-01-31 10:24:21 +01:00
Gilles Peskine
afc9db8bb7 Fix version number recognition heuristics
The regexp was wrong, for example it matched "2.20x" but failed to
match "3.1".

Some test cases:
>>> def f(title):
...  version_number = re.search(_version_number_re, title)
...  if version_number:
...      return not re.search(_incomplete_version_number_re,
...                                  version_number.group(0))
...  else:
...   return False
...
>>> [(s, f(s.encode('ascii'))) for s in ['foo', 'foo 3', 'foo 3.', 'foo 3.1', 'foo 3.14', 'foo 3.2.1', 'foo 3.2.1alpha', 'foo 3.1.a', 'foo 3.a', 'foo 3.x.1']]
[('foo', False), ('foo 3', False), ('foo 3.', False), ('foo 3.1', True), ('foo 3.14', True), ('foo 3.2.1', True), ('foo 3.2.1alpha', True), ('foo 3.1.a', False), ('foo 3.a', False), ('foo 3.x.1', False)]
2020-01-30 11:38:01 +01:00
Manuel Pégourié-Gonnard
2b9ebce4e1 Remove deprecated modules from config.py full 2020-01-30 10:16:15 +01:00
Gilles Peskine
a26079613a Create a new level-2 section if needed
Automatically create a level-2 section for unreleased changes if needed.
2020-01-28 19:58:17 +01:00
Gilles Peskine
da14e8225e Remove useless blank line removal in ChangeLog.write
The parsing functions eliminate blank lines, so there shouldn't be any
at this stage.
2020-01-28 19:27:54 +01:00
Gilles Peskine
37d670a1e1 Document read_main_file and simplify the logic a little 2020-01-28 19:27:54 +01:00
Gilles Peskine
974349d40e Style: follow PEP8 2020-01-28 19:00:59 +01:00
Gilles Peskine
d8b6c77388 Use OrderedDict instead of reinventing it 2020-01-28 18:57:47 +01:00
Gilles Peskine
2b242495e1 Add a sanity check on the output
Check that no line from any of the input files was lost.

This is not perfect for several reasons.
It doesn't check that the content goes to the desired location.
It doesn't check that sections are created as necessary.
It doesn't support whitespace normalization that the parsing code does.
But it's a good start.
2020-01-22 15:59:12 +01:00
Gilles Peskine
6e91009cfe Split strings on some very long lines 2020-01-22 15:59:12 +01:00
Gilles Peskine
566407d6f6 Simpler definition of a custom exception class 2020-01-22 15:59:12 +01:00
Gilles Peskine
8c4a84c5de Split read_main_file out of the ChangeLog constructor
Keep the constructor code simple.

No behavior change.
2020-01-22 15:48:58 +01:00
Gilles Peskine
5e39c9e94f Actually remove files
Minor rework of how files are removed. Actually do remove the
files (earlier I accidentally committed a debug version with removal
commented out).
2020-01-22 15:48:45 +01:00
Gilles Peskine
974232f045 Minor documentation improvements 2020-01-22 12:43:29 +01:00
Manuel Pégourié-Gonnard
ad6cb11461 Declare new config.h option MBEDTLS_SHA512_NO_SHA384 2020-01-06 11:40:23 +01:00
Gilles Peskine
180850a229
Merge pull request #291 from gilles-peskine-arm/ctr_drbg-test_aes_128
Test MBEDTLS_CTR_DRBG_USE_128_BIT_KEY
2019-12-20 10:43:44 +01:00
Janos Follath
df587ee6d6 Remove duplicate include statement
Now that the Error module has error codes as well and is processed by
the generate_errors script like any other module, we don't need to
include the header manually.
2019-12-19 10:27:57 +00:00
Jaeden Amero
795c6bab62
Merge pull request #323 from yanesca/iotcrypt-942-initialise-return-values
Initialise return values to an error
2019-12-19 10:24:22 +00:00
Janos Follath
2d20567122 Add two error codes to the Error module
One of the error codes was already reserved, this commit just makes it
explicit. The other one is a new error code for initializing return
values in the library: `MBEDTLS_ERR_ERROR_CORRUPTION_DETECTED` should
not be returned by the library. If it is returned, then it is surely a
bug in the library or somebody is tampering with the device.
2019-12-16 15:15:16 +00:00
Gilles Peskine
49af2d3a4f Support non-ASCII characters in headers
Filter out non-ASCII characters in automatically processed headers.

Do this in a way that minimizes the code change: keep manipulating
strings, but strip off non-ASCII characters when reading lines, which
should only remove characters in comments that we don't parse anyway.
2019-12-11 11:03:07 +01:00
Janos Follath
60f6b64b8f Add two error codes to the Error module
One of the error codes was already reserved, this commit just makes it
explicit. The other one is a new error code for initializing return
values in the library: `MBEDTLS_ERR_ERROR_CORRUPTION_DETECTED` should
not be returned by the library. If it is returned, then it is surely a
bug in the library or somebody is tampering with the device.
2019-12-03 16:01:06 +00:00
Gilles Peskine
987e271b16 If python3 fails, make it clear that this isn't fatal 2019-11-26 13:30:16 +01:00
Gilles Peskine
df78e496b7 Error out rather than call python on the perl script 2019-11-26 13:30:00 +01:00
Gilles Peskine
80a607171a config.pl full: exclude MBEDTLS_ENTROPY_FORCE_SHA256
This is a variant toggle, not an extra feature, so it should be tested
separately.

We test most of the effect of MBEDTLS_ENTROPY_FORCE_SHA256 (namely,
using SHA-256 in the entropy module) when we test the library with the
SHA512 module disabled (which we do at least via depends-hashes.pl).
This commit removes testing of the MBEDTLS_ENTROPY_FORCE_SHA256 option
itself, which should be added separately.
2019-11-21 13:52:09 +01:00
Gilles Peskine
24600e8290 Disable memory_buffer_alloc in the full config
Enabling MBEDTLS_MEMORY_BUFFER_ALLOC_C module together with
MBEDTLS_PLATFORM_MEMORY causes the library to use its own malloc
replacement. This makes memory management analyzers such as ASan
largely ineffective. We now test MBEDTLS_MEMORY_BUFFER_ALLOC_C
separately. Disable it in the "full" config.

This mirrors a change that was made in Mbed TLS on config.pl and had
not been ported to Mbed Crypto yet.

With this commit, config.py is aligned in Mbed Crypto and Mbed TLS.
2019-11-15 11:53:42 +01:00
Gilles Peskine
baf15df251 Compatibility redirect: add copyright notice 2019-11-13 14:33:34 +00:00
Gilles Peskine
518ce0beb3 Compatibility redirect: if python3 is not available, try python 2019-11-13 14:33:34 +00:00
Gilles Peskine
261742bd59 Fix config.py output when a symbol has acquired or lost a value
Normally a valueless symbol remains valueless and a symbol with a
value keeps having one. But just in case a symbol does get changed
from valueless to having a value, make sure there's a space between
the symbol and the value. And if a symbol gets changed from having a
value to valueless, strip trailing whitespace.

Add corresponding tests.

Also fix the case of a valueless symbol added with the set method,
which would have resulted in attempting to use None as a string. This
only happened with the Python API, not with the command line API.
2019-11-13 14:33:34 +00:00
Gilles Peskine
7f04013099 Documentation improvements 2019-11-13 14:33:34 +00:00
Gilles Peskine
2552bc73d4 Fix "#define ... not found" error when using the default file name
Also make that error message end with a newline.
2019-11-13 14:33:34 +00:00
Gilles Peskine
63cdb2855f Fix "--force set" without a value sneaking a None in 2019-11-13 14:33:34 +00:00
Gilles Peskine
a52f97d5a5 Fix --force requiring an argument 2019-11-13 14:33:34 +00:00