Testing the Software Tools Behind Safety-Critical Embedded Systems
What you'll learn:
- The difference between C and C++ standard libraries.
- Importance of library qualification.
- Testing C and C++ standard libraries with SuperGuard.
For anyone involved in the development of safety-critical products, particularly in the automotive sector, a wide range of safety standards apply. However, the ISO 26262 standard for functional safety stands above the rest.
Meeting the requirements of the standard is different from the usual task of achieving quality assurance. The reason is that the standard focuses on safety hazards stemming from the failure of the product itself or the underlying components. Complicating matters is the fact that identical products can have widely different impacts on safety depending on where they’re used.
When functional safety is on the line, the tools used to create such safety-critical components must be subjected to rigorous qualification. These include the compiler that generates code and the standard library—both core building blocks in modern software development kits. The standard library is the collection of headers and library functions defined by the C and C++ language definitions.
Library qualification is different from compiler qualification because library code inevitably becomes part of the application code that runs on top of the safety-critical component.
Standard Libraries: The Secret to C and C++ Programming
Most developers understand that C++ is based on the foundation of C and that it preserves the language’s predictable and high-speed performance. It’s one of the reasons why C++ has proved to be ideal for embedded applications. However, C++’s view of the standard library differs significantly from that of C.
C and C++ have a standard library that’s defined together with the programming language. The C language can be used efficiently without touching the library, which means developers need only worry about the quality of the compiler that implements the language. Many embedded applications operate in this way. But without the standard library, a C++ developer will struggle when writing code.
The reason is the C++ language provides new abstractions (implemented by the compiler) such as templates, constructors, and destructors, and the library offers features such as arrays, iterators, and sequences. The compiler-implemented abstractions, together with the feature set of the library, mean that switching from C to C++ makes sense. C++ is not just an extension of the C language.
The C++ compiler and its standard library are two sides of the same coin, to an even greater degree than is the case for C. To use it in safety-critical scenarios, both sides must be carefully qualified.
A whole new set of tests, test specifications, and requirements for the C++ standard library must be designed for use in high-quality and safety-critical applications. For instance, SuperGuard is a test package for C and C++ standard libraries designed by Solid Sands. It has full traceability between the requirements derived from the ISO C and C++ language definitions and the individual library tests.
It’s designed to support the qualification of implementations of the C and C++ standard libraries for safety-critical applications, and for third-party or self-developed or self-maintained library implementations.
Qualification of C and C++ Standard Libraries: The Why
The reality is that safety-critical applications are becoming more extensive today because affordable processing power and sensors are more prevalent. The result is that processing of 2D and 3D data (and the subsequent sensor fusion) requires software building blocks that are simply not available in C.
Having these building blocks in C++ raises the level of application development and makes C++ far more productive. For instance, the abstractions make it easier to avoid the common pitfalls of working with C while the library uses the new abstractions in a more straightforward manner.
One fact that must always be remembered is—unlike in C-based projects where it’s possible to skip the standard library and only qualify the compiler—library qualification can’t be avoided with C++. Anyone wanting to use C++ in a safety-critical project must ensure it can be used safely.
More than that, verification of the library must be done as rigorously as any other software in a safety-critical application. That’s why having a requirements-based test suite with full traceability between the C++ library specification and test results is essentially the only way to go.
Depending on the safety level you’re targeting, library qualification also requires 100% modified condition and decision coverage (MC/DC). Code coverage is a secondary concern (after requirements coverage) to show that the test suite is complete. Within SuperGuard, Solid Sands ensures code coverage analysis for the GNU compiler and LLVM C++ standard library implementations. Written in C++, LLVM is designed for compile-time, link-time, runtime, and idle-time optimization.
How to Effectively Test C and C++ Standard Libraries
How does a test regimen such as SuperGuard work? It provides a detailed breakdown of the ISO C and C++ library specifications into the requirements that must be met by implementation of the library.
These requirements are linked to test specifications that describe how a test verifies the requirements. In turn, the test specification is linked to tests in SuperGuard, creating a detailed path from the specification to the tests that’s easy to comprehend and verify. This builds confidence in the compliance of the C and C++ library implementations with their specifications.
SuperGuard is specially designed to achieve high structural code coverage of the target library implementation. The suite incorporates a software tool that reports which requirements are met and not met by verifying C and C++ library implementations in a run of the test suite for a specific use or configuration.
Other features include compliance with the requirements of functional-safety standards and parallel testing, which automatically splits validation runs into a preconfigured number of threads to speed up the validation process. The suite also ensures full control over test sets.
It’s also valuable for engineers to have access to C++ headers, specifically when it comes to the safety-critical automotive sector. In LLVM and GCC open-source compiler infrastructures, C++ headers simplify the process for users of C++ standard libraries having to prove their requirements and test specifications.
As safety-critical applications have become more widespread, it’s spurred on greater demand for C++ standard library qualification. A requirements-based test suite for C and C++ standard libraries is expected to play a vital role in ensuring that designers of safety-critical products can bring their solutions to the market faster and with greater confidence than previously possible.