Jasmine Williams, QA Systems, technical content specialist
The use of fault injection as a robustness testing technique is mandated for safety critical avionics software by the DO 178B/C safety standard ‘Software Considerations in Airborne Systems & Equipment Certification’. The technique is already used manually in airborne systems, including those such as nose wheel steering, FADEC, surface controls, radar, navigation, communications, power control, landing gear, hydraulics, FBW controls and life support systems.
Fault injection is a powerful technique for robustness testing fault handling mechanisms. It is often used to achieve 100% structural coverage, because lines of code can be dynamically tested that would not otherwise be accessible without breaking connected hardware or exhaustive black-box testing.
Custom code injection
The 8.0 release of QA System’s Cantata test tool provides Custom Code Injection to automate this technique and to ensure that deactivated code is not accidentally enabled in production.
Custom Code Injection automates fault injection by allowing user code to be inserted at any point in the source code without the need to modify the deliverable application. This makes it easy to test conditions such as interrupt handlers and breaking out of infinite loops.
Cantata function call control methods such as stubbing or wrapping can be used to set or change parameters, variables, or even registers, at call boundaries. This can force code down hard to reach paths for testing functionality such as error handling. Custom Code Injection takes this one step further by allowing these to be set at any point in the source code during execution of a test
Automating fault injection avoids problems often encountered when manually modifying code for testing. Traditionally, faults are injected using conditional compilation to add code that is only compiled during testing. Unfortunately, manually adding code in this way introduces the risk that mistakes are made, or that abnormal system conditions can cause unintended execution of the code. These types of error can result in potentially catastrophic behavior in the final code.
Avoid deactivated code
DO-178B/C classifies this conditionally compiled code added as ‘deactivated code’ because it is present in the source code but is not intended to be executed in the final product. The standard specifies that a combination of analysis and testing should ensure that deactivated code is not inadvertently executed.
DO-178B/C also requires these processes to be documented. Automating code injection avoids the need to use and justify deactivated code for fault injection entirely. Where deactivated test code is left in the production code, this increases code complexity making it harder to verify the design and ensure safety objectives are achieved. As code complexity increases, maintenance and requirements traceability also become more complex, adding time and cost as the project progresses.
A practical example of automating error injection in avionics is testing a nose wheel steering control system. A fault is injected using custom code injection to simulate the failure of one of the control lines between the nose wheel control unit and the landing gear components. This should cause the system to inform the pilot of the failure and produce an ACARS alert.
Where the control line failure condition affects a read of data within a function, rather than at a function call boundary, this data would have to be set by manually modifying the function to inject the control line failure.
However, using a controlled process of code instrumentation during testing, data can be modified, or interrupts fired before or after any statement within a function. Custom code snippets for injection in specific tests are stored in a file separate to both the code under test, and the test driver script. This eliminates the risks inherent in test code being accidentally or deliberately left in the final version. It also means that the same code can be reused to inject faults for multiple tests at the click of a button.
