Model-based testing on production hardware
Time Partition Testing (TPT) is a model-based testing technique, which is based on hybrid, hierarchical, parallel running automatons with continuous behavior. These use a real-time semantics, support continuous signal processes and―with the help of so-called variation points―enable modelling of a number of test cases in a common automaton. Hence, with TPT all test cases for a test object are managed in a common test model.
As a result of this approach, the overview still remains manageable and maintenance-friendly even for complex tests with hundreds of test cases, whereby the tests themselves are formally precise and automatically executable. A consistent solution ‒ from the requirements specification, through test modelling and automated test execution to test evaluation and documentation ‒ is possible by means of coupling with the requirements management tool DOORS (Dynamic Object-Oriented Requirements System) and an integrated automated test evaluation.
The advantage of TPT lies in the clear graphic modelling of signal-oriented behavior for automated executable and evaluable tests. Differences between the individual test cases are modelled by behavior variants on the states and transitions. Therefore, each state and each transition in the automaton cannot only define one, but potentially any number of alternative signal definitions or transition conditions, which in turn can stimulate different specific test procedures.
States and transitions, for which more than one variant was defined, are called variation points. With TPT, the specific test cases are defined by selecting and combining variants for all variation points. Different combinations define different test cases. Figure 1 shows a section of the main window of the TPT software with an example automaton.
With TPT, modelling of the test cases takes place irrespective of the platform during test execution. Therefore, the same tests can be reused in all phases of the development from early Model in the Loop (MiL), through Software in the Loop (SiL) and Processor in the Loop (PiL) to subsequent Hardware in the Loop (HiL) without having to make any alterations. TPT can be deployed for control of the target and for setting/reading values for HiL-tests with various run-time environments, for example, LABCAR from ETAS or dSPACE systems. Another advantage of the platform-independent test cases is that the results can directly be compared.
With support of the Universal Debug Engine (UDE) from PLS, users can now execute tests that were modeled and managed by means of TPT, even in real-time on the actual target hardware, for example, an embedded control device.
The basis for this is the software interface provided by the UDE. It is based on the Component Object Model (COM) from Microsoft and offers almost all functions of the debugger such as FLASH programming, run control, reading and writing of target memory in symbolic and numerical form, trace data acquisition and analysis, and much more (Figure 2).
Via the software interface, the debugger is controllable within the user interface by macros or externally by scripts. All scripting languages that can handle COM components ‒ such as JavaScript, Perl, Python, VB Script or also PowerShell ‒ can be used. Hence, programming of automatable processes therefore does not take place in a proprietary syntax, but in the language chosen by or familiar to the developer and, namely, by making full use of the objects, properties and methods provided by the UDE software application programming interface (API).
Furthermore, with the corresponding external tools, the interface is also useable with different programming languages such as C, C++, C# and other .NET frontends. A dynamic link library (DLL) written in C++ forms the connection between TPT and the Universal Debug Engine (UDE). This means that in every simulation step any debugger functions can be selected and, for example, breakpoints can be set fully automated. At each of these breakpoints, the user can precisely define which variables of the System under Test (SuT) can be read out or filled with predefined data. Furthermore, the execution can be continued at any position by changing the instruction pointer. This approach provides a great advantage over other couplings, which can only change and determine variables via the interface to the SuT. A great advantage is that additionally any memory spaces and registers of the target hardware, individual bit positions and even local variables from functions can also be set and read out. Hence, it is now possible for the first time to not only prove or refute the functional correctness of the SuT, but also at the same time to show why the SuT does not behave as expected. Since in each simulation step several actions of the UDE debugger can be executed on the target hardware, testing functional correctness of software/bypass hooks of the SuT is also possible, fully automated.
Figure 3 shows the configuration of UDE coupling with the TPT technique.
In the upper table, breakpoints on the target hardware are determined. In the table in the middle, actions to be performed by the UDE debugger per simulation step are determined. In the lower table, to be set and/or to be read out variables of the target hardware and of the SuT are determined. The breakpoints can be set by the user either at an absolute hexadecimal address, at the beginning of a function, at the end of a function – defined via the function name with hexadecimal offset or by means of keyword endof(<function-name>) – or at a specific line in a source code file.
A state machine is used for the description of the individual actions that are to be performed by the UDE per simulation step. In this state machine, the breakpoints are the states and state transitions are described by continuing the execution to a certain breakpoint (keyword: run) or by changing the instruction pointer (keyword: setNextStatement). With that, an action which should be executed one time, for example memory initialization, can be separated from the actual test execution because the control steps for the UDE will not processed line by line from the table ‘Program’ in the middle (Figure 3). In the given example the first three lines are executed once after program start and then with the next two lines the execution is continued. The test flow goes from function_start via function_middle to function_end. Through program execution as state machine, the subsequent steps are in the last line, followed by the previous. The test iterates again at function_start from this point.
Conclusion
Testing of highly-complex embedded software increasingly requires integrated approaches at the system level and model-based methods. The role that hardware-related debuggers will play in the future, which since some time now show an emerging paradigm shift, will depend particularly not least on how simply and reliably their functionality can be used by other programs within the tool chain. As clearly shown by the example of close interaction of TPT and UDE, entirely new workflows are conceivable here in the ideal situation. These new workflows open up completely new opportunities for developers, in particular with regard to automated test techniques.
About the authors:
Dirk Tetzlaff (Dr.-Ing.) completed his studies in computer science in 2007 at the Technische Universität Berlin (TU Berlin) in Germany. Following that he was on the scientific staff at the Institute of Programming Embedded Systems under the direction of Professor Dr. Sabine Glesner, where he received his doctorate in 2014. Since then, he works as a software engineer and consultant at PikeTec GmbH.
Heiko Riessland (Dipl.-Ing.) studied information technology at the Technische Universität Dresden (TU Dresden) in Germany. In 1992, he began his career as software developer at PLS. After several years he moved to sales of software development tools for 16-bit and 32-bit microcontrollers. During the past eight years, Heiko is heading product marketing at PLS GmbH.
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
