Functional safety considerations for ADAS designs using FPGAs
ADAS need to meet specific functional safety requirements. In 2011, the ISO26262 standard for passenger vehicles up to 3.5 tons was introduced to minimize the risk of a malfunctioning system to create a dangerous situation. This standard tries to address the reduction of systematic faults by implementing a rigorous design process as well as detection of random hardware faults during the application execution.
The developer of an application defines specific safety goals and assigns a specific Automotive Safety Integrity Level (ASIL) to each of the goals. The highest ASIL for an application usually defines the requirements to which the development and operation up to end of life of each component has to adhere. Figure 1 shows the current range of ASILs, seen from customer requirements, to which ADAS have to comply.
Figure 1. ADAS ASIL Market Requirements
ASIL-B is the lowest level in the market, but some applications require up to ASIL-D for certain functionality. With an increased ASIL come more stringent requirements. Also generalizing an ASIL on the component or even item (system) level may introduce unnecessary complexities in a specific implementation and can have impact on the cost and schedule of the development. Analyzing the system concept and the derived safety concept and requirements may make it possible to split the application into several different steps with varying ASILs and thus make it easier and more efficient to implement.
For example, a front camera application uses a single image sensor often found in ADAS. Figure 2 shows a high-level block diagram of the system.
Figure 2. High-Level Mono Front Camera ADAS
An image sensor is connected to the image processor, which, for example could be the Altera Cyclone V SoC. The signal processing chain and data flow is split into four parts. First, perform low-level processing on the pixel level by transforming the image into a more useful representation. Then perform mid-level processing on a line or block of the image, which will extract features like edges by using appropriate algorithms. Next is to perform high-level processing where data is extracted on a frame basis to detect and classify objects. The system then needs to track the objects, and if action needs to be taken, communicate with, for example, the braking and/or steering electronic control unit (ECU).
The low-level and mid-level processing can be implemented very efficiently on the FPGA, but users may also implement some of the mid-level processing on a CPU like the Cortex-A9 processor used in the hard processor system (HPS) of the Cyclone V SoC. The high-level processing, which is predominantly control code, could be mapped to one or both of the Cortex-A9 in the HPS. The last step in the processing chain, where the object tracking and decision making is done, could be implemented on an external microcontroller.
Throughout the processing, the input data set gets reduced by each step to more meaningful data, and the safety criticality increases with the reduction in data. Therefore, the low-level implementation could be classified to be either quality managed (QM) or a low ASIL (for example, ASIL-A). The reason for this is that faults introduced during a single pixel can have low or negligible impact on the performance of the subsequent algorithms. The mid-level processing is assumed in this example to comply with either ASIL-A or ASIL-B, and the high-level processing, where objects are identified and classified, could be seen as having to comply with ASIL-B. After the objects are classified, target lists are generated and provided to the microcontroller, which does the tracking of objects and decision-making. As this is the most critical portion of the signal chain, we assume it has to comply with ASIL-D as it can have direct impact on the behavior of the car.
In this kind of application, it is beneficial to do an even more thorough analysis of the data flow than done above, as the definition of the safety criticality for each stage can have a direct impact on the performance of the whole system. Putting too stringent safety requirements on the earlier parts of the computation stages could introduce challenges to meet the system performance goals, with very little impact on the overall safety of the system. There may, however, be faults in the lower stages of the processing chain that could have a high impact on the functionality or safety of the system. For example, a permanent fault in the low-level processing function may lead to a permanent corruption of data for the higher-level stage, but this can be easily detected with plausibility checks that have a relatively small impact on the performance of the system.
Figure 3. Mono front camera system example
Figure 3 shows a high-level block diagram of a mono front camera system example. An external power management circuit supplies power to the Cyclone V SoC. Separate voltage monitoring will generate a reset in case the supply voltages are not in the normal operating range. External non-volatile memory could be connected to the quad serial peripheral interface (quad SPI) module to be used during the booting process of the system to load the application and configure the FPGA. We use DDR memory for the application code execution and storage of data and image frames. The external microcontroller connected via an SPI provides the object tracking, final decision-making, and communication to the rest of the vehicle infrastructure via the CAN interface.
Figure 4. Cyclone V SoC view of modules
The image processor modules involved in our application are shown in Figure 4. Data from the image sensor is received by the video port and passed on to the image-preprocessing block. This block represents the low-level image processing stage. Once the data passed through the image-preprocessing block, it is written to the DDR memory via the FPGA-to-HPS (F2H) bridge in this example, but it could also be passed to the next stage for a more efficient implementation. The second stage, which is the mid-level processing, is performed by the miscellaneous image-processing block. It retrieves the data previously stored in the DDR memory via the HPS-to-FPGA (H2F) bridge, processes it, and writes it to the DDR memory again. The high-level processing stage is performed by the HPS in this example.
Now we will look at some of the diagnostics that could be used to detect faults in the different areas of the design. Some of the discussed diagnostics may be able to detect permanent faults while others only transient faults, or both. A transient fault is a fault that appears and subsequently disappears again. For the analysis, we need to consider faults in used memory for a specific function and also potential faults in the logic involved in performing the function.
The image sensor needs to be configured before it can be used by the application, and the configuration is constantly modified during the application execution to adapt it to different light conditions. As the image sensor is very critical for the application operation, it is advisable to check its configuration at least once during the Fault Tolerant Time Interval (FTTI). This will not cover all of the potential faults in the sensor, but will take care of the configuration register set. See Table 1.
Some of the sensors used in automotive also allow you to transmit certain configuration register data in auxiliary lines of each image frame. With this feature, you could check the settings of the sensor for each frame without having to read the registers via the I2C interface. This checking could even be implemented in the FPGA while streaming the frame data without overhead on the CPU.
Table 1. Summary of image sensor configuration diagnostics
With low-level image processing it is very unlikely that a change in a single pixel will significantly influence the behavior of the application, so such faults can in many cases be neglected. However, faults that could lead to missing or entirely corrupted frames should be examined.
Most image sensors include a feature to transmit a defined test frame instead of the normal image data. As the input data is defined, the output data of the image preprocessing block is also defined. This is followed by a subsequent test. For example, a cyclic redundancy check (CRC) of the output data can find any permanent faults in the system. This test could cover permanent faults in the complete datapath.
Data changes during transmission from one block to the next in the FPGA should be detectable as well. The test pattern or test frame methodology mentioned above covers most of the permanent faults, but they will not detect transient faults. These faults could be detected by diverse transmission or information redundancy.
The mid-level image processing implements edge or corner detection algorithms and could also apply feature extraction algorithms. Therefore, the generated data set is reduced by looking at only the interesting features of an image. With the reduction in data comes an increase in risk that a missed feature due to a fault could lead to a missed object at a higher processing step.
The high-level image processing stage includes the object detection and classification of objects. Speaking in software terms, this stage involves mainly control code so it is well suited to run on a CPU. The HPS incorporates several hardware features (e.g. ECC, MMU, watchdogs) that provide diagnostic capabilities of faults in the HPS.
Another important aspect of functional safety is to ensure systematic fault reduction. This is accomplished by using robust development processes and tools. The ISO26262 standard specifies in detail the requirements for the management of functional safety, such as the use of confirmation measures for the different activities during the safety lifecycle and supporting processes, like configuration and change management. Used tools need to be analyzed if they can contribute to potential failures of the application, and measures need to be put in place to mitigate the risk of this happening.
ADAS are the next wave of innovations to make driving on our more and more congested roads safer. These systems have performance requirements that challenge existing and future standard commercial-of-the-shelf (COTS) products and where the programmability of FPGAs can provide a tremendous advantage. Implementing application-specific diagnostics can increase the diagnostic coverage of the system. Many COTS products have not been designed with functional safety in mind, and using a platform, development environment, and partner who has the right expertise in functional safety can benefit the overall implementation of the system.
About the author:
Frank Noha is Safety Specialist, System Solutions Engineering, Industrial and Automotive Business Unit, Altera Corporation.
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
