Hybrid image sensor enables in-vehicle interior monitoring
STMicroelectronics has developed a hybrid image sensor operating on the rolling shutter and global shutter principle. The potential applications go beyond elementary driver monitoring and enable additional safety and comfort applications.
With the installation of driver-monitoring systems (DMS) becoming increasingly mandatory in key sales markets for motor vehicles, STMicroelectronics aims to provide automotive manufacturers with the necessary technologies. By recording the driving ability of the person behind the wheel, DMS solutions promise greater safety on the road. The next-generation dual image sensor now being unveiled monitors all of a vehicle’s occupants beyond the driver. New applications enabled by ST’s new image sensor include checking that seat belts are fastened, vital sign monitoring, detection of left-behind children, gesture recognition, and high-quality video and image recording.
According to STMicroelectronics, the strain gage market is currently experiencing growth rates of more than 10% annually. “Our new image sensor is expected to further fuel this growth by enabling automakers to realize new functionality with full, all-person interior monitoring and provide even greater value to vehicle occupants,” commented Eric Aussedat, executive vice president, imaging sub-group general manager at STMicroelectronics.
The VD/VB1940 sensor combines the sensitivity and high resolution of infrared imaging with high dynamic range (HDR) color imaging in a single device. It can capture images in either rolling shutter or global shutter mode and, with its 5.1-megapixel resolution, is capable of capturing the high-quality near-infrared (NIR) images of conventional DMS sensors as well as the color HDR images required for an occupant monitoring system (OMS). DMS applications use NIR images to detect the head and eye movements of the person behind the wheel, regardless of the prevailing lighting conditions.
The new image sensor is already being sampled, both as a wafer (VDB1940) and with a BGA package (VB1940). Mass production ramp-up is planned to meet demand for the 2024 model year vehicles currently under development.
The image sensor is based on ST’s second-generation 3D stacked BSI (back-side illuminated) wafer technology, which offers maximum optical area and chip-integrated processing relative to die dimensions. This allows the sensor to process sophisticated algorithms internally for optimal performance in generating color and NIR images, reducing power consumption and offloading the external coprocessor.
Algorithms processed on-chip include Bayer conversion and HDR merging for optimal image quality and frame rate. The on-chip Bayer processing makes it possible to convert the 4×4 structure of the RGB and NIR images into the RGGB format, which is compatible with a wide range of SoCs. Apart from that, local processing also handles independent pixel exposure optimization for color and NIR images to achieve the best possible image quality in both modes. Last but not least, intelligent upscaling is enabled to achieve maximum resolution of the NIR images by capturing additional NIR information from the RGB pixels.
The embedded processor is additionally used for improved cyber security to ensure data protection in connected vehicles. This involves mutual authentication, coupling of the camera and ECU, and authentication of the video stream.
The sensor captures NIR images in global shutter mode. This allows synchronization with the infrared LED to capture fast motion without blurring. With additional support for rolling shutter mode, in which pixels are read line-by-line, the VD/VB1940 delivers optimal color image performance. With chip-integrated HDR merging in rolling shutter mode, the VD/VB1940 produces full-resolution color images with a dynamic range of 100 dB.
The AEC-Q100 qualified VD/VB1940 is additionally ISO-26262 compliant, which facilitates its use in functional safety systems up to ASIL-B.
More information here
