Configurable image signal processor core for IoT smart camera chips

Configurable image signal processor core for IoT smart camera chips

New Products |
By Nick Flaherty

ARM has developed a configurable image signal processor core for embedded and surveillance chips.

The Mali-C55 is the company’s smallest and most configurable ISP core and has been licensed to Renesas Electronics.

The Mali-C55 core improves the image quality under a wide range of different lighting and weather conditions, and is designed to enable maximum performance and capability in area and power constrained applications, making it ideal for smart camera and edge AI vision use cases. It is also aimed at augmented reality (AR/VR) applications and smart displays.

The core supports up to 8 separate inputs and image resolutions up to 8K and a maximum image size up to 48 megapixels (MP), the Mali-C55 offers the most efficient combination of image quality, throughput, power consumption and silicon area.

The core builds on the previous Mali-C52 ISP, with improved tone mapping and spatial noise reduction, enhanced support for high dynamic range (HDR) sensors and seamless integration with machine learning accelerators to take advantage of neural networks for various de-noising techniques. By combining multiple Mali-C55 ISPs, larger image sizes can be achieved for applications that require greater than 48 MP capabilities, such as video conferencing.

For embedded and IoT vision applications, silicon footprint and cost are important factors, the Mali-C55 can be half of the silicon area size of previous generations and significantly lowers power consumption for extended battery life.

The core integrates with ARM Cortex-A or Cortex-M cores and machine learning accelerators for AI applications via AXI and AHB interfaces. The Mali-C55 provides a second output pipe that can output downscaled images suitable for input to any machine learning accelerator.

The added Iridix local tone mapping applies intensity transformations to images to achieve better visualization by using information gathered from local regions within images. Iridix defines these local regions in an image as grids with equal sizes. It extracts statistics from each grid to apply the collected statistics to the corresponding local regions in the image. Compared to Mali-C52, Mali-C55 improves the Iridix local tone mapping algorithm by smoothing each local tone curve therefore enabling a more natural fall-off around bright light sources.

Temper is a temporal noise reduction algorithm that improves the quality of images in low light conditions by combining consecutive frames. Mali-C55 not only improves the image quality with updated noise reduction algorithms but achieves this with up to 50% reduced memory bandwidth compared to Mali-C52.

Sinter 2.6 is an improved spatial noise reduction technique that improves the detail and noise balance in colour channels. Compared to Mali-C52, Mali-C55’s Sinter achieves better balance of detail by using specific registers for each colour channel.

The Temper and Sinter functional blocks were designed to work together for a significantly better image quality by sharing information between the modules to apply stronger noise reduction in various regions. The Temper and Sinter block order is switched in the pipeline compared with previous ISP designs. This way the input motion mask from Temper improves the overall motion-adaptive noise-reduction performance, while providing per-plane noise profiling.

For computer vision applications where high throughput and low latency is required, the Mali-C55 ISP can be configured by either enabling or disabling these features.

A complete software package is available for Mali-C55 licensees for controlling the ISP, Sensor Auto White Balance, Auto-Focus, and Auto Exposure as well as Bare-metal support and Linux (Video4Linux framework – V4L2).

ISP users also need the capability to tune both objectively and subjectively, and ARM provides a full set of tuning and calibration tools.

ARM also offers a bit-exact simulation model along with a reference platform, which enables the pre-built and pre-tuned evaluation of Mali-C55 image quality. ARM plans to include Mali-C55 in upcoming Total Solutions for IoT,  starting with a full reference design for vision systems. This will come with a pre-validated solution that will support a specific sensor and dual output mode which seamlessly connect to a machine learning accelerator that performs various functions.

Related articles

Other articles on eeNews Europe 

If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :    eeNews on Google News


Linked Articles