Imagination designed the PowerVR Series2 ‘Raptor’ imaging pipeline architecture from the ground-up to be optimized for integration into next-generation low power System-on-Chip (SoC) devices for a broad range of imaging and vision applications.
‘Raptor’ is also the first ISP designed to operate as a cooperative part of a heterogeneous computing platform for advanced functions and next-generation applications. The ‘Raptor’ architecture takes ISP to the next level, providing the basis for context-aware applications such as facial and gesture recognition, augmented reality and more.
Highly power-efficient, and with a comprehensive configurable pipeline, ‘Raptor’ is ideal for low-power products such as mobile and wearables, and can seamlessly scale to provide the performance needed for UltraHD video, multi-megapixel photography and higher pixel depth sensors. Because it was built for efficient extensibility, the ‘Raptor’ architecture easily extends to address these high-end products with higher pixel counts without impacting the efficiency of the base architecture.
"‘Raptor’ offers the ultimate in performance within the power envelope of a mobile device, providing a unique solution for next-generation applications," said Martin Ashton, EVP engineering for PowerVR Visual IP at Imagination. "Vision systems will be a major driver for future consumer technologies, and a primary differentiator for products such as smartphones, tablets, IoT and emerging automotive applications. With our PowerVR graphics and video processors and MIPS CPUs, Imagination already powers these types of products. ‘Raptor’ adds a critical component which will allow our licensees to implement entire multimedia systems where the whole is much more than the sum of the parts.”
“With its Raptor ISP, which has features such as simultaneous multi-sensor support, 16-bit capability, and a path to 4K, Imagination is demonstrating that it knows what is needed for next generation imaging. This is how you leap the crowd,” says Jon Peddie, president, Jon Peddie Research.
The ‘Raptor’ architecture provides the features and functionality needed for optimal integration with utilization of modern CMOS camera sensors, including features such as support for up to 16-bit pixel depth to provide appropriate dynamic range for automotive, security and other challenging real-world applications as well as the ability to configure the pixel depth of the pipeline to match the sensor, conserving power compared to DSPs with fixed datapath widths. Multiple sensor support has fast context switching, enabling stereo, multi-cam and vision array support for volumetric imaging, with a single ISP and there is the ability to output image statistics and meta data to an Imagination hardware video encoder for best-in-class encode quality and real-time bitrate control. There is also support for emerging 10-bit sensor outputs, key for 4K/Ultra-HD TV and full support for CMOS sensors from all major vendors
Today’s HD video cameras are expected to be small enough to be mounted on equipment such as helmets and skateboards, and DSLR (Digital Single-Lens Reflex) functions are migrating into compact and mirrorless format cameras. Smartphones and tablets are increasingly required to integrate full HD video and DSC (digital still camera) capabilities, and specialist technologies such as robotic vision for visually-aware apps. With these trends, companies are challenged to meet size, cost and power constraints. By integrating ISP functionality onto the SoC, companies can achieve high performance while saving cost and power.
Imagination takes a whole system approach to design of IP, creating optimizations between and within IP blocks to allow implementation of ultra-efficient subsystems. ‘Raptor’ provides flexible access to both programmable and hard wired system resources, enabling optimizations such as ‘zero memory’ output to video encode and vision accelerators for low-power, low-latency applications; fast, low power vision systems; and flexible sensor array support. There is optimized software to implement zero copy computational photography via standards extensions as well as a re-entrant streaming port to allow image data to be streamed to custom hardware vision processors and then re-inserted to the ISP pipe.
Close connections between ‘Raptor’ IP cores and other SoC resources can break down the barriers to enabling full support of Ultra-HD video and high pixel count photography in mobile devices and tablets. Close coupling to Imagination’s PowerVR VXE core reduces memory traffic and enables better allocation of bitrate to produce superior image quality for a given bitrate.
Small-format sensor limitations today mean that high pixel count photography requires a significant amount of additional processing. By enabling hardware-based computational photography on a core such as the PowerVR Series6 GPU, the ‘Raptor’ architecture can overcome the 13Mpixel barrier for mobile devices, providing better non-bayer pattern de-mosaicing, noise reduction and more.
Imagination is providing comprehensive software for ISP, lens and sensor tuning to enable customers to support a wide variety of industry-leading sensors and lenses such as those from Aptina, Omnivision, Sony and ST. Also included are software drivers to support Linux, Android and other operating systems including real-time operating systems.
In developing ‘Raptor’, Imagination used its deep and proven vision technology experience, gained through the acquisition of Californian startup Nethra Imaging’s ISP technology in February 2012. Imagination’s vision technologies have shipped in millions of products, including sports cameras and high-end cinema cameras, over multiple generations. Imagination’s unique expertise in tuning image pipelines for specific sensor and lens combinations is vital to enabling the next generation of products.
As part of its portfolio of market-leading PowerVR video and vision IP, Imagination will offer a range of synthesizable ISP intellectual property (IP) cores based on ‘Raptor.’ Targeting specific market segments, the new ISP cores will begin rolling out in the first quarter of 2014, and are available for licensing to lead partners today.