The company claims these are the world’s first image sensors with AI processors on board, although numerous startups have been prototyping event-based sensors that minimize data transfer and are well suited to machine vision.
The ability to put AI processing on the image sensor enables high-speed edge processing and provides the opportunity to develop smart cameras and industrial and commercial IoT applications, Sony said.
The sensors are made with a two-die, stacked configuration with pixel chip and logic chip. The signal acquired by the pixel sensor is analysed by the AI processor eliminating the need for high-performance processors or external memory and minimizing data transfer both within the system and also up to the cloud.
The sensor component output can then be metadata belonging to the image. Different AI models and machine learning can be used to customize the image sensor to different applications, lighting conditions and locations.
The iMX500 has 12.3 megapixel back-side illuminated image sensor and a Sony-developed logic chip including memory and image signal processor and DSP for AI signal processing. The ISP is used to select region-of-interest and to format images (YUV/RGB).
These latest sensors can perform AI processing in 3.1 milliseconds (MobileNet v1) on the logic chip. As this is within a single frame period it allows real-time tracking of objects while recording video at 30fps.
Users can write and rewrite their own AI models to the embedded memory allowing the sensor to be used for such applications as people-counting, stock monitoring, heat mapping of people in a store
The iMX500 is the bare die version of the product. The iMX501 is housed in a ceramic package 12.5mm by 15.0mm. The sample price of the bare die version is 10,000 yen (about US$94) and of the packaged version is 20,000 yen (about US$187).
Next: Key specificatons of iMX500.
Key specifications of iMX500: Source Sony Semiconductor Solutions.
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.