In each of their two principal eyes, jumping spiders sport a multi-tiered retina that simultaneously receives multiple images of a prey with different amounts of defocus, from which they can accurately decode the prey’s distance even with little brain power. Trying to emulate the multi-tiered retina optics, the researchers designed special metalens optics that split the light passing through an aperture, forming two differently defocused images at distinct regions of a single planar photosensor. Using fewer than 700 floating point operations per output pixel, they are then able to interpret the two images and build a depth map to represent object distance.
This depth extraction technique also known as “depth from defocus” is traditionally implemented with large cameras featuring motorized internal components that can capture differently focused images over time. But this cumbersome approach has speed limitations and is compute intensive.