Finnish startup beats human eye resolution for AR/VR
The company has built-up an augmented reality headset prototype codenamed “20|20” (as for perfect eye-sight) which it claims delivers an astounding 70 million pixels of effective resolution over a 100º field of view. This is to be compared with today’s Oculus Rift clocking at 1.2Mpixels over the same field of view. Coming out of stealth last June, Varjo (meaning “shadow” in Finnish) issued a somewhat enigmatic press release mentioning patented technology that replicates how the human eye naturally works by creating a super-high-resolution image matching the users’ gaze direction.
Contacting Varjo’s founder and CEO Urho Konttori to learn more about the company and its technical prowess, eeNews Europe asked if this vast improvement in resolution (beyond the reach of today’s state-of-the-art displays) relied on some form of foveated rendering.
Implemented in virtual reality and gaming to lower the computational workload, foveated rendering relies on an eye tracker to only display detailed graphics where the user gazes, while graphics are displayed in a lower resolution outside the zone gazed by the fovea. Varjo does that in a different way. It merges the images of two full HD micro-displays through a specially designed optical combiner. One image serves as the background image of the full scene displayed across the full field of view (which would be what today’s VR headsets merely do), while the other display only delivers a narrow field of view of the full scene, but at full resolution, right where the user gazes. Adding up these narrow high resolution spots across the full 100º field of view gets you to Varjo’s claim of a 70-fold improvement in effective viewing resolution.
“At any given time, you have the high resolution where you are looking, but we don’t have to render the full scene at 70M pixels, so we can run the graphics on a laptop and we are not limited by data bandwidth either”, explained Konttori.
So the 20|20 must be able to rotate the optical combiner synchronously with the user’s gaze so as to always deliver crisp details where they are needed the most. The CEO wouldn’t comment on whether Varjo would use a MEMS- or piezoelectric-based actuator for this, “but for what we want to achieve, it should be silent and faster than the human eye”, he noted, adding that Varjo had developed both its own hardware and algorithms for the custom optical eye tracker it uses so as to be in full control over its destiny.
“We had envisaged to work with Eye Tribe but it was recently acquired by Facebook, then our next potential partner SMI got acquired by Apple last month, so we’re glad in the end that we did it all ourselves” Konttori said (for its part Google has acquired Eye Tribe’s competitor Eyefluence).
“Delays and latency should not be worse than high-end VR headsets. The whole latency chain is tricky” admits the CEO, “but during an eye saccade which takes about 100ms, when the eye gazes from one location to the next, the eye is no longer sensitive to contrast and it takes another 50ms to regain contrast sensitivity, so we really have a 150ms time budget and we know we can do this in 100ms”.
Talking about how narrow the foveated spot should be, “we feel that we get a sufficient resolution by focusing on a spot not larger than 20º horizontally and less than 30º vertically”, said Konttori.
versus today’s Oculus Rift (bottom).
The interesting idea behind 20|20 is that it will naturally increase a system’s resolution as display technologies improve too. In an introduction to Varjo, Konttori told eeNews Europe that early on after the creation of the company, they had decided to focus on professional use cases. “Our system would be prohibitively expensive for consumers, probably several thousands of euros, and we wouldn’t want to cut corners on the bill of materials and downgrade the user experience, the wearer should not be able to tell apart what’s real and what’s virtual”.
“If you consider today’s AR/VR headsets, they only deliver about one percent of the human eye resolution. With that level of eye-sight, you’d be legally blind and certainly not allowed to drive a car. Imagine living your life with a pixelated display”, said Konttori.
“At the early stage of the company, we realized that if we didn’t innovate, we wouldn’t achieve a good enough resolution. Often in see-through applications, the field of view is narrow, or the view is clipped as you move around.
Over a Skype video call, the CEO shared some impressive video-see-through rendering examples whereby the objects being “floated” in augmented reality over a live video feed seamlessly reacted under the ambient light conditions, as if the real world light sources were directly fed into some sort of ray-tracing algorithms.
“We use two cameras for the see-through video feed as well as a depth camera” explained Konttori, then we use our own rendering algorithms to match the lighting of the objects to that of the real world context”. One example showed a translucent stained glass window letting the background shine through as well as occasionally reflecting the luminaires’ light. Another impressive example was a designer’s lamp shade (with a lit light bulb), whose shadows and light beam were effectively projected into real office-space and across walls, in real-time.
The company is hoping to secure its next round of investment soon, it is working with partners to finalize its prototype and help companies develop their software, preparing a development kit for different rendering engines and for third-party developers. It plans to bring its first developers’ hardware platform to market by the end of 2017, with a finalized product sometime next year.
The CEO is adamant Varjo will market its own branded headsets rather than license its IP. “We are too eager on quality that we wouldn’t trust other companies to do it better than us, and we’re not scared of manufacturing” Konttori said.
Talking about other solutions on the market, such as Daqri’s smart helmet and Microsoft’s Hololens both aimed at the professional market, Konttori doesn’t see them as competitors.
With the very high resolution it delivers, the startup is focused on creativity and realism, chasing industrial designers, architects, but also real-estate agents or even travel agents, and possibly also amusement parks, all professional use cases that could support the price premium of Varjo’s solution.
In the upcoming years, the company will work on improving dynamic range, field of view and lowering the price point. Created by a team of optical scientists, creatives and developers with former top positions at companies like Microsoft, Nokia, Intel, Nvidia and Rovio, the 19-staff company is hiring, “one person per week on average, we only want to get the best people” told us the CEO.
So, is being acquired by Facebook or Google an option?
“We plan to stay an independent company and our exit strategy will be an IPO, but that would be at least in a couple of years” concluded Konttori.
Varjo Technologies – www.varjo.com
Related articles:
A hard hat for ar industrialization
Augmented, virtual, annotated, mixed… a “$37 billion reality” by 2027
Microsoft shines light on Hololens
Flooding young minds through digital immersion
$793.5 million of C-level funding confirmed for Magic Leap
Finnish startup develops 1mm thin optics for augmented reality
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
