Light fields offer more realistic VR experience
Light fields are a set of advanced capture, stitching, and rendering algorithms that describe the amount of light flowing in every direction through every point in space. While they are still an experiment, says Google, they can create still captures that offer an extremely high-quality sense of presence by producing motion parallax and extremely realistic textures and lighting.
“To create the most realistic sense of presence, what we show in VR needs to be as close as possible to what you’d see if you were really there,” says Paul Debevec, Senior Researcher, Google VR. “When you’re actually in a place, the world reacts to you as you move your head around: light bounces off surfaces in different ways and you see things from different perspectives.”
“With light fields, nearby objects seem near to you – as you move your head, they appear to shift a lot,” says Debevec. “Far-away objects shift less and light reflects off objects differently, so you get a strong cue that you’re in a 3D space. When viewed through a VR headset that supports positional tracking, light fields can enable some truly amazing VR experiences based on footage captured in the real world.”
This is possible, says Google, because a light field records all the different rays of light coming into a volume of space. To record them, Google researchers modified a GoPro Odyssey Jump camera, bending it into a vertical arc of 16 cameras mounted on a rotating platform.
The rotating camera platform takes about a minute to swing around and record about a thousand outward-facing viewpoints on a 70-cm sphere, resulting in a two-foot wide diameter volume of light rays. This determines the size of the headspace that users have to lean around in to explore the scenes once they are processed.
To render views for the headset, rays of light are sampled from the camera positions on the surface of the sphere to construct novel views as seen from inside the sphere to match how the user moves their head. They’re aligned and compressed in a custom dataset file that’s read by special rendering software the Google researchers implemented as a plug-in for the Unity game engine.
To try out their light field-recording camera rig, the researchers recorded videos in several selected locations, including the Gamble House in Pasadena, the Mosaic Tile House in Venice, and St. Stephen’s Church in Granada Hills. In addition, they were given access to NASA’s Space Shuttle Discovery at the Smithsonian Institute’s National Air and Space Museum, enabling them to record an astronaut’s view inside the orbiter’s flight deck.
To demonstrate the potential of this technology, Google is releasing “Welcome to Light Fields” – a free app available on Steam VR for HTC Vive, Oculus Rift, and Windows Mixed Reality headsets. The app offers a seven-minute guided tour of the technology and the locations recorded by the Google researchers.
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
