Researchers bridge the gap between VR and AR interoperability for different users
This cross-world interaction would be particularly useful wherever collaboration is needed between the AR user (say a technician in the field) and an expert offering its guidance remotely from a VR scene recreating the local environment seen by the AR user.
In order to transfer a scene from the real world into virtual reality and vice-versa, the researchers used eight cameras (four stereoscopic pairs each) recording the scene from all sides and producing depth maps at a frame rate up to 30Hz. The AR user’s gestures and dynamic movements are also detected and the data is fused by algorithms, coded and transmitted in real time with the associated 3D textures to the VR station. At the same time, in the virtual scenario created from the 3D data, another 3D camera records the VR user’s position and gestures relative to the VR scene, so it can be fed back from the VR scene into the AR scene without disruptive controllers or markers.
The person in the VR world is represented there as a movable full-body avatar which the local AR user can interact with.
“Only the combination of the two technologies enables a unique solution for new mixed reality interaction and collaboration scenarios,” says Paul Chojecki, Project Manager at the Fraunhofer HHI.
The feedback from the virtual world is represented in the real scene by means of a projection. For this projected augmentation, the researchers used special image processing algorithms to ensure that the cues and controls are displayed with visual accuracy, even if the surfaces being projected upon are moving or tilting.
The types of applications for what Fraunhofer HHI describes as X-Reality include not only remote assistance, but also human-robot interaction, telecommunications, telepresence and interactive gaming. For example two spatially separated individuals could play a board game with each other as if seating around the same table.
Fraunhofer HHI – www.hhi.fraunhofer.de
Related articles:
Modular motion tracking glove has built-in haptics for ar/vr immersion
Feeling virtual objects at your fingertips
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
