The limitations of virtual reality may be slowly crumbling thanks to a demo by Microsoft Research that shows a user creating something in Minecraft with realistic interactions.
Microsoft is focused on augmented reality with its HoloLens headset, but the company is not ignoring the virtual reality boom either. VR is seen as the next frontier in entertainment and for other application, its limits are arguably without boundary and the technology could change the way we live our lives.
One of the biggest issues for virtual reality is that it immerses users in a completely fabricated environment, which is cool of course, but real interactions are impossible. Microsoft Research is on the way to solving that problem and used Minecraft to show off its progress so far, showing that VR could be more immersive and allow true interaction.
As is often the case, the solution is elegantly simple in tech terms. There were no new inventions, simple Microsoft Researchers using an Oculus Rift VR headset and one of its own Xbox Kinect v2 units. The Rift on the head and the Kinect attached to a ceiling tracked the hands and body of the wearer and the result was approaching real world interaction.
“Manipulating a virtual object with appropriate passive haptic cues provides a satisfying sense of presence in virtual reality. However, scaling such experiences to support multiple virtual objects is a challenge as each one needs to be accompanied with a precisely-located haptic proxy object.
We propose a solution that overcomes this limitation by hacking human perception. We have created a framework for repurposing passive haptics, called haptic retargeting, that leverages the dominance of vision when our senses conflict. With haptic retargeting, a single physical prop can provide passive haptics for multiple virtual objects.
We introduce three approaches for dynamically aligning physical and virtual objects: world manipulation, body manipulation and a hybrid technique which combines both world and body manipulation.
Our study results indicate that all our haptic retargeting techniques improve the sense of presence when compared to typical wand-based 3D control of virtual objects.”
As you can see in the demo video, the user was able to interact in a meaningful way with Minecraft and build a castle using blocks he could interact with and which felt real. Of course, the real feeling was simply one block on a table, but the VR experience “tricked” the user into thinking he was interacting with multiple blocks and building with them.
Of course, and it really is worth noting, this is just a very first step towards making virtual reality a little bit more realistic, and we could even argue that Microsoft's HoloLens already accomplishes realism in an invented reality more robustly. However, Microsoft Research is showing that virtual reality could be a lot more useful in the future if we can at least believe we are truly interacting with the environment.