In March, Microsoft introduced its Holoportation solution, a Microsoft Research project. Using high-quality 3D capture technology, it can transform teleconferencing and other services. In November, Microsoft revealed a version of Holoportation that is mobile. Around the same time Shahram Izadi demoed the technology at UIST'16, and here is the video (below) of that presentation.
The 26 minute discussion was part of the ACM Symposium on User Interface Software and Technology in Tokyo, Japan. The event run October 16-19 this year. Izadi is one of the Microsoft researchers who fronted the Holoportation project.
He left the company in May, 2016 and is now the co-founder and CTO of perceptiveIO. The new company specializes in real-time computer vision, applied machine learning, novel displays, sensing, and human-computer interaction. He and the company are now partner researchers to Microsoft.
During the discussion at UIST'16, Izadi touches on the challenges in the development stage of Microsoft Holoportation. As a demonstration of the aspirations of Holoportation, he uses several Star Wars movies. In those films, characters can conference visually or talk with other characters, even if they are on the other side of the galaxy.
This is achieved through holograms, which means some sort of holographic technology must be in place for both (all) speakers. Microsoft is pushing development towards this kind of ability. The result is an end-to-end system for relaying augmented and virtual reality telepresence holograms in real environments.
Holoportation not only recreates individual figures, but can render high-quality 3D reconstructions of a whole space in real-time. Izadi explains that this includes people, objects, and even furniture. A set of new depth cameras are needed to achieve this result.
While the Star Wars (to continue the example) movies present no wearable tech, Holoportation will require users to wear a virtual or augmented reality headset, like HoloLens. The video describes how this combines to create real-like experience:
“This allows users wearing virtual or augmented reality displays to see, hear and interact with remote participants in 3D, almost as if they were present in the same physical space. From an audio-visual perspective, communicating and interacting with remote users edges closer to face-to-face communication.”
Lastly, you can check out Microsoft's mobile solution for implementing the technology in the video below: