Microsoft has been building up the capabilities of HoloLens since it launched the developer edition last year. The headset is becoming a platform for media, productivity, and design. However, HoloLens is currently limited as a singular experience. Spectator View helps to change that, and now Microsoft has open sourced the feature.
If augmented and virtual reality are to change how we consume entertainment, companies need to find ways to allow multiple users to enjoy an experience. HoloLens is very expensive, so multiple people buying headset in one household is unlikely. A feature like Spectator View lets other enjoy the user experience without wearing the headset.
Spectator View works through a special camera rig mounted in a room. By combining the camera hardware with Microsoft's software, the rig allows other people to see a video feed of the HoloLens app being used, and the person interacting with it.
There are four steps to the functionality of the feature. The first is the specific application based on shared holographic. A desktop PC with the shared holographic application will composite what the HoloLens app and user into a third-person spectator video.
“Mixed reality capture (MRC) provides a video composite of what the HoloLens user is seeing from a first person point-of-view. Spectator view produces a video from a third-person perspective, allowing the video observer to see the environment with holograms and the user wearing a HoloLens device.”
Microsoft has open sourced the Spectator View and placed the documentation on GitHub. It is worth noting that this is very much a development solution that is at a very early stage. A full implementation for HoloLens still seems a way off, but it is a step in the right direction.
HoloLens Spectator View
Spectator View is not actually a new feature. It has been a part of Microsoft's HoloLens development since the start. Indeed, it has been the underlying technology behind all the company's HoloLens demonstrations.
This is very much a development feature. That's because the hardware setup requirements are long and very expensive. The eventual idea would be for the cost of the setup to become more affordable. In the meantime, the open sourced documentation allows developers to create their own Spectator View rig.
Microsoft explains how developers can leverage Spectator View:
“Photo capture – Using this technology, you can capture high resolution images of your holograms. These images can be used to showcase content at marketing events, send to your potential clients, or even submit your application to the Windows Store. You get to decide which photo camera you would like to use to capture these images, as such you may prefer a quality DSLR camera.
Live demonstrations – Spectator view is a preferred approach for live demonstrations as the camera position remains steady or controlled. Because you can use a high-quality video camera, you can also produce high-quality images meant for a big screen. This is also appropriate for streaming live demos on a screen, possibly to eager participants waiting in line for their turn.
Video capture – Videos are the best story telling mechanism for sharing a holographic app experience with many people. Spectator view lets you choose the camera, lens, and framing that best suits how you want to showcase your app. It puts you in control of the video quality based on the video hardware you have available.”