How-to: Spectator View, a new tool to help others see what you see in HoloLens
By Brandon Bray, Microsoft Principal Program Manager
February 14, 2017
Hello everyone! I wanted to take a quick moment to announce an exciting new tool we have developed to help bring to life the apps and experiences you have created for Microsoft HoloLens.
Since our launch last year, one of the pieces of feedback we have heard from many folks is, “How can I create a way for people to see what someone sees when they are wearing a HoloLens from a third-person perspective? I want people to not only see what I see when wearing a HoloLens, I want people to feel like they are part of the experience!”
So, what is a spectator view camera?
When we created HoloLens, we knew that we needed to create a way to connect the experience of someone wearing a HoloLens with that of an audience (whether that audience is 1 person or 100 people). Since a HoloLens can only be worn by one person, how do you open the view so everyone can see what that person sees? To solve this problem, we developed Mixed Reality Capture (MRC) to aid in the visualization of holograms that the wearer is seeing from a first-person perspective.
For many reasons MRC is a great way to let others see what you see when you are wearing a HoloLens. It is built right into the device so it’s always there and ready to use. It’s a very simple way to capture what is seen from the device so that your audience can follow along when you are giving a demo or enjoying a HoloLens app you want others to see.
Though MRC is easy to use, there are some constraints. Because MRC is built directly into the device it is subject to what’s available in the device, for example the specs of the photo/video camera, available processing resources to do capture, compositing, and more. It’s also a camera attached to the wearer’s head so it’s subject to the normally imperceptible head movements that can sometimes make the resulting video not as stable as some situations might require. If you have used MRC before, you may notice that sometimes even the slightest head movement by the wearer can make it hard for the audience to follow along.
We have developed the spectator view camera to specifically address those constraints.
A spectator view camera will allow your audience to do more than just see what you see when wearing a HoloLens. Yes, it allows others, who aren’t wearing HoloLens, to see the holograms you would see if you were wearing the device, but it also allows you to see what the people wearing HoloLens are doing and how they are interacting with their mixed reality experience. Additionally, because a spectator view camera is not captured with a camera attached to a human’s head, it won’t be subject to the normally imperceptible head movements so the resulting video footage will be much more stable and easier for the audience to follow along.
How would I use a spectator view camera?
Instead of me explaining it, here is a peek into what the world will look like once your spectator view camera setup is complete.
As you’ll see in the video, a spectator view camera can be used in three specific scenarios:
Great, so how do I get my hands on this?
To create your own spectator view camera, which is based off of the same technology we use to bring HoloLens demos to life for our audiences when we do presentations and keynotes, head on over to GitHub and check out the documentation needed to help you get started. But before you do that, here is some important background to understand.
As you can see from the video, you will need some development experience to build your own spectator view camera, and with all the parts in hand, you’d be able to erect one in less than a day. We designed the above video to help walk you through the steps to bring your spectator view camera to life. Think of video as a companion piece to the full documentation and GitHub step-by-step guidance we have provided. At a high level, here’s what you’ll have to do:
We recently shared the spectator view camera technology with a few of our partners and it has been great to hear from them first-hand how spectator view is opening new ways to show off their innovations. Here is what a couple of them had to say.
On behalf of everyone on our team we are thrilled to bring spectator view to you so that you can bring what YOU have created to more people. We can’t wait to see what you create with this new perspective!