360 degrees video is not new. In fact, there are dozens of 360 cameras for sale, with the minimalistic Ricoh Theta being one of my favorites. Other people might prefer the Geonaute or the Bubl.
These cameras use several lenses to capture a spherical 360 degrees video, which allows you to “look around” in a scene by clicking + dragging your mouse, by rotating your phone or tablet around, or by viewing the video in a virtual reality (VR) headset like the Oculus Rift or Google Cardboard.
This works great, and companies like Scopic have created some seriously impressive videos using a 360 camera rig with GoPro action cams in it. But while 360 video content can be impressive, it is not realistic enough to create an immersive virtual reality experience due to the lack of depth in the video, a significant fish-eye effect, and a rather low resolution. Let’s quickly see what can be done to improve this.
Adding depth
We can probably all agree, with Gravity being the exception, that 3D cinema has not been as breathtaking as we had hoped. Also, the 3D glasses are sort of a hassle. However, the idea of virtual reality is to create a virtual experience which comes close to a real world experience, and the real world contains depth. Therefore, 3D is crucial to trick your mind into believing you’re actually somewhere else.
Besides, with virtual reality you are already forced to wear a headset anyway, so content producers might as well add in some 3D effects while they’re at it. But how do you create a realistic 3D effect?
There are two major schools of thought:
1) Anaglyph. Two overlapping images with a complementary color, usually red/cyan. You need red/cyan 3D glasses to view the content.
2) Side-by-Side. Two images are shown next to each other. The lenses in the 3D glasses then turn them into a single image with depth in it, which is the way VR headsets do it.
Both of the above mentioned techniques are shot by mimicking the way our eyes work. The distance between the centre of your eyes is called the interpupillary distance (IPD), which is on average about 62mm. Because your eyes are apart, each eye views objects from a slightly different angle, and this is what allows you to see depth.
We can mimic this situation by shooting a scene with two regular cameras placed 62mm apart, where each camera represents one eye, and then displaying these images side-by-side in a virtual reality headset. This technique, used to create an illusion of depth, is called Stereoscopy. Stereoscopic 3D is the closest current technology is able to get to realistic depth effects in video content.
No fish-eye, por favor
A common complaint about 360 video is that the content contains significant distortions. The effect is similar to standing in front of a laughing mirror, where people in the scene get a massive head if they come too close to the camera. Generally speaking, this is not the desired effect when you’re aiming to create realistic content.
The cause of this distortion is the fact that as little as 2 lenses are used, in the case of the Ricoh Theta, to capture a complete video sphere. In other words, each lens needs to capture a 180 degrees Field of View (FOV), and the larger the FOV, the more rounded the lens needs to be, and the more pronounced the distortion gets.
The solution is rather simple though: use more lenses. Our virtual reality camera contains 16 GoPro action cameras, 2 on each side of the rig, which film 2 video spheres: one for the left-, and one for the right-eye. By using more lenses we are able to minimize the so called fish-eye effect and shoot in stereoscopic 3D at the same time.
Up the resolution
Another issue related to using only a couple of lenses is that the resolution of the captured images needs to be spread out over an entire sphere. So even if you shoot in 4K resolution, you will still be left with way less than HD quality once the video is placed on a sphere for viewing.
A benefit of this lower resolution, and therefore smaller file size, is that 360 video can be streamed reasonably well. This is not yet possible for high quality, stereoscopic VR video content. The software to stream VR video is there, see Vahana VR for a good example, but the bandwidth required is not broadly available yet.
Some say that a high resolution is not that important, because the current Oculus Rift DK2 only supports 960 x 1080 pixels per eye. While this is true, you want your content to be future proof, since it won’t be long before we are watching 4K resolution per eye. The reason for this is, again, that the real world is rather high definition, and so to maximize immersion the resolution of VR videos needs to be as crisp as it can get.
Immersive audio
So far we’ve discussed the differences between regular 360 video and VR video, also called cinematic VR. The main difference is that cinematic VR requires a much higher degree of realism in order to create a powerful experience for the viewer. Since VR is essentially about tricking your mind into believing you are part of the scene, the more senses you include, the more “present” you will feel. See the video below for a great, to-the-point explanation.
Virtual reality with purely visual cues stimulates the feeling of presence in your brain only half as much as in the real world. To create a fully immersive experience, additional senses, like balance, realistic sound, and possibly even smell and touch need to be included. This is why we place great importance on adding spatial 3D audio to our video productions, captured using an ambisonic microphone.
Ambisonic microphones do not simply record a stereo track, but instead they capture a so called sound field. Software then turns this ambisonic recording into a binaural sound experience, which dynamically changes the audio depending on where the user is looking, just like in the real world, and allows users to determine the direction and origin of sounds. By adding realistic audio, the feeling of presence gets significantly enhanced.
Final words
As you can see, cinematic VR is not easy. It requires a high degree of realism to actually create the illusion of being somewhere else. Regular 360 degrees video therefore simply does not cut it. Thanks for your time, and if you have questions or remarks after reading this article, feel free to leave a comment below.