The recent Consumer Electronics Show (CES) in Las Vegas made it clear: virtual reality is taking on serious forms! All the big names in tech are working on their own devices, including Apple, Sony, Samsung, and Google. Even Microsoft decided to join the club. 2015 is going to be an exciting year!
So far there are two major types of VR companies: those creating hardware, and those creating games and animated content. However, one underserved, yet fascinating niche with massive potential is cinematic VR; high quality, immersive video experiences which place the viewer in the middle of the action.
At this moment, the Palo Alto based Jaunt, which primarily targets the Hollywood market, is the undisputed leader when it comes to cinematic VR. We at Purple Pill learned a lot from Jaunt, and have developed our own VR camera system and workflow. While having a clear example to work towards helped us avoid many common mistakes, we still encountered plenty facepalm moments. That’s why we decided to share 10 easy-to-avoid mistakes which will save you days of frustration.
1: Label EVERYTHING!!
Like Jaunt, our camera contains a whole bunch of GoPros. 16 to be exact. When testing our very first prototype, we gathered some GoPros from friends and rented the rest. Keeping track of 16 camera’s, 16 SD cards, 32 batteries, and a big pile of camera accessories from several sources was a nightmare.
Do yourself a favor, and clearly label everything. Use a permanent marker to write the card number on your SDs. Format and rename each SD card so you can tell which number it is, and put a sticker on each GoPro as well. If you ask friends for a couple of cams, let them keep their accessories and SD cards to minimize the amount of stuff you have to manage.
2: Use identical hardware and identical settings
Yes, you can ask friends for some camera’s to test out your rig, but you absolutely need to purchase your own SD cards. Why? Because different cards have different writing speeds, which can mess up your recording. You need 100% identical cards of the same brand. Do not compromise on this. Also use identical cameras. So no mixing of HERO 3’s and HERO 4’s!
Possibly even more important than hardware are the capture settings. We used 1080p 60fps wide, protune ON, white balance CAM RAW (More on capture settings later). If just one of your cams is using different settings, it will not be possible to stitch the videos together during post-production. Some of our cams seemed to reset at some point, so always check each cam before every recording to ensure they are all using the correct settings.
3: Do not rush things
Our very first full-scale shoot was a beautiful opportunity to shoot on the field of the Amsterdam ArenA (thanks EMansion and ArenA!), which we definitely wanted to take. However, we had less than 24 hours to acquire 9 GoPros (thanks for the help Budgetcam!) and 9 SD cards.
This we managed to arrange, but then we had to put each of our total of 16 camera’s in the right settings during our brief train ride to the ArenA, after which we had about 10 minutes to synch every camera to our brand-new GoPro smart remote, put all of our SD cards into the cams and finally place each GoPro in the correct position in our camera rig before we had to start shooting.
Yes, I was stressing a bit at this point, but we were excited and good to go! Or so we thought..
When I pressed the ON button of the smart remote, only 5 of the 16 cams were recognized. With no time to fix this rather embarrassing hiccup, we had to manually start the recording of each camera, which cost us about a day extra in post-production to figure out which files belonged to which take. Afterwards we also found out that one of our cams had not been running at all during the shoot, which left an ugly black patch in our final video.
Take your time and double check if every cam is both working and in the correct setting before each recording, and always use a properly configured remote to start your cams. You may thank me later.
4: Do not run out of power
GoPro’s have a frustratingly short battery life of about an hour if you’re lucky, especially when shooting in the demanding settings required for high quality VR videos. After our first two takes we already saw that some of our cameras were struggling to stay alive. We hastily swapped some of the batteries, and then accidentally misplaced some of the cameras, adding to the confusion in post-production.
When you are not recording, turn off WIFI on your cams. Also, use external USB power banks to triple your battery life. Something like the PowerPad15 might be an even better solution, as it allows you to power your cams directly from the grid. We do not recommend the original GoPro Battery BacPac, because it dramatically increases the size of your cams, meaning you get a much less compact rig and more parallax. The before mentioned solutions can be hidden in the middle-, or underneath your rig.
5: Carefully calibrate your camera beforehand
Shooting in a major soccer stadium is great, but creating an accurate stitching template, with software like PTGui, for a video where nearly half the screen is composed of grass is a mission impossible.
In a stitching template you tell the computer that point X in video #1 is the same as point Y in video #2. You enter a couple of those points for each video, and then, in theory, software like Autopano Video can seamlessly stitch your separate recordings into a single, smooth equirectangular video file which can be viewed in any of the popular VR players. Unfortunately it is not possible to determine that this grass blade in the first video is the same as that grass blade in the second video, which led to a whole lot of distortion and an unclean stitch.
The solution is to create a camera rig in which the GoPros are completely fixed in place, so they are always in the exact same position, and then create a detailed stitch template in a specially outfitted room (using strokes of wallpaper can work wonders!). Now you can film wherever you want, even on a soccer pitch, and then simply apply the pre-calibrated stitch template to get a perfectly aligned result!
6: Know the limitations of your camera
Besides shooting on the actual pitch, we were also allowed to shoot from a walking bridge just below the roof. We placed our rig near the railing, since we thought that would give a powerful effect when you put on a VR headset. However, when stitching our footage, we noticed that it was impossible to get a clean stitch of the railing: it was simply too close to the camera.
You have to run tests to figure out how close to your camera you can get before stitching and parallax becomes an issue. As a general rule, do not get closer than 2 meters. Also know where your stitch lines will be, so you can place as little objects as possible on those places in your scene to minimize noticeable distortion, especially with moving objects.
7: Choose frames per second over resolution
As mentioned before, we shot our footage at 1080p 60fps wide mode. Because we stitch 16 of these full HD videos together, this results in a very high resolution of more than 4K per eye. This is a massive overkill, since the latest Oculus Rift Crescent Bay edition contains a 1440p display. However, Oculus does aim for content shown at 80fps! In addition, higher fps greatly improves the quality of your stitches, which is especially noticeable when objects are moving through your stitch lines.
Therefore we recommend to shoot in 960p 100fps on a GoPro HERO3 Black Edition, and 960p 120fps on the HERO4. Keep in mind though that shooting at the highest possible quality can lead to overheating of your cameras. Also make sure your SD cards are capable of handling those high fps settings.
8: Don’t underestimate the required GPU power
Do you know why so many Oculus Rift DK2’s can be found on eBay? It’s because most people are unable to run even the simplest demos on their machines. Oculus uses high frame rates and renders the entire scene twice, once for each eye. This requires massive GPU power! The same goes for stitching.
We used the most powerful MacBook Pro Retina to stitch our first small scene, which nearly resulted in lift-off into space judging from the noise coming from the fan of the video card. So unless you want to be rendering for days on end, get a powerful CUDA enabled video card. If possible, get the high-end NVidia GTX 980, or else any other NVidia card with at least 500 CUDA cores and 2GB or more of video memory.
9: Script & practice everything
Cinematic VR requires radically new ways of thinking about filming. I already mentioned some of the curious effects caused by parallax and stitch lines, and then I haven’t even talked about the obvious issue of where to hide the crew and their donut trucks, since the camera films on all sides simultaneously. This is why it is important to script everything beforehand.
Because our first shoot was so last-minute, we had to improvise. Someone was supposed to give a little talk in front of our camera, but before we managed to run and hide, he already started talking.. Also, he moved too close to the camera and started walking around it while talking, straight through all of our stitch lines. On top of this, the mic we used was set to capture audio from only one direction, so only a small part of his undoubtedly epic speech was audible.
So script & practice everything in detail, view it as a choreography, and make sure everyone knows exactly what to do and, even more important, what not to do.
10: Automate file management
This was possibly the most painful lesson to learn. Having 16 cameras which sometimes malfunctioned or were misplaced, with several takes on them, all started at different times because we had to set them manually, is more than enough to break anyone’s spirit. In fact, it took us several frustrating days to figure out which file belonged to which take and then prepare them for stitching. You have to automate this step, or you will be growing gray hairs faster than you could have imagined.
Most people use 360 CamMan, created by 360Heros. While this tool definitely works, I have to say that the interface is an absolute mess and the software is hugely overpriced for the functionality it offers. It can format your SD cards, copy files from your SD cards to your computer, and then put each video file in the correct take folder. Extremely useful, but not exactly rocket science.
That’s why have developed our own simplified tool, called ManyCams, which you can purchase in our shop.
Closing thoughts
Cinematic VR is a brand-new field and all of the rules still have to be written. The possibilities are endless and we are excited to start working on creating experiences that were never before possible. We do not want to simply exchange a regular camera for a VR camera and use head-tracking as a gimmick. No, we want to push cinematic VR to the limits and create immersive experiences which maximize the potential of this revolutionary medium.
Again, 2015 is going to be an exciting year!