During the Google I/O conference at the end of May, Google announced Jump; a revolutionary step in the world of 360 3D content!
The idea has three parts:
- Camera: an open-source 360 3D camera design using 16 synchronized GoPro HERO4 action cams
- Assembler: computer vision algorithms running on powerful Google servers which seamlessly stitch the 16 images and compute the 3D effect
- YouTube: yes, YouTube already existed, but will be fully integrated with Jump technology for easy playback and distribution of content
The power of Jump lies in the fact that it gives creators around the world access to top-of-the-line VR camera technology. No more tinkering with custom-built rigs and hours of painstaking manual video stitching. Jump is a dream come true!
However, the camera design is not yet available.. Let’s see what we can learn from analyzing the images and videos that have been released so far.
What we learned
- Not up and down
Looking at the rig, the first thing you might notice is that there are no cameras aimed up or down. This means that any footage shot with this rig will have an empty spot both at the bottom and top part of the video. In the demo footage shown by GoPro these blind spots have been masked with some clever Photoshopping.
However, this has most likely been done to simplify the stitching algorithm, which allowed a quicker launch of this awesome project. We expect that future versions will have the ability to film up as well.
- Software magic
Normally if you want to film in stereoscopic 3D, you need one camera per eye. That’s also the concept our 360 3D camera is based on. The only other way is to film in 2D and then let an extremely complex computer vision algorithm calculate the 3rd dimension. That’s the concept the camera of Jaunt VR, the current leader in cinematic VR technology, is based on.
Google took a similar approach as Jaunt, with the slight difference that Google is giving the public access to their algorithm! This is great news for content creators like ourselves, but bad news for 360 camera developers.
- Dimensions & angles
Because we know the size of a GoPro, we can use the images of the Jump rig to calculate the dimensions. The width of a GoPro is 41mm, and based on that we approximate that the lens-to-lens width of the entire rig is around 280mm.
Since we know that the Jump rig consists of 16 cams, we can calculate that they must by definition be positioned in 22.5° angles (360 / 16 = 22.5). However, if you look closely, you can see that the cams are not aimed straight from the center of the rig, they are placed slightly angled. The reason for this is to save space and make the rig more compact, which allows for more overlap in the videos and therefore better stitching.
- Hardware synchronization
While Google will release the design for a basic rig, GoPro goes one step further by adding hardware synchronization, which allows the 16 individual cams to function as one. This means you can easy set each cam to the same settings and make sure they all start recording on the exact same frame. This saves a lot of frustration in post-production!
Wish list
Google and GoPro have made a big step towards making 360 3D camera technology available for the masses. It will not be cheap, since you need to purchase 16 HERO4 cams at $500+ each, and you will still have to wait until next year for the public launch, but at least most of the technical hurdles will be taken care of for you.
However, over the past few months we have been working with our own 360 3D rig in the field and therefore know a thing or two about the limitations of using (multiple) GoPros. The main problem is overheating, leading to unreliable behavior and crashed cameras. If only one cam out of the 16 stops working while shooting, your entire scene is ruined. That’s why we have stripped our GoPros and added active cooling to our rig.
Another issue is the short battery life. We have wired all our cams into an industrial grade USB hub, which allows us to power our rig through mains current or a big car battery. This way we can film for 12 hours on end while keeping everything portable.
The current Jump design saves the video files to individual SD cards inside the GoPros. However, reading out this data and managing the videos is extremely time consuming. The USB hub allows us to read out all cameras simultaneously using a single cable. This way we do not have to eject every single SD card manually. Our custom file management software also helps us organize our files to significantly speed up our post-production workflow.
While we can’t be certain about this yet, it seems like Jump footage can only be played back on YouTube for now, which makes sense because it needs the Google algorithms to function. However, some companies might want to have a dedicated, branded VR application.
A final point of attention is that in VR, realistic audio is almost 50% of the experience. Therefore we, as well as Jaunt, added an ambisonic microphone to our rig. This allows viewers to hear where sounds are coming from, adding to a sense of presence.
Obviously the prototype revealed at Google I/O is only a first version. I’m sure future products will incorporate many of the above mentioned technology.
Closing thoughts
Yes, we at Purple Pill VR have developed our own 360 3D camera system, but our core mission is not to create hardware, but to produce high quality, immersive VR video content. From the beginning we knew that one day a camera would be released which would surpass our own, and now it’s (almost) here. We’re welcoming Jump and can’t wait to start using it!