Some of you may know we have written multiple blogs about 360 video encoding dating back to 2015, 2016 and 2017. These blogs serve as a good starting point for everybody that is new to 360° video compression or wants to learn more about the history of 360° video compression and how we got to this point. But since the industry has evolved rapidly over the last couple of months and our upload settings and cloud encoding profiles needed a well-deserved overhaul, here is our latest platform update on cloud encoding and 360 playback resolutions.

Stare at the progress bar and…. it’s failed again!

Encoding 360° videos will always be a major point of struggle in the production pipeline, there were a lot of times that I almost set fire to my computer after it failed on a 6 hour render at 5 AM the night before an important deadline. Rendering high resolution 360° video requires a lot of technical knowledge, a well thought out and tested encoding plan and a lot of patience. On top of that you also need deep pockets to afford the hardware you need to render faster and without errors especially when operating complicated VFX pipelines. 

Tools like FFmpeg are not easy to learn when you are new to video encoding which makes the whole process even more error-prone. In the past, we tried to simplify the process for ourselves as a studio with custom tools like VRencoder and our FFmpeg cheat sheet but they still require a lot of technical knowledge and even then we still got our occasional black frames or render artifacts.

So in search of a definitive solution, our latest Headjack update and this two part blog is completely devoted to the video processing pipeline. This update is the result of over 5 years of research and continuous optimization of our Cloud encoding profiles and a lot of back and forth with our clients and production team. This is our latest effort to get our Cloud encoding up to industry standards and give users a lot more flexibility on which video they want to use for final distribution, as well as unlocking new resolutions up to 8K x 8K for PC based headsets.

What is Headjack Cloud Encoding?

If you want to save yourself a lot of time or don’t have a technical background in video compression please select one of our pre-defined encoding profiles to make sure your video always plays for your chosen target device. Headjack Cloud encode runs on Brightcove: a cloud compression solution which enables us to set custom encoding profiles for specific device groups and run cloud encodes with the input of maximum 8K footage. This will save you hours of rendering time compared to running these renders on a local machine while minimizing risk on failed renders. Cloud encode also takes care of all the HLS streaming formats which are necessary for switching between different resolutions in varying network conditions.

We have created 4 different cloud encoding profiles each tailored to work seamlessly across multiple devices for maximum compatibility. You can use the corresponding FFmpeg commands to run the same compression profiles on your local machine for testing and comparison. All the other resolution marked in green can be targeted with the Use Original profile

Check out the comparison chart below to find our tested resolutions with the latest version of our Headjack Link application on the various VR headsets.

  • VR Headsets (Coming Soon)

Up to 5760×2880 resolution for monoscopic (mono) equirectangular video or 4096×4096 for stereoscopic (stereo), using H.264 and H.265 encoding at 30FPS. Specifically tested on most modern VR Headsets, this profile perfectly aligns with commonly used post-production formats while making the most out of current mobile VR device capabilities. Until this profile is available use the “use original” profile to target these resolutions!

ffmpeg -i "monoscopic_video.mp4" -c:v libx264 -crf 18 -x264-params "mvrange=511" -maxrate 120M -bufsize 150M -vf "scale=5760x2880" -pix_fmt yuv420p -c:a aac -b:a 192k -r 30 -movflags faststart "monoscopic_output.mp4"

ffmpeg -i "stereoscopic_video.mp4" -c:v libx265 -crf 17 -maxrate 120M -bufsize 150M -vf "scale=4096x4096" -pix_fmt yuv420p -c:a aac -b:a 192k -r 30 -movflags faststart "stereoscopic_output.mp4"

  • Mobile devices

This is a 4096×2048 mono or a 3840×2160 stereo 30FPS H.265 encode. Specifically designed to work on a wide range of mobile phones and (older) VR headsets. For a long time, this profile was our standard for video distribution and therefore tested over thousands of headsets over the last couple of years.

ffmpeg -i "monoscopic_video.mp4" -c:v libx265 -crf 17 -maxrate 80M -bufsize 100M -vf "scale=4096x2048" -pix_fmt yuv420p -c:a aac -b:a 192k -r 30 -movflags faststart "monoscopic_output.mp4"

  • Stream

The HLS video-on-demand stream uses the mobile-compatible 4096×2048 mono or 3840×2160 stereo 30FPS H.265 encode as its highest quality for streaming, dropping down to 1440p, 1080p or 720p if network conditions are poor.

ffmpeg -i "stereoscopic_video.mp4" -c:v libx265 -crf 20 -x265-params "keyint=60:min-keyint=60" -maxrate 25M -bufsize 35M -vf "scale=3840x2160" -pix_fmt yuv420p -c:a aac -b:a 192k -r 30 -g 60 "stereoscopic_output.mp4"

  • WebVR (BETA)

There is a 1080p version available optimized for distribution on websites with just a couple of lines of HTML. This 1080p version is also used as a fallback video format on devices that are unable to run any of the other formats for some reason.

ffmpeg -i "monoscopic_video.mp4" -c:v libx264 -crf 21 -maxrate 10M -bufsize 15M -vf "scale=1920x1080" -pix_fmt yuv420p -c:a aac -b:a 192k -r 30 -g 60 -keyint_min 60 "monoscopic_output.mp4"

Set your own encoding profile

By choosing the Use Original function instead of cloud encoding you are able to use your own encoding settings defined by the final export of your editing program. Use Original is both the easiest profile to explain and yet at the same time the hardest. Let me start with the easy part: we are not doing any processing whatsoever on your video file. So the file that will be served as a download in your app is the exact same video that was uploaded in the Headjack CMS. This has three big advantages. Firstly, it’s immediately available in the application as a download when the upload finishes. Secondly, you as a creator have complete control over the compression method used for the final distribution file. Lastly, it allows for custom encoding that falls out of the compression profiles defined by us so you can determine parameters like file size yourself.

This profile should be used for footage that needs to be distributed at 60FPS or higher, any resolution higher than 5.7K or footage with custom bitrate profiles or video containers. Since you are playing back your own video, you are responsible for playback. This means that we can’t guarantee that our players work with your video compression since variations in video format or compression settings could cause the player to fail. This profile is made for experts only.

Learn more on setting the right resolution for your 360 videos and get it ready for distribution with the Use Original feature please continue reading part two of our blog for an in-depth look on how we handle cloud encodes and how you can set-up your own video compression profiles.

Download, Package or Stream to the app

Before we get to compression in our next blog you need to know there are three different ways of getting the video from the Headjack CMS server to your device: 

  • Download the videos to the device from the app’s menu.
  • Package video in the App Package so users don’t have to download any additional content after the initial app is installed from the stores.
  • Stream content directly from our servers for instant playback up to 4K 

How to choose the right compression 

Before we go into an in-depth look of our cloud encoding solutions and their inner workings there are a couple of things to consider before locking down on a specific resolution. Video compression has a lot of levers to pull on and there is not really a right or wrong it all depends on the use case and which headsets you are targeting. Our Profiles can be seen as the most commonly used compression formats and video resolutions 

One of the most common myths is that higher resolution is always better: due to limitations in display technologies, going higher than standard 5.7K resolution on an Oculus GO, for example, can cause artifacts like moiré patterns. Super high 8K resolution only makes sense when you want to distribute 360° stereo video to a recent PC-based VR headset. Also, the file size vs bitrate trade-off is a major factor to consider when distributing on a large scale or when you have to deal with varying network conditions or very long videos and limited file storage on your device.

Learn more on setting the right resolution for your 360 videos and get it ready for distribution please continue reading part two of our blog for an in-depth look on how we handle cloud encodes.