The sheer abundance of 3D modeling tools and file formats is overwhelming, and picking the best 3D model format for your AR application can therefore be rather frustrating and confusing. Every file format has different features and limitations, some can only be exported from specific (and expensive) software tools, and not all of them can be imported into an app at runtime, to name just a few of the hurdles. This article explains what we learned from building our first AR applications, and which 3D file format proved to be the most powerful for the job.
Troubles
Our XR apps are powered by the popular Unity game engine, and our CMS allows us to import new assets into our apps while they are running (called “runtime”). For example, users install a small AR app, and only when the app starts up it downloads the necessary 3D models, images, videos, and texts from our server, and imports them into the app. This makes it easy to make changes to your app without having to push an entire app update to the stores.
The first step was to decide on a 3D file format to use, and one of the main criteria was that Unity should be able to ingest it. Looking at the Unity docs, we learned that FBX, DAE (Collada), 3DS, DXF, OBJ, and SKP files were supported. We then further narrowed down this list. For example, OBJ wasn’t an option, since it does not support animations, and 3DS only works with 3ds Max, so is too limited. We finally decided that FBX, a widely used 3D format owned by Autodesk, was the file type of choice, since it works with a wide variety of software packages, OBJ, DXF, DAE, and 3DS models can be easily converted to FBX using the SDK, and FBX models can be viewed in web using three.js.
A major problem we encountered, though, was that Unity does not like it when you try to import 3D models at runtime, since game developers usually have all their models imported in their Unity project before they build it (“edit time”). We found a 3D model importer asset on the Unity Asset Store, which promised to allow runtime importing of 40+ 3D file formats. However, the format we initially planned to use, FBX, was only partially supported, and so we were clueless as to how we were to import models at runtime.
What would Sketchfab do?
At Headjack, we initially specialized in 360º VR video, so handling 3D models was new to us. That’s why we decided to look at Sketchfab, one of the largest 3D model websites in the world, to see if we could learn something from what they are doing. What we found was a pleasant surprise…
When we tried to download a mech model from Sketchfab, we were presented with two options: 1) download the original file as uploaded by its creator, or 2) download a converted file in glTF format. glTF?! Never heard of that before! But if it is the format of choice for Sketchfab, there must be something to it, and some further digging revealed a lot of exciting information about this mysterious file format.
glTF: the JPEG of 3D
glTF is the self-proclaimed “JPEG of 3D”, and the first thing that excited us about glTF, was that it is a format created by the Khronos Group, which is the group behind the WebGL standard, and includes members like Apple, Microsoft, Google, NVIDIA and many more tech industry heavyweights. In the words of its creators:
“(GL Transmission Format) is a royalty-free specification for the efficient transmission and loading of 3D scenes and models by applications. glTF minimizes both the size of 3D assets, and the runtime processing needed to unpack and use those assets. glTF defines an extensible, common publishing format for 3D content tools and services that streamlines authoring workflows and enables interoperable use of content across the industry. ”
As you can read in the above statement, glTF is royalty-free, and so not tied to any specific software package. It also has a small file-size and amazing performance, making it an ideal format for mobile and web apps. The glTF spec also includes support for Physically-Based Rendering (PBR), which “enables a glTF file to be rendered consistently across platforms”. And as the cherry on the cake, there is an official library which allows glTF models to be imported into Unity at runtime!! In other words, glTF has everything we were looking for in a 3D file format, and then some.
We also view the fact that Sketchfab uses glTF as its format of choice as a major plus, because in the future we would love to allow users to import their 3D models into Headjack directly from their Sketchfab account. Even Facebook announced support for glTF!
A closer look
On a more technical note, what is a glTF file? Well, it is actually a combination of files, as you can see in the image below.
However, because we host these models on a server and have to import them into an app, it would be a lot easier if .gltf, the .bin, and the texture files could be combined into a single file. Luckily, the glTF standard has this option, in the form of a GLB file container with a .glb extension. This way we just have to deal with a single file, which makes things a lot easier, and is a major benefit over file formats like OBJ, which do not have this option.
As regards to compression, the Group has the following to say:
“Version 2.0 of glTF does not define compression for geometry and other rich data. However, the design team believes that compression is a very important part of a transmission standard, and there is already work underway to define compression extensions.”
For now, though, some compression can be achieved using WEB3D_quantized_attributes.
As is to be expected from a Khronos Group standard, there is widespreads support for the glTF format. Some notable examples of tools that already support it:
And Headjack can be added to that list once we roll out our AR features later this year!
By now you are hopefully as excited as we are about glTF, but how can you turn your models into glTF files? Well, one way is to upload your model to Sketchfab, and then download it as glTF. However, a better approach would be to use one of the several glTF exporters and converters out there. There are, for example, official Blender, 3ds Max and Sketchup exporters, as well as FBX2glTF or OBJ2glTF converters. These glTF models can then be viewed in Unity, Paint 3D, or the very impressive Clay viewer. As you can see, there is already a very impressive ecosystem for this still rather new format.
Since we had to integrate glTF into our Headjack workflow, we built (and open-sourced) our own glTF converter API. We also added a simple drag and drop UI so you can easily convert your FBX, OBJ and COLLADA models to glTF or GLB. Check it out at https://gltfapi.co
What about Alembic?
While I was writing this article, a colleague told me about yet another file format, called Alembic, which has been extensively used in Unity’s ADAM showcase project.
To learn more about Alembic, I suggest you read What’s So Hot About Alembic, but in short, it is a format developed by Sony and Lucasfilm, and is especially suited for real-time CG films. The Alembic website says this format could be used…
…To bake the results of an animated scene for hand-off to lighting & rendering
…To hand off an animated creature for cloth or flesh simulation
…To store the results of a cloth or flesh simulation for use in lighting & rendering
…To hand off animated geometry to a physical simulation engine
…To store the results of a physical simulation engine for use in lighting & rendering
In other words, it is a format that has very powerful animation capabilities, but in ADAM it was still used in conjunction with a body rig in FBX format. This means that the choice is not either glTF or Alembic, but it can be a combination of both, each with their own strengths. And the beauty is that both these formats are open standards, so there are no hefty licensing fees to be paid.
Apple’s 3D file format: USDZ
During Apple’s 2018 WWDC conference, a new 3D file format was announced in partnership with Pixar, called USDZ. USD, or Universal Scene Description, is a format already used in Pixar’s 3D animation pipeline. USDZ is an extension of this filetype, and combines all assets required for displaying a 3D object into a single uncompressed zip file. Besides Pixar, Apple also partnered with Adobe to implement USDZ into the Adobe suite, so it seems like Apple has big plans for this new file format.
The main reason for this is of course to have a good 3D file format for their ARkit platform. From iOS 12 onwards, i-devices support USDZ natively.
So is USDZ preferable over glTF? It’s really too early to say. USDZ has Apple, Pixar, and Adobe behind it, but glTF has Microsoft, Google, and Facebook. And, contrary to Apple’s usual MO, USDZ is actually an open format, just like glTF is. Time will tell which format will become the industry standard, but our money is on glTF.
Closing thoughts
I hope that sharing our path to figuring out the best 3D model file format to use for our AR applications has given you some useful information for your own projects. At the very least I hope it will save you a lot of time and frustration. Headjack’s support for 3D models and AR is still in early beta, but will become publicly available in the second half of 2018.