(Apologies for the reference, but it had to be done)
While everyone here in the lovely city of San Francisco was preparing Halloween costumes, metaio was at the International Symposium on Mixed and Augmented Reality presenting research that will not only enable an actual Augmented City, but will lead to more natural, more subtle augmentations of the physical world.
metaio has made substantial improvements in mobile optical tracking and recognition, having implemented both 3D object recognition and Gravity-Aligned feature descriptors (GAFD). Through complex algorithms and sensor-fusion, we are able to align the field of vision of the smartphone with gravity. That means no matter where or how the user is positioned, his or her smartphone will always understand “up” and “down” as constants instead of relative positions. Looking at the photo above, you can see that natural feature tracking fell short in trying to describe surfaces buildings- all the windows look the same and there is little variability or contrast in the overall facade. Before GAFD, identifying or isolating only one or two of those windows would have been nearly impossible- defining gravity as a constant however allows the tracking to easily identify each individual window. Check out the video below for a visual demonstration- you’ll notice that the tracking is faster, smoother, more robust, and above all more natural. Note also that we can align individual elements of a given 3D model with gravity, allowing 3D models and animations to behave more realistically when placed in the real world.
metaio also debuted 3D object recognition and tracking for the first time on a mobile device running on a consumer application: junaio. Not a specifically engineered research platform or prototype, but an iOS and Android application free to download and more importantly, free to develop. And in its first appearance, 3D tracking on junaio won the ISMAR Tracking Competition.
3D object tracking is the obvious next step for mobile AR- though printed 2D images provide a wealth of material for launching AR experiences, being able to track actual real-world objects will open up a whole world of opportunities for placing useful, engaging, and interactive content in the physical world. Combined with GAFD, objects like buildings, automobile engines, complex industrial prototypes, groceries, billboards, and public transportation vehicles become the canvases for the digital world.
In order to have real-time, natural and subtle augmentations of the real world it is imperative that we able to understand and digitally reconstruct our immediate environments. For instance, a realistic digital or virtual piece of furniture placed into a physical room needs to behave as though it is real. You would expect an object in front of it to obscure it, and vice versa. Until recently, the placing of virtual objects into the real world in real-time has been restricted to the foreground of the user’s perspective. metaio however presented at ISMAR research that show the instantaneous online, real-time learning of 3D environments in order to realistically display or remove virtual objects from the physical world. This kind of technology will eventually allow prospective homeowners to furnish an entire house or apartment the first time they step across the threshold, or allow for the construction of truly immersive, realistic AR gaming environments. Watch the video below to see it in action- note how the camera actually learns its surroundings in real-time, and within seconds adjusts to make a given virtual object or animation appear realistically in reference to surrounding real world objects.
If you build it…
metaio is well aware that in order to see stellar use cases, engaging content, and disruptive interfaces that this kind of technology needs to be in the hands of both creators and users. That’s why all of this research will soon be available through junaio (now, assuredly, the world’s most advanced AR browser) and the advanced metaio Mobile SDK (which if you haven’t heard has a free version coming soon). We here at metaio believe that Augmented Reality as a whole is far too vast, multi-faceted and diverse to be unified under a sole “Killer App”. That’s why we’re giving people the tools to investigate, experiment, and explore all aspects of this wonderful technology to expand on all potential applications of AR.
For more information, see the press release, as well as some of the articles written about our research in the past few days:
Singularity Hub, “The Latest Updates to Augmented Reality: 3D and Gravity”