We built this city on A and R



(Apologies for the reference, but it had to be done)

While everyone here in the lovely city of San Francisco was preparing Halloween costumes, metaio was at the International Symposium on Mixed and Augmented Reality presenting research that will not only enable an actual Augmented City, but will lead to more natural, more subtle augmentations of the physical world.



metaio has made substantial improvements in mobile optical tracking and recognition, having implemented both 3D object recognition and Gravity-Aligned feature descriptors (GAFD). Through complex algorithms and sensor-fusion, we are able to align the field of vision of the smartphone with gravity. That means no matter where or how the user is positioned, his or her smartphone will always understand “up” and “down” as constants instead of relative positions. Looking at the photo above, you can see that natural feature tracking fell short in trying to describe surfaces buildings- all the windows look the same and there is little variability or contrast in the overall facade. Before GAFD, identifying or isolating only one or two of those windows would have been nearly impossible- defining gravity as a constant however allows the tracking to easily identify each individual window.  Check out the video below for a visual demonstration- you’ll notice that the tracking is faster, smoother, more robust, and above all more natural. Note also that we can align individual elements of a given 3D model with gravity, allowing 3D models and animations to behave more realistically when placed in the real world.



metaio also debuted 3D object recognition and tracking for the first time on a mobile device running on a consumer application: junaio. Not a specifically engineered research platform or prototype, but an iOS and Android application free to download and more importantly, free to develop. And in its first appearance, 3D tracking on junaio won the ISMAR Tracking Competition.

3D object tracking is the obvious next step for mobile AR- though printed 2D images provide a wealth of material for launching AR experiences, being able to track actual real-world objects will open up a whole world of opportunities for placing useful, engaging, and interactive content in the physical world. Combined with GAFD, objects like buildings, automobile engines, complex industrial prototypes, groceries, billboards, and public transportation vehicles become the canvases for the digital world.


3D Reconstruction

In order to have real-time, natural and subtle augmentations of the real world it is imperative that we able to understand and digitally reconstruct our immediate environments. For instance, a realistic digital or virtual piece of furniture placed into a physical room needs to behave as though it is real. You would expect an object in front of it to obscure it, and vice versa. Until recently, the placing of virtual objects into the real world in real-time has been restricted to the foreground of the user’s perspective. metaio however presented at ISMAR research that show the instantaneous online, real-time learning of 3D environments in order to realistically display or remove virtual objects from the physical world. This kind of technology will eventually allow prospective homeowners to furnish an entire house or apartment the first time they step across the threshold, or allow for the construction of truly immersive, realistic AR gaming environments.  Watch the video below to see it in action- note how the camera actually learns its surroundings in real-time, and within seconds adjusts to make a given virtual object or animation appear realistically in reference to surrounding real world objects.



If you build it…

metaio is well aware that in order to see stellar use cases, engaging content, and disruptive interfaces that this kind of technology needs to be in the hands of both creators and users. That’s why all of this research will soon be available through junaio (now, assuredly, the world’s most advanced AR browser) and the advanced metaio Mobile SDK (which if you haven’t heard has a free version coming soon).  We here at metaio believe that Augmented Reality as a whole is far too vast, multi-faceted and diverse to be unified under a sole “Killer App”. That’s why we’re giving people the tools to investigate, experiment, and explore all aspects of this wonderful technology to expand on all potential applications of AR.

For more information, see the press release, as well as some of the articles written about our research in the past few days:

Singularity Hub, “The Latest Updates to Augmented Reality: 3D and Gravity”

TechCrunch, “Metaio Adds Gravity To Their Augmented Reality Platform”

6 Responses to We built this city on A and R

  1. Miguel Palma says:

    Great post Trak! Entertaining and liked the video references! :-)

  2. And all we need to do now to get these awesome experiences on-the-go is to wait until smartphones featuring a depth camera are released…

    …or until someone implements 3D object recognition based on moving images and perspective distortion. That would be awesome!

    In the mean time, here at PreviewLabs – a company specialized in rapid game prototyping – we’ve been playing around with Augmented Reality as well; check out our blog post featuring an augmented reality darts board video, running on a Google Nexus One Android device.

  3. [...] we’ve been quite busy with our research efforts, we still found time to release junaio 3.1 and the junaio Plugin (previously only available for [...]

  4. [...] went on to demonstrate cutting-edge research that would eventually be presented at ISMAR in October- inarguable evidence that metaio is looking [...]

  5. [...] nice development is going on… Augmented realtiy #tracking for #mobile [...]

  6. [...] reality is by nature a very technically-driven industry, and the R&D labs at metaio are constantly trying to devise new ways of using information from on-board sensors and mobile [...]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

Join 5,079 other followers

%d bloggers like this: