Technical insight into the 2014 DLD Museum Tour: An Augmented Reality exhibit for the Bavarian National Museum

April 17, 2014

This week we offered once again an Augmented Reality museum walk through together with the Bavarian National Museum in Munich and were overwhelmed by all the attention we got afterwards via different blogs (like in the Huffington Post Germany )and in the social networks. This attention shows that the topic of Augmented Reality is really interesting both for the museums themselves and also for the visitors. Therefore, we decided to give you a more technical insight into our museum project and asked our developers to talk about the development of our application. Here it is:

Alexei, Nicolai & Kevin - Developers of the Museums AR Experience

Alexei, Nicolai & Kevin – Developers of the Museums AR Experience

Hello, my name is Kevin and I am a member of the creative team here at Metaio.  I worked together with my colleagues Alexei and Nicolai to create the assets and code for the DLD Bavarian National Museum AR experience. 

Built in a very short time, the Bavarian National Museum application was a great opportunity to create a valuable cultural experience with AR technology. It aimed to create a balanced user experience between physical and digital content that could inform and entertain the average museum visitor. We  focused on five pre-selected exhibition pieces and  unveiled them as part of the DLD Conference in Munich.

The Metaio Toolbox: Easily creating 3-D tracking maps

The first step to develop the project was to visit the museum itself in order to come up with some ideas and generate our 3-D tracking maps with the Metaio Toolbox .  Many of the assets would be created off-site so establishing an accurate reference that we could use back in the office was important. This was easy to do with the Metaio Toolbox, and within 2-3 attempts we had a satisfactory 3-D tracking model that we could later use in the Metaio Creator . We could also load our point-clouds into Autodesk products for designing more complicated 3-D content by extracting them as an OBJ file from the Metaio Creator.

During our discussions with the museums curators there was concern that the new digital content would take attention away from the physical artifacts, so we set out to create a design that would keep a respectful balance between the cultural artifacts of the museum and the new digital information we were introducing into the environment. To achieve this we kept AR content to the sides of the physical objects, used discreet 3-D white lines as indicators and semi-transparent backgrounds for our buttons and texts.

The AR pieces in detail:

untitled-3756Tilman Riemenschneider: The Mary Magdalene

In the Mary Magdalene scenario we introduced to the user an audio explanation of the piece, some general background information and a photographic overlay that showed the sculpture’s past place of residence (a church altar that was color-corrected in order to better match the lighting scheme of the room).

 

untitled-3776Conrad Meit’s “Judith with the head of Holofernes.”

In the Judith scenario we connected pieces of explanatory text to the model with 3-D white lines. Supported by a particularly strong 3-D map, the experience provided a great sense of depth and space to the user without distracting from the physical object. In order to ensure that the lines were a pleasant shape and length we imported the reference point cloud into Maya before constructing the 3-D lines.

untitled-3788Jakob Sandtner: The Munich City Model

The Munich city model was a real challenge to us because the lighting conditions in the room were very difficult. Due to the sensitive state of some of the historical pieces, strong lights were not allowed in this particular room. This meant that getting a good 3-D map and lining up content to the physical model involved a lot of trial and error. But we managed, and in the end visitors could see an overlaying map of today’s Munich.

untitled-3803Christoph Jamnitzer: The Moor’s Head

The Moor’s head cup contained three nice reference images of the interior and base of the cup which were not viewable to the visitor. In order to display them and not take away from the physical model we created a thumbnail effect that shrank and grew the images when the user tapped on them. This was done by overlaying the images on to a 3-D object and adding a simple on-click animation in the Metaio Creator along with an additional piece of code to allow for a secondary on-click animation.

untitled-3794Hubert Gerhard: Flying Mercury

In the case of Flying Mercury we displayed large images of other artworks created by the artist. They are “floating” around the sculpture.

Balancing AR and non-AR content

AR tablet experiences have a short viewing time in comparison to other media due to the energy required for navigating the physical space. In a museum there are many different people who are enjoying the exhibition at different paces. To create a more fulfilling experience, we needed to support each AR scenario with a non-AR content section: something people could easily switch to while sitting down and relaxing. This non-AR section contained text, audio and video and was accessible through a button on the bottom of the AR viewing screen. It was built by creating an offline webpage that was then integrated into the AR experience.

A developer’s point of view

Nicolai’s thoughts:

This AR scenario was very generous from a developer’s point of view. Its value derives from the simplicity and unobtrusive way of visualizing the additional, well designed content. In this case it means to me that the design and arrangement of the content together with the stable tracking already contributes a lot to this experience. For this AR experience my colleagues finalized the concepts and designed the assets, as well as combined and positioned everything with the help of the Metaio Creator. The effort as a developer was therefore quite manageable.

untitled-3764My part in this was to integrate the designed webpages and connect them to the exhibits. Those webpages functioned as a user interface, enabling the user to see detailed information and multimedia items. The combination of those webpages into one user interface is fairly straightforward. The user interface basically is a single webpage on top of the camera feed. The Metaio AREL bridge allows you to connect to the underlying renderer with the included JavaScript code.

First of all, the information elements had to be connected to the exhibits. This is done by listening to the respective tracking event. When the event is fired, the according HTML elements are shown.  The difference from regular websites is mainly keeping everything within one HTML file. The CSS and JavaScript files can be included there as well. So instead of linking to another page, these other webpage elements were included into the one HTML overlay file. On request, certain elements are hidden or shown. When it comes to best practice, HTML editors and templates can be a great help when designing those HTML elements. Integrating those files for the overlay, however, can be slow due to the produced overhead in including styles and linking in the first place. Within the web overlay you have all the possibilities that HTML5 has to offer. Next to the AREL API this scenario uses standard HTML5 technology like video and audio tags to include even more media content.

In the end, the AREL Technology enables you to easily create slick and effective user interfaces for your AR scenarios.


Update: Audi eKurzinfo now availabe for Q3, RSQ3 and A3 g-tron

April 10, 2014

Audi lovers, get your mobile devices ready!

InsideAR_11

The latest update for the Audi eKurzinfo application is now available for the Audi Q3, RSQ3 und A3 g-tron and in even more languages such as Portuguese, Spanish, Italian and French.

The new app can recognize over 300 individual elements of the Audi A3 – from the insignia on the windshield wipers and entertainment system to actual engine components under the hood – in order to return relevant how-to information or even virtual overlays of maintenance instructions, animated in real-time and in 3-D. Metaio’s Cloud-based architecture pushes digital information directly to the device.

The design of the Audi eKurzinfo app brings the most intuitive user experience with it: using Metaio’s latest 2-D and 3-D Augmented Reality tracking technology, the user positions the camera of the mobile device directly over the individual vehicle elements, instantly detecting and returning information on the desired subject. For example, after scanning the engine compartment, the app will return information and an animated overlay showing how to locate the engine coolant and refill it to the appropriate level.


Come on baby, light my fire: Light switches in a new dimension

April 7, 2014

The release of the BUSCH-JAEGER light switch application magically projects virtual light switches onto the walls of your home. 

1-titel

Do you want to experience the light switch of your choice on your wall even before it is installed? Now, that’s possible with the new BUSCH-JAEGER light switch app. The new 3D-LiveView transforms a marker tacked to the wall into a deceptively realistic light switch. You can then look at the three-dimensional switch from different angles – amazingly believable. In addition to the 3-D-visualization of the complete BUSCH-JAEGER light switch line, the app also uses location based AR to determine and display sales partners in the area.  2

The marketing team of Q:marketing  has developed this Augmented Reality app with the help of the Metaio SDK.

“With augmented reality applications we offer our clients a new and innovative approach to a more intense and sustainable dialogue with their customers. Our main intention is to deliver digital added value. Metaio and its AR SDK are providing us with the technical framework for bringing our ideas to live.”, says Sascha Wenning, Q:marketing, Head of Department Screendesign & Mobile Development. 

The app is intended to strengthen the brand of BUSCH-JAEGER with an innovative approach to product presentation. The app is available for iOS and Android free of charge. The marker can be downloaded via link within the app or on the BUSCH-JAEGER website.

The application enjoys increasing popularity due to the added value it provides. It will receive further updates and new functions in the future. Up next is a tablet version for iOS.  BUSCH-JAEGER will certainly expand the AR features even further.

4


Weekest Links: First Week of April

March 31, 2014

We have to March into April first

Peter Meier 2

Peter Meier discusses the future of AR in light of the recent Facebook-Oculus purchase.

Metaio | Junaio Recap

#MetaioWearsEpson Sweepstakes! [Augmented Blog]

Virtual Technologies are on the move & Mark Zuckerberg dreams of Augmented Reality [Augmented Blog]

AVK Terway augments Tamaris’ Spring/Summer Collection [Junaio Blog]

Movin’ Klee: A Paul Klee ARt Installation [Augmented Blog]

SLAM: It’s about tracking and mapping your world [Augmented Blog]

Augmented Reality News

One of America’s Largest Hospitals Brings Google Glass to the ER [Fast Company]

Turning the Snake Game into a Location-Based Exergame that Encourages Walking [Youtube]

John Carmack speaks up about Facebook buying Oculus VR [Polygon]

This cool pad and pen lets you sketch in 3D using augmented reality [SPLOID]

Jeri Ellsworth talks castAR’s accidental beginnings and its augmented reality future [Engadget]

Tomorrow’s Cargo Ships Will Use Augmented Reality to Sail the Seas [WIRED]

How Real is Real Time Marketing [Gartner Blog]

From science-fiction to reality — augmented reality that is [Deseret News]

Seebright Headset Creates Smartphone-Powered Virtual Reality [Tom's Guide]

USC is Offering a Google Glass Course for Journalism [Mashable]

Facebook buying Oculus VR for $2 billion [The Verge]

Augmented Reality Is About to Turn Football Into a Real-Life Videogame [WIRED]

Upcoming Events

AWE is just around the corner, will you be attending? [Augmented World Expo]

No webinar this week folks, apologies. Did we mention our giveaway?

Pick of the Week

Even with all the buzz surrounding Facebook’s purchasing Oculus VR and Sony’s Project Morpheus, we can’t help to be excited about the Yelpulus Rift. We’re well on our way to producing some amazing applications for this state of the art device. Look out Oculus and Sony, Yelp’s got your number and they’re coming for you!


Movin’ Klee: A Paul Klee ARt Installation

March 26, 2014

devices_MK

IMG_0485

A group of students from the University of Applied Sciences in Augsburg are using Augmented Reality to expand a series of secret paintings created by Paul Klee while he served as the treasurer’s typist at the Royal Bavarian Flying School in the early- to mid- 1900s.

The students created their own application as part of the Master’s course “Interactive Media” in cooperation with the Paul Klee exhibition “Mythos Fliegen” (“The myth of flying”). They choose eleven different pieces of art from the Klee exhibition to enhance with color and animation, with the goal being that visitors should experience something entirely new while visiting the exhibition and still being able to appreciate the original artwork.

Image Courtesy: Movin'Lab

Image Courtesy: Movin’Lab

The Movin’ Klee application was developed for iOS and Android devices and related animated content is overlaid over images from the real paintings. Due to safety reasons in the exhibition, the AR installation could not be directly integrated into the museum walk through, but is rather set up just outside of the main gallery. The application was well publicized to visitors, and they could even  use the museum’s internet connection to download and install the application on site.

IMG_0484

Movin’ Klee consists of a native application as well as a Junaio plugin that interface with augmented reality forms. The native apps are written in the corresponding programming languages for Android and iOS. The augmented reality part, which is the located on the Junaio-server channels, is developed in the programming languages ​​PHP, HTML, AREL and jQuery. This channel represents a type of browser where the augmented reality objects (animations) are loaded and played.  Britta Diehm, Xiaomeng Jiang, Yue Ma and Kerstin Vierthaler developed the native application as part the project group Movin’Lab.

IMG_8339

IMG_0469

Kerstin Vierthaler, one of the developers, wrote her master’s thesis on the integration of Augmented Reality into museums and came to an interesting conclusion:

“The study of the animated Movin’ Klee application shows that customers remembered the Augmented Reality related paintings better than non-animated pictures, since animations raise the viewers’ attention and perception. Besides the better memorization of paintings, the rather unknown technology itself was very interesting for the visitors. [...] Furthermore, the intuitive usability makes the application accessible for visitors who are not experienced with applications. [...] The evaluation reveals that it is important to inform the visitor about digital offerings within the exhibition. In addition to the technical challenge the museum’s internal integration and communication plays an important role.”

For more information about Movin’ Klee, please visit their website.

 


Weekest Links – Mid March

March 17, 2014

Let’s find four-leaf AR clover

Gearing up to present at the AEC Hackathon. So many cool projects!

Metaio | Junaio Recap

An SAP Warehouse Concept for Error Detection in Warehouse Picking Processes [Augmented Blog]

SunHub – Explore the Sun’s Trajectory [Augmented Blog]

Augmented Reality Software Developer Takes B2B Sales Globally [Internet Retailer]

Augmented Reality Across the World

[AR]e We There Yet? [Storyboard]

Winnipeg Game Developers Creating a New Reality [CBC News]

Skully AR Helmets Win the Wearable Tech Award at SXSW 2014 [Augmented Reality Trends]

Augmented Reality is Blurring the Line Between Mobile Gaming and Reading [Mobile Commerce Press]

Make Friends with a Brazilian, Via Coke’s World Cup Mini-Bottles [Creativity Online]

Epson Showcases Its Augmented Reality Smart Glasses [MCAD Café]

This Is the Future of Retail: Robotic Fitting Rooms and Magic Augmented Reality Mirrors [TheNextWeb]

Measuring Human Motion with a Soft, Wearable Sensors Kit [Digital Journal]

Upcoming Events

Metaio at Laval Virtual [Laval Virtual]

InsideAR Tokyo, Register Today! [InsideAR]

Webinar- Junaio 5.3: The Basics [Register Here]

Pick of the Week

Coca-cola has all sorts of ways to make people smile. In preparation for this year’s World Cup, Coca-cola Brazil has released an app to help you make friends in Brazil using their Coke Minibottles. Create your own personal augmented reality avatar and share with the world. For more information check out the video below.


Leviathan – A Whale of a Tale

February 25, 2014

LEVIATHAN was originally inspired by the steampunk universe of the Scott Westerfeld novel of the same name- an alternate universe where the British empire genetically engineers wondrous whale-like beasts that soar amid hot-air-balloon jellyfish. Though fantastical and as far as one can get from consumer electronics, this was the back drop for what transpired at the 2014 CES Intel booth.

The Cinematic Art World Building Media Lab of the University of Southern California (USC) worked closely with Intel Labs, the innovation arm of the giant chipset company, to bring the world of Leviathan to Las Vegas in a narrative-driven experience that no visitor would forget.

CES attendees and visitors to the Intel Booth were treated to daily shows were they could watch a digital story unfold before them. Starting on the main screen, the giant Leviathan “whale” would swim through the sky, only to turn and head directly for the foreground of the video screen. Visitors were instructed to hold up the Intel Ultrabooks that had been passed out to them and watch as the creature emerged from the video screen and proceeded to drift slowly over the entire audience.

leviathan1

Unlike any other augmented reality experience, every single visitor with an Ultrabook was viewing the same exact content but from their own relative position. It was as if a portal had opened into a different universe, accessible only through the wonder of the participants and the mysterious magic of emerging technology.

It’s clear from Intel’s press release that this is exactly the kind of effect they want to bring consumers in the near future: LEVIATHAN is a new storyworld, powered by Intel technology and USC imagination. We, along with other explorers in the entertainment industry, are building these world and beginning to deliver them to audiences. In five years, will your living room become a portal to these new storyworlds? Will you be able to step inside these worlds like Alice stepping through the looking glass, and explore new kinds of stories. We think so.

Tawny Schlieski of Intel Labs, one of the researchers responsible for launching the experience, said “Intel is exploring this new narrative platform with tools that enable artists and technologists to collaborate, integrate, and create immersively from the earliest inception of story, while giving fans the ability to move and play naturally built virtual worlds.”

Metaio utilized cutting-edge sensor fusion and the latest tracking technology in order to bring this world to reality. In order for each Ultrabook to sync with the augmented content, Metaio developed indoor-position protocols that would enable a virtually seamless user experience.

leviathan3

According to Intel and USC, this is the first of “many storyworlds” the two organizations plan to build and install. Metaio looks forward to working with them again, and hopes to be part of something as unique and immersive as the LEVIATHAN experience in the near future.

For more information on upcoming projects, check out http://digitalrim.org.


Meet us at the Mobile World Congress! #MWC14

February 21, 2014

It’s time for Barcelona!

Kristina_1

On Monday morning the 2014 Mobile World Congress starts and we prepared thrilling demos to showcase the latest innovations in the Augmented Reality business to you – for smartphones, for wearables (Google Glass, Vuzix & Epson glasses), for any possible device. We will show 3-D camera interfaces for augmented reality along with the latest in enterprise and wearable experiences. Our visitors to the Metaio booth in the App Planet, Hall 8.1, Booth 8.1G47, will get see how Augmented Reality is truly “creating what’s next” at the world’s largest mobile conference.

tabletThis year’s Mobile World Congress arrives at the dawn of the 3-D camera. Apple has acquired PrimeSense; NVIDIA has announced partnerships with SoftKinetic for embedded cameras; and Intel has announced the future embedding of RealSense 3-D cameras in Ultrabooks and other devices. Adding depth to interactive mobile experiences will provide all new capabilities in augmented reality, including 3-D reconstruction, dynamic occlusion and brand-new tracking for consumer and enterprise experiences.

“3-D cameras will soon arrive on mobile devices,” said Metaio CTO Peter Meier. “Developers and businesses alike will be able to take advantage of this new technology through the support for 3-D cameras in the Metaio SDK. This year in Barcelona we are showcasing how we have adapted our core technology to support this new wave of 3-D integration, while also demonstrating the power of silicon integration, especially with regard to a wearable future. AR has already shown usefulness and value in both enterprise and consumer sectors, and we will continue to lead the way in innovation for 2014.”

3-D integration opens up numerous possibilities for everyday use: consumers could visualize products in home without the need for markers or printouts; utilize gestures to interact with their devices and the real world; reconstruct or modify entire environments; or even create 3-D models on the fly that could be pushed to 3-D printing devices.

Metaio is already working with companies like SAP and Intel to bring the next generation of augmented reality to developers and users through mobile devices, wearable technology and embedded 3-D cameras.

Intel_games

(Image courtesy: Intel)

“Intel has been collaborating this year with Metaio to showcase the power of combining Intel platforms with augmented reality, which can be seen at MWC 2014,” said Prasad Modali, Intel Principle Engineer. “We look forward to continuing these efforts to further blend the virtual and real worlds in the near future.”

Metaio collaborated with Intel to produce groundbreaking apps specifically for Intel mobile devices, which visitors will experience at the Intel booth, App Planet: Hall 8.1, booth #E41:

  • “Portal Run”: a brand-new digital AR table-top game that utilizes instant SLAM tracking to allow players to play in any environment, anywhere. No board, marker (or even table) necessary.

Visitors to the Metaio booth at MWC will experience:

    • The first hands-free augmented reality maintenance utilizing wearable computing- step-by-step instructions visualized through Epson Moverio BT-200 and Google Glass
  • The new AREngine, the next generation of the first ever AR hardware IP, with 6000% faster initialization and 80% reduction in power consumption on silicon
  • Next-generation AR experiences that utilize 3-D and depth cameras for real-time 3-D tracking and reconstruction
  • The latest in enterprise and consumer demos, featuring experiences from IKEA, Audi, and Mitsubishi Electric

To try out the technology behind the booth demonstrations, sign up for a free Metaio account and download the Metaio SDK at http://my.metaio.com.

If you cannot attend MWC, you have at least the chance to stay up to date with our latest issue of the InsideAR magazine: here.

See you in Barcelona and keep following us on Facebook for pictures from MWC! 


Weekest Links: Mid February

February 18, 2014

Even the Simpsons are getting their AR on. 

Image from Buzzfeed.com

Image from Buzzfeed.com

Metaio | Junaio Recap

Metaio featured in BBC Click and ARTE Futuremag [Augmented Blog]

InsectARium – Beyond a Bug in a Box [Junaio Blog]

OGC, Wikitude, Layar, and Metaio invite Mobile World Congress attendees to AR Interoperability Demo [Wikitude]

Augmented Reality Across the World

Savvy Brands Are Using Apps Instead of Ads to Get Messages Across [Time]

Making Computing More Human with Wearable Tech [Designers of Things]

Beyond Glass: Inside Epson’s Scheme to Make the De-Facto Smart Glasses [SlashGear]

Augmented Reality is a Missed Opportunity for Marketers [The Guardian]

The latest marketing issues blog, brought to you by The Marketing Agencies Association.

Only Genuine Augmented Reality Eyewear App to Deliver Richest Experience for Motorcyclists [Digital Journal]

Augmented Reality Innovators, 6 Companies to Look Out For [Neon Tommy]

castAR rocks the Augmented World [AR Dirt]

Upcoming Events

Mobile World Congress. Come visit us next week in Hall 8.1 booth 8.1G47! [Mobile World Congress]

Webinar –  Continuous Visual Search: A Basic Introduction [Metaio Developer Portal]

Pick of the Week

There isn’t anything as fun as hanging out with the guys from BBC Click. Check out Dan Simmons’s trip to Audi’s research headquarters here and at 14 minutes in the video below.


Weekest Links: February 10th

February 10, 2014

Love is in the AR.

nitro-pos-innen-pic

Wicked Slopes Bro.

Metaio | Junaio Recap

Quantified Reality: How AR is teaching us more about our customers [Augmented Blog]

Augmented Reality Raises GERD Awareness [Augmented Blog]

Nitro Snowboard Co. Spices Up Marketing with Augmented Reality [Junaio Blog]

Webinar: How Will Augmented Reality Change Business in 2014? (Will be up on the Dev Portal soon!) [Youtube]

Interesting Stuff Happening

Perceptual Computing: From Science Fiction to Science Fact [Technology Spectator]

Word Lens adds the Russian language for the Olympics [Android Authority]

Interview with Technical Illusions about CastAR [VentureBeat]

Amazon adds Flow to its Shopping App [CNET]

AR Hackathon in Vancouver [Eventbrite]

Upcoming Events

Mobile World Congress: We’ll be there, want to come with us? [Facebook]

Sorry folks, no webinar this week. Hopefully this will make up for it.

Pick of the Week

I’m pretty sure Indiana Jones was really looking for this Ark. An awesome Augmented Reality projection-mapping project in Oaxaca by AntiVJ. They’ve done some amazing work so be sure to check out their website.


Follow

Get every new post delivered to your Inbox.

Join 4,074 other followers