Technical insight into the 2014 DLD Museum Tour: An Augmented Reality exhibit for the Bavarian National Museum

April 17, 2014

This week we offered once again an Augmented Reality museum walk through together with the Bavarian National Museum in Munich and were overwhelmed by all the attention we got afterwards via different blogs (like in the Huffington Post Germany )and in the social networks. This attention shows that the topic of Augmented Reality is really interesting both for the museums themselves and also for the visitors. Therefore, we decided to give you a more technical insight into our museum project and asked our developers to talk about the development of our application. Here it is:

Alexei, Nicolai & Kevin - Developers of the Museums AR Experience

Alexei, Nicolai & Kevin – Developers of the Museums AR Experience

Hello, my name is Kevin and I am a member of the creative team here at Metaio.  I worked together with my colleagues Alexei and Nicolai to create the assets and code for the DLD Bavarian National Museum AR experience. 

Built in a very short time, the Bavarian National Museum application was a great opportunity to create a valuable cultural experience with AR technology. It aimed to create a balanced user experience between physical and digital content that could inform and entertain the average museum visitor. We  focused on five pre-selected exhibition pieces and  unveiled them as part of the DLD Conference in Munich.

The Metaio Toolbox: Easily creating 3-D tracking maps

The first step to develop the project was to visit the museum itself in order to come up with some ideas and generate our 3-D tracking maps with the Metaio Toolbox .  Many of the assets would be created off-site so establishing an accurate reference that we could use back in the office was important. This was easy to do with the Metaio Toolbox, and within 2-3 attempts we had a satisfactory 3-D tracking model that we could later use in the Metaio Creator . We could also load our point-clouds into Autodesk products for designing more complicated 3-D content by extracting them as an OBJ file from the Metaio Creator.

During our discussions with the museums curators there was concern that the new digital content would take attention away from the physical artifacts, so we set out to create a design that would keep a respectful balance between the cultural artifacts of the museum and the new digital information we were introducing into the environment. To achieve this we kept AR content to the sides of the physical objects, used discreet 3-D white lines as indicators and semi-transparent backgrounds for our buttons and texts.

The AR pieces in detail:

untitled-3756Tilman Riemenschneider: The Mary Magdalene

In the Mary Magdalene scenario we introduced to the user an audio explanation of the piece, some general background information and a photographic overlay that showed the sculpture’s past place of residence (a church altar that was color-corrected in order to better match the lighting scheme of the room).

 

untitled-3776Conrad Meit’s “Judith with the head of Holofernes.”

In the Judith scenario we connected pieces of explanatory text to the model with 3-D white lines. Supported by a particularly strong 3-D map, the experience provided a great sense of depth and space to the user without distracting from the physical object. In order to ensure that the lines were a pleasant shape and length we imported the reference point cloud into Maya before constructing the 3-D lines.

untitled-3788Jakob Sandtner: The Munich City Model

The Munich city model was a real challenge to us because the lighting conditions in the room were very difficult. Due to the sensitive state of some of the historical pieces, strong lights were not allowed in this particular room. This meant that getting a good 3-D map and lining up content to the physical model involved a lot of trial and error. But we managed, and in the end visitors could see an overlaying map of today’s Munich.

untitled-3803Christoph Jamnitzer: The Moor’s Head

The Moor’s head cup contained three nice reference images of the interior and base of the cup which were not viewable to the visitor. In order to display them and not take away from the physical model we created a thumbnail effect that shrank and grew the images when the user tapped on them. This was done by overlaying the images on to a 3-D object and adding a simple on-click animation in the Metaio Creator along with an additional piece of code to allow for a secondary on-click animation.

untitled-3794Hubert Gerhard: Flying Mercury

In the case of Flying Mercury we displayed large images of other artworks created by the artist. They are “floating” around the sculpture.

Balancing AR and non-AR content

AR tablet experiences have a short viewing time in comparison to other media due to the energy required for navigating the physical space. In a museum there are many different people who are enjoying the exhibition at different paces. To create a more fulfilling experience, we needed to support each AR scenario with a non-AR content section: something people could easily switch to while sitting down and relaxing. This non-AR section contained text, audio and video and was accessible through a button on the bottom of the AR viewing screen. It was built by creating an offline webpage that was then integrated into the AR experience.

A developer’s point of view

Nicolai’s thoughts:

This AR scenario was very generous from a developer’s point of view. Its value derives from the simplicity and unobtrusive way of visualizing the additional, well designed content. In this case it means to me that the design and arrangement of the content together with the stable tracking already contributes a lot to this experience. For this AR experience my colleagues finalized the concepts and designed the assets, as well as combined and positioned everything with the help of the Metaio Creator. The effort as a developer was therefore quite manageable.

untitled-3764My part in this was to integrate the designed webpages and connect them to the exhibits. Those webpages functioned as a user interface, enabling the user to see detailed information and multimedia items. The combination of those webpages into one user interface is fairly straightforward. The user interface basically is a single webpage on top of the camera feed. The Metaio AREL bridge allows you to connect to the underlying renderer with the included JavaScript code.

First of all, the information elements had to be connected to the exhibits. This is done by listening to the respective tracking event. When the event is fired, the according HTML elements are shown.  The difference from regular websites is mainly keeping everything within one HTML file. The CSS and JavaScript files can be included there as well. So instead of linking to another page, these other webpage elements were included into the one HTML overlay file. On request, certain elements are hidden or shown. When it comes to best practice, HTML editors and templates can be a great help when designing those HTML elements. Integrating those files for the overlay, however, can be slow due to the produced overhead in including styles and linking in the first place. Within the web overlay you have all the possibilities that HTML5 has to offer. Next to the AREL API this scenario uses standard HTML5 technology like video and audio tags to include even more media content.

In the end, the AREL Technology enables you to easily create slick and effective user interfaces for your AR scenarios.


Weekest Links: First Week of April

March 31, 2014

We have to March into April first

Peter Meier 2

Peter Meier discusses the future of AR in light of the recent Facebook-Oculus purchase.

Metaio | Junaio Recap

#MetaioWearsEpson Sweepstakes! [Augmented Blog]

Virtual Technologies are on the move & Mark Zuckerberg dreams of Augmented Reality [Augmented Blog]

AVK Terway augments Tamaris’ Spring/Summer Collection [Junaio Blog]

Movin’ Klee: A Paul Klee ARt Installation [Augmented Blog]

SLAM: It’s about tracking and mapping your world [Augmented Blog]

Augmented Reality News

One of America’s Largest Hospitals Brings Google Glass to the ER [Fast Company]

Turning the Snake Game into a Location-Based Exergame that Encourages Walking [Youtube]

John Carmack speaks up about Facebook buying Oculus VR [Polygon]

This cool pad and pen lets you sketch in 3D using augmented reality [SPLOID]

Jeri Ellsworth talks castAR’s accidental beginnings and its augmented reality future [Engadget]

Tomorrow’s Cargo Ships Will Use Augmented Reality to Sail the Seas [WIRED]

How Real is Real Time Marketing [Gartner Blog]

From science-fiction to reality — augmented reality that is [Deseret News]

Seebright Headset Creates Smartphone-Powered Virtual Reality [Tom's Guide]

USC is Offering a Google Glass Course for Journalism [Mashable]

Facebook buying Oculus VR for $2 billion [The Verge]

Augmented Reality Is About to Turn Football Into a Real-Life Videogame [WIRED]

Upcoming Events

AWE is just around the corner, will you be attending? [Augmented World Expo]

No webinar this week folks, apologies. Did we mention our giveaway?

Pick of the Week

Even with all the buzz surrounding Facebook’s purchasing Oculus VR and Sony’s Project Morpheus, we can’t help to be excited about the Yelpulus Rift. We’re well on our way to producing some amazing applications for this state of the art device. Look out Oculus and Sony, Yelp’s got your number and they’re coming for you!


Movin’ Klee: A Paul Klee ARt Installation

March 26, 2014

devices_MK

IMG_0485

A group of students from the University of Applied Sciences in Augsburg are using Augmented Reality to expand a series of secret paintings created by Paul Klee while he served as the treasurer’s typist at the Royal Bavarian Flying School in the early- to mid- 1900s.

The students created their own application as part of the Master’s course “Interactive Media” in cooperation with the Paul Klee exhibition “Mythos Fliegen” (“The myth of flying”). They choose eleven different pieces of art from the Klee exhibition to enhance with color and animation, with the goal being that visitors should experience something entirely new while visiting the exhibition and still being able to appreciate the original artwork.

Image Courtesy: Movin'Lab

Image Courtesy: Movin’Lab

The Movin’ Klee application was developed for iOS and Android devices and related animated content is overlaid over images from the real paintings. Due to safety reasons in the exhibition, the AR installation could not be directly integrated into the museum walk through, but is rather set up just outside of the main gallery. The application was well publicized to visitors, and they could even  use the museum’s internet connection to download and install the application on site.

IMG_0484

Movin’ Klee consists of a native application as well as a Junaio plugin that interface with augmented reality forms. The native apps are written in the corresponding programming languages for Android and iOS. The augmented reality part, which is the located on the Junaio-server channels, is developed in the programming languages ​​PHP, HTML, AREL and jQuery. This channel represents a type of browser where the augmented reality objects (animations) are loaded and played.  Britta Diehm, Xiaomeng Jiang, Yue Ma and Kerstin Vierthaler developed the native application as part the project group Movin’Lab.

IMG_8339

IMG_0469

Kerstin Vierthaler, one of the developers, wrote her master’s thesis on the integration of Augmented Reality into museums and came to an interesting conclusion:

“The study of the animated Movin’ Klee application shows that customers remembered the Augmented Reality related paintings better than non-animated pictures, since animations raise the viewers’ attention and perception. Besides the better memorization of paintings, the rather unknown technology itself was very interesting for the visitors. [...] Furthermore, the intuitive usability makes the application accessible for visitors who are not experienced with applications. [...] The evaluation reveals that it is important to inform the visitor about digital offerings within the exhibition. In addition to the technical challenge the museum’s internal integration and communication plays an important role.”

For more information about Movin’ Klee, please visit their website.

 


Weekest Links – Mid March

March 17, 2014

Let’s find four-leaf AR clover

Gearing up to present at the AEC Hackathon. So many cool projects!

Metaio | Junaio Recap

An SAP Warehouse Concept for Error Detection in Warehouse Picking Processes [Augmented Blog]

SunHub – Explore the Sun’s Trajectory [Augmented Blog]

Augmented Reality Software Developer Takes B2B Sales Globally [Internet Retailer]

Augmented Reality Across the World

[AR]e We There Yet? [Storyboard]

Winnipeg Game Developers Creating a New Reality [CBC News]

Skully AR Helmets Win the Wearable Tech Award at SXSW 2014 [Augmented Reality Trends]

Augmented Reality is Blurring the Line Between Mobile Gaming and Reading [Mobile Commerce Press]

Make Friends with a Brazilian, Via Coke’s World Cup Mini-Bottles [Creativity Online]

Epson Showcases Its Augmented Reality Smart Glasses [MCAD Café]

This Is the Future of Retail: Robotic Fitting Rooms and Magic Augmented Reality Mirrors [TheNextWeb]

Measuring Human Motion with a Soft, Wearable Sensors Kit [Digital Journal]

Upcoming Events

Metaio at Laval Virtual [Laval Virtual]

InsideAR Tokyo, Register Today! [InsideAR]

Webinar- Junaio 5.3: The Basics [Register Here]

Pick of the Week

Coca-cola has all sorts of ways to make people smile. In preparation for this year’s World Cup, Coca-cola Brazil has released an app to help you make friends in Brazil using their Coke Minibottles. Create your own personal augmented reality avatar and share with the world. For more information check out the video below.


Leviathan – A Whale of a Tale

February 25, 2014

LEVIATHAN was originally inspired by the steampunk universe of the Scott Westerfeld novel of the same name- an alternate universe where the British empire genetically engineers wondrous whale-like beasts that soar amid hot-air-balloon jellyfish. Though fantastical and as far as one can get from consumer electronics, this was the back drop for what transpired at the 2014 CES Intel booth.

The Cinematic Art World Building Media Lab of the University of Southern California (USC) worked closely with Intel Labs, the innovation arm of the giant chipset company, to bring the world of Leviathan to Las Vegas in a narrative-driven experience that no visitor would forget.

CES attendees and visitors to the Intel Booth were treated to daily shows were they could watch a digital story unfold before them. Starting on the main screen, the giant Leviathan “whale” would swim through the sky, only to turn and head directly for the foreground of the video screen. Visitors were instructed to hold up the Intel Ultrabooks that had been passed out to them and watch as the creature emerged from the video screen and proceeded to drift slowly over the entire audience.

leviathan1

Unlike any other augmented reality experience, every single visitor with an Ultrabook was viewing the same exact content but from their own relative position. It was as if a portal had opened into a different universe, accessible only through the wonder of the participants and the mysterious magic of emerging technology.

It’s clear from Intel’s press release that this is exactly the kind of effect they want to bring consumers in the near future: LEVIATHAN is a new storyworld, powered by Intel technology and USC imagination. We, along with other explorers in the entertainment industry, are building these world and beginning to deliver them to audiences. In five years, will your living room become a portal to these new storyworlds? Will you be able to step inside these worlds like Alice stepping through the looking glass, and explore new kinds of stories. We think so.

Tawny Schlieski of Intel Labs, one of the researchers responsible for launching the experience, said “Intel is exploring this new narrative platform with tools that enable artists and technologists to collaborate, integrate, and create immersively from the earliest inception of story, while giving fans the ability to move and play naturally built virtual worlds.”

Metaio utilized cutting-edge sensor fusion and the latest tracking technology in order to bring this world to reality. In order for each Ultrabook to sync with the augmented content, Metaio developed indoor-position protocols that would enable a virtually seamless user experience.

leviathan3

According to Intel and USC, this is the first of “many storyworlds” the two organizations plan to build and install. Metaio looks forward to working with them again, and hopes to be part of something as unique and immersive as the LEVIATHAN experience in the near future.

For more information on upcoming projects, check out http://digitalrim.org.


Weekest Links: Mid February

February 18, 2014

Even the Simpsons are getting their AR on. 

Image from Buzzfeed.com

Image from Buzzfeed.com

Metaio | Junaio Recap

Metaio featured in BBC Click and ARTE Futuremag [Augmented Blog]

InsectARium – Beyond a Bug in a Box [Junaio Blog]

OGC, Wikitude, Layar, and Metaio invite Mobile World Congress attendees to AR Interoperability Demo [Wikitude]

Augmented Reality Across the World

Savvy Brands Are Using Apps Instead of Ads to Get Messages Across [Time]

Making Computing More Human with Wearable Tech [Designers of Things]

Beyond Glass: Inside Epson’s Scheme to Make the De-Facto Smart Glasses [SlashGear]

Augmented Reality is a Missed Opportunity for Marketers [The Guardian]

The latest marketing issues blog, brought to you by The Marketing Agencies Association.

Only Genuine Augmented Reality Eyewear App to Deliver Richest Experience for Motorcyclists [Digital Journal]

Augmented Reality Innovators, 6 Companies to Look Out For [Neon Tommy]

castAR rocks the Augmented World [AR Dirt]

Upcoming Events

Mobile World Congress. Come visit us next week in Hall 8.1 booth 8.1G47! [Mobile World Congress]

Webinar –  Continuous Visual Search: A Basic Introduction [Metaio Developer Portal]

Pick of the Week

There isn’t anything as fun as hanging out with the guys from BBC Click. Check out Dan Simmons’s trip to Audi’s research headquarters here and at 14 minutes in the video below.


DLD Conference and Augmented Reality Experts at Metaio presented interactive Museum Tour in Munich

January 22, 2014

In collaboration with the Digital Life Design (DLD) Conference taking place this week in Munich, we presented a guided and interactive tour through the Bavarian National Museum. On January 19th DLD visitors and journalists joined us for the chance to see live how Augmented Reality builds a bridge between historical pieces of art and the digitalisation of our everyday lives.

DLD visitors were able to experience Augmented Reality with mobile devices and even a pair of Google Glass to get additional information about the art pieces on show. Five exhibition pieces from different periods and artistic styles were augmented with rich virtual information via the latest 3D object tracking technologies from Metaio.

untitled-3803untitled-3788untitled-3794untitled-3776

For us, the most amazing piece of art was the St. Maria Magdalene statue: Legend has it that Maria Magdalene lived for years as a penitent in the desert clad only in her long hair until her entire body became hirsute. In the 18th century, objections around her were raised and she was removed from the famous Riemenschneider altar in 1756. We wanted to show St. Marie Magdalene in her original setting so we reconstructed a virtual altar visible around her. And the result is amazing! 

untitled-3756

DLD Panel Discussion: “From Museums to Playstations”

On Tuesday morning, Dr. Thomas Alt, CEO and Co-Founder of Metaio, joined  curator and museum director Max Hollein from Frankfurt as well as Sophia George, an artist from Great Britain,  on a panel at DLD Conference to discuss the possibilities of integrating Augmented Reality in museums. “Building virtual exhibitions doesn’t do the trick, ” Thomas Alt said: “What Metaio is trying to do, is to teach mobile devices to recognize the environment. The idea behind is to create context for the user in a real environment and to offer them additional content.” He also emphasized that Augmented Reality cannot replace the real object in a museum, but it can actually enrich the visitor’s personal visit and experiences.  

DLD Conference already celebrated ten years of bringing together the most influential opinion-makers, industry leaders, start-ups and digital giants in Munich. We were very proud to join them as partners this year and want especially thank Steffi Czerny (the very charming organizer of the DLD Conference), her great team and in particular the National Bavarian Museum to realize this idea so fast! 

untitled-3584 untitled-3575 untitled-3596 untitled-3599 untitled-3629 untitled-3696 untitled-3708 untitled-3760


Happy Augmented Holidays from Metaio!

December 23, 2013

Before everyone runs off for the holidays, we decided to send out a little holiday cheer! No, it’s not eggnog. Instead, we put together a little piece to highlight one way augmented reality can bring everyone closer this time of year. We’re happy to present: Happy Augmented Holidays from Metaio!

And here’s this little gem in case you haven’t seen it.

Happy Holidays!


Xcope – Creating an Immersive AR|VR Experience

November 29, 2013

Smart devices . . . smart devices everywhere.

With all the talk of smart glasses and smart watches and smart bras one can’t help but wonder what might happen to their predecessor: smart phones. Sure, phones have been around for years and having the most advanced, smartest phone is possibly the most technologically trendy device a person could own. But something designed specifically for your hands can’t be used anywhere else . . . could it?

A New Challenger Approaches . . .

This time let go of your conscious self and act on instinct.

Meet Xcope, the Virtual & Augmented Reality Headset for Smartphones designed by Xyson Labs. Xcope is a headset specifically designed for smartphones to create an immersive augmented or virtual reality experience with just your smartphone.

Borrowed a little visual.

Xcope works by slipping your smart phone into a fitted slit which goes over your eyes much like a pair of goggles. Your field of vision is limited to whatever you can see on your device: game, movie, video, whatever you’re engaged with. The monoscope optics give a theater-like feel with a lens designed to improve screen size and quality making it great for watching shows or using AR channels (*Junaio*). The stereoscope optics separates the your eyes’ vision and shifts their perspectives slightly to create 3-D images for 3-D movies and virtual reality. Add a bluetooth controller or another input device and you’re set for a full VR experience.

Unlike Google Glass, castAR and other smart glasses technologies, Xcope takes an already powerful, widely used technology – smart phones – and adapts them for similar results. Of course there are obvious drawbacks but  unlike other phone headsets, Xcope is specifically designed for developers so they can eventually take the technology where they want, especially for the stereoscopic fitting. Imagine adding eye tracking, bluetooth controllers and head motion sensitive tracking to your favorite game and suddenly you’ve got developers doing something even more amazing. Interested? Check out their website and Kickstarer page for more information.

Double, double toil and trouble.


Vanatur and WARNER present AURYN’s Augmented Reality Debut

November 20, 2013

I’m singing in the rain.

plantilla fan edition.fh11

The Metaio SDK hits the Spanish pop scene with boy band AURYN and their new self titled app. Developed by Vanatur World Mobile Solutions, the app comes with five main features – Social, Videos, Radio, an amazing Augmented Reality Experience and + Auryn – to fully immerse fans in everything AURYN.

Vanataur says about the AR function: “The Augmented Reality functionality will give fans even more surprises over time, connecting the online and offline world with “Gamification” strategies. The app is able to recognize certain images caused by actions such as video streaming, links to special content, etc.”

Besides providing an easy way to access updates from AURYN, the app takes the group’s ANTI-HEROES album cover along with various posters and pictures published to their social networks and embeds exclusive content only accessible through the AURYN app.

Image Courtesy of Vanataur

Image Courtesy of Vanataur

The app is free to download on Google Play and iTunes. For a peak at one of the AR exclusives use the AURYN app on the photo above or check the band’s promotional video below.

Every real story is a never ending story.


Follow

Get every new post delivered to your Inbox.

Join 4,074 other followers