Thermal Touch: The Future of Wearable User Interfaces

June 3, 2014

GAL-thermal-touch-004Have you ever worn a wearable headset? Have you tried Google Glass or any of its myriad competitors? After the initial (and deserved) sense of wonder and awe wore off from perceiving digital and virtual content overlaid on the real world, have you found yourself strangely frustrated at just using the device itself?

You wouldn’t be alone – smartphone companies (Apple chiefly among them) have labored diligently to irrevocably addict you to touchscreens and touch interfaces. Not unkindly, touchscreens have largely replaced mobile keyboards and are largely to thank for the meteoric rise and massive adaptation of smart devices. But insidiously secreted away amid marketing language and shiny rectangles is the sentiment that gestures like “pinch-to-zoom” and “swiping” are only natural- if not pure instinct.  GAL-thermal-touch-003

Imagine an iPhone- without a touchscreen. Imagine a tablet, and no amount of swiping or pinching will allow you to manipulate its contents. This is the reality of wearable computing and augmented reality devices – they’ve removed the necessity of touch. But then how to use an application more than passively? How to navigate to a different screen?

chess-5885Wearable augmented reality devices rely on vision to display content. There are already forays into voice navigation (along with infuriating buttons and swiping motions on the glasses themselves; some clever companies are utilizing Low Bluetooth Energy to pair companion smartphones or new devices like the Enimai “Nod” companion “ring” to activate in-app features. So what then – projectors, 3-D cameras for “finger tracking”? It’s hard to imagine a future where everyone is wearing AR glasses while obnoxiously yelling commands and waving their hands around in front of their faces or furiously trying to dial phone numbers on their hands.

Okay- so it’s not that difficult.

But what if we could bypass all of that? What if we could use camera technology to get even cleverer with reality interaction? Enter Thermal Touch – a technology that will enable interaction with nearly any object or surface.

Thermal Touch – Turning your whole world into a touchscreen

Pitch_picThermal Touch is a radical new approach to wearable headset graphical user interfaces (GUIs). It utilizes infrared cameras to register and track minute thermal imprints left by the heat signature of a finger. Touch your desk – you’re leaving imperceptible (and impermanent) heat maps each time your finger touches the surface. Combining a thermal camera with a normal camera, and developing AR tracking in conjunction with thermal heat tracking, Metaio can now turn anything into a touchscreen.

Trak Lord, Head of Metaio US Marketing, sat down with Daniel Kurz, lead Metaio R&D Engineer and creator of the Thermal Touch prototype, to talk about the future of human-computer interaction.

Daniel_KurzWhere did the idea for Thermal Touch originate?

It was happy coincidence that we got our hands on a thermographic camera and played around with it in the lab. Our R&D team had already been tasked with developing natural and intuitive ways of interacting with Augmented Reality applications when using head-mounted displays. After measuring the temperature of my coffee mug and my display, and after discovering interesting temperature patterns in my face, I noticed that wherever I left my hand resting on the desk, residual heat would become apparent in the thermal image. Brief experiments with different objects showed that this is not a unique property of my desk but most objects exhibit warm spots after touching them. The camera module further included a visible light camera, which allows recognizing and tracking objects in its field of view. Putting one and one together, this is how the idea arose that the combination of detecting touches in the thermal image and detecting and tracking the touched objects in the visible image would enable a natural way to interact with those objects and digital information associated to it – particularly for wearable headsets.

Can you describe how we built the prototype?

Our mobile prototype is based on a tablet PC to which we attached a combined thermal and visible light camera module. The fixture is simply a joist hanger I bought at the next do-it-yourself store. Our proof-of-concept software implementation is based on the Metaio SDK and therefore features the latest tracking capabilities for dealing with both planar and three-dimensional objects. It further provides the functionality to render virtual objects registered with the tracked objects. We had to extend the Metaio SDK to support capturing images from the thermographic camera and I developed a prototypical touch detection algorithm. All in all it wasn’t really that much work, because most pieces already existed in our SDK. The last thing to do was then creating some exemplary applications to demonstrate the versatile opportunities this technology offers in different use cases.

Ideally, what will Thermal Touch look like in the future? How many years are we from embedded infrared cameras?

This new way of interacting with Augmented Reality is clearly meant for wearable computers and head-mounted displays. These devices become increasingly important not only in the context of Augmented Reality, and as they do not have touch screens and they leave the hands free, our technology is a perfect fit. We keep working on improving our prototype in terms of robustness and latency and we are looking into how this fundamental approach can allow more advanced interaction techniques. For example, touching an object with different fingers might have different effects. Of course, it will take a couple of years until the first head-mounted devices will include a thermographic camera. But the current trend clearly is that these cameras become available at a small form factor and an affordable price. A mobile phone add-on enabling mobile thermal imaging will become available this year, and this is only the beginning. Once wearables are really being used ubiquitously, their hardware should be ready for Thermal Touch.

Though it may be years ahead in the future, embedding infrared cameras into wearable computing is not beyond the realm of possibility, especially in an industry that is still iterating on form factor and hardware, let alone the ideal graphical user interface.


City Visions – An Augmented Reality Developer Challenge

April 30, 2014

This year the Augmented World Expo will be hosting an open challenge for augmented reality developers: City Visions. The goal of the challenge is to get developers to think about the future of an augmented urban environment. Developers are encouraged to use their imagination to and design an augmented reality experience focused architecture, transportation, and art in a future city. Expect to use the mural below as the main tracking target.

awe_cityVisions_2014_v1

Rules, Guidelines and Submissions can be found here. Any questions or suggestions can be sent to heavy[at]theheavyprojects.com. Best of luck!

 


Weekest Links: Mid-April

April 21, 2014

Augmented Easter Eggs, Augmented Easter Eggs everywhere

samsung-earphone-diagram-from-patent-application

Image courtesy of EE Times

Metaio | Junaio Recap

Bringing history and the future to life with augmented reality [Phys.org]

Technical insight into the 2014 DLD Museum Tour: An Augmented Reality exhibit for the Bavarian National Museum [Augmented Blog]1

Happy Easter to all of you! [Junaio Blog]

Argyle Social shuts down, Metaio bids farewell to an amazing service [Augmented Blog]

Leviathan – A Whale of a Tale [iQ]

Into the Wild: Metaio goes Re:publica [Augmented Blog]

How to Leverage AR and Multimodal Interaction in Your Mobile Projects [SAP Community Network]

Red Cell White Cell – An AR Book [Junaio Blog]

Update: Audi eKurzinfo now available for Q3, RSQ3 and A3 g-tron [Augmented Blog]

AR Technology of the Week: Augmented Reality SDK for Virtual Shopping by Metaio [Smart Reality]

Land Rover – Augmented Cars in a New Way [Augmented Blog]

Shopping on the run: The new Decathlon brochure uses Augmented Reality [Junaio Blog]

3 Reasons to Work for Metaio – Impressions for a “Newbie” [Augmented Blog]

Augmented Reality News

Ghostman Augmented Reality System Lets You Learn from a Teacher’s Perspective [Technabob]

BMW vision future luxury integrates augmented reality display [Designboom]

How a Mobile Device Can Save Your Life: Augmented Reality Surgery [iQ]

Nike fires majority of FuelBand team, will stop making wearable hardware [CNET]

Mashing-Up Augmented Reality and Cutting-Edge Technology [Manufacturing.net]

Samsung Patent Leaks Point to Google Glass Competitor [EE Times]

Toyota Has A Tron Bike [Popular Science]

Leviathan: The Future of Storytelling [The Creators Project]

Land Rover’s ‘invisible bonnet’ technology [The Telegraph]

Upcoming Events

Metaio at Re:publica 14 [Re:publica]

Pick of the WeeK

With the Augmented World Expo coming just around the corner, everyone is getting geared up to see all the new augmented reality technologies. If you’re interested in seeing where augmented reality will go in the future check out AWE: NY’s The Future of Augmented Reality Panel. Steve Feiner, Ken Perlin, John Havens, and Trak Lord provide some very insightful commentary about augmented reality its future.


Weekest Links, End of March

March 24, 2014

Welcome to the winning March Madness Bracket. Occupation: not us. 

 

Trak visits  Seebright at GDC.

Metaio | Junaio recap

The Case for Wearable Computing [Augmented Blog]

Metaio releases newest version of Junaio AR Browser with new design, real-time POI visualization and browser interoperability [Augmented Blog]

Metaio releases newest version of the Junaio AR Browser [CIOL]

Metaio at hy! Summit (See Episode 2, Minute 18) [hy! Summit]

Junaio Augmented Reality Browser Revealed [Ubergizmo]

Augmented Reality World News

Pugwash: Augmented reality can be both useful and dangerous [The Tartan]

Augmented Reality and Cloud Gaming [Cloud Tweaks]

“UK’s first” augmented reality property app raises $500,00 [Startups.co.uk]

10 Forthcoming Augmented Reality & Smart Glasses You Can Buy [Hongkiat.com]

Augmented Reality Is About to Turn Football Into a Real-Life Videogame [Wired]

Seebright Reveals Industry’s First Smartphone Integrated AR/VR Head-Mounted Display Platform With Wireless Controller [PR Newswire]

Headset combines virtual and augmented reality with holodeck-like results [Engadget]

Winnipeg company goes for gold with video game [CBC News]

Google reveals Android Wear, an operating system for smartwatches [The Verge]

Upcoming Events

Metaio at Augmented World Expo: NY [AWE-NY]

Webinar this week – Metaio Trivia: Technical Edition [Register]

Pick of the Week

Pepsi delivers its own augmented reality experience, only it’s at a bus stop in London. The AR bus shelter uses some clever camera work to surprise people and they get some fun results. Enjoy!


Weekest Links, Solstice Edition

June 24, 2013

*Translated from Japanese.

Augmented Penguins with Junaio

It seems AR navigation app that penguins us to guide to Sunshine Aquarium. I want to use!* [article]

Metaio | Junaio Recap

  • Metaio wants to bring Augmented Reality everywhere [Mashable]
  • Trak Lord discusses the impact of AR on everyday life in this short documentary on the Augmented Reality industry [video]
  • What’s up with Metaio? [Wired - Beyond the Beyond]
  • Augmented Reality as a maintenance and service tool for BOSCH [Junaio blog]
  • Augmented Reality made easier with the Metaio Creator [Moosylvania]

Events

  • If you’re in San Francisco this week, don’t miss Metaio at the 2013 Mobile Commerce World [event]
  • Check out Metaio at the 2013 SIGGRAPH conference at NVIDIA’s theater, July 23 [event]
  • There’s no place like InsideAR, there’s no place like InsideAR, there’s no place like InsideAR [Oct 10-11]

More AWE 2013 stuff

  • Metaio CTO Peter Meier presents on the AREngine and the importance of form factors [video]
  • Director of Metaio R&D Ben Blachnitzky talks about the Metaio SDK at AWE 2013 [video]
  • Jacob Ervin of Metaio discusses proper content management at AWE 2013 [video]
  • Live demo of Clandestine Anomaly, an amazing augmented reality game from Zenfri built on the Metaio SDK and Unity [video]
  • Founder of Zenfri, Corey King’s presentation at the 2013 Augmented World Expo [video]
  • Gabe Weiss of Mitsubishi Electric presents the most outstanding enterprise application of AR, powered by Metaio [video]
  • Naomi Kawase of NGRAIN showcases their 3-D industrial visualization solution, powered by Metaio tracking & recognition [video]

There are things that are happening

  • In the shadow of Google Glass, an Augmented Reality industry revs its engines [The Verge]
  • Could Google Glass be the next iPhone? [CNET]

Pick of the week: Infinity AR Concept Video

Remember that creepy dystopian-future-of-augmented-reality video, Sight? This is like that, but through rose-colored glasses.

Congrats to Helen and her team on the 1.5M views. 


Weekest Links 514 – 5/20

May 20, 2013

Don’t be a glasshole

AVK Terwey Zoo Monster Augmented Reality project with Junaio

Zoo Monster: An Augmented Reality Project with Junaio, by Certified Developers AVK Terwey [video]

Metaio | Junaio

  • Vote for the Metaio SDK, the Mitsubishi Enterprise MeView, The IKEA Catalog and the AREngine in the 2013 Augmented World Expo Auggies Competition! [Vote]
  • Developers: Sign up for this week’s webinar, Augmented Reality Content Creation [sign up]
  • Even more stuff for developers: Submit your intent to compete in the first-ever Metaio International Developers Contest, with grand prizes including software licenses and a trip to Munich to present at the 2013 InsideAR [contests]
  • Intelligent Computing meets Augmented Reality: Brutus guides you through Ohio State University [article]

There are things that are happening

  • 7 Standout Google Glass apps you can download right now [Mashable]
  • Forget Google Glass: Recon debuts Android-friendly glasses at I/O [CNET]
  • Epson tries its hand at high-tech glasses [USA Today]
  • Meta 1 augmented reality headset launches on Kickstarter [Slashgear]
  • How two Valve Engineers walked away with the company’s augmented reality glasses [The Verge]

Events

  • We like to party- are you in the Silicon Valley area on June 3? Come to an Augmented Evening with Metaio (tickets are limited) [registration]
  • Join the Ninth annual AR standards meeting at Columbia University, May 29-30 [event site]
  • Interested in speaking, sponsoring or attending InsideAR, the most exciting event in the augmented reality industry? Contact us! [event site]
  • Meet Metaio at the Augmented Reality Summit, June 20th in London [event site]

pick of the week: Google Glass from the eyes of the toddler

“Lego stuff…and blocks and stuff….”


Vote For Metaio for the 2013 Auggies!

May 17, 2013

Sorta reminds me of this.

It’s that time of year again, as the Augmented World Expo has renewed and expanded the annual award for the best of the best in the augmented reality industry, the “Auggies”. Previously consisting of only a single award, the competition now has 6 categories, and Metaio and its partners are competing in 4 of them!

For the next two weeks, awards will be open to public voting in order to narrow down the entries for each category to just 5 finalists. We’re competing against some of the best and most creative apps in the world, we’d love your support. Please take a moment to vote for our entries (individual links are listed below), or just click on one of the category buttons below to go straight to the entry.

    

And while you’re at it, we’d love for you to support our amazing partners working with our technology:

Here’s a look back at the our winning submission from the 2012 Auggies, 3-D object recognition and tracking:

Once again, thanks for your support in making Metaio the best Augmented Reality Company out there!


Follow

Get every new post delivered to your Inbox.

Join 5,057 other followers