Augmented reality (AR) is a term for a live direct or indirect view of a physical real-world environment whose elements are merged with (or augmented by) virtual computer-generated imagery – creating a mixed reality. The augmentation is conventionally in real-time and in semantic context with environmental elements, such as sports scores on TV during a match. With the help of advanced AR technology (e.g. adding computer vision and object recognition) the information about the surrounding real world of the user becomes interactive and digitally usable. Artificial information about the environment and the objects in it can be stored and retrieved as an information layer on top of the real world view. The term augmented reality is believed to have been coined in 1990 by Thomas Caudell, an employee of Boeing at the time.
Mobile augmented reality (AR) brings computer-generated multimedia into an end- user’s literal field-of-vision. It merges real-time digital information with the user’s perceptions of their immediate physical surroundings. The mobile AR user simultaneously experiences physical reality and digital media consumption.
There are two main classes of mobile AR: marker-based and markerless. In the first instance of marker-based mobile AR, the handset’s camera captures an image (“marker”) that might be a 2-D barcode or a specific geometric icon. The mobile AR client senses the marker through image recognition technologies and uses that data capture to pull information from the system to display it. Museum exhibits in which there an icon or code in the corner of the painting that unlocks information about the artist is a familiar example of marker-based mobile AR.
A second category of mobile AR focuses on markerless systems. In this case, the mobile AR client senses its target object or location through location technologies rather than image recognition technologies. Instead of “seeing” the object or location, the mobile AR client uses latitude/longitude/altitude technologies to know where it is.
Then the mobile AR client uses magnetometers (e.g. compass) to understand in which direction it faces the target. Finally, the mobile AR client uses accelerometers, inclinometers and other sensors to understand its orientation (distance/rotation/tilting, also known as roll/pitch/yaw) to the object or location.
Mobile Augmented Reality uses location technologies to orient a smartphone’s spatial relationship to physical objects and geographic points in order to create a real-time, blended output of the physical world and digital media accessible via the smartphone’s camera.
At the time of publication, the best level of accuracy a mobile user can expect is around 10-20 meters from the target location or object based on standard GPS and compass functionality in today’s smartphones. Many mobile AR browsers therefore must apply additional processing and filtering to error-correct geolocation data because raw GPS/compass data coming from the handset can make the user experience wobbly or, worse, dead wrong.
Currently, there are four main categories of mobile AR applications: navigation, location overlays, geo-information services, and gaming.
In the case of mobile AR navigation, the goal isn’t to replace existing mobile mapping services like Google Maps or MapQuest so much as to help users find what they want more efficiently.
A second category of mobile AR apps related to navigation involves overlaying locations with objects (2D and 3D) or information services. Monocle is a mobile AR app for iPhone 3GS that overlays Yelp member reviews onto buildings as seen with the camera.
Other mobile AR apps are expanding the remit of geo-location beyond finding the near tube or providing restaurant reviews to creating geotagged information services.
Wikitude is an iPhone 3GS and Android app created by Austria-based Mobilizy.
A fourth category of mobile AR apps focuses on gaming. Not surprisingly, this area has seen some of the most development work. At the E3 conference in June 2009, Sony gave a demo of a Playstation Portable game called Invizimals. Targeted at kids, the game uses the camera on a PSP to capture special images that, when printed out and scattered on the floor, activate on-screen 3-D animals for players to trap.
If you ask most long-time AR technologists, they’ll probably tell you that mobile AR is not ready for prime time. According to some experts, the required location accuracy for aligning digital information with a real world location isn’t close to what’s needed technically for a seamless mobile AR experience.
If mobile AR 1.0 can address some of its current challenges with location accuracy, while growing its audience during 2010, people will care less about its early days of placing a 3-D T-Rex on top of a landmark.