Augmented Reality: A New Vision

Abstract

Augmented Reality (AR) is a technology that combines computer generated and real world data by integrating real objects or space with virtual contextual information. As a technology, AR has been around for over 30 years. According to Scientific American, the term Augmented Reality was first coined in 1990 by a scientist at Boeing where they developed a prototype solution to help workers construct wiring harnesses. Applications are being developed in the medical, military, manufacturing, and entertainment industries that will improve productivity and processes via high-tech innovation. As production costs fall and technology improves, the adoption of AR will have a profound effect on our society; the way we interact with computers, one another, and the way we learn and work.

Introduction

Augmented Reality (AR) is a specific example of what Fred Brooks called Intelligence Amplification (IA): using the computer as a tool to make a task easier for a human to perform (Brooks, 1996). An AR user such as a technician or mechanic might wear a Head Mounted Display (HMD) through which he could see the real world, as well as computer-generated schematics and repair information projected on top of what he is seeing in real time (see Figure 1.). An augmented reality system is one that combines real and virtual, is interactive in real time, and is registered in 3D. AR attempts not only to superimpose graphics over a real environment in real-time, but also change those graphics to accommodate a user's head- and eye- movements, so that the graphics always fit the perspective. Three components needed to make an augmented-reality system work are head-mounted displays, tracking systems, and mobile computing power (Bonsor, n.d.). Over the past decade, there has been a boom in AR research as hardware costs have fallen enough to make the necessary lab equipment affordable.

Figure 1- Optical see-through HMD conceptual diagram (Azuma, 1992).

Image Guided Surgery (IGS) is the typical application area where virtual objects and real must be merged into a single unified scene, calling for Augmented Reality (AR) techniques. As minimally invasive surgeries gradually phase out open invasive surgeries, IGS systems will experience greater uptake among surgeons. Pre-operative imaging studies, such as CT or MRI scans, provide the surgeon a view of the internal anatomy to help better plan the surgery. Augmented reality can be applied so that the surgical team sees the CT or MRI data correctly registered on the patient in the operating room while the procedure is progressing. From these images the surgery is planned by creating a 3D model from the multiple views and slices in the pre-operative study (see Figure 2.). AR can be applied so that the surgical team can see the CT or MRI data displayed on the patient in the operating room while the procedure is ongoing (Vallino, 1998). These devices provide surgeons an unbeatable advantage by aiding real-time navigation and offering a 3D virtual representation.

Figure 2 - Virtual fetus inside womb of pregnant patient.

(Courtesy UNC Chapel Hill Dept. of Computer Science.) (Azuma, 1992).

Being able to accurately register the images at this point will enhance the performance of the surgical team and may eliminate the need for the painful and cumbersome stereotactic frames currently used for registration (Mellor, 1995).

How it Works

IGS systems are comprised of several features: a tracking device or camera; the hardware and software components; and a registration device, which is there to align the patient’s image with preoperative images. The most important step is registration. A special probe fitted with infrared-light-emitting diodes (LEDs) is used to teach the computer the exact location of the patient. The camera tracks the LEDs emitted by the probe as the surgeon touches different points on the patient's spinal anatomy. This builds a one-to-one relationship between the computer and patient. During surgery, the patient's anatomy moves, such as during breathing. The connection between patient and computer established during registration to automatically update images each time the anatomy moves. Throughout the surgery, the computer can generate images of the patient's anatomy as it relates to the surgeon's instrument.

Benefits

There are many significant benefits to implementing an IGS system. Image-guided procedures are substantially less invasive than traditional surgery. Because the image-guided technology is so precise and accurate, surgeons can decide how best to get to a targeted area and avoid healthy tissue before an incision is ever made. This ability allows surgeons to perform fewer revision procedures, because they’re able to achieve pinpoint accuracy the first time. This accuracy creates alluring incentives for hospitals that are looking to maximize reimbursement.

Hospitals that invest their money in image-guided surgery (IGS) systems are wagering on cost savings related to more accurate surgeries and briefer recovery times for patients. The potential payouts rely on everything from fewer malpractice suits to increased revenues from previously inoperable conditions.

The benefit for the patient is that, because it is minimally invasive, there is a smaller incision and the surgery is shorter and safer. Because of the precision that image-guided surgery technology provides, surgeons are able to create an exact, detailed plan for the surgery вЂ" where the best spot is to make the incision, the optimal path to the targeted area, and what critical structures must be avoided. It also means that patients with conditions considered inoperable in the past now have an option (Fisk, 2004).

Challenges

One of the major challenges in the past for AR systems was how to combine the real world and virtual world into a single environment while maintaining the user’s illusion that the virtual objects are part of the real world, a process called registration. If the real world and virtual objects are not properly aligned with respect to one another, the illusion that the two coexist will be compromised. Other areas that caused problems in earlier AR systems were a lack of reference to global coordinates in the Virtual Reality Modeling Language (VRML) standard, lack of wireless internet connectivity, and lack of mass marketing. (Bonsor, n.d.)

Additionally, there

...