AR System Interacts with Virtual Organ Replicas

A novel augmented reality (AR) system allows users to manipulate, navigate, and demonstrate patient data using customized 3D virtual models and the patients' electronic medical record (EMR).

Under development by Birmingham City University (BCU; United Kingdom), the system is based on motion detecting sensors and freehand interaction to create a more realistic experience in virtual mixed reality environments, bridging the gap between users and technology. Using freehand motions and gestures, users can visually demonstrate medical problems, the areas where surgery will be conducted, improvements that could be made following treatment, or the damage caused by harmful addictive substances such as tobacco.


It could also assist practitioners to view patient data in an array of settings and showcase customized medical models, procedures, and treatment effects. Surgeons would also be able to interact with images of the patients' bodies to view and manipulate them during procedures, without the need to remove their scrubs and gloves in sterilized environments. According to the researchers, in the future the system will be upgraded to replicate injuries, mobility problems, or illnesses, and show changes that could be made through lifestyle choices or medical procedures that can be shared with the patient.

“We are developing this system as a platform to allow medical professionals to interact with genuine patient data and manipulate it by hand to educate and inform patients,” said Ian Williams, PhD, of the BCU Digital Media Technology (DMT) lab. “The real advantages this brings are being able to visually demonstrate parts of the anatomy, using virtual models which can be customized for each patient and show how they have been impacted by lifestyle choices, or how they may be changed following treatments or surgery.”

AR is a term that describes live direct or indirect view of a physical, real-world environment whose elements are augmented by computer-generated sensory input such as sound, video, graphics, or GPS data. It is related to a more general concept called mediated reality, in which a view of reality is modified--possibly even diminished rather than augmented--by a computer. As a result, the technology functions by enhancing one’s current perception of reality. By contrast, virtual reality (VR) replaces the real world with a simulated one.

Related Links:
Birmingham City University