Telementoring System Supports Surgeons in the Battlefield

A novel augmented reality system could provide effective support to surgeons on the battlefield or other remote locations from specialists located thousands of kilometers away.

Developed by researchers at Purdue University (Lafayette, IN, USA) and Indiana University School of Medicine (Indianapolis, USA), the System for Telementoring with Augmented Reality (STAR) integrates mentor annotations and illustrations directly into the field of view of the surgeon, using a simulated transparent display created by using two networked conventional tablets. Mentor tablet annotations thus remain anchored to the surgical field even when the trainee tablet moves or as the surgical field deforms or becomes occluded.


While existing telestrator-based surgical telementoring systems require a trainee surgeon to shift focus frequently between the operating field and a nearby monitor to acquire and apply instructions from a remote mentor, the STAR superimposes annotations directly onto the surgical field using the transparent tablet. Ongoing research is now focused on improving the robustness of annotation anchoring and the transparent display simulation fidelity. For example, since the surgeon's hands are between the camera and the surgical field, they can momentarily obstruct the mentor's view. An algorithm could render the surgeon's hands semi-transparent.

The researchers are now testing the system while performing procedures commonly carried out in operating rooms, such as cricothyrotomy, in which a tube is inserted into the throat to establish an airway; laparotomy, in which an incision is made in the abdomen to examine internal organs and structures in the abdomen; and fasciotomy, a limb-saving procedure that involves cutting the fascia, a layer of fibrous connective tissue surrounding muscles, to relieve pressure. The study describing the STAR system was published in the October 2015 issue of the Visual Computer.

“Telementoring is widely used, but it's still primitive in that it has not kept pace with advances in information technology and computer graphics,” said senior author associate professor of industrial engineering Juan Pablo Wachs, PhD, of Purdue University. “It is usually done with a telestrator displaying a video of the surgery overlaid with graphical annotations, which requires the surgeon to look away from the operating table while receiving mentoring advice.”

“Optimal surgery and trauma treatment integrates different surgical skills frequently unavailable in rural and field hospitals, and telementoring can provide the missing expertise,” added study coauthor Prof. Gerardo Gomez, MD, director of the division of trauma, surgical critical care, and emergency surgical services at the Indiana University School of Medicine. “Two primary applications are surgeries in the battlefield and in rural regions where specialists might not be available.”

Related Links:
Purdue University
Indiana University School of Medicine