Touchless Technology Enters the Operating Theater

A pioneering technology enables surgeons to view, control, and manipulate medical images in the operating room (OR) without contact.

Researchers at King’s College London (KCL; United Kingdom) and Guy’s and St Thomas’ NHS Foundation Trust (London, United Kingdom), in conjunction with Microsoft Research (St Thomas’, Cambridge, United Kingdom) and other institutions, developed the software for the imaging surgery system to help surgeons during complex aneurysm procedures. The gesture-based system uses Microsoft Kinect for Windows hardware and the Kinect for Windows Software Development Kit (SDK), to allow the vascular surgery team to maintain a sterile environment, whilst being able to view and manipulate medical images through a combination of gesture and voice control.


In the operating room, the setup consists of a Windows PC connected to a fluoroscopy scanner through an external video-capturing device, a Kinect for Windows sensor attached to the PC with a USB cable, and a monitor. The Kinect for Windows sensor is placed below or above one of the computer monitors that the surgeons use for visualizing medical images. Key three-dimensional (3D) medical images are acquired before the surgery using a computed tomography (CT) scanner. During surgery, the fluoroscopy scanner captures real-time X-ray images, which are overlaid on one another, registered, and displayed in an image viewer. A surgeon then uses gestures to rotate images, zoom into details, and more.

The system is currently under trial on vascular patients at St Thomas’, with a view to expanding to the manipulation of 3D volumetric models of the brain for neurosurgery at Addenbrooke’s Hospital (Cambridge, United Kingdom). The ultimate aim is to develop a touchless interaction in surgery toolkit that can be used in any hospital or system interested in applying touchless interaction to their imaging system.

“With Kinect, we could revolutionize the way we do complex operations. Patients and clinical teams will spend less time in theater, and surgeons will be more in control of the information they need,” said Tom Carrell, MD, a senior lecturer at KCL and a vascular surgeon at Guy’s and St Thomas’. “Touchless interaction means there is no compromise in the sterility of the operating field or in patient safety.”

“The idea of being able to control almost anything with just a wave of the hand really speaks to how surgeons want to feel like they have control over things,” added Helena Mentis, PhD, a postdoctoral researcher with the Socio-Digital Systems group at the Microsoft Cambridge lab. “They want to be able to save as much time as possible and to be able to get a patient in and out of anesthesia as quickly as possible. Those all are things that they’re motivated by, so they think this is the greatest thing ever.”

Related Links:
King’s College London
Guy’s and St Thomas’ NHS Foundation Trust
Microsoft Research Cambridge