AI-Powered Wearable Camera System Detects Errors in Medication Delivery

Drug administration errors are the most commonly reported critical incidents in anesthesia and represent the leading cause of serious medical errors in intensive care. Overall, it is estimated that 5% to 10% of all medications administered are linked to errors. Adverse events related to injectable medications are thought to impact 1.2 million patients each year, incurring costs of approximately USD 5.1 billion. Syringe and vial-swap errors frequently occur during intravenous injections, where a clinician must transfer medication from a vial to a syringe and then to the patient. About 20% of these mistakes are substitution errors, where the wrong vial is chosen or a syringe is mislabeled. An additional 20% of errors arise when the drug is correctly labeled but administered incorrectly. While safety measures, such as barcode systems that quickly read and confirm a vial’s contents, are implemented to prevent such incidents, practitioners may occasionally overlook this check-in high-stress situations, as it adds an extra step to their workflow. A research team has now developed the first wearable camera system that, aided by artificial intelligence (AI), identifies potential errors in medication delivery.

Researchers at the University of Washington School of Medicine (Seattle, WA, USA) aimed to create a deep-learning model that, when paired with a GoPro camera, could effectively recognize the contents of cylindrical vials and syringes and issue warnings before the medication is administered to the patient. Training the model took several months. The team collected 4K video footage of 418 drug draws performed by 13 anesthesiology providers in various operating room setups and lighting conditions. The videos captured clinicians handling vials and syringes of specific medications. These video snippets were logged, and the contents of the syringes and vials were annotated to train the model to recognize both the contents and the containers. The video system does not read the text on each vial directly; instead, it identifies visual cues such as the size and shape of the vials and syringes, the color of the vial caps, and the size of printed labels.

Image: Still images from video snippets show how AI identifies in real-time what a clinician is holding (Photo courtesy of Paul G. Allen School of Computer Science & Engineering)

Additionally, the computational model was trained to focus on medications in the foreground while ignoring vials and syringes in the background. In a test whose results were published in npj Digital Medicine, the video system demonstrated high proficiency in recognizing and identifying medications being drawn in busy clinical environments. The AI achieved a sensitivity of 99.6% and a specificity of 98.8% in detecting vial-swap errors. This research highlights the potential of AI and deep learning to enhance safety and efficiency across various healthcare practices. Researchers are only beginning to explore this potential and anticipate that the system will serve as a vital safeguard, particularly in operating rooms, intensive care units, and emergency medicine settings.

“The thought of being able to help patients in real-time or to prevent a medication error before it happens is very powerful,” said co-lead author Dr. Kelly Michaelsen, an assistant professor of anesthesiology and pain medicine at UW Medicine. “One can hope for a 100% performance but even humans cannot achieve that. In a survey of more than 100 anesthesia providers, the majority desired the system to be more than 95% accurate, which is a goal we achieved.”