AI Detects Serious Neurologic Changes in NICU Infants Using Only Video Data

Every year, more than 300,000 newborns are admitted to neonatal intensive care units (NICUs) across the United States. Infant alertness is a key indicator of neurological health, reflecting the overall function of the central nervous system. Neurological decline in NICUs can occur suddenly, with serious consequences. However, while cardiorespiratory telemetry has been widely used to monitor heart and lung function continuously in NICUs, neurotelemetry has not been implemented similarly, despite advances in electroencephalography (EEG) and specialized neuro-NICUs. Neurological assessments are still typically performed intermittently through physical exams, which can be inaccurate and may miss subtle changes. Now, a deep learning pose-recognition algorithm trained on video feeds of infants in the NICU can track their movements and accurately measure key neurological metrics.

This AI-powered tool, developed by a team of clinicians, scientists, and engineers at Mount Sinai (New York, NY, USA), offers the potential for continuous, minimally invasive monitoring of neurological health in NICUs. It could provide real-time, critical insights into infant health that have previously been difficult to obtain. The team at Mount Sinai theorized that using computer vision to track infant movements could help predict neurological changes in NICU patients. The method, called “Pose AI,” utilizes machine learning to track anatomic landmarks from video data—a technique that has already revolutionized fields like athletics and robotics. The researchers trained the AI model using over 16,938,000 seconds of video footage from 115 NICU infants at The Mount Sinai Hospital who were also undergoing continuous video EEG monitoring.

The results showed that Pose AI was able to accurately track infant landmarks and use this data to predict two key conditions—sedation and cerebral dysfunction—with high accuracy. The team was surprised by the algorithm's ability to function effectively across various lighting conditions (day, night, and during phototherapy) and from different angles. Additionally, they found that their Pose AI movement index was associated with both gestational age and postnatal age. However, the study did have limitations, as the AI models were trained using data from a single institution, meaning further evaluation is necessary with video data from other hospitals and using different camera setups. The team plans to test the technology in more NICUs and develop clinical trials to assess its impact on patient care. They are also exploring its potential for diagnosing other neurological conditions and expanding its use to adult populations, as detailed in the research published in Lancet's eClinicalMedicine.

“Our study shows that applying an AI algorithm to cameras that continuously monitor infants in the NICU is an effective way to detect neurologic changes early, potentially allowing for faster interventions and better outcomes,” said Felix Richter, MD, PhD, senior author of the paper and Instructor of Newborn Medicine in the Department of Pediatrics at Mount Sinai. “We envision a future system where cameras continuously monitor infants in the NICU, with AI providing a neuro-telemetry strip similar to heart rate or respiratory monitoring, with alert for changes in sedation levels or cerebral dysfunction. Clinicians could review videos and AI-generated insights when needed, offering an intuitive and easily interpretable tool for bedside care.”