Consumer Ratings Do Not Reflect Medical App Accuracy
By HospiMedica International staff writers Posted on 20 Jun 2018 |
Image: The now-withdrawn IBP app claimed to accurately measure blood pressure (Photo courtesy of AuraLife).
A new study suggests that positive user reviews, comments, and ratings of popular medical apps do not necessarily mean that they actually work.
Researchers at the University of Vermont (UVM, Burlington, USA), Johns Hopkins University (JHU; Baltimore, MD, USA), and other institutions conducted a study of all user reviews of the now-defunct AuraLife Instant Blood Pressure (IBP) app in order to examine if they correctly reflected the proven inaccuracy of the unregulated blood pressure (BP) measuring app, to understand the drivers for high and low ratings, and to understand if company disclaimers prevented medical use.
To do so, 261 iTunes app reviews for IBP were downloaded and assessed for themes by two reviewers, with summary statistics for themes tabulated with their associated star ratings. The average star rating of the latest version of the app was four out of five stars, and 59% of the reviews assigned the app five stars. Commentary praising the accuracy of the app based on anecdotal experience comprised 42% of the reviews, and 10% of the reviews mention inaccuracy. Six reviews came from people who claimed to be health care professionals, who gave an average rating of 4.2 stars out of five.
While the IBP app included a disclaimer that it shouldn't be used as a medical device and is intended for recreational purposes only, 24 reviews (10%) claimed to use the app for medical purposes, with 11 people using it to manage their high BP treatments, one using it to manage kidney disease, and another using it to monitor BP after a heart transplant. Eleven reviews came from people who said their health care provider approved of the app. Two people said professionals disapproved of the app, and gave a one-star rating. The study was published on June 7, 2018, in the Journal of the American Medical Informatics Association.
“People tend to trust user reviews when shopping online and use them to decide which products to purchase, but that doesn't cut it for medical apps. There are certain thresholds of accuracy that need to be maintained, and a five-star rating doesn't replace clinical validation studies and FDA review,” said study author Timothy Plante, MD, MHS, of the University of Vermont. “Our take-home message to health care providers is to make sure you hold apps to the highest standards, and not just give out recommendations willy-nilly for apps that you think are neat.”
The IBP app sold for USD 4.99 on iTunes and was downloaded 140,000 times before it was withdrawn. It ranked among the best-selling apps on all of iTunes for much of its stretch on the app store. After the app was withdrawn and following the study, the U.S. Federal Trade Commission settled with AuraLife for USD 600,000, with payment suspended for lack of funds in the company. Many other medical apps are still available for consumers, some downloading as many as 1,000,000 times each.
Related Links:
University of Vermont
Johns Hopkins University
Researchers at the University of Vermont (UVM, Burlington, USA), Johns Hopkins University (JHU; Baltimore, MD, USA), and other institutions conducted a study of all user reviews of the now-defunct AuraLife Instant Blood Pressure (IBP) app in order to examine if they correctly reflected the proven inaccuracy of the unregulated blood pressure (BP) measuring app, to understand the drivers for high and low ratings, and to understand if company disclaimers prevented medical use.
To do so, 261 iTunes app reviews for IBP were downloaded and assessed for themes by two reviewers, with summary statistics for themes tabulated with their associated star ratings. The average star rating of the latest version of the app was four out of five stars, and 59% of the reviews assigned the app five stars. Commentary praising the accuracy of the app based on anecdotal experience comprised 42% of the reviews, and 10% of the reviews mention inaccuracy. Six reviews came from people who claimed to be health care professionals, who gave an average rating of 4.2 stars out of five.
While the IBP app included a disclaimer that it shouldn't be used as a medical device and is intended for recreational purposes only, 24 reviews (10%) claimed to use the app for medical purposes, with 11 people using it to manage their high BP treatments, one using it to manage kidney disease, and another using it to monitor BP after a heart transplant. Eleven reviews came from people who said their health care provider approved of the app. Two people said professionals disapproved of the app, and gave a one-star rating. The study was published on June 7, 2018, in the Journal of the American Medical Informatics Association.
“People tend to trust user reviews when shopping online and use them to decide which products to purchase, but that doesn't cut it for medical apps. There are certain thresholds of accuracy that need to be maintained, and a five-star rating doesn't replace clinical validation studies and FDA review,” said study author Timothy Plante, MD, MHS, of the University of Vermont. “Our take-home message to health care providers is to make sure you hold apps to the highest standards, and not just give out recommendations willy-nilly for apps that you think are neat.”
The IBP app sold for USD 4.99 on iTunes and was downloaded 140,000 times before it was withdrawn. It ranked among the best-selling apps on all of iTunes for much of its stretch on the app store. After the app was withdrawn and following the study, the U.S. Federal Trade Commission settled with AuraLife for USD 600,000, with payment suspended for lack of funds in the company. Many other medical apps are still available for consumers, some downloading as many as 1,000,000 times each.
Related Links:
University of Vermont
Johns Hopkins University
Latest Health IT News
- Machine Learning Model Improves Mortality Risk Prediction for Cardiac Surgery Patients
- Strategic Collaboration to Develop and Integrate Generative AI into Healthcare
- AI-Enabled Operating Rooms Solution Helps Hospitals Maximize Utilization and Unlock Capacity
- AI Predicts Pancreatic Cancer Three Years before Diagnosis from Patients’ Medical Records
- First Fully Autonomous Generative AI Personalized Medical Authorizations System Reduces Care Delay
- Electronic Health Records May Be Key to Improving Patient Care, Study Finds
- AI Trained for Specific Vocal Biomarkers Could Accurately Predict Coronary Artery Disease
- First-Ever AI Test for Early Diagnosis of Alzheimer’s to Be Expanded to Diagnosis of Parkinson’s Disease
- New Self-Learning AI-Based Algorithm Reads Electrocardiograms to Spot Unseen Signs of Heart Failure
- Autonomous Robot Performs COVID-19 Nasal Swab Tests
- Statistical Tool Predicts COVID-19 Peaks Worldwide
- Wireless-Controlled Soft Neural Implant Stimulates Brain Cells
- Tiny Polymer Stent Could Treat Pediatric Urethral Strictures
- Human Torso Simulator Helps Design Brace Innovations
- 3D Bioprinting Rebuilds the Human Heart