Human Physicians Outperform Diagnostic Apps by Wide Margin

Doctor’s diagnoses remain vastly superior to those of 23 commonly used symptom-checker apps, according to a new study.

Researchers at Harvard Medical School (HMS; Boston, MA, USA), the Human Diagnosis Project (Washington, DC, USA), and other institutions conducted a study in which 234 internal medicine physicians evaluated 45 clinical cases of both common and uncommon conditions, with varying degrees of severity. For each scenario, the physicians had to identify the most likely diagnosis along with two additional possible diagnoses. Each clinical vignette was solved by at least 20 physicians.

The researchers then compared the diagnostic accuracy of the physicians with computer algorithm symptom checker applications. The results showed that the physicians outperformed the symptom-checker apps, listing the correct diagnosis first 72% of the time, compared with 34% of the time for the digital platforms. In addition, 84% of clinicians listed the correct diagnosis in the top three possibilities, compared with 51% for the digital symptom-checkers.

The difference between physician and computer app performance was most dramatic in more severe and less common conditions; it was smaller for less acute and more common illnesses. But despite outperforming the apps by a wide margin, physicians still made errors in about 15% of cases. According to the researchers, developing computer-based algorithms to be used in conjunction with human decision-making may help further reduce diagnostic errors. The study was published on October 10, 2016, in JAMA Internal Medicine.

“While the computer programs were clearly inferior to physicians in terms of diagnostic accuracy, it will be critical to study future generations of computer programs that may be more accurate,” said senior author associate professor of health care policy Ateev Mehrotra, PhD, of HMS. “Clinical diagnosis is currently as much art as it is science, but there is great promise for technology to help augment clinical diagnoses. That is the true value proposition of these tools.”

Related Links:
Harvard Medical School
Human Diagnosis Project