AI Outperforms Humans in Diagnosis of Skin Lesions

A new study shows that artificial intelligence (AI) machine-learning (ML) classifiers outperform human experts in the diagnosis of pigmented skin lesions.

Researchers at the Medical University of Vienna (MedUni; Austria), the University of Queensland (UQ; Brisbane, Australia), Tel Aviv University (TAU; Israel), and other member institutions of the International Skin Imaging Collaboration (ISIC) organized an international challenge to compare the diagnostic skills of 511 physicians with 139 computer algorithms from 77 different ML labs. A database of more than 10,000 images was used as a training set for the machines.

The database includes both benign and malignant pigmented lesions, which fell into one of seven pre-defined disease categories. These included intraepithelial carcinoma, including actinic keratoses and Bowen's disease; basal cell carcinoma; benign keratinocytic lesions, including solar lentigo, seborrheic keratosis and lichen planus-like keratosis; dermatofibroma; melanoma; melanocytic nevus; and vascular lesions. The two main outcomes were the differences in the number of correct specific diagnoses per batch between all human readers and the top three algorithms, and between human experts and the top three algorithms.

The results revealed that when comparing all human readers with all ML algorithms, the algorithms achieved a mean of 2.01 more correct diagnoses than the human readers. The 27 human experts with more than 10 years of experience achieved a mean of 18.78 correct answers, compared with 25.43 correct answers for the top three machine algorithms. For images in the test set that were collected from sources not included in the training set, humans still underperformed, but the difference was lower, at 11.4%. The study was published on June 11, 2019, in The Lancet Oncology.

“Two thirds of all participating machines were better than humans; this does not mean that the machines will replace humans in the diagnosis of skin cancer. The computer only analyzes an optical snapshot and is really good at it. In real life, however, the diagnosis is a complex task,” said lead author Philipp Tschandl, PhD, of MedUni Vienna. “Physicians usually examine the entire patient and not just single lesions. When humans make a diagnosis they also take additional information into account, such as the duration of the disease, whether the patient is at high or low risk, and the age of the patient.”

The rising popularity of ML techniques for medical applications is evident from the increasing amount of research, the number of products obtaining regulatory approvals, and entrepreneurial efforts over the past few years. Venture capital funding for healthcare AI startup companies was about USD 3.6 billion in the last five years, underscoring the increasing appreciation of the value that ML can potentially bring to the medical community.

Related Links:
Medical University of Vienna
University of Queensland
Tel Aviv University