Investigating the Performance of VisualDx on Common Dermatologic Conditions in Skin of Color
Main Article Content
Keywords
artificial intelligence, generative models, skin of color, skin tone diversity, diverse skin phenotypes
Abstract
Background: Artificial intelligence (AI) has been used to create diagnostic models, such as VisualDx, to assist in rapidly diagnosing skin conditions. AI diagnostic models are typically trained on image databases of dermatologic conditions, which are known to underrepresent patients with richly pigmented skin.
Objectives: We investigated whether VisualDx performed differently when classifying conditions across different skin phenotypes and whether images of conditions processed to resemble richly pigmented skin impacts diagnostic accuracy.
Methods: Our image dataset consisted of sixteen common conditions. For each condition, three subgroups were curated: “Fitzpatrick I-III”, “Fitzpatrick IV-VI”, and “Processed”. The “Processed” subgroup contained images from the “Fitzpatrick I-III” subgroup altered to resemble richly pigmented skin. Images were processed by VisualDx to obtain a differential diagnosis list and diagnostic performance was analysed.
Results: Across all subgroups, the highest sensitivity ( 97%) was seen in hidradenitis suppurativa, prurigo nodularis, and tinea versicolor. Atopic dermatitis, post-inflammatory hyperpigmentation, and basal cell carcinoma demonstrated the lowest sensitivity (23%, 23%, and 27%, respectively). Significantly greater diagnostic sensitivity was noted for all conditions in the “Fitzpatrick I-III” subgroup (p < 0.001) except acanthosis nigricans, melasma, and melanoma compared to the “Fitzpatrick IV-VI” and “Processed” subgroups. For all conditions, a reduction in sensitivity and specificity was observed in processed images (p < 0.001).
Conclusion: Overall, VisualDx demonstrated diagnostic bias for images in the “Fitzpatrick I-III” subgroup, and colour-editing reduced diagnostic accuracy. These results suggest comprehensive databases should be used for future training of AI diagnostic tools to improve performance in all skin phototypes.
References
2. Adelekun A, Onyekaba G, Lipoff JB. Skin color in dermatology textbooks: An updated evaluation and analysis. J Am Acad Dermatol. 2021;84(1):194-196. doi:10.1016/j.jaad.2020.04.084
3. Daneshjou R, Smith MP, Sun MD, Rotemberg V, Zou J. Lack of Transparency and Potential Bias in Artificial Intelligence Data Sets and Algorithms: A Scoping Review. JAMA Dermatol. 2021;157(11):1362-1369. doi:10.1001/jamadermatol.2021.3129
4. Marchetti MA, Adamson AS, Halpern AC. Melanoma and Racial Health Disparities in Black Individuals-Facts, Fallacies, and Fixes. JAMA Dermatol. 2021;157(9):1031-1032. doi:10.1001/jamadermatol.2021.2215
5. Liu Y, Jain A, Eng C, et al. A deep learning system for differential diagnosis of skin diseases. Nat Med. 2020;26(6):900-908. doi:10.1038/s41591-020-0842-3
6. Marchetti MA, Liopyris K, Dusza SW, et al. Computer algorithms show potential for improving dermatologists’ accuracy to diagnose cutaneous melanoma: Results of the International Skin Imaging Collaboration 2017. J Am Acad Dermatol. 2020;82(3):622-627. doi:10.1016/j.jaad.2019.07.016
7. Haenssle HA, Fink C, Toberer F, et al. Man against machine reloaded: performance of a market-approved convolutional neural network in classifying a broad spectrum of skin lesions in comparison with 96 dermatologists working under less artificial conditions. Ann Oncol. 2020;31(1):137-143. doi:10.1016/j.annonc.2019.10.013
8. Vardell E, Bou-Crick C. VisualDx: a visual diagnostic decision support tool. Med Ref Serv Q. 2012;31(4):414-424. doi:10.1080/02763869.2012.724287
9. Daneshjou R, Vodrahalli K, Novoa RA, et al. Disparities in dermatology AI performance on a diverse, curated clinical image set. Sci Adv. 2022;8(32):eabq6147. doi:10.1126/sciadv.abq6147
10. Tschandl P, Rosendahl C, Kittler H. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci Data. 2018;5:180161. doi:10.1038/sdata.2018.161
11. Schlessinger DI, Chhor G, Gevaert O, Swetter SM, Ko J, Novoa RA. Artificial intelligence and dermatology: opportunities, challenges, and future directions. Semin Cutan Med Surg. 2019;38(1):E31-37. doi:10.12788/j.sder.2019.
12. Chan S, Reddy V, Myers B, Thibodeaux Q, Brownstone N, Liao W. Machine Learning in Dermatology: Current Applications, Opportunities, and Limitations. Dermatol Ther Heidelb. 2020;10(3):365-386. doi:10.1007/s13555-020-00372-0
13. Eman Rezk ME. Improving Skin Color Diversity in Cancer Detection: Deep Learning Approach. JMIR Dermatol. 2022;5(3). doi:10.2196/39143
14. Akrout M et al. Diffusion-Based Data Augmentation for Skin Disease Classification: Impact Across Original Medical Datasets to Fully Synthetic Images. Presented at: International Conference on Medical Image Computing and Computer-Assisted Intervention; 2024.
15. DermNet NZ. DermNet Image Library.
16. Global Skin Atlas. Diagnosis Index.
17. Donkor CMYA ABJ. Malignancies. In: Atlas of Dermatological Conditions in Populations of African Ancestry. 1st ed. Springer; 2021:227-230.
18. Susan C. Taylor APK. Taylor and Kelly’s Dermatology for Skin of Color. 2nd ed. McGraw-Hill Education; 2016.
19. Brunner PM, Guttman-Yassky E. Racial differences in atopic dermatitis. Ann Allergy Asthma Immunol. 2019;122(5):449-455. doi:10.1016/j.anai.2018.11.015
20. Tokura Y, Hayano S. Subtypes of atopic dermatitis: From phenotype to endotype. Allergol Int. 2022;71(1):14-24. doi:10.1016/j.alit.2021.07.003
21. Czarnowicki T, He H, Krueger JG, Guttman-Yassky E. Atopic dermatitis endotypes and implications for targeted therapeutics. J Allergy Clin Immunol. 2019;143(1):1-11. doi:10.1016/j.jaci.2018.10.032
22. Chiricozzi A, Maurelli M, Calabrese L, Peris K, Girolomoni G. Overview of Atopic Dermatitis in Different Ethnic Groups. J Clin Med. 2023;12(7). doi:10.3390/jcm12072701
23. Rajan Ramji & Amanda Oakley. Basal cell carcinoma in skin of colour.
24. Alvarado SM, Feng H. Representation of dark skin images of common dermatologic conditions in educational resources: A cross-sectional analysis. J Am Acad Dermatol. 2021;84(5):1427-1431. doi:10.1016/j.jaad.2020.06.041
25. Stuart CA, Driscoll MS, Lundquist KF, Gilkison CR, Shaheb S, Smith MM. Acanthosis nigricans. J Basic Clin Physiol Pharmacol. 1998;9(2-4):407-418. doi:10.1515/JBCPP.1998.9.2-4.407
26. Stuart CA, Pate CJ, Peters EJ. Prevalence of acanthosis nigricans in an unselected population. Am J Med. 1989;87(3):269-272. doi:10.1016/s0002-9343(89)80149-4
27. Ball Arefiev KL, Hantash BM. Advances in the treatment of melasma: a review of the recent literature. Dermatol Surg. 2012;38(7 Pt 1):971-984. doi:10.1111/j.1524-4725.2012.02435.x
28. Sheth VM, Pandya AG. Melasma: a comprehensive update: part I. J Am Acad Dermatol. 2011;65(4):689-697. doi:10.1016/j.jaad.2010.12.046
29. Brunsgaard EK, Wu YP, Grossman D. Melanoma in skin of color: Part I. Epidemiology and clinical presentation. J Am Acad Dermatol. 2023;89(3):445-456. doi:10.1016/j.jaad.2022.04.056
30. Taylor SC. Epidemiology of skin diseases in people of color. Cutis. 2003;71(4):271-275.
Article Sidebar
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Creative Commons Attribution (CC BY): lets others distribute and copy the article, to create extracts, abstracts, and other revised versions, adaptations or derivative works of or from an article (such as a translation), to include in a collective work (such as an anthology), to text or data mine the article, even for commercial purposes, as long as they credit the author(s), do not represent the author as endorsing their adaptation of the article, and do not modify the article in such a way as to damage the author's honor or reputation.
Authors retain copyright in their work and license the publisher to publish the content. The publisher is the National Society for Cutaneous Medicine, with production and publishing support provided by OJS, Open Journal Systems,see https://pkp.sfu.
Prior to January 2022, authors transferred copyright to the National Society for Cutaneous Medicine. However, all articles are freely available to anyone in the world. There are no subscription fees, page charges, or article processing charges.