Skip to main content
Erschienen in: European Radiology 7/2023

09.01.2023 | Head and Neck

Clinical value of artificial intelligence in thyroid ultrasound: a prospective study from the real world

verfasst von: Yingying Li, Yihao Liu, Jing Xiao, Lin Yan, Zhen Yang, Xinyang Li, Mingbo Zhang, Yukun Luo

Erschienen in: European Radiology | Ausgabe 7/2023

Einloggen, um Zugang zu erhalten

Abstract

Objective

To evaluate the diagnostic performance of a commercial artificial intelligence (AI)–assisted ultrasonography (US) for thyroid nodules and to validate its value in real-world medical practice.

Materials and methods

From March 2021 to July 2021, 236 consecutive patients with 312 suspicious thyroid nodules were prospectively enrolled in this study. One experienced radiologist performed US examinations with a real-time AI system (S-Detect). US images and AI reports of the nodules were recorded. Nine residents and three senior radiologists were invited to make a “benign” or “malignant” diagnosis based on recorded US images without knowing the AI reports. After referring to AI reports, the diagnosis was made again. The diagnostic performance of AI, residents, and senior radiologists with and without AI reports were analyzed.

Results

The sensitivity, accuracy, and AUC of the AI system were 0.95, 0.84, and 0.753, respectively, and were not statistically different from those of the experienced radiologists, but were superior to those of the residents (all p < 0.01). The AI-assisted resident strategy significantly improved the accuracy and sensitivity for nodules ≤ 1.5 cm (all p < 0.01), while reducing the unnecessary biopsy rate by up to 27.7% for nodules > 1.5 cm (p = 0.034).

Conclusions

The AI system achieved performance, for cancer diagnosis, comparable to that of an average senior thyroid radiologist. The AI-assisted strategy can significantly improve the overall diagnostic performance for less-experienced radiologists, while increasing the discovery of thyroid cancer ≤ 1.5 cm and reducing unnecessary biopsies for nodules > 1.5 cm in real-world medical practice.

Key Points

The AI system reached a senior radiologist-like level in the evaluation of thyroid cancer and could significantly improve the overall diagnostic performance of residents.
The AI-assisted strategy significantly improved1.5 cm thyroid cancer screening AUC, accuracy, and sensitivity of the residents, leading to an increased detection of thyroid cancer while maintaining a comparable specificity to that of radiologists alone.
The AI-assisted strategy significantly reduced the unnecessary biopsy rate for thyroid nodules > 1.5 cm by the residents, while maintaining a comparable sensitivity to that of radiologists alone.
Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Chmielik E, Rusinek D, Oczko-Wojciechowska M et al (2018) Heterogeneity of thyroid cancer. Pathobiology 85:117–129CrossRefPubMed Chmielik E, Rusinek D, Oczko-Wojciechowska M et al (2018) Heterogeneity of thyroid cancer. Pathobiology 85:117–129CrossRefPubMed
2.
Zurück zum Zitat Sung H, Ferlay J, Siegel RL et al (2021) Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 71:209–249CrossRefPubMed Sung H, Ferlay J, Siegel RL et al (2021) Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 71:209–249CrossRefPubMed
4.
Zurück zum Zitat Kitahara C, Sosa J (2016) The changing incidence of thyroid cancer. Nat Rev Endocrinol 12:646–653CrossRefPubMed Kitahara C, Sosa J (2016) The changing incidence of thyroid cancer. Nat Rev Endocrinol 12:646–653CrossRefPubMed
5.
Zurück zum Zitat Haugen B, Alexander E, Bible K et al (2016) 2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid 26:1–133CrossRefPubMedPubMedCentral Haugen B, Alexander E, Bible K et al (2016) 2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid 26:1–133CrossRefPubMedPubMedCentral
6.
Zurück zum Zitat Kim JH, Baek JH, Lim HK et al (2018) 2017 thyroid radiofrequency ablation guideline: Korean Society of Thyroid Radiology. Korean J Radiol 19:632–655CrossRefPubMedPubMedCentral Kim JH, Baek JH, Lim HK et al (2018) 2017 thyroid radiofrequency ablation guideline: Korean Society of Thyroid Radiology. Korean J Radiol 19:632–655CrossRefPubMedPubMedCentral
7.
Zurück zum Zitat Gharib H, Papini E, Garber J et al (2016) American Association of Clinical Endocrinologists, American College of Endocrinology, and Associazione Medici Endocrinologi medical guidelines for clinical practice for the diagnosis and management of thyroid nodules--2016 update. Endocr Pract 22:622–639CrossRefPubMed Gharib H, Papini E, Garber J et al (2016) American Association of Clinical Endocrinologists, American College of Endocrinology, and Associazione Medici Endocrinologi medical guidelines for clinical practice for the diagnosis and management of thyroid nodules--2016 update. Endocr Pract 22:622–639CrossRefPubMed
8.
Zurück zum Zitat Haddad R, Nasr C, Bischoff L et al (2018) NCCN guidelines insights: thyroid carcinoma, version 2.2018. J Natl Compr Cancer Netw 16:1429–1440CrossRef Haddad R, Nasr C, Bischoff L et al (2018) NCCN guidelines insights: thyroid carcinoma, version 2.2018. J Natl Compr Cancer Netw 16:1429–1440CrossRef
9.
Zurück zum Zitat Melany M, Chen S (2017) Thyroid cancer: ultrasound imaging and fine-needle aspiration biopsy. Endocrinol Metab Clin N Am 46:691–711CrossRef Melany M, Chen S (2017) Thyroid cancer: ultrasound imaging and fine-needle aspiration biopsy. Endocrinol Metab Clin N Am 46:691–711CrossRef
10.
Zurück zum Zitat Ozel A, Erturk SM, Ercan A et al (2012) The diagnostic efficiency of ultrasound in characterization for thyroid nodules: how many criteria are required to predict malignancy? Med Ultrason 14:24–28PubMed Ozel A, Erturk SM, Ercan A et al (2012) The diagnostic efficiency of ultrasound in characterization for thyroid nodules: how many criteria are required to predict malignancy? Med Ultrason 14:24–28PubMed
11.
Zurück zum Zitat Choi S, Kim E, Kwak J, Kim M, Son E (2010) Interobserver and intraobserver variations in ultrasound assessment of thyroid nodules. Thyroid 20:167–172CrossRefPubMed Choi S, Kim E, Kwak J, Kim M, Son E (2010) Interobserver and intraobserver variations in ultrasound assessment of thyroid nodules. Thyroid 20:167–172CrossRefPubMed
12.
Zurück zum Zitat Wienke JR, Chong WK, Fielding JR, Zou KH, Mittelstaedt CA (2003) Sonographic features of benign thyroid nodules: interobserver reliability and overlap with malignancy. J Ultrasound Med 22:1027–1031CrossRefPubMed Wienke JR, Chong WK, Fielding JR, Zou KH, Mittelstaedt CA (2003) Sonographic features of benign thyroid nodules: interobserver reliability and overlap with malignancy. J Ultrasound Med 22:1027–1031CrossRefPubMed
13.
Zurück zum Zitat Kim SH, Park CS, Jung SL et al (2010) Observer variability and the performance between faculties and residents: US criteria for benign and malignant thyroid nodules. Korean J Radiol 11:149–155CrossRefPubMedPubMedCentral Kim SH, Park CS, Jung SL et al (2010) Observer variability and the performance between faculties and residents: US criteria for benign and malignant thyroid nodules. Korean J Radiol 11:149–155CrossRefPubMedPubMedCentral
14.
Zurück zum Zitat Lee H, Yoon D, Seo Y et al (2018) Intraobserver and interobserver variability in ultrasound measurements of thyroid nodules. J Ultrasound Med 37:173–178CrossRefPubMed Lee H, Yoon D, Seo Y et al (2018) Intraobserver and interobserver variability in ultrasound measurements of thyroid nodules. J Ultrasound Med 37:173–178CrossRefPubMed
15.
Zurück zum Zitat Sakorafas GH (2010) Thyroid nodules; interpretation and importance of fine-needle aspiration (FNA) for the clinician - practical considerations. Surg Oncol 19:e130–e139CrossRefPubMed Sakorafas GH (2010) Thyroid nodules; interpretation and importance of fine-needle aspiration (FNA) for the clinician - practical considerations. Surg Oncol 19:e130–e139CrossRefPubMed
16.
Zurück zum Zitat Tuttle RM, Zhang L, Shaha A (2018) A clinical framework to facilitate selection of patients with differentiated thyroid cancer for active surveillance or less aggressive initial surgical management. Expert Rev Endocrinol Metab 13:77–85CrossRefPubMedPubMedCentral Tuttle RM, Zhang L, Shaha A (2018) A clinical framework to facilitate selection of patients with differentiated thyroid cancer for active surveillance or less aggressive initial surgical management. Expert Rev Endocrinol Metab 13:77–85CrossRefPubMedPubMedCentral
17.
Zurück zum Zitat Durante C, Grani G, Lamartina L, Filetti S, Mandel SJ, Cooper DS (2018) The diagnosis and management of thyroid nodules: a review. JAMA 319:914–924CrossRefPubMed Durante C, Grani G, Lamartina L, Filetti S, Mandel SJ, Cooper DS (2018) The diagnosis and management of thyroid nodules: a review. JAMA 319:914–924CrossRefPubMed
18.
Zurück zum Zitat Choi Y, Baek J, Park H et al (2017) A computer-aided diagnosis system using artificial intelligence for the diagnosis and characterization of thyroid nodules on ultrasound: initial clinical assessment. Thyroid 27:546–552CrossRefPubMed Choi Y, Baek J, Park H et al (2017) A computer-aided diagnosis system using artificial intelligence for the diagnosis and characterization of thyroid nodules on ultrasound: initial clinical assessment. Thyroid 27:546–552CrossRefPubMed
19.
Zurück zum Zitat Yoo Y, Ha E, Cho Y, Kim H, Han M, Kang S (2018) Computer-aided diagnosis of thyroid nodules via ultrasonography: initial clinical experience. Korean J Radiol 19:665–672CrossRefPubMedPubMedCentral Yoo Y, Ha E, Cho Y, Kim H, Han M, Kang S (2018) Computer-aided diagnosis of thyroid nodules via ultrasonography: initial clinical experience. Korean J Radiol 19:665–672CrossRefPubMedPubMedCentral
20.
Zurück zum Zitat Jeong E, Kim H, Ha E, Park S, Cho Y, Han M (2019) Computer-aided diagnosis system for thyroid nodules on ultrasonography: diagnostic performance and reproducibility based on the experience level of operators. Eur Radiol 29:1978–1985CrossRefPubMed Jeong E, Kim H, Ha E, Park S, Cho Y, Han M (2019) Computer-aided diagnosis system for thyroid nodules on ultrasonography: diagnostic performance and reproducibility based on the experience level of operators. Eur Radiol 29:1978–1985CrossRefPubMed
21.
Zurück zum Zitat Chung SR, Baek JH, Lee MK et al (2020) Computer-aided diagnosis system for the evaluation of thyroid nodules on ultrasonography: prospective non-inferiority study according to the experience level of radiologists. Korean J Radiol 21:369–376CrossRefPubMedPubMedCentral Chung SR, Baek JH, Lee MK et al (2020) Computer-aided diagnosis system for the evaluation of thyroid nodules on ultrasonography: prospective non-inferiority study according to the experience level of radiologists. Korean J Radiol 21:369–376CrossRefPubMedPubMedCentral
22.
Zurück zum Zitat Wei Q, Zeng S, Wang L et al (2020) The value of S-Detect in improving the diagnostic performance of radiologists for the differential diagnosis of thyroid nodules. Med Ultrason 22:415–423CrossRefPubMed Wei Q, Zeng S, Wang L et al (2020) The value of S-Detect in improving the diagnostic performance of radiologists for the differential diagnosis of thyroid nodules. Med Ultrason 22:415–423CrossRefPubMed
23.
Zurück zum Zitat Cibas ES, Ali SZ (2017) The 2017 Bethesda system for reporting thyroid cytopathology. Thyroid 27:1341–1346CrossRefPubMed Cibas ES, Ali SZ (2017) The 2017 Bethesda system for reporting thyroid cytopathology. Thyroid 27:1341–1346CrossRefPubMed
24.
Zurück zum Zitat Han M, Ha E, Park J (2021) Computer-aided diagnostic system for thyroid nodules on ultrasonography: diagnostic performance based on the thyroid imaging reporting and data system classification and dichotomous outcomes. AJNR Am J Neuroradiol 42:559–565CrossRefPubMedPubMedCentral Han M, Ha E, Park J (2021) Computer-aided diagnostic system for thyroid nodules on ultrasonography: diagnostic performance based on the thyroid imaging reporting and data system classification and dichotomous outcomes. AJNR Am J Neuroradiol 42:559–565CrossRefPubMedPubMedCentral
25.
Zurück zum Zitat Kim H, Ha E, Han M (2019) Real-world performance of computer-aided diagnosis system for thyroid nodules using ultrasonography. Ultrasound Med Biol 45:2672–2678CrossRefPubMed Kim H, Ha E, Han M (2019) Real-world performance of computer-aided diagnosis system for thyroid nodules using ultrasonography. Ultrasound Med Biol 45:2672–2678CrossRefPubMed
26.
Zurück zum Zitat Chambara N, Ying M (2019) The diagnostic efficiency of ultrasound computer-aided diagnosis in differentiating thyroid nodules: a systematic review and narrative synthesis. Cancers (Basel) 11 Chambara N, Ying M (2019) The diagnostic efficiency of ultrasound computer-aided diagnosis in differentiating thyroid nodules: a systematic review and narrative synthesis. Cancers (Basel) 11
27.
Zurück zum Zitat Zhao WJ, Fu LR, Huang ZM, Zhu JQ, Ma BY (2019) Effectiveness evaluation of computer-aided diagnosis system for the diagnosis of thyroid nodules on ultrasound: a systematic review and meta-analysis. Medicine (Baltimore) 98:e16379CrossRefPubMed Zhao WJ, Fu LR, Huang ZM, Zhu JQ, Ma BY (2019) Effectiveness evaluation of computer-aided diagnosis system for the diagnosis of thyroid nodules on ultrasound: a systematic review and meta-analysis. Medicine (Baltimore) 98:e16379CrossRefPubMed
28.
Zurück zum Zitat Barczyński M, Stopa-Barczyńska M, Wojtczak B, Czarniecka A, Konturek A (2020) Clinical validation of S-Detect mode in semi-automated ultrasound classification of thyroid lesions in surgical office. Gland Surg 9:S77–S85CrossRefPubMedPubMedCentral Barczyński M, Stopa-Barczyńska M, Wojtczak B, Czarniecka A, Konturek A (2020) Clinical validation of S-Detect mode in semi-automated ultrasound classification of thyroid lesions in surgical office. Gland Surg 9:S77–S85CrossRefPubMedPubMedCentral
Metadaten
Titel
Clinical value of artificial intelligence in thyroid ultrasound: a prospective study from the real world
verfasst von
Yingying Li
Yihao Liu
Jing Xiao
Lin Yan
Zhen Yang
Xinyang Li
Mingbo Zhang
Yukun Luo
Publikationsdatum
09.01.2023
Verlag
Springer Berlin Heidelberg
Erschienen in
European Radiology / Ausgabe 7/2023
Print ISSN: 0938-7994
Elektronische ISSN: 1432-1084
DOI
https://doi.org/10.1007/s00330-022-09378-y

Weitere Artikel der Ausgabe 7/2023

European Radiology 7/2023 Zur Ausgabe

Mammakarzinom: Brustdichte beeinflusst rezidivfreies Überleben

26.05.2024 Mammakarzinom Nachrichten

Frauen, die zum Zeitpunkt der Brustkrebsdiagnose eine hohe mammografische Brustdichte aufweisen, haben ein erhöhtes Risiko für ein baldiges Rezidiv, legen neue Daten nahe.

„Übersichtlicher Wegweiser“: Lauterbachs umstrittener Klinik-Atlas ist online

17.05.2024 Klinik aktuell Nachrichten

Sie sei „ethisch geboten“, meint Gesundheitsminister Karl Lauterbach: mehr Transparenz über die Qualität von Klinikbehandlungen. Um sie abzubilden, lässt er gegen den Widerstand vieler Länder einen virtuellen Klinik-Atlas freischalten.

Klinikreform soll zehntausende Menschenleben retten

15.05.2024 Klinik aktuell Nachrichten

Gesundheitsminister Lauterbach hat die vom Bundeskabinett beschlossene Klinikreform verteidigt. Kritik an den Plänen kommt vom Marburger Bund. Und in den Ländern wird über den Gang zum Vermittlungsausschuss spekuliert.

Darf man die Behandlung eines Neonazis ablehnen?

08.05.2024 Gesellschaft Nachrichten

In einer Leseranfrage in der Zeitschrift Journal of the American Academy of Dermatology möchte ein anonymer Dermatologe bzw. eine anonyme Dermatologin wissen, ob er oder sie einen Patienten behandeln muss, der eine rassistische Tätowierung trägt.

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.