(0837) The Artificial Laryngologist: The Current Diagnostic Capabilities of Artificial Intelligence for Laryngoscopy Physical Examination

Introduction: As the utilization of artificial intelligence (AI) becomes increasingly ubiquitous in medicine, its diagnostic accuracy is not fully elucidated. The purpose of this study is to examine closed source artificial intelligence responses to common laryngoscopy findings compared to responses of ENT specialists.

Methods: This cohort comparative study was conducted in January 2024Twenty-seven flexible laryngoscopy images, 2 normal control images and 25 pathologic, were obtained from clinical sources. The pathologic images were given to a laryngologist and ChatGPT with a prompt to identify the most likely pathology. The complexity of each case was assessed by 4 Otolaryngologists. The ChatGPT responses were then assessed for correctness and consistency with a validated AI scale from 0-3. To verify stability of AI responses, the ChatGPT image evaluation was repeated for 5 consecutive days.

Results: The complexity of the cases were rated as: 5 low, 8 moderate and 12 high complexity. The mean consistency and correctness ratings of the ChatGPT responses were 0.55 (STD 0.71) and 0.72 (STD 0.69), respectively. The mean correctness for low complexity versus high complexity cases was 0.78 and 0.40 (p=0.13), whereas they were 0.79 and 0.54 (p=0.03) for consistency. There was no statistically significant difference in consistency and correctness between complexities. The total intraclass correlation coefficient (ICC) for the stability of the correctness and consistency of Chat GPT was 0.369 and 0.318, respectively. There was no statistically significant difference between the low and high complexity ICC.

Conclusions: The results presented here indicate that web-based AI evaluation of laryngoscopy pathologic images is currently lacking requiring specialist guidance. Specific programming to guide AI has shown promise, but current closed source AI laryngology evaluation is weak.