Skip to main content
AAO-HNSF 2024 Annual Meeting & OTO EXPO Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.
Back

Late-Breaking Scientific Oral Presentations

Care Equity, Quality, and Safety

Session: Late-Breaking Scientific Orals: The Otolaryngology Community and Systems

Triaging Emergency Cases in Otolaryngology Using Different Artificial Intelligence Chatbots

Monday, September 30, 2024
10:18 AM – 10:24 AM EDT
Location: MBCC, 222
Disclosure(s):
Chandala Chitguppi, MD:

faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

Introduction: Triaging emergency consults is one of the most challenging aspects of otolaryngology (ORL) residency training for a junior trainee on overnight calls and there can be a steep learning curve. AI platforms may be able to assist junior trainees with clinical decision-making. This study assesses the accuracy of different AI-supported chatbots for triaging 15 commonly seen emergencies in ORL.

Methods: Different AI chatbots (ChatGPT 4.0, Google Gemini, Bing Chat) were provided with 15 clinical scenarios of commonly seen ORL emergencies. The age, gender, and duration of symptoms were kept constant to reduce the confounding. Scenarios were divided into 5 groups: neurotology (O), laryngology (L), pediatric ORL (P), rhinology/skull base (R), and facial plastics (F). Each group had 3 scenarios: one life threatening emergency, one non-life threatening but time sensitive, one scenario which needed treatment but not very time sensitive. Another group (Mixed-M) was created wherein all 5 life-threatening emergencies from each group mentioned above were included. Each group was tested twice after changing the order of the scenarios. Accuracy was calculated on a scale (1-3 or 1-5) with each number representing the number of correct responses.

Results: On assessing the accuracy of responses: for ChatGPT 4.0: O (3/3), R (3/3), L (3/3), P (3/3), F (3/3) and M (5/5), for Google Gemini: O (1/3), F (3/3), R (1/3), P (1/3), L (0/3), and M (2/5), for Bing chat: O (1/3), R (0/3), F (3/3), L (3/3), P (3/3) and M (5/5). The order of triage and urgency levels did not always correlate in Bing chat.

Average response time was the least for ChatGPT (p < 0.05). Additional information provided by the AI chatbot regarding differential diagnosis and rationale for the triage was most accurate for Chat GPT (p < 0.05)

Conclusions: ChatGPT responses for triaging ORL emergencies are superior in quality compared to Google Gemini and Bing Chat responses. Junior trainees may benefit from using ChatGPT to triage different ORL emergency scenarios as a supplementary guide. However, it should not replace the guidance of senior trainees and attending otolaryngologists.

    Presenting Author(s)

  • Chandala Chitguppi, MD

    Resident
    University of Maryland
    Baltimore, MD, United States

    Disclosure(s):

    faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

    • Senior Author(s)

  • GS

    Grace E. Snow, MD

    University of Maryland School of Medicine, United States

    Disclosure information not submitted.