Skip to main content
AAO-HNSF 2024 Annual Meeting & OTO EXPO Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.
Back

Scientific Oral Presentations

Comprehensive Otolaryngology

Session: Scientific Oral Presentations 3 - Comprehensive Otolaryngology

Performance and Consistency of GPT-4 vs Otolaryngologists: A Clinical Case-Series

Monday, September 30, 2024
1:24 PM – 1:30 PM EDT
Location: MBCC, 203-205
Disclosure(s):
Antonino Maniaci, MD, PhD:

faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

Jerome R. Lechien, MD, PhD, MS:

faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

Introduction: The purpose of this study was to study the performance of ChatGPT-4 in the management of cases in otolaryngology head and neck surgery.

Methods: The study design was a prospective case series in 2023. The setting was in multicenter university hospitals. History, clinical, physical, and additional examinations of outpatients consulting in otolaryngology departments of CHU Saint-Pierre and Dour Medical Center were presented to ChatGPT-4, which was interrogated for differential diagnoses, management, and treatment(s). According to specialty, the ChatGPT-4 responses were assessed by two blinded board-certified otolaryngologists with the Artificial Intelligence Performance Instrument (AIPI).

Results: A total of 100 cases were presented to ChatGPT-4. ChaGPT-4 indicated a mean of 3.34 (95%CI: 3.09, 3.59) additional examinations per patient versus 2.10 (95%CI: 1.76, 2.34; p = 0.001) for the practitioners. There was strong consistency (k > 0.6) between otolaryngologists and ChatGPT-4 for the indication of upper aerodigestive tract endoscopy, PET-CT, audiometry, tympanometry, and psychophysical evaluations. Primary diagnosis was correctly performed by ChatGPT-4 in 38% to 86% of cases, depending on subspecialty. Additional examinations indicated by ChatGPT-4 were pertinent and necessary in 8% to 31% of cases, while the treatment regimen was pertinent in 12% to 44% of cases. The performance of ChatGPT-4 was not influenced by the human-reported level of difficulty of clinical cases.

Conclusions: ChatGPT-4 may be a promising adjunctive tool in otolaryngology, providing extensive documentation about additional examinations, primary and differential diagnoses, and treatments. ChatGPT-4 is more effective in providing a primary diagnosis, and less effective in the selection of additional examinations and treatments.

    Presenting Author(s)

  • AM

    Antonino Maniaci, MD, PhD

    MD, PhD, Assistant Professor
    University of Enna Kore
    Enna, Sicilia, Italy

    Disclosure(s):

    faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.

    Resources

    • Co-Author(s)

  • CC

    Carlos Chiesa Estomba, MD, PhD

    University of San Sebastian, Spain

    Disclosure information not submitted.

  • LV

    Luigi Varia, MD

    Dr
    Sassari University, ITA

    Disclosure information not submitted.

    • Senior Author(s)

  • JL

    Jerome R. Lechien, MD, PhD, MS

    Professor of Surgery
    University of Mons
    Sombreffe, Namur, Belgium

    Disclosure(s):

    faculty for this accredited education activity has no relationship(s) with ineligible companies to disclose.


Back to Top