(0745) The Use of Chatbots in Head and Neck Mucosal Malignancy Treatment Recommendations

Introduction: As cancer patients increasingly use chatbots, it is crucial to recognize ChatGPT's potential in enhancing health literacy while ensuring validation to prevent misinformation. This study aims to assess ChatGPT-3.5’s capability to provide appropriate staging and treatment recommendations for head and neck mucosal malignancies.

Methods: Forty distinct prompts were introduced into ChatGPT to inquire about staging and treatment recommendations for head and neck mucosal malignancies. Prompts were created based on head and neck cancer (HNC) disease descriptions (cancer location, tumor size, lymph node involvement, and symptoms). Staging and treatment recommendations according to the 2021 National Comprehensive Cancer Network (NCCN) guidelines were scored by three fellowship-trained HNC surgeons from two separate tertiary care institutions. HNC surgeons assessed the accuracy of staging and treatment recommendations, such as the completeness of surgery and the appropriateness of treatment modality.

Results: The majority of ChatGPT's responses aligned with the NCCN 2021 guidelines for mucosal HNC by recommending the correct first-line treatment (95%), but 55% of responses contained inaccurate staging. Neck dissection was incorrectly omitted from treatment recommendations in 50% of the cases. Moreover, 40% of ChatGPT's treatment recommendations were deemed unnecessary.

Conclusions: This study emphasizes ChatGPT's potential in HNC patient education, aligning with NCCN guidelines for mucosal malignancies, but highlights the importance of ongoing refinement and scrutiny due to observed inaccuracies in TNM staging, incomplete surgery options, and inappropriate treatment recommendations. Otolaryngologists can use this information to caution patients, families, and trainees regarding the use of ChatGPT for HNC education without expert guidance.