(0312) Enhancing Multilingual Patient Education: ChatGPT’s Accuracy for Sudden Sensorineural Hearing Loss Queries in English and Spanish

Introduction: The increasing prevalence of ChatGPT in medicine has raised considerable apprehension regarding its precision in dissemination of information, particularly in non-English languages. This study investigates the effectiveness of ChatGPT in responding to common patient queries on sensorineural hearing loss (SSNHL) in English and Spanish. The objective is to address concerns regarding its proficiency in addressing medical inquiries when presented in a language divergent from its primary programming.

Methods: Employing ChatGPT 3.5, questions from the AAO-HNSF guidelines were presented in English and Spanish. Two distinct otolaryngologists, each proficient in either language, submitted 7 questions to the chatbot through separate accounts. To guarantee linguistic accuracy, a two-stage translation process was implemented. Subsequently, the users thoroughly reviewed and graded the responses using a 4-point Likert scale. To ensure uniform application of the Likert scale between the two reviewers, a third independent evaluator reviewed the consistency in grading.

Results: Across both languages, the responses displayed a native-level language proficiency. Further, there was uniformity in the answers to 6/7 questions in both English and Spanish, indicating high consistency. In terms of accuracy, 2/7 perfectly adhered to the guidelines (grade 4/4), 4/7 were mostly accurate with minor details missing (grade 3/4), and the remaining question was partially adequate (grade 2/4) in Spanish and mostly adequate (grade 3/4) in English.

Conclusions: ChatGPT offers patients comprehensive and guideline-conforming answers to their medical queries, whether in English or Spanish. The utilization of ChatGPT in these two widely spoken American languages can be safely endorsed to patients, in conjunction with professional medical advice.