(0314) Evaluation of Patient Education Material on Myringotomy Tubes for Recurrent Otitis Media Infection

Introduction: Over a million children undergo myringotomy and ear tube surgery annually, prompting parents to seek information online about their child’s procedure. This study aims to compare the quality and readability of patient education materials from artificial intelligence (AI) chatbots ChatGPT and Google Bard and information obtained through Google search.

Methods: Three questions (“What are ear tubes?”, “Why are ear tubes recommended?”, “What is the surgery and recovery for ear tubes?”) were asked in ChatGPT and Google Bard addressing three domains (“Condition,” “Investigation,” and “Treatment”). Google was also queried for “Ear tubes,” “Myringotomy and tubes,” and “Tympanostomy tubes.” Website quality was assessed using the DISCERN instrument. Readability was assessed using the Flesch-Kincaid Grade Level and Flesch-Kincaid Reading Ease scores.

Results: The average DISCERN score for websites was 52 (SD = 13.1, Median = 55.5), out of 80. The mean Flesch-Kincaid Reading Grade Level was 8 (SD = 7.1, Median = 3), and the mean Flesch-Kincaid Reading Ease score was 55 (SD = 12.3, Median = 57.7). ChatGPT and Google Bard’s “Condition” responses each had a DISCERN score of 46. ChatGPT’s response had a Flesch-Kincaid Grade Level of 13.1 and Google Bard had 9.5. The Flesch-Kincaid Reading Ease score was 41 for ChatGPT and 61 for Google Bard. The “Investigation” responses had DISCERN scores of 46 for ChatGPT and 66 for Google Bard. The Flesch-Kincaid Grade Level was 13.9 for ChatGPT and 12.4 for Google Bard, with a Flesch-Kincaid Reading Ease of 38.9 for ChatGPT and 34.9 for Google Bard. For the “Treatment” question, ChatGPT had a DISCERN score of 45 and Google Bard 34. ChatGPT’s Flesch-Kincaid Grade Level was 15.7 and Google Bard’s 9.8, with Flesch-Kincaid Reading Ease scores of 36.2 for ChatGPT and 53.9 for Google Bard.

Conclusions: Sites and generative AI sources educating patients and families regarding myringotomy and ventilation tubes are of “fair” quality and at readability levels far above the recommended 6th grade reading level. Google search results were superior to generative AI in terms of readability with a mean 8th grade level.