Artificial Intelligence Model Versus Health Care Providers: Predicting Cochlear Implant Eligibility

Introduction: Cochlear implant (CI) underuse has been attributed to difficulty identifying patients who would benefit from the lengthy eligibility testing. We generated an AI model that predicts CI eligibility from basic audiometry data and aim to evaluate its performance against human providers to determine if it can facilitate appropriate referrals.

Methods: With IRB approval, 50 de-identified basic audiometry data sets were pulled from patients at our institution that underwent CI eligibility testing and weren't included in the AI model’s training set. A survey distributed to providers at our institution randomly presented this data and collected predictions of CI eligibility test scores (AZBio and CNC), proxies for prediction confidence, demographics and more. T-tests and mixed-effects modeling with post-hoc analysis was used for data analysis.

Results: From 44 respondents, 151 independent predictions were received. Most respondents practice in the context of otolaryngology (70%). Overall, the AI model made significantly more accurate predictions of AZBio and CNC scores than providers (differences in MAE: 6.1, 5.5; p = 0.005, 0.005). The AI model was significantly more accurate than providers that make referrals infrequently (p = 0.003, 0.01), but not those that make referrals frequently. After seeing the AI model’s predictions, providers predict increased probabilities that eligible patients will have eligible scores (mean change for AZBio, CNC: 2.2, 1.4, p = 0.13, 0.25) and decreased probabilities that ineligible patients will have eligible scores (mean change for AZBio, CNC: -18.4, -6.7, p = 0.0005, 0.087). Providers did not become significantly more likely to appropriately refer or not refer a patient for CI eligibility testing.

Conclusions: Our AI model outperforms providers at predicting cochlear implant eligibility from basic audiometry data. This model is unlikely to change providers' decisions to refer a patient for CI eligibility testing; however, automating predictions of CI eligibility at the time of audiometry testing may signal providers to consider making that decision and boost their confidence.