May AI robots provide accurate information about SSHL? A comparative analysis of ChatGPT and Gemini

Abstract

Aim: This study aimed to compare the accuracy and comprehensiveness of answers provided by the artificial intelligence (AI) models ChatGPT 3.5 and Gemini in response to medical inquiries concerning sudden sensorineural hearing loss (SSHL).

Materials and Methods: The researchers created a series of 20 open-ended questions derived from the 2019 guidelines of the American Academy of Otolaryngology-Head and Neck Surgery and evaluated the accuracy and completeness of the AI-generated responses.

Results: Gemini achieved higher average scores in both completion and accuracy compared to ChatGPT. While the difference in accuracy scores was not statistically significant, the difference in completion scores was found to be statistically significant. Both AI models were able to provide accurate answers (scoring 5 or 6 on a 6-point scale) to the majority of the questions, with Gemini achieving a higher success rate than ChatGPT.

Conclusion: The study highlights the potential of AI models to provide useful medical information, but also emphasizes the need for caution and oversight when relying on these technologies, particularly in the medical field. The authors recommend educating healthcare professionals about the limitations of AI, obtaining patient consent for AI-assisted medical decisions, and integrating ethical principles into the development and deployment of these technologies.