Evaluation of large language models’ responses to patient questions about avascular necrosis of the femoral head

Evaluation of large language models’ responses to patient questions about avascular necrosis of the femoral head. Evaluation of large language models’ responses to patient questions about avascular necrosis of the femoral head Article Open access Published: 01 November 2026 Cite this article You have full access to this open access article Download PDF View saved research Evaluation of large language models’ responses to patient questions about avascular necrosis of the femoral head Download PDF Abstract Artificial intelligence chatbots are rapidly gaining popularity for acquiring health information, but the quality of responses seems to differ between platforms. This study investigated the responses of ChatGPT, Gemini, Copilot, Claude, and DeepSeek to 15 frequently asked questions regarding femoral head avascular necrosis.

Each platform answered each question once, for a total of 75 responses. Four orthopedic surgeons assessed the response using the JAMA benchmark criteria, DISCERN, an adapted five-point scale of clinical accuracy, and a binary safety score. Readability was measured by Flesch reading ease score (FRES) and Flesch-Kincaid grade level (FKGL).

Models were compared by using Friedman tests and Bonferroni-corrected Wilcoxon signed-rank tests. Claude ranked the highest by JAMA and DISCERN scores and that DeepSeek outperformed ChatGPT, Gemini, and Copilot in terms of DISCERN. Clinical accuracy differed significantly among the models ( p < 0.001).

Medical Disclaimer: This article is for informational purposes only and does not constitute medical advice. Always consult with qualified healthcare professionals for medical decisions and treatment options.