The repetitive forces exerted on the ulnar collateral ligament (UCL) by the overhead throwing motion place it at significant risk for injury [1-5]. Although UCL reconstruction has long been considered the gold standard treatment for these injuries and produces excellent return to sport (RTS) rates, typical rehabilitation timelines exceed 12 months [6]. In recent years, UCL primary repair with suture augmentation has gained traction [7-10]. In a recent comparison of UCL primary repair versus reconstruction, Dugas et al. [2] reported no difference in RTS rates or patient reported outcomes between groups but found that UCL repair patients were able to return to practice (6.7 months vs. 10.2 months) and competition (9.2 months vs. 13.4 months) significantly faster than those who underwent UCL reconstruction. Given the potential for an expedited return to competition after UCL repair, patients are increasingly interested in pursuing this procedure following injury.

UCL injuries primarily affect young, active patient populations, and prior research has shown that such patients often use the internet to find health information [11]. Recent years have seen a dramatic advance in the commercial availability of generative artificial intelligence (AI) large language models (LLMs) such as ChatGPT (OpenAI, San Fransico, California), Gemini (Google, Mountain View California), and Grok (xAI, Palo Alto, California) [12]. These AI LLMs can rapidly provide information about a wide array of topics, including medicine [13]. However, those responses are not vetted by subject-level experts and can include inaccuracies or potentially misleading statements [14]. A flurry of investigations evaluating the accuracy and readability of generative AI responses to common patient questions has recently been published about a range of orthopedic topics, including UCL reconstruction [13,15,16]. Those studies have reported that although AI responses to questions about UCL reconstruction are generally accurate, nuanced questions might be answered insufficiently, and the level of readability generally exceeds what is recommended for patient education materials. To our knowledge, no studies have yet evaluated AI-generated responses to common patient questions specifically about UCL repair. Given the increasing interest in this procedure and the nuances surrounding its indications, it is essential to assess whether generative AI models provide accurate and readable information to patients.

Our objective in this prospective study was to evaluate AI-generated responses to common patient questions about UCL repair. It was hypothesized that the AI LLMs would generate reasonably accurate answers to general questions but that their answers to more nuanced questions would lack the detail needed to make clinical decisions and that the language used would be more complex than what is recommended for patient education materials.

This study did not involve human participants or identifiable patient data; therefore, institutional review board approval and informed consent were not required.

The most significant finding of this study is that all three AI LLMs produced generally accurate information about UCL repair, but at a reading level well above recommended standard for patient education. Additionally, comparisons between models revealed that Gemini produced more accurate information than the other AI LLMs but with a tradeoff of reduced readability. Gemini was also the only AI LLM included in this study that routinely cited reputable sources for each response. To our knowledge, this is the first study to evaluate AI-generated responses about UCL primary repair with suture augmentation and how it differs from reconstruction.

These findings are consistent with previous studies that evaluated AI-generated responses to UCL reconstruction questions. In an analysis of ChatGPT 3.5 and ChatGPT 4’s responses to UCL reconstruction related–questions, Shaari et al. found that although most AI responses were generally accurate, the newer ChatGPT 4 model produced more accurate answers than ChatGPT 3.5, indicating that AI-generated information about UCL injuries might be improving over time [15]. In a 2024 study, Johns et al. [16] evaluated ChatGPT 3.5 responses to ten common patient questions about UCL reconstruction and reported that its response to the question “Should I have a UCL reconstruction or repair?” was “unsatisfactory requiring substantial clarification” (CRRS=4) due to a significant oversimplification of the complexities of UCL repair indications. In this study, both ChatGPT 4o (Q7: CRRS=2, Q8: CRRS=2) and Gemini 2.5 Flash (Q7: CRRS=1.67, Q8: CRRS=2) provided satisfactory information requiring no more than minimal clarification about the difference between UCL reconstruction and repair and the indications for UCL repair. It is possible that AI-generated responses to questions about the nuances of UCL repair are rapidly improving over time.

However, a major concern about the utility of AI LLMs as patient education tools remains the readability of responses. In this study, every response from all three models was written far above the AMA/NIH 6th grade reading level recommendation, with most responses being written at a college level or higher. Previous studies have also found that AI-generated responses to orthopedic questions are written above what many patients can comprehend [15,17]. Although it is encouraging that the accuracy of AI-generated responses seems to be improving, that improvement is of little value to patients if they cannot comprehend the information as it is presented. Additionally, there appears to be a tradeoff between accuracy and readability. This study found that although Gemini was the most accurate AI LLM tested, its responses were also written at the most complex reading level. Caution and an individualized approach must be taken when recommending the use of AI LLMs because some patients could be overwhelmed by the complexity of the information provided by these models. Clinicians should offer patients guidance on the strengths and limitations of AI and serve as the final expert resource who provides clarification as needed to prevent misinformation and frustration.

In terms of comparison among the three tested AI LLMs, the findings of this study are largely consistent with the literature. Other studies evaluating Gemini’s performance on orthopedic topics, including anterior cruciate ligament reconstruction and Achilles tendon rupture, similarly found it to outperform ChatGPT and Grok in terms of accuracy and clarity [20,21]. In this study, Gemini not only produced the most accurate responses but was also the only AI LLM that included sources for each response. The sources cited by Gemini were generally reputable, such as Massachusetts General Hospital, the American Academy of Orthopedic Surgeons, and peer-reviewed medical journals [26-28]. Citation of sources is imperative because it not only provides interested patients with a roadmap for source information but can also give clinicians an idea of where AI is falling short by allowing them to reference alternative trustworthy sources [13]. Although readability remains a concern, Gemini might have the greatest current utility for patient education due to its superior accuracy and consistent citation of reputable sources in this study.

This study is not without limitations. First and foremost, the accuracy ratings were the subjective opinions of the raters. To limit bias, all raters were fellowship trained in either sports medicine or shoulder/elbow surgery and blinded to each other’s ratings, and the three independent ratings were averaged for each question during the data analysis. Additionally, AI LLMs are developing rapidly. They continue to change over time and might not provide the same responses if prompted at the time of publication. Therefore, further advances in AI could limit the long-term applicability of these findings. Finally, this study did not assess patient comprehension directly, which would be an important direction for future work.

AI models generated generally accurate information about UCL repair but at reading levels far above the AMA/NIH recommendations. In this study, Gemini was the most accurate model, but it produced the least readable content.