Evaluation of online video content related to reverse shoulder arthroplasty: a YouTube-based study
Article information
Abstract
Background
Reverse shoulder arthroplasty (RSA) has evolved continuously over recent years, with expanded indications and better outcomes. YouTube is one of the most popular sources for health-related information available to patients. Evaluating the quality of YouTube videos concerning RSA is important to ensure proper patient education.
Methods
YouTube was queried for the term “reverse shoulder replacement.” The first 50 videos were evaluated using three different scores: Journal of the American Medical Association (JAMA) benchmark criteria, the global quality score (GQS), and the reverse shoulder arthroplasty-specific score (RSAS). Multivariate linear regression analyses were conducted to determine the presence of a relationship between video characteristics and quality scores.
Results
The average number of views was 64,645.78±264,160.9 per video, and the average number of likes was 414 per video. Mean JAMA, GQS, and RSAS scores were 2.32±0.64, 2.31±0.82, and 5.53±2.43, respectively. Academic centers uploaded the highest number of videos, and surgical techniques/approach videos was the most common video content. Videos with lecture content predicted higher JAMA scores whereas videos uploaded by industry predicted lower RSAS scores.
Conclusions
Despite its massive popularity, YouTube videos provide a low quality of information on RSA. Introducing a new editorial review process or developing a new platform for patients’ medical education may be necessary.
Level of evidence
Not applicable.
INTRODUCTION
Shoulder arthroplasty has witnessed continuous evolution over recent decades with continuous developments in techniques and instrumentation [1,2]. Different shoulder arthroplasty procedures have been established, all of which have provided pain relief and restoration of function for a substantial number of patients. As such, a dramatic increase in these procedures has been witnessed over the past decade [2]. Specifically, the reverse shoulder arthroplasty (RSA) has seen a significant growth recently, mainly due to its expanded indications, successful outcomes, and improved implant technology, making it favored over other shoulder procedures [2,3]. Wagner et al. [3] noted a 191.3% increase in the number of reverse shoulder arthroplasty-specific score (RSAS) performed in the United States from 2011 to 2017, indicating a prominent rise in the popularity and utility of this procedure in orthopedic surgery.
Approximately 56% of the total world population used the internet in 2022, compared to around 5% back in 2000 [4]. Seventy-five percent of patients reportedly base their health-related decisions on information acquired through the internet [5]. As such, it is of pivotal importance to have access to reliable online information regarding medical procedures so that patients can make well-informed decisions related to their health care. Among the different sources of information present on the internet, YouTube is one of the most popular with billions of reported users each month and one billion hours of video watching recorded each day [6]. Given that it provides content that can educate patients in an audiovisual format, it comes to no surprise that YouTube constitutes a favorable source of patient education regarding different medical procedures, including those relating to orthopedics like shoulder replacement surgeries [7,8].
As shared decision-making has become a central tenet in the healthcare field, patients play a major role in deciding to undergo an RSA, which is essentially an elective procedure design to reduce pain and restore function in the glenohumeral joint [9]. As such, evaluating the quality of online sources of information like YouTube videos can give insight into what content patients are basing their health care decisions on. Several studies have evaluated video content on YouTube regarding different orthopedic topics [7,8,10,11]. Nevertheless, and to our knowledge, none have explored the quality of content pertaining to RSA. Accordingly, and considering the high rise in utility and popularity of RSA in recent years, the purpose of our study was to evaluate the reliability and educational content of YouTube videos concerning the procedure.
METHODS
YouTube Search
The YouTube.com online database was queried on October 14, 2022, using the keywords “reverse shoulder replacement.” Similar to previous YouTube-based studies in orthopedic literature, the videos that appeared based on the keywords were sorted by relevance, and the first 50 videos were evaluated [8,12,13]. A video was excluded if it was not in English. Recorded variables included video duration, number of views, upload source, content category, days since video upload, view ratio (number of views/number of days since upload), and number of likes.
Video Upload Sources and Content Categories
We categorized and assessed the video sources/uploaders according to the following categories: academic (e.g., universities, colleges, medical centers with academic affiliations), medical centers (e.g., centers or health care systems not academically affiliated), physicians or health care providers (e.g., affiliated with independent physicians or physical therapists), medical industry (e.g., companies that manufacture surgical devices), and other (sources that do not fall in any of the aforementioned categories).
Video content was categorized into one of the following groups: patient education (informative videos targeting patients and general public), lectures (informative videos targeting medical professionals, often in the form of academic talks or presentations), patient experience (videos showcasing the experience of patients post-surgery), and surgical technique or approach (videos focusing primarily on the technique and surgical concepts of RSA).
The Assessment of Video Reliability and Educational Content
The Journal of the American Medical Association (JAMA) benchmark criteria were used to analyze the credibility of the videos [14]. JAMA benchmark criteria (Table 1) are objective tools that use a point system. A point is assigned for each of the four core principles that can be identified in online videos. A maximum score on this scale is a score of four, indicating a highly credible source, whereas a minimum score of zero indicates a questionable source with poor credibility. Although this tool is unvalidated, numerous YouTube-based cases in previous orthopedic literature use these criteria to judge the usefulness, credibility, and reasonableness of sources [10,15].
To assess the nonspecific educational quality and video flow, the global quality score (GQS) was used, which uses a five-point scale to determine the educational value and quality of online content (Table 2). The importance of the presented information, the amount of relevant information, the thoroughness of the descriptions and explanations, and the flow of the video were taken into account when using the GQS tool. A maximum score of five indicates excellent quality and flow. Although the GQS assessment is unvalidated, this tool has been used in numerous YouTube-based studies in the orthopedic literature [16,17].
To evaluate the quality of the reverse total shoulder arthroplasty-specific educational content, we created the RSAS composed of sixteen items determined using the guidelines published by the American Academy of Orthopedic Surgeons (Table 3) [18]. Although this assessment tool is unvalidated and novel, numerous YouTube-based studies on the educational value of online resources used unvalidated topic-based tools in the literature [7,8,16,19-21]. To use the RSAS tool, an observer assigns one point for each item present in the video. A maximum score of fifteen indicates a video with high reverse total shoulder arthroplasty-specific educational quality. Two independent observers (JK and PB) scored each of the videos according to the JAMA, GQS, and RSAS scoring tools, and an average score from each tool was obtained.
Statistical Analysis
To determine the educational quality and credibility of the video content, descriptive statistics were used to quantify the video characteristics. Continuous variables were expressed as means with a standard deviation while providing maximum and minimum values. Categorical variables were expressed as relative frequencies and percentages. The kappa statistic was used to measure the inter-rater reliability of the categorical variables in the manuscript. To determine whether the credibility or educational content quality were influenced by either (1) the video source/uploader or (2) the content, a Kruskal-Wallis test was used for the non-normal data. Multivariate linear regression analyses were conducted to determine whether a relationship was present between specific video characteristics and the credibility of the video (JAMA score) and the quality of the content (GQS and RSAS). Statistically significant results were indicated by a P-value below 0.05.
RESULTS
The first 50 videos that appeared in our initial search were included in our analysis. Video duration ranged between 35 seconds and 62 minutes, 47 seconds, with a mean of 12 minutes and 15 seconds (±14 minutes, 56 seconds) (Table 4). The oldest video was uploaded 4,215 days prior to the search, and the most recent video was uploaded 34 days prior to the search. On average, the included videos were uploaded 1,526±1,079.6 days prior to our search (Table 4). The total collective number of views was 3,232,289 views, and total number of likes was 20,684 likes, leading to an average of 64,645.78±264,160.9 views and 414 likes per video (Table 4). The average view ratio was 32.96±127.7, and the average number of likes was 413±1,848.6 per video. There were significant between-group effects observed for the view ratio with the uploader source (P=0.025), with industry videos having the highest view ratio (149.2±335.9).
With regard to video sources, academic centers had the highest number of uploaded videos at sixteen videos (32%), whereas industry-produced videos had the lowest number at seven videos (14%). The most commonly encountered video content was that of patient education with 18 videos (36% each), whereas the least common video content was patient experience with eight videos (16%).
Video Reliability and Educational Content Analysis
When assessing for inter-rater reliability, the kappa value was 0.831 for JAMA scores (near perfect agreement), 0.623 for GQS (substantial agreement), and 0.502 for RSAS (moderate agreement). The mean JAMA score was 2.32±0.64, the mean GQS was 2.31±0.82, and the mean RSAS was 5.53±2.43 (Tables 5 and 6). A Kruskal-Wallis test for non-normal data was used to determine whether the video reliability and the quality of educational content differed by upload source and by content classification. Significant between-group effects were observed for the JAMA score based on the content category (P=0.006), with lecture videos conferring the highest mean JAMA score (Table 6). No significant between-group effects were observed for the JAMA score based on the video upload source (P=0.082). Similarly, between-group effects were not observed for the GQS or RSAS by either content category (P=0.481 and P=0.247, respectively) or video upload source (P=0.916 and P=0.836, respectively).
Predictors of Video Reliability and Educational Content Quality
The influence of video characteristics, the video content category, and the video upload source on the JAMA score, GQS, and RSAS were investigated using multivariate linear regression models including all collected video characteristics, content categorizations, and upload source variables. These models identified lecture content as a significant predictor of a higher JAMA score (β=0.777, P=0.033). Meanwhile, videos uploaded by independent sources (“other”) were negative predictors of the GQS score (β=–1.20, P=0.039) and the RSAS score (β=–3.69, P=0.023). Finally, videos uploaded by industry channels were negative predictors of the RSAS score (β=–3.08, P=0.032).
DISCUSSION
Our study found that the top 50 YouTube videos on RSA accumulated around 3.2 million views, with industry videos acquiring the highest number of views. The videos had mean JAMA, GQS, and RSAS scores of 2.32±0.64, 2.31±0.82, and 5.53±2.43, respectively. Academic centers were the video source with the highest number of videos uploaded, and surgical techniques/approach videos were the most common according to video content. Lecture videos showed and predicted higher JAMA scores, whereas industry videos predicted lower RSAS scores. Considering the prominent involvement of online resources in directing patient health care decisions, addressing the inaccurate content on online platforms is essential.
It is evident through our findings that RSA has been a popular topic among YouTube users, as shown by the large number of views accrued in our study. This falls in line with the rising popularity and prevalence of RSA in shoulder surgery and how it’s garnering much more interest from patients, surgeons, and industry personnel [2,3]. This is also expected given the established popularity of YouTube and the availability of numerous videos and content that convey information and insight regarding different surgical procedures and medical interventions [5]. The ease of the search, as well as the audiovisual presentation of different pathologies and their treatments, renders this tool a valuable resource for people who are planning on making health care decisions [5,22]. In addition, the upload sources with the highest number of views were medical/orthopedic industry sources, and this reflects the increasing commercial and financial interest in this procedure. Uploaded sources related to the medical industry were also found to be negative predictors for RSAS, which is alarming yet expected, given that the main interest of these sources resides in promotion and marketing rather than delivery of quality patient education [23].
In general, YouTube videos proved to be a poor online source for information regarding RSA, as shown by the poor quality scores attained. This falls in line with other YouTube-based studies in the literature that reported a low quality of information for orthopedic-related pathologies and procedures [7,16,20,21,24]. The unrestricted ability to upload videos and the lack of editorial evaluation on accuracy and comprehensiveness of content are potential contributors to the low scores exhibited in our study [8]. This raises concerns for the status of patient education in orthopedics and the ability to make well-informed health care decisions based on the available online video content. It also highlights the importance of proper patient education during clinic visits and appropriate counseling toward reliable and accurate sources of information.
The Kruskal-Wallis test in our study failed to demonstrate significant differences in quality scores between the different uploaded source types. That is surprising considering that academic and physician sources were expected to provide higher quality information than other sources, given the involved knowledge and expertise [8]. Videos with lecture content generally had higher quality scores than did other content types. Lecture videos often involve physicians and surgeons with high knowledge and expertise in their respective fields. These videos are usually thorough, comprehensive, and accurate in delivering information [25]. As such, it is not surprising that these had higher scores in our study. Nevertheless, the concerns with lecture videos are that they are often cater to medical personnel rather than patients, and this limits their appeal to patients, who might prefer shorter videos with simpler concepts and terminologies.
Our study demonstrates important findings regarding the educational value of YouTube videos related to RSA. Given the high number of views and likes accrued by these videos, it is evident that this informative video format is appealing to the RSA patient. Nevertheless, this low informative quality needs to be addressed as online sources play a huge role in patient education and, accordingly, patient-related health care decisions [5]. Introducing an editorial process that monitors medical videos on YouTube may be helpful in limiting the spread of inaccurate or harmful information. If that is not possible, introducing a new platform that publishes reliable expert-reviewed content may be the tool to provide proper education on orthopedic-related topics to patients. In addition, as lectures were shown to generally have better quality scores than other content categories, uploading lecture-based content that caters to patients rather than professionals can be helpful in disseminating credible educational content.
Several limitations exist for this study. While the query “reverse shoulder replacement” may not have provided a thorough representation of all videos targeting this topic, we decreased selection bias by systematically analyzing the first 50 videos in our search. Most internet users do not go beyond the first two pages populated by a search, and this is in accordance with the methodology implemented in our study. Moreover, our study involved the use of an unvalidated quality assessment tool, though several similar studies used similar tools to assess and evaluate online resources with acceptable inter-observer reliability for both the validated and unvalidated tools both in the literature and in our study [7,11,12,16,19-21]. This indicates that the findings in our study relating to low quality of information is, in fact, accurate.
CONCLUSIONS
Based on the number of views and likes, our study showed that YouTube videos focusing on RSA are popular and attractive to the prospective patient. Nevertheless, these videos provide a low quality of information on the surgical procedure and do not offer an appropriate educational value. More lectures, which generally had higher quality scores in our study, need to be published, and these need to cater to patients more than to medical professionals. Editorial reviewing of videos that showcase medical content may be required to ensure proper informative material is being disseminated. If that is not possible on YouTube, a new patient-friendly online video sharing platform, with expert peer review, should be developed to ensure accurate and reliable information is available to patients in need.
Notes
Author contribution
Conceptualization: MYF, JK, PB, JS, ASV, JAA. Data curation: JK, PB. Formal Analysis: ASV.Investigation: ASV, JAA. Methodology: MYF, JK, PB, JS. Supervision: JAA. Writing – original draft: MYF, JK, PB. Writing – review & editing: MYF, JS, ASV, JAA.
Conflict of interest
JAA would like to disclose royalties from: DJO Global, Zimmer-Biomet, Smith and Nephew, Stryker, Globus Medical, Inc.; research support as a PI from: Lima Corporate—Italy, Orthofix, Arthrex, OREF; royalties, financial or material support from: Wolters Kluwer; and board member/committee appointments for: American Shoulder and Elbow Society, Pacira. No other potential conflicts of interest relevant to this article were reported.
Funding
None.
Data availability
Contact the corresponding author for data availability.
Acknowledgments
None.