Evaluation of online video content related to reverse shoulder arthroplasty: a YouTube-based study

Article information

Clin Shoulder Elb. 2023;26(2):162-168
Publication date (electronic) : 2023 May 16
doi : https://doi.org/10.5397/cise.2022.01452
Division of Shoulder and Elbow Surgery, Rothman Orthopedic Institute, Philadelphia, PA, USA
Correspondence to: Mohamad Y. Fares Division of Shoulder and Elbow Surgery, Rothman Orthopedic Institute, 925 Chestnut St 5th Floor, Philadelphia, PA 19107, USA. Tel: +1-26-7905-0552 Email: mohamadfaresmd@gmail.com
Received 2022 December 22; Revised 2023 January 31; Accepted 2023 February 4.

Abstract

Background

Reverse shoulder arthroplasty (RSA) has evolved continuously over recent years, with expanded indications and better outcomes. YouTube is one of the most popular sources for health-related information available to patients. Evaluating the quality of YouTube videos concerning RSA is important to ensure proper patient education.

Methods

YouTube was queried for the term “reverse shoulder replacement.” The first 50 videos were evaluated using three different scores: Journal of the American Medical Association (JAMA) benchmark criteria, the global quality score (GQS), and the reverse shoulder arthroplasty-specific score (RSAS). Multivariate linear regression analyses were conducted to determine the presence of a relationship between video characteristics and quality scores.

Results

The average number of views was 64,645.78±264,160.9 per video, and the average number of likes was 414 per video. Mean JAMA, GQS, and RSAS scores were 2.32±0.64, 2.31±0.82, and 5.53±2.43, respectively. Academic centers uploaded the highest number of videos, and surgical techniques/approach videos was the most common video content. Videos with lecture content predicted higher JAMA scores whereas videos uploaded by industry predicted lower RSAS scores.

Conclusions

Despite its massive popularity, YouTube videos provide a low quality of information on RSA. Introducing a new editorial review process or developing a new platform for patients’ medical education may be necessary.

Level of evidence

Not applicable.

INTRODUCTION

Shoulder arthroplasty has witnessed continuous evolution over recent decades with continuous developments in techniques and instrumentation [1,2]. Different shoulder arthroplasty procedures have been established, all of which have provided pain relief and restoration of function for a substantial number of patients. As such, a dramatic increase in these procedures has been witnessed over the past decade [2]. Specifically, the reverse shoulder arthroplasty (RSA) has seen a significant growth recently, mainly due to its expanded indications, successful outcomes, and improved implant technology, making it favored over other shoulder procedures [2,3]. Wagner et al. [3] noted a 191.3% increase in the number of reverse shoulder arthroplasty-specific score (RSAS) performed in the United States from 2011 to 2017, indicating a prominent rise in the popularity and utility of this procedure in orthopedic surgery.

Approximately 56% of the total world population used the internet in 2022, compared to around 5% back in 2000 [4]. Seventy-five percent of patients reportedly base their health-related decisions on information acquired through the internet [5]. As such, it is of pivotal importance to have access to reliable online information regarding medical procedures so that patients can make well-informed decisions related to their health care. Among the different sources of information present on the internet, YouTube is one of the most popular with billions of reported users each month and one billion hours of video watching recorded each day [6]. Given that it provides content that can educate patients in an audiovisual format, it comes to no surprise that YouTube constitutes a favorable source of patient education regarding different medical procedures, including those relating to orthopedics like shoulder replacement surgeries [7,8].

As shared decision-making has become a central tenet in the healthcare field, patients play a major role in deciding to undergo an RSA, which is essentially an elective procedure design to reduce pain and restore function in the glenohumeral joint [9]. As such, evaluating the quality of online sources of information like YouTube videos can give insight into what content patients are basing their health care decisions on. Several studies have evaluated video content on YouTube regarding different orthopedic topics [7,8,10,11]. Nevertheless, and to our knowledge, none have explored the quality of content pertaining to RSA. Accordingly, and considering the high rise in utility and popularity of RSA in recent years, the purpose of our study was to evaluate the reliability and educational content of YouTube videos concerning the procedure.

METHODS

YouTube Search

The YouTube.com online database was queried on October 14, 2022, using the keywords “reverse shoulder replacement.” Similar to previous YouTube-based studies in orthopedic literature, the videos that appeared based on the keywords were sorted by relevance, and the first 50 videos were evaluated [8,12,13]. A video was excluded if it was not in English. Recorded variables included video duration, number of views, upload source, content category, days since video upload, view ratio (number of views/number of days since upload), and number of likes.

Video Upload Sources and Content Categories

We categorized and assessed the video sources/uploaders according to the following categories: academic (e.g., universities, colleges, medical centers with academic affiliations), medical centers (e.g., centers or health care systems not academically affiliated), physicians or health care providers (e.g., affiliated with independent physicians or physical therapists), medical industry (e.g., companies that manufacture surgical devices), and other (sources that do not fall in any of the aforementioned categories).

Video content was categorized into one of the following groups: patient education (informative videos targeting patients and general public), lectures (informative videos targeting medical professionals, often in the form of academic talks or presentations), patient experience (videos showcasing the experience of patients post-surgery), and surgical technique or approach (videos focusing primarily on the technique and surgical concepts of RSA).

The Assessment of Video Reliability and Educational Content

The Journal of the American Medical Association (JAMA) benchmark criteria were used to analyze the credibility of the videos [14]. JAMA benchmark criteria (Table 1) are objective tools that use a point system. A point is assigned for each of the four core principles that can be identified in online videos. A maximum score on this scale is a score of four, indicating a highly credible source, whereas a minimum score of zero indicates a questionable source with poor credibility. Although this tool is unvalidated, numerous YouTube-based cases in previous orthopedic literature use these criteria to judge the usefulness, credibility, and reasonableness of sources [10,15].

The Journal of the American Medical Association benchmark criteria

To assess the nonspecific educational quality and video flow, the global quality score (GQS) was used, which uses a five-point scale to determine the educational value and quality of online content (Table 2). The importance of the presented information, the amount of relevant information, the thoroughness of the descriptions and explanations, and the flow of the video were taken into account when using the GQS tool. A maximum score of five indicates excellent quality and flow. Although the GQS assessment is unvalidated, this tool has been used in numerous YouTube-based studies in the orthopedic literature [16,17].

The global quality score criteria

To evaluate the quality of the reverse total shoulder arthroplasty-specific educational content, we created the RSAS composed of sixteen items determined using the guidelines published by the American Academy of Orthopedic Surgeons (Table 3) [18]. Although this assessment tool is unvalidated and novel, numerous YouTube-based studies on the educational value of online resources used unvalidated topic-based tools in the literature [7,8,16,19-21]. To use the RSAS tool, an observer assigns one point for each item present in the video. A maximum score of fifteen indicates a video with high reverse total shoulder arthroplasty-specific educational quality. Two independent observers (JK and PB) scored each of the videos according to the JAMA, GQS, and RSAS scoring tools, and an average score from each tool was obtained.

The reverse shoulder arthroplasty-specific score

Statistical Analysis

To determine the educational quality and credibility of the video content, descriptive statistics were used to quantify the video characteristics. Continuous variables were expressed as means with a standard deviation while providing maximum and minimum values. Categorical variables were expressed as relative frequencies and percentages. The kappa statistic was used to measure the inter-rater reliability of the categorical variables in the manuscript. To determine whether the credibility or educational content quality were influenced by either (1) the video source/uploader or (2) the content, a Kruskal-Wallis test was used for the non-normal data. Multivariate linear regression analyses were conducted to determine whether a relationship was present between specific video characteristics and the credibility of the video (JAMA score) and the quality of the content (GQS and RSAS). Statistically significant results were indicated by a P-value below 0.05.

RESULTS

The first 50 videos that appeared in our initial search were included in our analysis. Video duration ranged between 35 seconds and 62 minutes, 47 seconds, with a mean of 12 minutes and 15 seconds (±14 minutes, 56 seconds) (Table 4). The oldest video was uploaded 4,215 days prior to the search, and the most recent video was uploaded 34 days prior to the search. On average, the included videos were uploaded 1,526±1,079.6 days prior to our search (Table 4). The total collective number of views was 3,232,289 views, and total number of likes was 20,684 likes, leading to an average of 64,645.78±264,160.9 views and 414 likes per video (Table 4). The average view ratio was 32.96±127.7, and the average number of likes was 413±1,848.6 per video. There were significant between-group effects observed for the view ratio with the uploader source (P=0.025), with industry videos having the highest view ratio (149.2±335.9).

Video characteristics of the YouTube videos included

With regard to video sources, academic centers had the highest number of uploaded videos at sixteen videos (32%), whereas industry-produced videos had the lowest number at seven videos (14%). The most commonly encountered video content was that of patient education with 18 videos (36% each), whereas the least common video content was patient experience with eight videos (16%).

Video Reliability and Educational Content Analysis

When assessing for inter-rater reliability, the kappa value was 0.831 for JAMA scores (near perfect agreement), 0.623 for GQS (substantial agreement), and 0.502 for RSAS (moderate agreement). The mean JAMA score was 2.32±0.64, the mean GQS was 2.31±0.82, and the mean RSAS was 5.53±2.43 (Tables 5 and 6). A Kruskal-Wallis test for non-normal data was used to determine whether the video reliability and the quality of educational content differed by upload source and by content classification. Significant between-group effects were observed for the JAMA score based on the content category (P=0.006), with lecture videos conferring the highest mean JAMA score (Table 6). No significant between-group effects were observed for the JAMA score based on the video upload source (P=0.082). Similarly, between-group effects were not observed for the GQS or RSAS by either content category (P=0.481 and P=0.247, respectively) or video upload source (P=0.916 and P=0.836, respectively).

Quality metrics by upload source

Quality metrics by video content

Predictors of Video Reliability and Educational Content Quality

The influence of video characteristics, the video content category, and the video upload source on the JAMA score, GQS, and RSAS were investigated using multivariate linear regression models including all collected video characteristics, content categorizations, and upload source variables. These models identified lecture content as a significant predictor of a higher JAMA score (β=0.777, P=0.033). Meanwhile, videos uploaded by independent sources (“other”) were negative predictors of the GQS score (β=–1.20, P=0.039) and the RSAS score (β=–3.69, P=0.023). Finally, videos uploaded by industry channels were negative predictors of the RSAS score (β=–3.08, P=0.032).

DISCUSSION

Our study found that the top 50 YouTube videos on RSA accumulated around 3.2 million views, with industry videos acquiring the highest number of views. The videos had mean JAMA, GQS, and RSAS scores of 2.32±0.64, 2.31±0.82, and 5.53±2.43, respectively. Academic centers were the video source with the highest number of videos uploaded, and surgical techniques/approach videos were the most common according to video content. Lecture videos showed and predicted higher JAMA scores, whereas industry videos predicted lower RSAS scores. Considering the prominent involvement of online resources in directing patient health care decisions, addressing the inaccurate content on online platforms is essential.

It is evident through our findings that RSA has been a popular topic among YouTube users, as shown by the large number of views accrued in our study. This falls in line with the rising popularity and prevalence of RSA in shoulder surgery and how it’s garnering much more interest from patients, surgeons, and industry personnel [2,3]. This is also expected given the established popularity of YouTube and the availability of numerous videos and content that convey information and insight regarding different surgical procedures and medical interventions [5]. The ease of the search, as well as the audiovisual presentation of different pathologies and their treatments, renders this tool a valuable resource for people who are planning on making health care decisions [5,22]. In addition, the upload sources with the highest number of views were medical/orthopedic industry sources, and this reflects the increasing commercial and financial interest in this procedure. Uploaded sources related to the medical industry were also found to be negative predictors for RSAS, which is alarming yet expected, given that the main interest of these sources resides in promotion and marketing rather than delivery of quality patient education [23].

In general, YouTube videos proved to be a poor online source for information regarding RSA, as shown by the poor quality scores attained. This falls in line with other YouTube-based studies in the literature that reported a low quality of information for orthopedic-related pathologies and procedures [7,16,20,21,24]. The unrestricted ability to upload videos and the lack of editorial evaluation on accuracy and comprehensiveness of content are potential contributors to the low scores exhibited in our study [8]. This raises concerns for the status of patient education in orthopedics and the ability to make well-informed health care decisions based on the available online video content. It also highlights the importance of proper patient education during clinic visits and appropriate counseling toward reliable and accurate sources of information.

The Kruskal-Wallis test in our study failed to demonstrate significant differences in quality scores between the different uploaded source types. That is surprising considering that academic and physician sources were expected to provide higher quality information than other sources, given the involved knowledge and expertise [8]. Videos with lecture content generally had higher quality scores than did other content types. Lecture videos often involve physicians and surgeons with high knowledge and expertise in their respective fields. These videos are usually thorough, comprehensive, and accurate in delivering information [25]. As such, it is not surprising that these had higher scores in our study. Nevertheless, the concerns with lecture videos are that they are often cater to medical personnel rather than patients, and this limits their appeal to patients, who might prefer shorter videos with simpler concepts and terminologies.

Our study demonstrates important findings regarding the educational value of YouTube videos related to RSA. Given the high number of views and likes accrued by these videos, it is evident that this informative video format is appealing to the RSA patient. Nevertheless, this low informative quality needs to be addressed as online sources play a huge role in patient education and, accordingly, patient-related health care decisions [5]. Introducing an editorial process that monitors medical videos on YouTube may be helpful in limiting the spread of inaccurate or harmful information. If that is not possible, introducing a new platform that publishes reliable expert-reviewed content may be the tool to provide proper education on orthopedic-related topics to patients. In addition, as lectures were shown to generally have better quality scores than other content categories, uploading lecture-based content that caters to patients rather than professionals can be helpful in disseminating credible educational content.

Several limitations exist for this study. While the query “reverse shoulder replacement” may not have provided a thorough representation of all videos targeting this topic, we decreased selection bias by systematically analyzing the first 50 videos in our search. Most internet users do not go beyond the first two pages populated by a search, and this is in accordance with the methodology implemented in our study. Moreover, our study involved the use of an unvalidated quality assessment tool, though several similar studies used similar tools to assess and evaluate online resources with acceptable inter-observer reliability for both the validated and unvalidated tools both in the literature and in our study [7,11,12,16,19-21]. This indicates that the findings in our study relating to low quality of information is, in fact, accurate.

CONCLUSIONS

Based on the number of views and likes, our study showed that YouTube videos focusing on RSA are popular and attractive to the prospective patient. Nevertheless, these videos provide a low quality of information on the surgical procedure and do not offer an appropriate educational value. More lectures, which generally had higher quality scores in our study, need to be published, and these need to cater to patients more than to medical professionals. Editorial reviewing of videos that showcase medical content may be required to ensure proper informative material is being disseminated. If that is not possible on YouTube, a new patient-friendly online video sharing platform, with expert peer review, should be developed to ensure accurate and reliable information is available to patients in need.

Notes

Author contribution

Conceptualization: MYF, JK, PB, JS, ASV, JAA. Data curation: JK, PB. Formal Analysis: ASV.Investigation: ASV, JAA. Methodology: MYF, JK, PB, JS. Supervision: JAA. Writing – original draft: MYF, JK, PB. Writing – review & editing: MYF, JS, ASV, JAA.

Conflict of interest

JAA would like to disclose royalties from: DJO Global, Zimmer-Biomet, Smith and Nephew, Stryker, Globus Medical, Inc.; research support as a PI from: Lima Corporate—Italy, Orthofix, Arthrex, OREF; royalties, financial or material support from: Wolters Kluwer; and board member/committee appointments for: American Shoulder and Elbow Society, Pacira. No other potential conflicts of interest relevant to this article were reported.

Funding

None.

Data availability

Contact the corresponding author for data availability.

Acknowledgments

None.

References

1. Guareschi AS, Eichinger JK, Friedman RJ. Patient outcomes after revision total shoulder arthroplasty in an inpatient vs. outpatient setting. J Shoulder Elbow Surg 2023;32:82–8.
2. Day JS, Lau E, Ong KL, Williams GR, Ramsey ML, Kurtz SM. Prevalence and projections of total shoulder and elbow arthroplasty in the United States to 2015. J Shoulder Elbow Surg 2010;19:1115–20.
3. Wagner ER, Farley KX, Higgins I, Wilson JM, Daly CA, Gottschalk MB. The incidence of shoulder arthroplasty: rise and future projections compared with hip and knee arthroplasty. J Shoulder Elbow Surg 2020;29:2601–9.
4. Internet World Stats. Internet usage statistics for all the Americas [Internet]. Internet World Stats; 2022. [cited 2023 Feb 4]. Available from: https://www.internetworldstats.com/stats2.htm.
5. Madathil KC, Rivera-Rodriguez AJ, Greenstein JS, Gramopadhye AK. Healthcare information on YouTube: a systematic review. Health Informatics J 2015;21:173–94.
6. YouTube Official Blog. YouTube has billions of monthly logged-in users [Internet]. YouTube; 2022. [cited 2023 Feb 4]. Available from: https://blog.youtube/press/.
7. Erdem MN, Karaca S. Evaluating the accuracy and quality of the information in kyphosis videos shared on YouTube. Spine (Phila Pa 1976) 2018;43:E1334–9.
8. Kunze KN, Alter KH, Cohn MR, et al. YouTube videos provide low-quality educational content about rotator cuff disease. Clin Shoulder Elb 2022;25:217–23.
9. Smithers CJ, Young AA, Walch G. Reverse shoulder arthroplasty. Curr Rev Musculoskelet Med 2011;4:183–90.
10. Cassidy JT, Fitzgerald E, Cassidy ES, et al. YouTube provides poor information regarding anterior cruciate ligament injury and reconstruction. Knee Surg Sports Traumatol Arthrosc 2018;26:840–5.
11. Gokcen HB, Gumussuyu G. A quality analysis of disc herniation videos on YouTube. World Neurosurg 2019;124:e799–804.
12. Kunze KN, Krivicich LM, Verma NN, Chahla J. Quality of online video resources concerning patient education for the meniscus: a YouTube-based quality-control study. Arthroscopy 2020;36:233–8.
13. Kunze KN, Cohn MR, Wakefield C, et al. YouTube as a source of information about the posterior cruciate ligament: a content-quality and reliability analysis. Arthrosc Sports Med Rehabil 2019;1:e109–14.
14. Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor: let the reader and viewer beware. JAMA 1997;277:1244–5.
15. Etzel CM, Bokshan SL, Forster TA, Owens BD. A quality assessment of YouTube content on shoulder instability. Phys Sportsmed 2022;50:289–94.
16. Tang K, Azhar U, Babar M, et al. Assessing the quality of YouTube videos on adhesive capsulitis. Cureus 2022;14e27406.
17. Singh AG, Singh S, Singh PP. YouTube for information on rheumatoid arthritis: a wakeup call. J Rheumatol 2012;39:899–903.
18. OrthoInfo. Reverse total shoulder replacement [Internet]. American Academy of Orthopaedic Surgeons; 2022. [cited 2023 Feb 4]. Available from: https://orthoinfo.aaos.org/en/treatment/reverse-total-shoulder-replacement/.
19. Kwak D, Park JW, Won Y, Kwon Y, Lee JI. Quality and reliability evaluation of online videos on carpal tunnel syndrome: a YouTube video-based study. BMJ Open 2022;12e059239.
20. Ovenden CD, Brooks FM. Anterior cervical discectomy and fusion YouTube videos as a source of patient education. Asian Spine J 2018;12:987–91.
21. Addar A, Marwan Y, Algarni N, Berry G. Assessment of “YouTube” content for distal radius fracture immobilization. J Surg Educ 2017;74:799–804.
22. Drozd B, Couvillon E, Suarez A. Medical YouTube videos and methods of evaluation: literature review. JMIR Med Educ 2018;4:e3.
23. Khan SN, Mermer MJ, Myers E, Sandhu HS. The roles of funding source, clinical trial outcome, and quality of reporting in orthopedic surgery literature. Am J Orthop (Belle Mead NJ) 2008;37:E205–12.
24. Kunze KN. Editorial commentary: YouTube videos provide poor-quality medical information: don’t believe what you watch! Arthroscopy 2020;36:3048–9.
25. Szpiro KA, Harrison MB, Van Den Kerkhof EG, Lougheed MD. Patient education in the emergency department: a systematic review of interventions and outcomes. Adv Emerg Nurs J 2008;30:34–49.

Article information Continued

Table 1.

The Journal of the American Medical Association benchmark criteria

Criteria Description
Authorship Provides the affiliations and credentials of the authors and contributors
Attribution Clearly lists references, sources, and copyright information
Currency Provides the initial date of posted content and dates of subsequent updates to the content
Disclosure Fully discloses website "ownership," conflicts of interest, sponsorship, commercial funding, and advertising

Table 2.

The global quality score criteria

Grading Description of quality
1 Poor quality and flow, unlikely to be useful for patients since most information is missing.
2 Generally poor quality and flow, is of limited use to patient, limited information and misses many important topics.
3 Moderate quality and flow, somewhat useful to patients, some important information is covered.
4 Good quality and flow, most of the relevant and important information is discussed, useful to patients.
5 Excellent quality and flow, highly useful for patient education

Table 3.

The reverse shoulder arthroplasty-specific score

Criteria Grading
Patient presentation 1
 Describes patient symptomatology 1
Information about reverse shoulder arthroplasty 6
 Discusses the materials used 1
 Discusses the duration of the procedure 1
 Discusses the steps of the procedure 1
 Discusses the advantages of the treatment 1
 Discusses the anatomy/function of the deltoid muscle 1
 Discusses the anatomy/function of the rotator cuff 1
Diagnosis and evaluation 4
 Mentions physical examination findings 1
 Describes surgical indications 1
 Describes surgical contraindications 1
 Describes the use of imaging 1
Postoperative course 4
 Discusses possible surgical complications and outcomes 1
 Mentions physical therapy 1
 Mentions physical restrictions 1
 Mentions the recovery timeline 1
RSAS score 15

RSAS: reverse shoulder arthroplasty-specific score.

Table 4.

Video characteristics of the YouTube videos included

Characteristic Mean Standard deviation Minimum Maximum
Video duration 12 min and 15 sec 14 min and 56 sec 35 sec 62 min and 47 sec
Views 64,646 264,161 110 1,861,102
Days since upload 1,526 1,080 34 4,215
View ratio 33.0 127.7 0.5 908.3
Likes 413 1,849 0 13,000

Table 5.

Quality metrics by upload source

Academic center (n=16) Industry (n=7) Medical center (n=10) Other (n=6) Physician/HCP (n=11) Overall (n=50)
JAMA 2.16±0.44 2.71±0.76 2.05±0.50 2.67±0.52 2.36±0.84 2.32±0.64
GQS 2.31±0.70 2.14±0.48 2.40±0.74 2.17±0.68 2.41±1.28 2.31±0.82
RSAS 5.44±2.26 4.64±2.01 5.70±1.86 5.17±1.47 6.27±3.66 5.53±2.43

Values are presented as mean±standard deviation. The P-values for video source between-group effects: JAMA=0.082, GQS=0.916, RSAS=0.836.

HCP: health care provider, JAMA: Journal of the American Medical Association, GQS: global quality score, RSAS: reverse shoulder arthroplasty-specific score.

Table 6.

Quality metrics by video content

Education patient (n=18) Lecture (n=9) Patient experience (n=8) Surgical technique or approach (n=15) Overall (n=50)
JAMA 2.08±0.43 2.94±0.64 2.13±0.44 2.33±0.73 2.32±0.64
GQS 2.33±0.94 2.50±0.75 1.88±0.84 2.40±0.69 2.31±0.82
RSAS 5.17±2.17 6.11±2.34 4.25±2.10 6.30±2.77 5.53±2.43

Values are presented as mean±standard deviation. The P-values for video content between-group effects: JAMA=0.006, GQS=0.481, RSAS=0.247.

JAMA: Journal of the American Medical Association, GQS: global quality score, RSAS: reverse shoulder arthroplasty-specific score.