Online resources for information on shoulder arthroplasty: an assessment of quality and readability
Article information
Abstract
Background
Many patients use online resources to educate themselves on surgical procedures and make well-informed healthcare decisions. The aim of our study was to evaluate the quality and readability of online resources exploring shoulder arthroplasty.
Methods
An internet search pertaining to shoulder arthroplasty (partial, anatomic, and reverse) was conducted using the three most popular online search engines. The top 25 results generated from each term in each search engine were included. Webpages were excluded if they were duplicates, advertised by search engines, subpages of other pages, required payments or subscription, or were irrelevant to our scope. Webpages were classified into different source categories. Quality of information was assessed by HONcode certification, Journal of the American Medical Association (JAMA) criteria, and DISCERN benchmark criteria. Webpage readability was assessed using the Flesch reading ease score (FRES).
Results
Our final dataset included 125 web pages. Academic sources were the most common with 45 web pages (36.0%) followed by physician/private practice with 39 web pages (31.2%). The mean JAMA and DISCERN scores for all web pages were 1.96±1.31 and 51.4±10.7, respectively. The total mean FRES score was 44.0±11.0. Only nine web pages (7.2%) were HONcode certified. Websites specified for healthcare professionals had the highest JAMA and DISCERN scores with means of 2.92±0.90 and 57.96±8.91, respectively (P<0.001). HONcode-certified webpages had higher quality and readability scores than other web pages.
Conclusions
Web-based patient resources for shoulder arthroplasty information did not show high-quality scores and easy readability. When presenting medical information, sources should maintain a balance between readability and quality and should seek HONcode certification as it helps establish the reliability and accessibility of the presented information.
Level of evidence
IV.
INTRODUCTION
Shoulder arthroplasty, also known as shoulder replacement surgery, is an effective and recognized therapeutic procedure for different glenohumeral joint pathologies [1]. Over the past several decades, shoulder arthroplasty has witnessed a surge in numbers, with different innovations and improvements emerging year by year. Day et al. [2] explored the prevalence of shoulder arthroplasty between 2000 and 2010 and noted a 7% to 13% increase per year. With increasing use of reverse shoulder arthroplasty and steady use of anatomic total shoulder, certain models predict approximately 350,558 shoulder arthroplasties to be conducted by 2025 [3-5].
As with other orthopedic procedures, shoulder arthroplasty is often indicated in certain scenarios where therapeutic goals and modalities are set according to physician guidance and patient expectations [6]. As such, and even though medical decisions are majorly influenced by surgeon choices, the patient remains a prominent part of the planning and decision-making regarding treatment strategies [7]. These patient health-related decisions are often governed by the patient’s awareness of the procedure and knowledge with regards to its indications, prognosis, and outcomes [7].
In 2013, it was reported that around 83.8% of U.S. households owned computers and 74.4% used the internet [8]. Recently, the percentage of internet users in the US exceeded 90%, and approximately 77% of the population owns a smartphone device [9-11]. With modern technological advancements and easy accessibility to online information, the internet is increasingly becoming the standard for gaining knowledge and information regarding medical procedures [12]. Accordingly, it is important for the information on the internet to be accurate, accessible, and of good quality.
Considering the elective nature of shoulder arthroplasty and the major role played by patients in determining whether to move forward with a procedure, evaluating the quality of online information and sources is necessary for the establishment of well-informed patient health-related decisions. There are ample of studies discussing the quality and reliability of YouTube video content on shoulder arthroplasty [13-16], but to the best of our knowledge, there are no studies in the literature appraising online sources for information regarding shoulder arthroplasty. As such, the aim of our study was to assess the quality and readability of online resources present on shoulder arthroplasty procedures using well-established scoring tools. We hypothesize that online web-based sources on shoulder arthroplasty information do not offer high-quality, reliable knowledge to prospective patients.
METHODS
This study appraises web pages that are publicly available online. It does not deal with patients or patient data, nor does it report it. Hence, no institutional review board approval or informed consent is required.
Search Strategy
On the 14th of July 2022, an internet search was conducted using Google Chrome browsing software. Searches were performed in a private browsing window, after clearing history and cache from the browser. For the sake of our study, we considered shoulder arthroplasty procedures to include total anatomic shoulder arthroplasty, reverse shoulder arthroplasty, and shoulder hemiarthroplasty. As such, search terms used were “Shoulder arthroplasty,” “Shoulder replacement,” “Anatomic shoulder arthroplasty,” “Anatomic shoulder replacement,” “Reverse shoulder arthroplasty,” “Reverse shoulder replacement,” “Shoulder hemiarthroplasty,” and “Partial shoulder replacement.” The search was done using the English language using the three most commonly used internet search engines: Google, Yahoo, and Bing [17,18]. We included the top 25 results generated by each term in each search engine, which was in accordance with other studies in the literature that noted patients are unlikely to view results beyond that number [19-21]. Exclusion criteria from the resultant webpages included duplicate webpages, webpages advertised by search engines, webpages that are subsections of others, webpages with subscription requirements or paywall access, and webpages of forums that did not fall within the scope of our study.
Webpage Classification
Webpages were classified by a physician into the following categories: academic, physician/private practice, commercial, healthcare professionals (HCP), and unspecified. Academic webpages were those that belonged to academic medical centers and institutions or those that showcased peer-reviewed scholarly work. Physician/private practice webpages were those that belonged to the practice of a physician or a private group of specialized physicians. Commercial webpages were those that either promoted a design or implant or were heavily inclusive of advertisements. HCP webpages were those that offered educational and informational content mainly catering for the HCP reader. Unspecified webpages were those that did not fall into any of the aforementioned categories.
Quality and Readability Assessment Tools
Two shoulder surgery research fellows did the quality assessment using three validated methods: presence of the Health on the Net Foundation (HONcode) certification, the Journal of the American Medical Association (JAMA) Benchmark Criteria (score ranges from 0 to 4), and the DISCERN instrument (score ranges from 0 to 80) [21-23].
HONcode certification is one of the most used and trusted seals for web-based health-related information. It is an initiative that was created in affiliation with the World Health Organization, and its strict standards governing the publication and presentation of medical information ensures proper quality, transparency, and objectivity. Hence, the included webpages were screened to check whether they complied and were certified with the HONcode seal. The JAMA and DISCERN scores are credible tools used to measure informational quality. JAMA criteria are used to evaluate webpages based on their appropriate display of authorship, references, disclosures, and currency [21]. For our study, webpages were considered current if they were dated or updated within the past 5 years. The DISCERN criteria are used to judge consumer health information based on 16 questions that pertain to quality and reliability [23]. Achieving higher scores on both JAMA and DISCERN is often associated with higher quality and accuracy of medical information. Two independent authors reviewed and scored the webpages according to the JAMA and DISCERN criteria. The results were reviewed by a third author to assess for any significant discrepancies in scoring. The mean JAMA and DISCERN scores were calculated and used for data analysis.
For assessing readability, the Flesch reading ease score (FRES) (score ranges from 0 to 100) was used [24]. This test is used to assess the readability and accessibility of a text or passage in the English language. Higher scores are associated with easier and more accessible text [24].
Statistical Analyses
All statistical analyses were performed with Stata ver. 16.1 (StataCorp.). Descriptive statistics were used to collect and present the data, with means and standard deviations for continuous variables and counts with frequency percentages for categorical variables. Given the data were non-normally distributed, Kruskal-Wallis tests were used to compare JAMA, DISCERN, and FRES between webpage categories. A multivariable linear regression analysis was conducted to determine (1) the association between webpage group and each collected score and (2) the influence of HONcode status on this relationship. Statistical significance was defined as P<0.05.
RESULTS
A total of 125 webpages were included (Fig. 1, Supplementary Table 1). The distribution of webpages was as follows: 36.0% (n=45) academic, 31.2% (n=39) physician or private practice, 15.2% (n=19) commercial, 9.6% (n=12) HCP, and 8% (n=10) unspecified. The average JAMA and DISCERN scores for all webpages were 1.96±1.31 and 51.4±10.7 with medians of 1.50 (range, 0–4) and 51.5 (range, 30.5–71.5), respectively. The average FRES score was 44.0±11.0 with a median of 45.4 (range, 14.5–71.1). Only 7.2% (n=9) of all webpages presented the HONcode seal. Likewise, only 7.2% (n=9) were deemed acceptable for a minimum of high school reading standards. Complete webpage characteristics are summarized in Table 1.
Webpage selection process.
Summary of the characteristics of the included webpages pertaining to shoulder arthroplasty information
HCP webpages had the highest JAMA score with a mean of 2.92±0.90, while physician/private practice webpages had the lowest scores with a mean of 0.94±0.60 (P<0.001). Similarly, for the DISCERN tool, HCP webpages had the highest score with a mean of 57.96±8.91, while physician/private practice webpages had the lowest scores with a mean of 45.88±7.98 (P<0.001). With regards to FRES scoring, HCP webpages had the lowest scores compared with all other groups, with a mean of 30.93±8.47 (P<0.001) (Table 2). Finally, those with a HONcode seal had significantly greater JAMA, DISCERN, and FRES scores compared with those that did not (P<0.05 for all).
JAMA, DISCERN and FRES scores of the included webpages pertaining to shoulder arthroplasty information
Multivariable linear regression models controlling for HONcode status were constructed to determine the influence of webpage category on each of the included scores with academic webpages as our reference variable. Both the JAMA and DISCERN models demonstrated statistically significant positive associations with HONcode status (P<0.01 for both) and statistically significant negative associations with each webpage type except for HCP (P<0.01 for all). Only HCP webpages demonstrated a significant relationship with the FRES score; HCP webpages were found to be negatively associated with readability by 13.20 points (95% confidence interval, –19.77 to –6.63; P<0.001).
DISCUSSION
Our study showed that academic sources constituted the most commonly available webpages for shoulder arthroplasty information online. The average accessible webpage for shoulder arthroplasty information had mean JAMA and DISCERN scores of 1.96 and 51.5, respectively, and only 7.2% of the webpages were HONCode certified. The average readability of available webpages was considered difficult according to FRES scoring. Webpages targeting HCPs had the highest quality scores but the lowest readability scores in our study.
Online sources for shoulder arthroplasty information exhibited a mean DISCERN score of 51.5, which is slightly higher than the scores reported in similar studies exploring online information on hip arthroplasty with a mean DISCERN score of 46.9, and revision hip arthroplasty with a mean DISCERN score of 43 [25,26]. The low quality scores reported in this study are similar to those found in studies appraising YouTube video content related to shoulder arthroplasty, indicating a problem for patient education on this topic across different online platforms [13-16]. Nevertheless, and considering the increasing rise and interest in shoulder arthroplasty procedures, targeting higher quality scores is essential for better patient education.
Only 7.2% of the included webpages were HONCode certified, which raises concerns over the ethical standards being adhered in these sources [27]. HONcode certification implies that webpages are offering quality health information that adheres to ethical principles in a transparent and objective manner [22]. This leads to more credible content with higher quality scores, as supported by our study, which showed a significantly positive correlation between HONcode status and JAMA and DISCERN scores.
Our study showed that different sources contribute to the available online information on shoulder arthroplasty, which is expected given how healthcare topics have been an area of interest for different institutions of different backgrounds [28,29]. Academic institutions and scholarly publications constituted the largest source of online information in our study. This is reflective of the efforts being put by surgeons and scientists in introducing and describing different shoulder arthroplasty procedures to the general public, as well as those exploring the outcomes of these procedures to synthesize better management protocols and strategies [30,31]. That said, other sources provided a notable number of webpages on shoulder arthroplasty as well. The high interest in publishing shoulder arthroplasty information by academic sources can be explained by the rise in shoulder arthroplasty procedures in recent decades and the incorporation of healthcare in different scientific and economic sectors [28,29,32]. Nevertheless, the differences in purposes and aims of these webpages had great implications on the quality, credibility, and readability of the content presented.
It is reasonable to predict that different sources would present information and content of varying quality due to their different purposes and agendas. Our study showed that webpages targeting HCPs had the highest average scores for JAMA and DISCERN, and this is founded, given that these webpages cater to educate medical professionals. As such, it is more likely that these webpages employ a holistic editorial review process for publishing their articles and would reflect positively on quality scores when compared to other sources [33,34]. On the other hand, webpages pertaining to physicians and private practices had the lowest quality scores in our study. As described earlier, these webpages often describe procedures to prospective patients similarly to the way that webpages of academic institutions do, but the former are often void of the editorial processing and reviewing exhibited at academic institutions. Due to the lack of resources or understanding required to stringently govern and review the content posted by the majority of these practices, the quality of the information may, unfortunately, be compromised.
Finally, while having high quality scores is reflective of reliable and objective shoulder arthroplasty content, little utility can be garnered from an online source if it is not accessible and reachable to the average patient seeking information [35,36]. It is important that the content be presented in a coherent manner, devoid of complex and sophisticated terminology, to allow the uninformed patient to gain knowledge about different aspects of a procedure—in our case, shoulder arthroplasty [37]. As such, several concerns can be derived from our results. Using FRES scoring criteria, we showed that the average webpage presenting information on shoulder arthroplasty procedures was difficult to read. In addition, while webpages pertaining to HCPs had the highest quality scores in our study, they had the lowest FRES scores, exhibiting a negative association with webpage readability on our multivariate linear regression analysis. Patients can be intimidated by sophisticated terminology, which may lead them to resort to online sources that are less transparent, objective, and reliable [37]. In order to ensure access to proper patient sources, a balance should be maintained between the quality, readability, and accessibility of the presented information.
The findings of our study allow us to provide several recommendations. Our study shows a prominent gap in patient education from online resources on shoulder arthroplasty, and several issues need to be highlighted. The low portion of HONcode-certified webpages in our study constitutes grounds for concern regarding the quality and validity of the medical information presented online. Educating people and patients on the relevance and importance of such certifications allows them to discern between webpages that are reliable and those that are not. In addition, education would add pressure to webpages to seek certification from these organizations. Moreover, the fact that webpages catering to healthcare professionals had the highest quality scores but the lowest readability scores shows that good, reliable information is often not accessible to the average patient. This stresses the need for online resources presenting reliable high-quality information that caters to the patient in an accessible manner. Physician webpages and webpages catering for private practices need to work on improving the quality scores of their educational content and ensuring the information present is objective, referenced, and reliable. Generally, patients should seek to obtain procedural information primarily from their physicians, who are well-acquainted with the patient's history and desired surgical operation. Physicians should also recommend high quality, reliable, and readable online resources for patients who desire additional information, especially because access to physicians is often limited. When these options are not available, resorting to HONcode-certified web pages may provide the best source for information on a specific procedure for prospective patients.
To the best of our knowledge, this is the first study to assess the quality of online information regarding shoulder arthroplasty. Nevertheless, several limitations exist. First, we conducted our search in the United States, and even though significant overlap would be present, conducting the search from different countries and regions worldwide might show different results. Second, we used eight terms to conduct our search, a high number when compared to other similar studies. However, even though these terms were very inclusive and representative of the procedure, some patients may have used different terms to explore the procedure online. Third, many web pages did not have information on authors, and hence, analysis of quality and readability based on author credentials was not possible.
CONCLUSIONS
There is a gap in web-based patient education regarding shoulder arthroplasty procedures. Most available resources do not have high quality information, and the webpages that do have high quality scores are difficult to read. A balance between readability and quality should be established and maintained when presenting medical information online. Additional healthcare-related websites should seek HONcode certification as it ensures the accuracy and reliability of presented information. Finally, patients should be educated on the different aspects governing online medical information to ensure well-informed patient-centered management decisions.
Notes
Author contributions
Conceptualization: MYF, JAA. Data curation: MYF, JS, ASV, JK, PB. Formal analysis: MYF, J Singh, ASV. Investigation: MYF, JS, JAA. Methodology: MYF, JS, ASV, JK, PB, JAA. Supervision: JAA. Writing - original draft: MYF. Writing - review & editing: MYF, JS, JK, PB, JAA.
Conflict of interest
JAA would like to disclose royalties from: DJO Global, Zimmer-Biomet, Smith and Nephew, Stryker, Globus Medical, Inc.; research support as a PI from: Lima Corporation - Italy, Orthofix, Arthrex, OREF; royalties, financial or material support from: Wolters Kluwer; and board member/committee appointments for: American Shoulder and Elbow Society, Pacira.
Funding
None.
Data availability
Contact the corresponding author for data availability.
Acknowledgments
None.
SUPPLEMENTARY MATERIALS
Supplementary materials can be found via https://doi.org/10.5397/cise.2023.00290.
Final list of unique webpages included in our study