Assessment of medical information on irritable bowel syndrome information in Wikipedia and Baidu Encyclopedia: comparative study

Data sources

The articles analyzed in this research were acquired from the English Wikipedia and Chinese Baidu Encyclopedia on December 21, 2022. Wikipedia and the Baidu Encyclopedia were searched for articles based on Rome IV Criteria (Drossman & Hasler, 2016), which are globally recognized guidelines for diagnosing and classifying IBS and other FGIDs (Rome Foundation, 2023). All possible synonyms and derived words for every term were searched. Two reviewers with decades of experience in gastroenterology and competence in the diagnosis and treatment of IBS and FGIDs evaluated these articles from Baidu Encyclopedia and Wikipedia. If there were any disagreements, the third examiner reviewed and arbitrated the dispute.

Assessment of the quality of the research articles

The quality of disease-related articles was assessed using the JAMA scoring system (Silberg, Lundberg & Musacchio, 1997), the DISCERN instrument (Charnock et al., 1999), and GQS (Bernard et al., 2007). Appendix S1 displays the specific contents of these scoring instruments. The DISCERN instrument evaluates health information quality. It has 15 questions, which are divided into three sections: reliability (Section 1), treatment choices (Section 2), and overall quality (Section 3). Each question is scored from 1 to 5. The total scores can range from 16 to 80. A score of 63–80 means excellent quality, 51–62 indicates good quality, 39–50 is fair quality, and 16–38 indicates poor quality results (Charnock et al., 1999). We set the cutoff value at 3 for each question, based on previous research, a score above 3 (60%) indicated good quality (Ma et al., 2022). The JAMA scoring system assesses the quality of health-related websites based on four criteria: author, attribution, disclosure, and currency. The GQS assesses each website’s overall quality with a subjective measure. It uses a 5-point scale to evaluate how easy and smooth the website is to use (Bernard et al., 2007).

Statistical analysis

We checked data normality using the Shapiro–Wilk test. We reported descriptive analyses as means and SDs for normally distributed variables, and as medians and IQRs for non-normally distributed ones. We assessed interobserver reliability with the intraclass correlation coefficient (ICC), which ranges from 0 (untrusted) to 1 (fully trusted), and discussed any values below 0.75 with the research team. We compared different ranks with the Mann–Whitney U test for nonparametric tests using SPSS 22.0. We used Fisher’s exact test to compare the frequency distribution of the DISCERN scores. We performed statistical analyses and created figures with Prism 8.4.0 (GraphPad Software, San Diego, CA, USA). P < 0.05 was as the significance level for the statistical analysis.

Overall scores for the Baidu Encyclopedia and Wikipedia

To assess the inter-rater reliability of the two independent reviewers, a 2-way mixed/random effects model was applied. Ratings for GQS, JAMA, and DISCERN scores were highly consistent between the reviewers, with ICC values of 0.759, 0.955, and 0.759, respectively.

The scores for total diseases (containing IBS, GD, BD, and AD) showed that Wikipedia outperformed the Baidu Encyclopedia in terms of quality, readability and usability of health information; we applied three different tools: the DISCERN, JAMA, and GQS to assess the contents of both sites (Fig. 1A). The DISCERN tool consists of three sections: reliability, treatment choices, and overall rating. Wikipedia scored significantly higher than the Baidu Encyclopedia in all three sections, indicating that it provides more reliable and objective information, better guidance for patients’ decision making, and higher overall quality of the publication (Fig. 1B). The JAMA and GQS tools also indicated that Wikipedia had better reading fluency and was easier use than the Baidu Encyclopedia. The DISCERN General score reflected the overall quality of the websites. Wikipedia scored significantly higher, suggesting that it is a more trustworthy and helpful source of health information.

Overall quality comparison for the theme of IBS between Wikipedia and the Baidu Encyclopedia

The classical symptoms of IBS include abdominal pain and bloating associated with a change in stool form or frequency. It is divided into four subtypes by the fecal trait: IBS-C, IBS-D, IBS-M, and IBS-U. Wikipedia scored higher in the total DISCERN score on IBS articles, but there were no obvious discrepancies in the JAMA and GQS scores (Fig. 2A). Wikipedia scored higher according to DISCERN Sections 1, 2, 3, and the General section (Fig. 2B). These results indicate a notable difference between the two sites on IBS content.

Overall quality comparison for the theme of GD between Wikipedia and the Baidu Encyclopedia

GD includes dyspepsia (or indigestion), flatulence of the gastrointestinal tract, and functional gastrointestinal disease. Wikipedia scored higher in the total DISCERN scores. Both sites had no obvious discrepancies in the JAMA and GQS scores (Fig. 3A). Further analysis of DISCERN scores found that Section 1, Section 3 and General had a significant difference (Fig. 3B).

Overall quality comparison for the theme of BD between Wikipedia and the Baidu Encyclopedia

Another subtype of FGIDS is BD, comprising functional gastrointestinal disorders, constipation, functional constipation, neurogenic bowel dysfunction, intestinal malrotation, dysbiosis, small intestinal bacterial overgrowth, splenic flexure syndrome, functional diarrhea, allergic colitis, and intestinal spasm. Wikipedia scored higher in the total DISCERN and JAMA scores (Fig. 4A). For GQS scores, the two sites had no obvious differences. Wikipedia scored remarkably higher in DISCERN Section 1 and 3 scores.

Overall quality comparison for the theme of AD between Wikipedia and the Baidu Encyclopedia

AD includes anal spasm, intestinal dilatation, proctalgia fugax, and levator ani syndrome. Wikipedia scored higher in the total DISCERN and JAMA scores on AD articles. For GQS scores, the two sites had no obvious differences (Fig. 5A). Moreover, there were obvious section differences between the two sites in the DISCERN scores and the General scores (average scores of three sections). This discrepancy was mainly derived from Section 1, indicating more reliability in Wikipedia articles.

Distribution of the DISCERN scores

Wikipedia’s overall quality for total diseases, IBS, GD, BD, and AD was generally higher than Baidu Encyclopedia’s, but some scores only differed numerically and were not statistically significant. Hence, we further investigated the score distributions for the two sites. As mentioned above, we considered a score above 3 (60%) points as an indicator of good quality. Table 1 shows how the scores for each section were distributed. Wikipedia had more General DISCERN and Section 1 scores above 3 points, either in overall disease or in IBS, GD, BD, AD, respectively. The Baidu Encyclopedia was lacking in information for some GD and BD characteristics in terms of epidemiology, pathophysiology, prevention, reference, and even some core subjects such as examination, diagnosis, and treatment. For example, the term “functional diarrhea” was lacking. The Baidu Encyclopedia also had a much longer update interval than Wikipedia. However, both sites had less Section 2 scores above 3 points for IBS, GD, BD, and AD. Wikipedia had more Section 3 scores above 3 points in the overall disease. Likewise, the two sites had no difference in the proportion of Section 3 scores above 3 points in IBS, GD, BD, and AD respectively, like “功能性腹泻” (functional diarrhea) in the Baidu Encyclopedia, which rated “good quality” (Table 1).

Principal findings

Over the last two decades, patients have been increasingly using the Internet to seek diagnosis and treatment information for diseases, especially for functional disorders such as IBS and constipation, which lack biomarkers that are universally accepted for diagnosing these conditions and whose diagnosis depends on self-reported symptom clusters. Most IBS patients do not receive adequate education from doctors due to its complex pathophysiology and limited time in the outpatient setting. This leads them to seek information online. In the general population, self-reported IBS has triple the prevalence compared to clinical diagnostic criteria (Van den Houte et al., 2019). An online search helps patients decide when to see a doctor and may improve their health outcomes and care quality. However, false or misleading information may cause misdiagnosis, treatment delays, and serious health risks (Suarez-Lledo & Alvarez-Galvez, 2021). Online resources vary in their credibility, accuracy, and readability. Evaluating these aspects is urgent, especially for multiple gastroenteric malignancy mimic disorders. Our research compared the Baidu Encyclopedia and Wikipedia on the reliability, readability, and objectivity of their articles about total diseases, IBS, GD, BD, and AD. Compared to the Baidu Encyclopedia, Wikipedia clearly scored higher in total DISCERN, DISCERN Section 1 and Section 3; there were more total DISCERN and Section 1 scores based on the “above 3 (60%) rule”; and it scored higher in JAMA, suggesting it might provide more reliable, higher quality, and more objective information; and it scored higher in GQS, suggesting it might be better flow or easier to apply. Most DISCERN Section 2 scores distribution for both sites were below 3 points, suggesting a similar and mediocre influence on patients’ treatment choices.

According to surveys, 239.33 million people in China (Yacob et al., 2020), and half of American adults (Zhao & Zhang, 2017) used online resources for medical information. However, online information quality about IBS varies widely (Levine et al., 2020). A total of 3%–89% of the retrieved IBS content was found to be relevant to IBS (Cruz et al., 2019), with similar problems in the health contents in Wikipedia (Levine et al., 2020) and the Baidu Encyclopedia (Ma et al., 2022). Wikipedia excels in accuracy, breadth, depth, and informativeness for GI diseases (containing IBS, GD, BD, and AD), which is displayed in Figs. 1–5 (Good et al., 2012). One possible reason for the accuracy gap between the two sites is that Baidu Encyclopedia does not have a well-developed consultation system to address issues that may arise from editorial mistakes (Zhenyu He, 2019; Jiang et al., 2022). Editors may be motivated by personal interests and information may not be rigorously vetted. Wikipedia offers a talk page for editors to discuss and correct errors, which enhances its accuracy (Shang, 2018). Secondly, the breadth gap found in the Baidu Encyclopedia may be due to its poor organization and single classification, while Wikipedia has multiple classifications (Shang, 2018). Thirdly, there is a talk page in Wikipedia, which enables editors to exchange information and facilitate the participation of editors with higher academic achievement (Heilman & West, 2015). The academic weakness of modern medicine in China also leads to less valuable Chinese sources for editors. Fourthly, the informativeness gap is partly due to the motivation of the editors. Editors who work for traditional media tend to omit information that is not relevant to their interests, while Wikipedia editors can use the talk page to facilitate the addition of information. Traditional Chinese medicine lacks sufficient studies in the Western world because of the scarcity of ancient books and the language barrier. This hinders Wikipedia’s performance on this topic. The interpretation of these four features are consistent with the DISCERN scores, and indicate that Wikipedia publications have more reliability and a better impact on patients’ treatment choices. It was gratifying that recent updates to the Baidu Encyclopedia have increased the involvement of medical professionals in reviewing or writing the content, which enhances the DISCERN and JAMA scores significantly.

Accurate disease information found in online encyclopedias can enable patients to gain a comprehensive understanding of medical information and avoid misdiagnosis and treatment delays. The diagnosis of IBS is primarily based on symptoms, and the wide range of symptoms increases the probability of a misdiagnosis (Lacy et al., 2021). Hence, patients without a clear diagnosis of IBS are more likely to look up their symptoms online (Van den Houte et al., 2019). Our research showed that a differential diagnosis or possible overlapping diagnosis of IBS was not mentioned in the Baidu Encyclopedia, however, it was listed briefly in Wikipedia (Ford et al., 2020). For example, one disease of GD, “消化不良” (indigestion or dyspepsia), lacks a differential diagnosis and management in primary care in both sites. However, the guidelines emphasized differentiating between dyspepsia and IBS with regard to the hierarchical diagnosis and treatment of dyspepsia (Black et al., 2022). Therefore, both sites had poor scores in JAMA, GQS and Section 2 DISCERN for GD. For BD, with regard to “小肠菌群过度生长综合征” (small intestinal bacterial overgrowth) and “神经源性肠道” (neurogenic bowel), Wikipedia showed detailed content on the definition, diagnosis and treatment, while the Baidu Encyclopedia only presented the definition of the two entries. For AD, both sites had little information on treatment and differential diagnosis, such as “肛提肌综合征” (levator ani syndrome). The lack of information may result from the need to search for a very specific disease name “肛提肌综合征” (levator ani syndrome) for which there is little awareness.

By comparing IBS, GD, BD, and AD content on both sites by the above 3 (60%) rule, Wikipedia content generally received “good quality” ratings, whereas the Baidu Encyclopedia content mostly got “fair quality” ratings. For example, both reviewers rated “功能性腹泻” (functional diarrhea) in the Baidu Encyclopedia as “poor quality”. The web page for “functional diarrhea” lacked essential information on epidemiology, pathophysiology, prevention, and reference. It also omitted most of the core content for examination, diagnosis, and treatment and did not provide any precautions for drug use, such as drug contraindications. Furthermore, “functional diarrhea” had a much longer update interval in the Baidu Encyclopedia than in Wikipedia. The former updated this entry on October 3, 2017, while the latter did so on December 13, 2022. In addition, the former included no pictures, while the latter had four, indicating Wikipedia was more informative in general. Hence, measures should be taken to enhance content regulation, provide for a review mechanism for interest-related content, and to update the content of IBS articles promptly in the Baidu Encyclopedia.

In the Baidu Encyclopedia, reviewers rated the information for “肠易激综合征” (irritable bowel syndrome) and “功能性腹泻” (functional diarrhea) as “good quality”. The good quality rating is possibly due to the high quality of the content provider. Most providers came from unregistered individuals in the past, whose contributions often lack references, in-depth information, and quality in the Baidu Encyclopedia. We also found only one advertising link on the two web pages, and the article content was objective and scientific, this is superior to the previous report: more than 25 irrelevant advertising links on a Baidu Encyclopedia web page, and the article was biased and unscientific (Ma et al., 2022). This improvement might be a result of the Baidu Encyclopedia “rainbow plan”, which restricts the editing and revision of medical entries to certified medical experts (Beijing News, 2023). Hence, more medical professionals may be motivated or invited to edit or proofread the disease content in the Baidu Encyclopedia, ensuring objectivity and comprehensiveness.

Most Wikipedia articles were close to “good quality” but not “excellent quality”, and required more improvement. Louis Levine revealed that Wikipedia was the most commonly viewed of 14 websites in searches for IBS, the highest scoring resource for relevant IBS content, it scored higher (74.3 out of 93) than other websites (Suarez-Lledo & Alvarez-Galvez, 2021). It has two content quality evaluation systems: the “Wiki-Project article quality grading scheme” and “Wiki-Project priority assessments”. However, these systems cannot fully address the challenges of content accuracy and readability that Wikipedia faces (Wikipedia, 2023b; Wikipedia, 2023c). To enhance accuracy, it may benefit from more expert input, diverse edits and external collaborations (Shafee et al., 2017; Ma & Zhang, 2020; Wang et al., 2020). To improve the experience of new editors and external collaborations, encyclopedia norms and bureaucracy should be clearer and more straightforward.

Based on the theory that the higher the content score of the resource, the more difficult it is to read (Wong & Cheung, 2019), Wikipedia health content needs to be simple and easy to understand. For instance, one effective diet treatment for IBS is described as “a low FODMAPs diet” in Wikipedia, it explains “a low FODMAPs diet” as a type of short-chain carbohydrate that the small intestine absorbs poorly. The explanation seems obscure and professional. While the Baidu Encyclopedia explains that “减少产气食物 (奶制品，大豆，扁豆等)” (reduces gas-producing foods (dairy, soybeans, lentils)), which seems easy to understand. Hence, automatic feedback on the readability can make Wikipedia more user-friendly to patients.