Inflammatory bowel disease: An evaluation of health information on the internet

Abstract

AIM

To evaluate the quality and accuracy of websites written to the public on inflammatory bowel disease (IBD) (Crohn's disease and ulcerative colitis) and assess their readability level.

METHODS

Google™, Bing™, and Yahoo™ search engines were searched independently by three researchers in December 2014. Only English-language websites were selected on the basis of predetermined inclusion and exclusion criteria. Researchers independently evaluated the quality of each website by using the DISCERN and the HONcode instruments. The readability levels were calculated using two formulas; the Flesch-Kincaid Grade Level Index, and the Coleman-Liau Readability Index. The agreement between the evaluators was calculated using Cohen kappa coefficient.

RESULTS

Eighty-four websites were finally identified. Scores varied from a minimum DISCERN score of 18 to a maximum of 68 [mean ± SD, 42.2 ± 10.7; median = 41.5, interquartile range, interquartile range (IQR) = 15.8] and a minimum score of HONcode of 0.14 and a maximum of 0.95 (mean ± SD, 0.16 ± 0.19; median = 0.45, IQR = 0.29). Most of these websites were reviewed in 2014 and 2015 (n = 51). The creators of these websites were: universities and research centers (n = 25, 30%), foundations and associations (n = 15, 18%), commercial and pharmaceutical companies (n = 25, 30%), charities and volunteer work (n = 9, 10%), and non-university educational bodies (n = 10, 12%). The Flesch-Kincaid Grade Level readability score (mean ± SD) was 11.9 ± 2.4 and the Coleman-Liau Readability Index score was 12.6 ± 1.5. Significant correlation was found between the two readability scores (R² = 0.509, P = 0.001). The overall agreement between evaluators measured by Cohen kappa coefficient was in the range of 0.804-0.876; rated as "Good".

CONCLUSION

The DISCERN and the HONcode scores of websites varied and the readability levels of most websites were above the public readability level. The study highlights the areas that need further improvement and development in patient education online materials about IBD.

Key Words: Inflammatory bowel disease; The internet; Patients' information; Evidence; Patients' education; online resources

Core tip: This is a comprehensive study analyzing the quality and accuracy of content and the readability level of websites in the English language on inflammatory bowel disease dedicated to the public. Two standardized instruments were used in assessing quality and accuracy and two methods were used in calculating readability level. The study showed variability in scores and the readability levels of most websites were above that for the public. Based on evidence, the study highlights the need for improving online patient education.

INTRODUCTION

Inflammatory bowel disease (IBD) refers to two chronic inflammatory disorders, Crohn's disease and ulcerative colitis. Both are life long, relapsing disorders of unknown etiology; possibly the result of interaction between genetic and environmental factors[1]. The diagnosis of these disorders is based on clinical features, endoscopy, and histological changes[2]. Crohn's disease may affect any part of the gastrointestinal tract but most commonly affects the distal ileum and proximal colon. The disease is characterized by inflammatory changes involving all the layers of the affected regions. In contrast, ulcerative colitis is characterized by continuous ulceration starting in the rectum and limited to the colonic mucosa[3]. IBD occurs worldwide with the highest incidence in developed countries mainly North America, United Kingdom and northern Europe. The incidence of ulcerative colitis in North America is approximately 19.5 per 100000 person years and 243 per 100000 person years in Europe while the incidence of Crohn's disease in North America is approximately 20.2 per 100000 person years and 12.7 per 100000 person years in Europe[4]. The aims of treatment are to induce remission in active disease and to maintain remission/prevent relapse. Therapeutic modalities include lifestyle modification, nutritional support, and medications. Surgery is reserved for the treatment of complications or when the medical therapy is ineffective. In addition to other complications, patients with IBD are at a higher risk of developing colorectal cancer. Therefore, patients have to undergo to regular checkup for early detection of the development of colon cancer[5].

With this information in mind, patients with IBD, as it is the case with other chronic diseases, usually seek information about the nature of the disease, its causes, investigations needed to diagnose the disease and therapeutic options. The advances in treatment modalities and options, and the increasing desire for patients to participate in decision-making about treatment choices necessitate the need for resources to support these decisions. Nowadays, patients have increasingly used the Internet as a source of health information because of its global accessibility, speed, and cost effectiveness[6]. Approximately 80% of the Internet users look for medical or health-related information through the Internet[7]. The topics most searched were information about specific disease or medical condition, treatment options, diet and nutrition, exercise and fitness and medications[7]. The increasing use of the Internet embraces a variety of aspects of topics searched, which gives the person an opportunity to investigate their questions from several resources. However, with the abundance of such information there is concern about the quality, accuracy, and readability level of the information available on the Internet about health care[8].

Therefore, the aims of this study were: to evaluate the quality, and accuracy of web-based information about IBD using two instruments, the DISCERN and the HONcode, as well as calculate the readability level by using two formulas, the Flesch-Kincaid Grade Level Index, and the Coleman-Liau Readability Index. The rationales for the study were to assess the educational usefulness of web-based information on IBD particularly their quality, accuracy and areas of deficiencies that need improvement. Also to assess whether these resources are easily read and understood by the public. Therefore our research questions are: (1) for the websites targeting the public and patients with IBD, what is the accuracy and the quality of these information resources? and (2) does the readability level of these online resources match with the recommended level for the public?

MATERIALS AND METHODS

Search design

In this study we assessed websites written for patients and the public on IBD by searching three search engines (Google™, Bing™ and Yahoo™), the selection of these three search engines was based on current statistical information that showed that these engines are the most searched by the public for health information[9]. The quality and accuracy of information provided on websites were assessed using two instruments: the DISCERN (http://www.discern.org.uk) and HONcode (http://www.hon.ch/honcode/) instrument. Details about these instruments and the justification for selecting them are discussed later. The readability of the websites was assessed using two methods: the Flesch-Kincaid Grade Readability Level and the Coleman-Liau Readability Index. After piloting the work and ensuring satisfactory use of these instruments by researchers, the work was carried out to assess the quality of websites. The Institutional Review Board, College of Medicine King Saud University, has approved the project and the approval number: F06/2014.

Searching the internet

Using the following key words: "inflammatory bowel disease", "Crohn's disease", "ulcerative colitis", "inflammatory bowel disease patient information", "Crohn's disease patient information", and "ulcerative colitis patient information", three search engines (Google™, Bing™ and Yahoo™) were searched. Researchers independently from 1 to 20 December 2014 conducted the search. Information for each website was recorded; these included: website title, website URL, name of creator, year of publication on the Internet, last date updated, and the objectives of the website. This information about each website was collected using the following online meter: http://whois.domaintools.com/. The data collected were evaluated on the bases of the inclusion and the exclusion criteria.

Inclusion and exclusion criteria

The inclusion criteria included: (1) websites covering public education about Crohn's disease, ulcerative colitis or IBD; and (2) websites focusing on patient education and in the English language. The excision criteria comprised: (1) websites addressing doctors or health professionals; (2) lectures, and advertisement on IBD; (3) websites in languages other than English; and (4) presentations at conferences.

Assessing accuracy and quality of information

Two instruments were used in assessing the quality and accuracy of information provided, namely the DISCERN instrument and the HONcode instrument. These two instruments have been widely used in the literature in assessing information on the Internet particularly health related issues and patients' education online resources[8,10-12]. More details about these two instruments can be summarized as follows:

DISCERN instrument: This instrument is a standardized set of criteria for judging the quality of health information and is written for the public to assess treatment options[8,10-12]. The DISCERN instrument was created by the University of Oxford, and the project was funded by the British Library and the National Health Service (NHS) Research & Development Programme[13] The instrument consists of 15 questions plus an overall quality rating question. The questions can be grouped under the three key topics as follows: Questions 1 to 8 addressing reliability, Questions 9 to 15 addressing specific detail about the information provided and treatment choices, and Q16 covering the overall quality rating[14]. The instrument has been used to assess healthcare-related websites and online resources. For example, the quality of patients' information on surgical treatment of haemorrhoids[12], and colorectal cancer information[11].

HONcode instrument: The Health on the Net (HON) Foundation, a non-profit, and non-government organization created this instrument in 1995. The instrument focuses on key questions on the provision of health information available on the Net, and provides a code of conduct addressing eight principles: (1) authoritative (indicates the qualifications of the authors); (2) complementarity (Information should support, not replace, the doctor-patient relationship); (3) privacy (Respect the privacy and confidentiality of personal data submitted to the site by the visitor); (4) attribution [cite the source(s) of published information, date medical and health pages]; (5) justifiability (site must back up claims relating to benefits and performance); (6) transparency (Accessible presentation, accurate email contact); (7) financial disclosure (Identify funding sources); and (8) advertising policy (Clearly distinguish advertising from editorial content)[15-17]. To earn HONcode certification, a website must conform to the eight principles of the HONcode of Conduct. An HONcode expert then assesses the candidate website using precise guidelines for each principle. Recently, the HON Foundation has developed an automated system to assist in detecting a website's HONcode conformity. Therefore, the automated assistance in conducting HONcode reviews can expedite the current time-consuming tasks of HONcode certification and ongoing surveillance. A recent study showed that there is concordance between automated and expert manual compliance detection for the criteria[18]. In this research we have used the electronic system available at: http://www.readabilityformulas.com/free-readability-formula-tests.php.

The HONcode has been widely used in the literature in assessing health-related websites[19]. The two instruments, the DISCERN and HONcode, do not exactly cover the same issues/topics, although there are some overlaps. Therefore, using these two instruments with these differences in mind could provide a better evaluation of the websites.

Piloting the study

The aims of piloting the study were: (1) to introduce the two instruments to the researchers and orient them on how to use each instrument in assessing the websites; and (2) identify difficulties facing the researchers on applying the two instruments and the sources of disagreements among them. Such exercise prior to the implementation of the two instruments was vital for ensuring optimal use of the instruments and maximizing the degree of agreement among evaluators when they apply these two instruments in the actual research. The piloting part was conducted as follows: (1) approximately 10 websites other than those identified for the research study were evaluated independently by three researchers using the two instruments; (2) the results of their evaluation were discussed with the aim to identify sources for difficulties/disagreements; (3) the identified differences were resolved after discussing them reaching to a solution; and (4) the same process was repeated on another 10 websites until the agreement between the researchers reached to an optimal level[20].

Conduction of the study

Along with the same approach described under piloting the study, the researchers evaluated the websites identified by applying the two instruments on each website and giving a score. The process was conducted by each researcher independently first by applying the DISCERN instrument then the HONcode instrument. The results of the assessment were placed on an Excel sheet for each researcher. The degree of agreement was measured using Cohen kappa coefficient[21].

Calculating website readability

The aims of calculating readability level of websites was to assess if they were written at the readability level of the general public and patients; should not exceed the 6^th grade readability level[22,23]. Two methods were used to calculate readability: The Flesch-Kincaid Grade Level Index[24], and the Coleman-Liau Readability Index[25,26]. It was decided to use these two methods rather than one method so that we can compare the readability scores and examine if there were an agreement between the two methods, and hence strengthening the outcomes of our readability assessment and our conclusions. The two methods can be summarized as follows:

Flesch-Kincaid grade level index: This test helps in indicating how difficult a reading passage in the English language to understand. The test was developed by Rudolf Flesch and finalized by J Peter Kincaid for use by the United States Navy, hence the name of the test[24]. The test is based on the word length and the sentence length and is based on the following formula:

0.39 × [(total words)/(total sentences)] + 11.8 × [(total syllables)/(total words)] - 15.59

This method has been widely used in assessing the readability of websites and educational material[27].

Coleman-Liau readability index: This test differs from the above method in relying on characters instead of syllables per word. It enables the users however to grade the readability level. It has been widely used in assessing the readability of educational material[27].

We used a free online calculator (http://www.readabilityformulas.com) to calculate the readability level using the two readability methods. As per instructions provided by the website, the top, middle and bottom 150-200 words of each website were placed in the calculator and then the text readability was checked by calculating the number of sentences, words, syllables, and characters in the sample. A sufficient sample size of four to five full sentences; approximately 200-500 words in total were used. The scores recorded for each website were placed on an Excel sheet and reviewed by two other researchers before conducting final analysis for the means and standard deviations.

Grouping the websites under five categories

Assessment of the identified websites revealed variability in their creators. These can be grouped into 5 categories: (1) university, affiliated hospitals, and research centres; (2) foundations and associations; (3) commercial and pharmaceutical companies; (4) charities and volunteer works; and (5) non-university educational bodies such as colleges, academies, and councils. The grouping of websites under these five categories was carried out by researchers independently and was reviewed in a meeting for any disagreements.

Statistical analysis

The collected data were placed on an Excel Sheet (Microsoft Excel for Mac 2011, Microsoft Corporation, Redmond, WA, United States). All analysis was conducted by using SPSS software (SPSS Statistics version 22 for Mac, IBM Corporation, Armonk, NY, United States). For the data collected from measuring website accuracy, and the readability scores, the means, standard deviations, the median and interquartile range (IQR) were calculated. Pearson correlation studies and P-values for significance were calculated to examine if there were correlations between the scores obtained from the two readability methods[28]. A P-value of < 0.05 was considered significant. The agreement between the evaluators measured by the degree of inter-rater agreement using Cohen kappa coefficient was also carried out using SPSS software. This has been interpreted as "Poor", if the results in the range: 0.21-0.40; "Fair" 0.41-0.60; "Moderate" 0.61-0.80; "Good" 0.81-1.00.

RESULTS

General information about websites

The search of the three databases, Google™, Bing™ and Yahoo™, resulted in the identification of 300 websites. After the duplicates were removed we ended with 210 websites. On applying the inclusion and exclusion criteria, 84 websites were finally identified and included in the study (Figure 1).

Figure 1 PRISMA flowchart showing the websites on inflammatory bowel disease searched on the Internet and those finally included in the study.

Table 1 summarizes the general information about the 84 websites, including: website title, URL, author/ownership, year created, last updated, number of pages, number of tables, images and illustrations. The oldest two websites were created by the University of North Carolina (UNC), School of Medicine, North Carolina, United States and the Department of Surgery, University of California, California, United States, while the most recent was published in 2013 and created by New Health Guide, United States.

Table 1 Summarizes general information about websites on inflammatory bowel disease included in the study.

No.	Website title, Organisation	URL	Authority/ownership, state, country	Year created	Last updated	Number of pages	Number of tables	Number of images	Number of illustrations
1	Inflammatory Bowel Disease (IBD), Mayo Clinic	http://www.mayoclinic.org/diseases-conditions/inflammatory-bowel-disease/basics/definition/con-20034908	Mayo Foundation for Medical Education and Research, Arizona, United States	1997	04 Feb 2014	11	0	0	0
2	Inflammatory Bowel Disease Health Center, WebMd	http://www.webmd.com/ibd-crohns-disease/	WebMD, Inc., Georgia, United States.	1998	06 Sep 2013	2	0	3	0
3	Inflammatory Bowel Disease (IBD), Center for Disease Control and Prevention (CDC)	http://www.cdc.gov/ibd/	Centers for Disease Control and Prevention (CDC), Georgia, United States.	1999	04 Sep 2014	3	0	1	1
4	Inflammatory Bowel Disease, NHS Choices	http://www.nhs.uk/conditions/inflammatory-bowel-disease/pages/introduction.aspx	NHS England, Wakefield, United Kingdom.	1996	29 Apr 2013	2	0	0	0
5	What are Crohn's and Colitis? Crohn's and Colitis Foundation	http://www.ccfa.org/what-are-crohns-and-colitis/	Crohns and Colitis Foundation of America, New York, United States.	1996	14 Apr 2014	1	0	1	0
6	Inflammatory Bowel Disease, KidsHealth	http://kidshealth.org/parent/medical/digestive/ibd.html	Kids Health Organisaion, The Nemours Foundation, Orlando, United States.	1995	27 Jan 2015	4	0	1	0
7	Inflammatory Bowel Disease (IBD). FamilyDoctor	http://familydoctor.org/familydoctor/en/diseases-conditions/inflammatory-bowel-disease.html	American Academy of Family Physicians, New Jersey, United States.	1998	22 Jan 2015	7	0	0	0
8	Inflammatory Bowel Disease, Healthline	http://www.healthline.com/health/inflammatory-bowel-disease#Overview1	Healthline, California, United States.	2004	29 Nov 2011	6	0	0	0
9	Crohn's and Colitis, Australia	https://www.crohnsandcolitis.com.au/about-crohns-colitis/inflammatory-bowel-disease/	Crohn's and Colitis Australia, Victoria, Australia.	2009	24 Jul 2014	3	0	1	0
10	Crohn's and Colitis UK	http://www.crohnsandcolitis.org.uk/information-and-support/information-about-ibd/what-is-IBD	Crohn's and Colitis UK, United Kingdom.	2010	13 Nov 2014	2	0	1	0
11	Inflammatory Bowel Disease (IBD) (Intestinal Problems of IBD), MedicineNet	http://www.medicinenet.com/inflammatory_bowel_disease_intestinal_problems/article.htm	Medicine.Net.com, WebMed Network, New York, United States.	1995	6 Sep 2013	12	0	27	0
12	Inflammatory Bowel Disease Center, Cedars-Sinai	http://www.cedars-sinai.edu/Patients/Programs-and-Services/Inflammatory-Bowel-Disease-Center/	Cedars-Sinai Medical Center, California, United States.	1992	5 Jul 2013	2	0	0	0
13	Inflammatory Bowel Diseases Symptoms and Treatment: Livescience	http://www.livescience.com/39880-inflammatory-bowel-disease.html	Tanya Lewis, LiveScience Contributor, New York, United States.	2001	10 Apr 2014	11	0	1	0
14	Inflammatory Bowel Diseases Program, Penn Medicine.org	http://www.pennmedicine.org/gastroenterology/patient-care/gi-diseases/inflammatory-bowel-disease-ibd/	Penn Medicine, Pennsylvania, United Stats.	2003	6 Nov 2014	4	0	0	0
15	Inflammatory Bowel Diseases Support Groups, IBDsupport.org	http://www.ibdsupport.org/	IBD support.org, Utah, United States.	2011	30 May 2014	2	0	0	0
16	Inflammatory Bowel Disease (IBD), ABC Health and wellbeing	http://www.abc.net.au/health/library/stories/2012/02/22/3435688.htm	Australian Broadcasting Corporation, NSW, Australia.	2001	3 Dec 2014	4	0	1	0
17	Inflammatory Bowel Disease (IBD), GIKids	http://www.gikids.org/content/7/en/IBD	GIKids and The NASPGHAN Foundation, Pennsylvania, United States.	2009	11 Jun 2013	2	0	1	0
18	Inflammatory Bowel Disease, Vitamin D Council	https://www.vitamindcouncil.org/health-conditions/inflammatory-bowel-disease/	The Vitamin D Council, California, United States.	2007	17 Jun 2011	6	0	0	0
19	Inflammatory Bowel Disease Symptoms and Diagnosis, Seattle children's	http://www.seattlechildrens.org/medical-conditions/digestive-gastrointestinal-conditions/ibd-symptoms/	Children's Hospital and Regional Med. Ctr, Washington, United States.	1993	23 Apr 2014	2	0	0	0
20	Crohn's Disease, Patient.co.uk	http://www.patient.co.uk/health/crohns-disease-leaflet	Patient, Patient information Publications, Leeds, United Kingdom.	1997	05 Mar 2013	8	0	0	1
21	Patient Information Crohn Disease (Beyond and the Basics), Uptodate	http://www.uptodate.com/contents/crohn-disease-beyond-the-basics	UpToDate, Wolters Kluwer Health, Illinois, United States.	1998	29 Jul 2014	5	0	0	0
22	Crohn's Disease, Centre for digestive diseases	http://www.cdd.com.au/pages/disease_info/crohns_disease.html	The Centre for Digestive Diseases, NSW, Australia.		08 Jul 2013	4	0	0	0
23	Crohn's Disease, Patients: British Society for Gastroenterology	http://www.bsg.org.uk/patients/general/crohn-s-disease.html	British Society of Gastroenterology, London, United Kingdom.	1996	05 Aug 2014	6	0	0	0
24	Crohn's Disease, Bupa	http://www.bupa.co.uk/health-information/directory/c/crohns-disease	The British United Provident Association Ltd, London, United Kingdom	1996	16 Jun 2014	6	0	0	1
25	Crohn's Disease, University of Maryland Medical Center	http://umm.edu/health/medical/reports/articles/crohns-disease	University of Maryland Medical Center, Maryland, United States	1996	19 Sep 2013	20	0	0	0
26	Crohn's Disease, Symptoms, Diagnosis, Treatment, Southern Cross	https://www.southerncross.co.nz/AboutTheGroup/HealthResources/MedicalLibrary/tabid/178/vw/1/ItemID/523/Crohns-disease-symptoms-diagnosis-treatment.aspx	Southern Cross Healthcare Group; Auckland, New Zealand.	1998	01 Feb 2015	5	0	0	0
27	Crohn's Disease, American family physician	http://www.aafp.org/afp/2011/1215/p1379.html	American Academy of Family Physicians, Kansas, United States	1995	02 Jul 2014	2	0	0	0
28	What is Crohn's Disease? What Causes Crohn's Disease? MNT	http://www.medicalnewstoday.com/articles/151620.php	Christian Nordquist, MNT, Sussex, United Kingdom.	2003	02 Jan 2014	7	0	0	0
29	Crohn's Disease, Netdoctor	http://www.netdoctor.co.uk/diseases/facts/crohnsdisease.htm	NetDoctor.co. Ltd, London, United Kingdom.	1998	24 Aug 2014	7	0	1	0
30	Crohn's Disease, UCSF medical center	http://www.ucsfhealth.org/conditions/crohns_disease/	University of California San Francisco Medical Center, California, United States.	2000	04 Jan 2012	2	0	0	0
31	Crohn's Disease Symptoms and Treatment, US.news Wellness	http://health.usnews.com/health-news/health-wellness/articles/2013/08/03/crohns-disease-symptoms-and-treatment	Guido Zanni, US News, New York, United States,	1995	22 Jan 2015	3	0	0	0
32	What are the treatments for Crohn's disease? Beth Israel Deaconess Medical Center	http://www.bidmc.org/Centers-and-Departments/Departments/Digestive-Disease-Center/Inflammatory-Bowel-Disease-Program/Crohns-Disease/What-are-the-treatments-for-Crohns-disease.aspx	Beth Israel Deaconess Medical Center, Massachusetts, United States.	2002	16 Mar 2006	15	0	4	1
33	Diagnosing Crohn's, Crohn's and Me.	http://www.crohnsandme.com/crohns-information/crohns-disease-diagnosis.aspx	UCB Multinational Biopharmaceutical Company, Brussels, Belgium	2005	29 Apr 2014	9	1	1	1
34	Understanding Crohn's Disease, Crohn's and Colitis.	http://www.crohnsandcolitisinfo.com/Crohns/What-is-Crohns-Disease	Crohn's and Colitis, Illinois, United States	2011	02 Oct 2014	8	0	1	1
35	Learning About Crohn's Disease, National Human Genome Research Institute	http://www.genome.gov/25521854	National Human Genome Research Institute, Massachusetts, United States		27 Sep 2011	2	0	0	0
36	Crohn's Disease, UPMC Life Changing Medicine	http://www.upmc.com/services/digestive-disorders-center/services/ibd/conditions/pages/crohns-disease.aspx	UPMC Digestive Disorders Center, UPMC Presbyterian, Pennsylvania, United States	1999	04 Mar 2014	3	0	0	0
37	Crohn's Disease, Cincinnati Children's	http://www.cincinnatichildrens.org/health/c/crohns/	Cincinnati Children's Hospital Medical Center, Ohio, United States	1998	15 may 2012	4	0	1	0
38	Crohn's Disease, Cleveland clinic	http://my.clevelandclinic.org/health/diseases_conditions/hic_Inflammatory_Bowel_Disease_IBD_QandA/hic_Crohns_Disease	The Cleveland Clinic Foundation, Ohio, United States	1998	02 Jan 2015	2	0	0	0
39	Treatment of Crohn's Disease, UNC Multidisciplinary Center for IBD Research and Treatment	http://www.med.unc.edu/gi/specialties/ibd/about-ibd/treatment-of-ibd-1/treatment-of-crohns-disease	UNC, School of Medicine, North Carolina, United States	1986	07 Mar 2013	9	0	9	10
40	Crohn's Disease, Emedicine health	http://www.emedicinehealth.com/crohn_disease/article_em.htm	EMedicine.com Inc, WebMD Network, Georgia, United States	2003	06 Sep 2013	10	0	0	1
41	Understanding Crohn's Disease and Ulcerative Colitis. Australian Gastroenterology Institute.	http://www.nevdgp.org.au/info/gastro/crohns.htm	Australian Gastroenterology Institute, Digestive Health Foundation, New South Wales, Australia		02 Jun 2014	3	0	0	0
42	Crohn's Disease, Health Centers	http://www.drweil.com/drw/u/ART00339/Crohns-Disease.html	Weil's Foundation, Arizona, United States	1999	16 Jan 2012	3	0	1	0
43	Crohn's Disease- An Overview, the Royal Children's Hospital Melbourne	http://www.rch.org.au/kidsinfo/fact_sheets/Crohns_Disease_an_overview/	The Royal Children's Hospital Melbourne, Victoria, Australia		19 Jun 2014	3	0	0	0
44	Fighting Inflammatory Bowel Disease Together, the Irish Society for Colitis and Crohn's Disease	http://www.iscc.ie/page.php?id=18&title=What%20is%20IBD	The Irish Society for Colitis and Crohn's Disease, Dublin, United Kingdom	2000	05 Jan 2014	4	0	10	1
45	Crohn's Disease Diagnosis, New health guide	http://www.newhealthguide.org/Crohn%27s-Disease-Diagnosis.html	New Health Guide, United States	2013	10 Feb 15.	3	0	1	0
46	Crohn's Disease, HealthDay.	http://consumer.healthday.com/encyclopedia/digestive-health-14/digestion-health-news-200/crohn-s-disease-644392.html	HealthDay, New York, United States	2002	15 Apr 2014	4	0	0	0
47	Ulcerative Colitis, Wikipedia	http://en.wikipedia.org/wiki/Ulcerative_colitis	Wikimedia Foundation, Inc, California, United States	2001	08 May 2012	14	6	6	1
48	Living with UC, Do You Know Your Treatment Options?	http://www.livingwithuc.ca/	Janssen Inc., Canada	2010,	May 2014	8	4	0	2
49	What Is Ulcerative Colitis? Everyday Health	http://www.everydayhealth.com/conditions/ulcerative-colitis	Everyday Health Media, LLC, New York, United States	2004	10 Feb 2014	14	0	0	0
50	What Is Ulcerative Colitis? News Medical	http://www.news-medical.net/health/What-is-Ulcerative-Colitis.aspx	The AZO Network, New South Wales, Australia	2004	26 Feb 2015	2	0	0	0
51	Crohn's Disease and Ulcerative Colitis, Better Health Channel	http://www.betterhealth.vic.gov.au/bhcv2/bhcarticles.nsf/pages/Crohn%27s_disease_and_ulcerative_colitis	Crohn's and Colitis, The State Government of Victoria, Victoria, Australia	2007	28 Sep 2014	4	0	0	0
52	Ulcerative Colitis, Jackson Siegelbaum Gastroenterology	http://gicare.com/diseases/ulcerative-colitis/	Jackson Gastroenterology Ltd, Central Pennsylvania, Pennsylvania, United States	1997	07 April 2014	3	0	0	3
53	Ulcerative Colitis, Crohn's and Colitis Canada	http://www.crohnsandcolitis.ca/site/c.dtJRL9NUJmL4H/b.9012449/k.C223/Ulcerative_Colitis.htm	Crohn's and Colitis, Canada	2008	19 Dec 2013	12	0	0	1
54	Ulcerative Colitis, Healthgrades	http://www.healthgrades.com/conditions/ulcerative-colitis	HealthGrades, Inc, Colorado, United States	1999	08 May 2014	4	0	0	0
55	Information for Those with Ulcerative Colitis, Colitis UK	http://www.ulcerativecolitis.org.uk/	Colitis UK, Buckinghamshire, United Kingdom	2005	20 Oct 2013	10	0	0	0
56	Colitis and Chronic Ulcerative Colitis, Virginia Mason	https://www.virginiamason.org/ColitisandChronicUlcerativeColitis	Virginia Mason Medical Center, Washington, United States	1998	07 May 2008	3	0	0	0
57	Ulcerative Colitis, GastroNet	http://www.gastro.net.au/diseases/ulcerativecolitis.html	GastroNet Australia Pty Ltd, Canberra, Australia	2009	12 May 2014	5	2	4	5
58	Ulcerative Colitis, Ulcerative Colitis Net.	http://www.ulcerativecolitis.net/	Serovera, Florida, United States	2000	28 Feb 2014	9	3	1	0
59	Inflammatory Bowel Disease, Lab Tests Online	http://labtestsonline.org/understanding/conditions/inflammatory-bowel	American Association for Clinical Chemistry (AACC), Washington, DC, United States	2001	08 Nov 2010	3	3	0	2
60	Inflammatory Bowel Disease Fact Sheet, Womenshealth.gov	http://www.womenshealth.gov/publications/our-publications/fact-sheet/inflammatory-bowel-disease.html	Womenshealth.gov, the US Department of Health and Human Services	1995	29 Nov 2014	3	0	0	3
61	Inflammatory Bowel Disease (IBD), Innerbody	http://www.innerbody.com/diseases-conditions/ibd	InnerBody, California, United States	1996	02 Oct 2012	5	2	3	4
62	Inflammatory Bowel Disease (IBD), Rightdiagnosis	http://www.rightdiagnosis.com/i/inflammatory_bowel_disease/intro.htm	Rightdiagnosis. com, United States	2005	11 Jul 2013	2	1	5	0
63	Inflammatory Bowel Disease, Lifescript.com	http://www.lifescript.com/health/centers/digestive/related_conditions/inflammatory_bowel_disease.aspx	LifeScript, California, United States	1999	05 May 2014	1	5	2	1
64	Inflammatory Bowel Disease (IBD), MUSC Health	http://www.ddc.musc.edu/public/symptomsDiseases/diseases/smallBowel/IBD.html	Digestive Disease Center, The Medical University of South Carolina, South Carolina, United States	1990	21 May 2013	1	1	1	0
65	Inflammatory Bowel Disease (IBD): Ulcerative Colitis, Crohn's Disease, New York-Presbyterian Digestive Diseases.	http://nyp.org/services/digestive/ibd.html	NewYork-Presbyterian Hospital, New York, United States	1998	24 Aug 2007	1	2	0	0
66	Inflammatory Bowel Disease, Human Diseases and Conditions Forum.	http://www.humanillnesses.com/original/Her-Kid/Inflammatory-Bowel-Disease.html	Human Diseases and Conditions Forum, United States	2006	16 Oct 2013	2	3	1	0
67	Facts About Crohn's Disease, US. Food and Drug Administration	http://www.fda.gov/ForConsumers/ConsumerUpdates/ucm107358.htm	US Food and Drug Administration, The US Department of Health and Human Services, Maryland, United States	1997	14 Oct 2014	1	3	2	1
68	Crohn's Disease Symptoms and warning Signs, SymptomFind	http://www.symptomfind.com/diseases-conditions/crohns-disease-symptoms-warning-signs/	Symptom.Find.Com. United States	2008	21 Jun 2014	3	2	1	2
69	Crohn's Disease-At a Glance, SixPartsWater.Org	http://www.sixpartswater.org/knowledge-centre/crohns-disease/glance	SixPartsWater.Org, United Kingdom	2007	02 Oct 2012	2	1	3	0
70	Ulcerative Colitis, eMedTV	http://colitis.emedtv.com/ulcerative-colitis/ulcerative-colitis.html	eMedTV, Washington, United States	2005	07 May 2014	3	0	5	2
71	Crohn's Disease, Department of Surgery, University of California.	http://colorectal.surgery.ucsf.edu/conditions--procedures/crohns-disease.aspx	Department of Surgery, University of California, California, United States	1986	10 Oct 2013	1	1	4	1
72	Crohn's Disease, the National Institute of Diabetes and Digestive and Kidney Diseases	http://www.niddk.nih.gov/health-information/health-topics/digestive-diseases/crohns-disease/Pages/facts.aspx	The National Institutes of Diabetes and Digestive and Kidney Diseases, NIDDK, Maryland, United States	2002	20 Jul 2014	1	1	2	0
73	Crohn's Disease, Patient Education Center	http://www.patienteducationcenter.org/articles/crohns-disease/	Patient Education Center, Harvard Medical School, Harvard Medical Publications, Massachusetts, United States	2003	20 Jun 2014	2	0	3	0
74	Crohn's Disease Information, alot health	http://health.alot.com/conditions/crohns-disease-information--163	Alot Health.Com, Arkansas, United States	1994	15 Aug 2014	3	1	1	1
75	Crohn's Disease, Diagnose-me.Com	http://www.diagnose-me.com/symptoms-of/crohns-disease.html	Diagnose-me.Com, Hawaii, United States	2002	21 Jan 2014	1	3	4	1
76	Crohns Disease Information: Is Colon Cleansing the Answer, Colon Cleanse Information	http://www.colon-cleanse-information.com/crohns-disease-information.html	Colon-Cleanse-Information.Com, MKR Concepts, Oregon, United States	2007	11 Nov 2014	3	0	5	0
77	Crohn's Disease or Regional Enteritis, MD India.	http://www.medindia.net/patients/patientinfo/Crohns-Disease.htm	Medindia4u.com Pvt. Ltd, Chennai, India	2000	21 Nov 2014	5	0	1	0
78	Crohn's Disease Information, Digestive Disorders	http://www.articleinsider.com/health-and-fitness/digestive-disorders/crohns-disease-information	Digestive Disorders, United States	2003	03 Sep 2014	2	1	2	1
79	Inflammatory Bowel Disease, Patient Center, American College of Gastroenterology	http://patients.gi.org/topics/inflammatory-bowel-disease/	Patient Center, American College of Gastroenterology, Maryland, United States	1996	16 May 2013	4	3	1	2
80	Crohn's Disease:Symptoms, Diagnosis and Treatment, Disabled World.Com	http://www.disabled-world.com/health/digestive/crohns-disease/	Disabled World.Com, New York, United States	2004	29 Oct 2012	1	3	4	1
81	Crohn's Disease, Nutritionist Resource	http://www.nutritionist-resource.org.uk/articles/crohns-disease.html	Nutritionist Resource, Surrey, United Kingdom	2010	9 Feb 15	3	1	3	0
82	Crohn's Disease: Symptoms, Diagnosis and Treatment, verywell.com	http://seniorhealth.about.com/cs/digestivetract/a/crohns_2.htm	Verywell.com part of about.com, Inc., United States	1999	10 Feb 14	6	0	4	0
83	Ulcerative Colitis, Halyard Surgical.	http://www.ulcerative-colitis.org/	Halyard Surgical, New South Wales, Australia	2003	7 Nov 14	2	0	6	1
84	Ulcerative Colitis Overview, Health Communities.com	http://www.healthcommunities.com/colitis/ulcerative-colitis-overview.shtml	Healthcommunities.com, New York, United States	1998	10 May 13	3	2	0	0

IBD: Inflammatory bowel disease; NHS: National Health Service; UNC: University of North Carolina.

For other websites, four websites were published in 1987-1994, 44 were published in 1995-2002, and 29 were published in the years 2003-2011. Only four websites were difficult to identify the exact year of their publication. Websites were updated regularly, 51 websites were updated in 2015 and 2014, while 33 websites were updated earlier, including one website was updated in 2006.

Of the 84 websites, 60 websites comprised 1-5 pages, 16 websites had 6-10 pages, 8 had more than 11 pages. The website titled Crohn's disease by the University of Maryland medical center had the highest number of pages, 20 pages. The number of tables varied from zero to 6. Out of the 84 websites only 24 websites used tables to explain their content. The total number of tables in these websites was 55. The number of images varied from zero to 27. Out of the 84 websites only 42 websites had images to explain the content. The total number of images in these websites was 141. Again the number of illustrations varied from zero to 10. Out of the 84 websites, only 28 had illustrations to explain the content. The total number of illustrations was 53.

DISCERN and the HONcode scores of websites

In order to calculate the accuracy of the websites, we used two instruments, the DISCERN and the HONcode instruments. Table 2 summarizes the scores calculated from applying the DISCERN and the HONcode scores expressed as mean ± SD for each website. The DISCERN scores varied from a minimum of 18 to a maximum of 68 (mean ± SD, 42.2 ± 10.7; median = 41.5, IQR = 15.8). The lowest DISCERN score was scored by the website, Crohn's Disease Diagnosis, New health guide, while the highest DISCERN score was scored by the website, Crohn's Disease, the National Institute of Diabetes and Digestive and Kidney Diseases. The HONcode trust worthy scores also varied from a minimum of 0.14 to a maximum of 0.95 (mean ± SD, 0.16 ± 0.19; median = 0.45, IQR = 0.29). The lowest HONcode score was scored by the website, Crohn's Disease, American family physician, while the maximum score was scored by the website, Crohn's Disease, the National Institute of Diabetes and Digestive and Kidney Diseases. Along with the HONcode trust worthy scores, HONcode certificate was indicated for websites that have received such certificates, Table 2.

Table 2 Summarizes websites included in the study on inflammatory bowel disease included in the study: The accuracy scores (calculated using the DISCERN score and the HONcode score) and the readability scores.

No.	Website title, Organisation	URL	Accuracy scores		Readability scores
			The DISCERN Score (mean ± SD)	The HONcode score (Out of 100)*	The Flesch-Kincaid Grade Level Index	The Coleman-Liau Readability Index
1	Inflammatory Bowel Disease (IBD), Mayo Clinic	http://www.mayoclinic.org/diseases-conditions/inflammatory-bowel-disease/basics/definition/con-20034908	65.0 ± 1.0	0.86*	13.1 ± 0.0	15.0 ± 0.0
2	Inflammatory Bowel Disease Health Center, WebMd	http://www.webmd.com/ibd-crohns-disease/	41.3 ± 1.1	0.70*	10.5 ± 3.2	13.3 ± 2.5
3	Inflammatory Bowel Disease (IBD), Center for Disease Control and Prevention (CDC)	http://www.cdc.gov/ibd/	28.7 ± 0.6	0.90	10.9 ± 2.7	13.3 ± 0.6
4	Inflammatory Bowel Disease, NHS Choices	http://www.nhs.uk/conditions/inflammatory-bowel-disease/pages/introduction.aspx	53.7 ± 0.6	0.63	12.2 ± 1.6	12.3 ± 1.5
5	What are Crohn's and Colitis? Crohn's and Colitis Foundation	http://www.ccfa.org/what-are-crohns-and-colitis/	50.3 ± 0.6	0.81	16.2 ± 9.3	13.7 ± 0.6
6	Inflammatory Bowel Disease, KidsHealth	http://kidshealth.org/parent/medical/digestive/ibd.html	50.7 ± 0.6	0.45	12.0 ± 0.7	11.7 ± 0.6
7	Inflammatory Bowel Disease (IBD). FamilyDoctor	http://familydoctor.org/familydoctor/en/diseases-conditions/inflammatory-bowel-disease.html	45.0 ± 1.0	0.63*	9.6 ± 1.4	10.7 ± 0.6
8	Inflammatory Bowel Disease, Healthline	http://www.healthline.com/health/inflammatory-bowel-disease#Overview1	41.0 ± 1.0	0.59*	8.7 ± 0.6	11.7 ± 0.6
9	Crohn's and Colitis, Australia	https://www.crohnsandcolitis.com.au/about-crohns-colitis/inflammatory-bowel-disease/	41.7 ± 0.6	0.34	13.6 ± 3.2	13.0 ± 1.0
10	Crohn's and Colitis UK	http://www.crohnsandcolitis.org.uk/information-and-support/information-about-ibd/what-is-IBD	60.7 ± 0.6	0.45	10.4 ± 1.6	10.3 ± 3.1
11	Inflammatory Bowel Disease (IBD) (Intestinal Problems of IBD), MedicineNet	http://www.medicinenet.com/inflammatory_bowel_disease_intestinal_problems/article.htm	57.3 ± 1.5	0.75*	13.3 ± 1.5	14.3 ± 1.5
12	Inflammatory Bowel Disease Center, Cedars-Sinai	http://www.cedars-sinai.edu/Patients/Programs-and-Services/Inflammatory-Bowel-Disease-Center/	25.7 ± 1.1	0.27	13.1 ± 2.9	14.0 ± 3.6
13	Inflammatory Bowel Diseases Symptoms and Treatment: Livescience	http://www.livescience.com/39880-inflammatory-bowel-disease.html	39.3 ± 0.6	0.27	11.1 ± 1.4	11.3 ± 1.5
14	Inflammatory Bowel Diseases Program, Penn Medicine.org	http://www.pennmedicine.org/gastroenterology/patient-care/gi-diseases/inflammatory-bowel-disease-ibd/	38.0 ± 0.0	0.54	15.1 ± 2.6	13.0 ± 2.0
15	Inflammatory Bowel Diseases Support Groups, IBDsupport.org	http://www.ibdsupport.org/	52.0 ± 1.7	0.61*	12.8 ± 3.7	13.0 ± 3.6
16	Inflammatory Bowel Disease (IBD), ABC Health and wellbeing	http://www.abc.net.au/health/library/stories/2012/02/22/3435688.htm	34.0 ± 0.0	0.45	11.1 ± 0.3	11.3 ± 0.6
17	Inflammatory Bowel Disease (IBD), GIKids	http://www.gikids.org/content/7/en/IBD	31.0 ± 0.0	0.52	12.5 ± 0.3	12.7 ± 0.6
18	Inflammatory Bowel Disease, Vitamin D Council	https://www.vitamindcouncil.org/health-conditions/inflammatory-bowel-disease/	31.3 ± 1.1	0.43	11.0 ± 10.4	10.7 ± 1.5
19	Inflammatory Bowel Disease Symptoms and Diagnosis, Seattle children's	http://www.seattlechildrens.org/medical-conditions/digestive-gastrointestinal-conditions/ibd-symptoms/	37.3 ± 0.6	0.43	8.9 ± 1.0	11.0 ± 1.0
20	Crohn's Disease, Patient.co.uk	http://www.patient.co.uk/health/crohns-disease-leaflet	55.7 ± 1.5	0.59*	8.5 ± 2.0	10.0 ± 1.7
21	Patient Information Crohn's Disease (Beyond and the Basics), Uptodate	http://www.uptodate.com/contents/crohn-disease-beyond-the-basics	36.7 ± 1.1	0.70	11.2 ± 1.9	12.3 ± 0.6
22	Crohn's Disease, Centre for digestive diseases	http://www.cdd.com.au/pages/disease_info/crohns_disease.html	34.7 ± 0.6	0.52	12.0 ± 0.6	14.3 ± 1.1
23	Crohn's Disease, Patients: British Society for Gastroenterology	http://www.bsg.org.uk/patients/general/crohn-s-disease.html	49.7 ± 0.6	0.56	11.3 ± 1.7	10.7 ± 1.1
24	Crohn's Disease, Bupa	http://www.bupa.co.uk/health-information/directory/c/crohns-disease	52.0 ± 0.0	0.77*	8.2 ± 0.5	9.0 ± 1.7
25	Crohn's Disease, University of Maryland Medical Center	http://umm.edu/health/medical/reports/articles/crohns-disease	64.7 ± 1.5	0.81	15.3 ± 4.8	15.0 ± 1.0
26	Crohn's Disease, Symptoms, Diagnosis, Treatment, Southern Cross	https://www.southerncross.co.nz/AboutTheGroup/HealthResources/MedicalLibrary/tabid/178/vw/1/ItemID/523/Crohns-disease-symptoms-diagnosis-treatment.aspx	42.7 ± 0.6	0.40	13.8 ± 0.3	13.7 ± 0.6
27	Crohn's Disease, American family physician	http://www.aafp.org/afp/2011/1215/p1379.html	42.3 ± 1.1	0.18	9.0 ± 2.9	12.3 ± 4.2
28	What is Crohn's Disease? What Causes Crohn's Disease? MNT	http://www.medicalnewstoday.com/articles/151620.php	45.7 ± 1.5	0.50*	11.2 ± 2.1	10.7 ± 1.5
29	Crohn's Disease, Netdoctor	http://www.netdoctor.co.uk/diseases/facts/crohnsdisease.htm	43.0 ± 0.0	0.59	10.0 ± 1.6	12.3 ± 1.5
30	Crohn's Disease, UCSF medical center	http://www.ucsfhealth.org/conditions/crohns_disease/	41.0 ± 1.0	0.40	11.4 ± 0.5	11.3 ± 0.6
31	Crohn's Disease Symptoms and Treatment, US.news Wellness	http://health.usnews.com/health-news/health-wellness/articles/2013/08/03/crohns-disease-symptoms-and-treatment	48.3 ± 0.6	0.50	6.7 ± 0.8	12.3 ± 2.5
32	What are the treatments for Crohn's disease? Beth Israel Deaconess Medical Center	http://www.bidmc.org/Centers-and-Departments/Departments/Digestive-Disease-Center/Inflammatory-Bowel-Disease-Program/Crohns-Disease/What-are-the-treatments-for-Crohns-disease.aspx	41.7 ± 1.5	0.59	14.3 ± 2.2	14.3 ± 1.1
33	Diagnosing Crohn's, Crohn's and Me.	http://www.crohnsandme.com/crohns-information/crohns-disease-diagnosis.aspx	43.3 ± 1.1	0.27	13.6 ± 1.1	12.7 ± 2.5
34	Understanding Crohn's Disease, Crohn' and Colitis.	http://www.crohnsandcolitisinfo.com/Crohns/What-is-Crohns-Disease	41.7 ± 0.6	0.45	12.4 ± 2.0	12.0 ± 2.6
35	Learning About Crohn's Disease, National Human Genome Research Institute	http://www.genome.gov/25521854	29.0 ± 0.0	0.40	10.8 ± 0.5	12.3 ± 1.5
36	Crohn's Disease, UPMC Life Changing Medicine	http://www.upmc.com/services/digestive-disorders-center/services/ibd/conditions/pages/crohns-disease.aspx	44.3 ± 0.6	0.27	20.3 ± 9.5	15.0 ± 2.0
37	Crohn's Disease, Cincinnati Children's	http://www.cincinnatichildrens.org/health/c/crohns/	44.7 ± 0.6	0.40	9.3 ± 1.1	10.7 ± 0.6
38	Crohn's Disease, Cleveland clinic	http://my.clevelandclinic.org/health/diseases_conditions/hic_Inflammatory_Bowel_Disease_IBD_QandA/hic_Crohns_Disease	26.3 ± 1.1	0.50*	12.9 ± 1.2	12.3 ± 2.3
39	Treatment of Crohn's Disease, UNC Multidisciplinary Center for IBD Research and Treatment	http://www.med.unc.edu/gi/specialties/ibd/about-ibd/treatment-of-ibd-1/treatment-of-crohns-disease	42.3 ± 0.6	0.45	12.3 ± 0.5	13.3 ± 0.6
40	Crohn's Disease, Emedicine health	http://www.emedicinehealth.com/crohn_disease/article_em.htm	59.3 ± 1.2	0.86*	13.0 ± 1.2	13.0 ± 1.7
41	Understanding Crohn's Disease and Ulcerative Colitis. Australian Gastroenterology Institute.	http://www.nevdgp.org.au/info/gastro/crohns.htm	42.3 ± 1.1	0.27	10.4 ± 0.7	11.3 ± 0.6
42	Crohn's Disease, Health Centers	http://www.drweil.com/drw/u/ART00339/Crohns-Disease.html	37.0 ± 0.0	0.50	11.9 ± 1.1	12.7 ± 1.5
43	Crohn's Disease- An Overview, the Royal Children's Hospital Melbourne	http://www.rch.org.au/kidsinfo/fact_sheets/Crohns_Disease_an_overview/	33.7 ± 1.1	0.50	7.7 ± 2.0	11.3 ± 1.5
44	Fighting Inflammatory Bowel Disease Together, the Irish Society for Colitis and Crohn's Disease	http://www.iscc.ie/page.php?id=18&title=What%20is%20IBD	37.7 ± 0.6	0.40	11.6 ± 1.4	10.7 ± 0.6
45	Crohn's Disease Diagnosis, New health guide	http://www.newhealthguide.org/Crohn%27s-Disease-Diagnosis.html	18.3 ± 0.6	0.65	13.5 ± 0.7	13.7 ± 1.1
46	Crohn's Disease, HealthDay.	http://consumer.healthday.com/encyclopedia/digestive-health-14/digestion-health-news-200/crohn-s-disease-644392.html	61.0 ± 1.0	0.86*	10.9 ± 1.4	12.0 ± 0.0
47	Ulcerative Colitis, Wikipedia	http://en.wikipedia.org/wiki/Ulcerative_colitis	54.0 ± 0.0	0.63	15.2 ± 2.1	14.0 ± 2.6
48	Living with UC, Do You Know Your Treatment Options?	http://www.livingwithuc.ca/	44.3 ± 1.1	0.68	12.2 ± 1.6	11.3 ± 2.3
49	What Is Ulcerative Colitis? Everyday Health	http://www.everydayhealth.com/conditions/ulcerative-colitis	40.0 ± 0.0	0.54*	14.1 ± 0.7	13.0 ± 1.0
50	What Is Ulcerative Colitis? News Medical	http://www.news-medical.net/health/What-is-Ulcerative-Colitis.aspx	46.0 ± 1.0	0.59	11.9 ± 0.2	13.0 ± 1.0
51	Crohn's Disease and Ulcerative Colitis, Better Health Channel	http://www.betterhealth.vic.gov.au/bhcv2/bhcarticles.nsf/pages/Crohn%27s_disease_and_ulcerative_colitis	34.3 ± 1.1	0.31	9.8 ± 2.4	11.3 ± 1.5
52	Ulcerative Colitis, Jackson Siegelbaum Gastroenterology	http://gicare.com/diseases/ulcerative-colitis/	52.3 ± 0.6	0.36	10.6 ± 1.4	11.7 ± 2.1
53	Ulcerative Colitis, Crohn's and Colitis Canada	http://www.crohnsandcolitis.ca/site/c.dtJRL9NUJmL4H/b.9012449/k.C223/Ulcerative_Colitis.htm	19.7 ± 0.6	0.31	9.2 ± 1.8	10.0 ± 1.0
54	Ulcerative Colitis, Healthgrades	http://www.healthgrades.com/conditions/ulcerative-colitis	52.7 ± 1.1	0.68	14.5 ± 3.5	14.0 ± 1.0
55	Information for Those with Ulcerative Colitis, Colitis UK	http://www.ulcerativecolitis.org.uk/	49.0 ± 1.7	0.45	14.5 ± 2.0	11.7 ± 1.1
56	Colitis and Chronic Ulcerative Colitis, Virginia Mason	https://www.virginiamason.org/ColitisandChronicUlcerativeColitis	40.7 ± 0.6	0.31	16.7 ± 3.9	15.0 ± 2.0
57	Ulcerative Colitis, GastroNet	http://www.gastro.net.au/diseases/ulcerativecolitis.html	26.7 ± 0.6	0.60	13.1 ± 1.1	13.3 ± 2.3
58	Ulcerative Colitis, Ulcerative Colitis Net.	http://www.ulcerativecolitis.net/	41.3 ± 0.6	0.30	11.4 ± 3.3	12.0 ± 2.0
59	Inflammatory Bowel Disease, Lab Tests Online	http://labtestsonline.org/understanding/conditions/inflammatory-bowel	38.3 ± 0.6	0.32*	13.0 ± 0.7	12.0 ± 1.0
60	Inflammatory Bowel Disease Fact Sheet, Womenshealth.gov	http://www.womenshealth.gov/publications/our-publications/fact-sheet/inflammatory-bowel-disease.html	55.7 ± 1.5	0.40	8.8 ± 0.7	10.0 ± 1.0
61	Inflammatory Bowel Disease (IBD), Innerbody	http://www.innerbody.com/diseases-conditions/ibd	39.0 ± 1.0	0.25	17.2 ± 5.7	16.7 ± 1.5
62	Inflammatory Bowel Disease (IBD), Rightdiagnosis	http://www.rightdiagnosis.com/i/inflammatory_bowel_disease/intro.htm	38.0 ± 1.0	0.33	15.9 ± 4.1	15.7 ± 2.1
63	Inflammatory Bowel Disease, Lifescript.com	http://www.lifescript.com/health/centers/digestive/related_conditions/inflammatory_bowel_disease.aspx	32.0 ± 1.0	0.50	15.1 ± 2.8	12.7 ± 1.5
64	Inflammatory Bowel Disease (IBD), MUSC Health	http://www.ddc.musc.edu/public/symptomsDiseases/diseases/smallBowel/IBD.html	22.7 ± 1.1	0.27	14.2 ± 2.2	15.7 ± 2.9
65	Inflammatory Bowel Disease (IBD): Ulcerative Colitis, Crohn's Disease, New York-Presbyterian Digestive Diseases	http://nyp.org/services/digestive/ibd.html	32.3 ± 0.6	0.27	13.2 ± 0.5	12.7 ± 0.6
66	Inflammatory Bowel Disease, Human Diseases and Conditions Forum.	http://www.humanillnesses.com/original/Her-Kid/Inflammatory-Bowel-Disease.html	35.3 ± 1.5	0.14	10.7 ± 0.9	13.3 ± 2.3
67	Facts About Crohn's Disease, US. Food and Drug Administration	http://www.fda.gov/ForConsumers/ConsumerUpdates/ucm107358.htm	34.7 ± 2.1	0.36	10.6 ± 2.6	13.7 ± 2.1
68	Crohn's Disease Symptoms and warning Signs, SymptomFind	http://www.symptomfind.com/diseases-conditions/crohns-disease-symptoms-warning-signs/	26.0 ± 1.0	0.26*	10.0 ± 1.9	13.7 ± 2.3
69	Crohn's Disease-At a Glance, SixPartsWater.Org	http://www.sixpartswater.org/knowledge-centre/crohns-disease/glance	39.7 ± 0.6	0.15	18.3 ± 3.4	12.7 ± 0.6
70	Ulcerative Colitis, eMedTV	http://colitis.emedtv.com/ulcerative-colitis/ulcerative-colitis.html	55.3 ± 0.6	0.32*	12.1 ± 0.8	11.0 ± 2.0
71	Crohn's Disease, Department of Surgery, University of California.	http://colorectal.surgery.ucsf.edu/conditions--procedures/ulcerative-colitis.aspx	51.0 ± 1.0	0.73	12.4 ± 0.8	12.3 ± 2.5
72	Crohn's Disease, the National Institute of Diabetes and Digestive and Kidney Diseases	http://www.niddk.nih.gov/health-information/health-topics/digestive-diseases/crohns-disease/Pages/facts.aspx	68.3 ± 1.2	0.95	13.6 ± 3.9	13.7 ± 5.5
73	Crohn's Disease, Patient Education Center	http://www.patienteducationcenter.org/articles/crohns-disease/	38.0 ± 1.0	0.30	8.9 ± 0.8	12.3 ± 1.5
74	Crohn's Disease Information, alot health	http://health.alot.com/conditions/crohns-disease-information--163	32.3 ± 0.6	0.22	10.4 ± 1.5	12.0 ± 2.6
75	Crohn's Disease, Diagnose-me.Com	http://www.diagnose-me.com/symptoms-of/crohns-disease.html	32.0 ± 1.0	0.28	12.5 ± 1.6	14.3 ± 1.1
76	Crohns Disease Information: Is Colon Cleansing the Answer, Colon Cleanse Information	http://www.colon-cleanse-information.com/crohns-disease-information.html	35.7 ± 2.5	0.36	13.2 ± 0.5	14.7 ± 1.5
77	Crohn's Disease or Regional Enteritis, MD India.	http://www.medindia.net/patients/patientinfo/Crohns-Disease.htm	46.3 ± 1.1	0.54	11.8 ± 1.3	14.0 ± 2.6
78	Crohn's Disease Information, Digestive Disorders	http://www.articleinsider.com/health-and-fitness/digestive-disorders/crohns-disease-information	33.3 ± 0.6	0.60	10.2 ± 0.6	12.3 ± 0.6
79	Inflammatory Bowel Disease, Patient Center, American College of Gastroenterology	http://patients.gi.org/topics/inflammatory-bowel-disease/	58.3 ± 1.2	0.80	12.7 ± 1.3	13.0 ± 1.0
80	Crohn's Disease:Symptoms, Diagnosis and Treatment, Disabled World.Com	http://www.disabled-world.com/health/digestive/crohns-disease/	26.3 ± 2.3	0.19	8.6 ± 0.6	12.00 ± 1.73
81	Crohn's Disease, Nutritionist Resource	http://www.nutritionist-resource.org.uk/articles/crohns-disease.html	39.3 ± 0.6	0.57	10.2 ± 2.6	11.3 ± 1.1
82	Crohn's Disease: Symptoms, Diagnosis and Treatment, verywell.com	http://seniorhealth.about.com/cs/digestivetract/a/crohns_2.htm	44.3 ± 1.5	0.37	12.6 ± 0.7	11.0 ± 1.0
83	Ulcerative Colitis, Halyard Surgical.	http://www.ulcerative-colitis.org/	51.7 ± 1.5	0.25	12.1 ± 0.6	12.7 ± 2.5
84	Ulcerative Colitis Overview, Health Communities.com	http://www.healthcommunities.com/colitis/ulcerative-colitis-overview.shtml	52.7 ± 1.5	0.23*	12.6 ± 1.2	12.7 ± 1.1

^*Websites that received HONCode certificates. IBD: Inflammatory bowel disease; NHS: National Health Service; UNC: University of North Carolina.

The top ten websites on IBD as per the DISCREN scores were in the following order: The Crohn's Disease, the National Institute of Diabetes and Digestive and Kidney Diseases (scored 68), Inflammatory Bowel Disease, MayoClinic (Scored 65), Crohn's Disease, University of Maryland Medical Center (scored 64), Crohn's Disease, HealthDay (scored 61), Crohn's Disease and Colitis UK (scored 60), Crohn's Disease, eMedicine health (scored 59), Inflammatory Bowel Disease, Patient Center, American College of Gastroenterology (scored 58), Inflammatory Bowel Disease, MedicineNet (scored 57), Inflammatory Bowel Disease, Fact Sheet, Womenshealth.gov (scored 55), Ulcerative Colitis, eMedTV (scored 55). The top ten websites as per the HONcode tool were in the following order: Crohn's Disease, the National Institute of Diabetes and Digestive and Kidney Diseases (scored 0.95), Inflammatory Bowel Disease, Center for Disease Control and Prevention (scored 0.90), Inflammatory Bowel Disease, MayoClinic (scored 0.86), Crohn's Disease, eMedicine health (scored 0.86), Crohn's Disease, HealthDay (scored 0.86), What are Crohn's & Colitis? Crohn's & Colitis Foundation (scored 0.81), Crohn's Disease, University of Maryland Medical Center (scored 0.81), Inflammatory Bowel Disease, Patient Center, American College of Gastroenterology (scored 0.80), Crohn's Disease, Bupa (scored 0.77), and Inflammatory Bowel Disease, MedicineNet (scored 0.75). It is interesting to note that the website, Crohn's Disease, the National Institute of Diabetes and Digestive and Kidney Diseases was ranked number one as per the two instruments. Seven websites in total were among the top ten websites as per both the DISCERN and the HONcode scores. Nine out of the ten websites were created in United State.

Grouping the websites under five categories

Table 3 summarizes the grouping of the 84 websites under five categories on the basis of the website creators. Universities and research centers created 25 (%), professional foundations and associations created 15 (%), commercial and pharmaceutical companies created 25 (%), charities and volunteers contributed to 9 (%), non-university educational bodies such as colleges, academies, councils, WebMed contributed to 10 (%). Further analysis revealed that there was no significant differences in the DISCERN scores between the groups (P = 0.472) but the HONcode scores were different (P = 0.041). Examples of content deficiencies or scientific content inaccuracies and suggestions for improvement are shown on Table 4.

Table 3 Grouping the websites on inflammatory bowel disease included in the study under five categories.

Category	Number	DISCERN score		HONcode score
		mean ± SD6	95%CI for means	mean ± SD7	95%CI for means
Universities and Research Centers1	25	41.3 ± 11.3	36.6-45.9	0.46 ± 0.20	0.37-0.54
Foundations and Associations2	15	44.5 ± 10.5	38.6-50.2	0.53 ± 0.16	0.44-0.62
Commercial and Pharmaceutical Companies3	25	40.4 ± 10.1	36.3-44.6	0.44 ± 0.18	0.36-0.52
Charities and Volunteer work4	9	40.2 ± 11.9	31.0-49.4	0.39 ± 0.17	0.26-0.52
Non-university Educational Bodies5	10	46.7 ± 9.7	39.7-53-6	0.63 ± 0.20	0.48-0.76
Total	84	42.1 ± 10.7	39.8-44.5	0.48 ± 0.19	0.44-0.52

¹This category includes university-affiliated centers, state hospitals, national or state research centers;

²This category includes gastroenterological societies, foundations, and associations- most were on inflammatory bowel disease, Crohn's disease or ulcerative colitis;

³This category includes industrial bodies, commercial and pharmaceutical companies aiming at serving the community and patients with inflammatory bowel disease;

⁴This category includes charities and websites created by individuals, or groups;

⁵This category includes all other non-university educational bodies including colleges, academies, councils, WebMed, etc.;

⁶The DISCERN scores were not significantly different between the groups as per ANOVA (combined, P = 0.472) or (linear term, P = 0.475);

⁷The HONcode scores were significantly different between the groups as per ANOVA (combined, P = 0.041). The linear term, P = 0.228.

Table 4 Examples of assessment of the content of some websites on inflammatory bowel disease.

Website Number	Title	Areas of deficiencies	Suggestions for improvement
45	Crohn's Disease Diagnosis, New health guide	Symptoms of Crohn's disease are briefly mentioned. Some details are needed to explain the common presenting symptoms. No mention of differential diagnosis. No mention of investigations needed to confirm the diagnosis. Nothing is mentioned about treatment of Crohn's disease.	Symptoms may include abdominal pain, typically in the right lower quadrant, diarrhoea, some blood may be present in stools, fatigue. In more severe disease fever, and weight loss may be present. Some patients may have nausea, and abdominal distention together with abdominal pain. It is worth to mention that Crohn's disease is a lifelong illness (chronic disease). People who have Crohn's will experience periods of flare-ups, when their symptoms are active, and other times when their symptoms go into remission. Up to 30% of patients may have changes in the area around the anus including anal fistulas (internal tracts connecting the anal lumen with the skin around the anus), abscess, skin tags, and anal fissures. About 10%-20% of patients also have joint pains, lower back pain, skin rash known as erythema nodosum, and eye changes. (images showing some of these changes will enhance this part). A section discussing investigations should be added. In addition to detailed medical history, the treating doctor will initiate the evaluation by testing for infectious conditions that can cause inflammation of the colon, screen for endocrine-metabolic disorders such as excessive activity of the thyroid gland. Therefore biochemical tests and stool tests are needed. Endoscopic evaluation (colonoscopy) should be carried out in patients who have symptoms suggestive of inflammatory bowel disease and no evidence for an infection to explain symptoms. Small bowel images, computed tomography (CT) enterography may also be needed. Nutritional changes, medical and surgical treatment should be briefly discussed.
53	Ulcerative Colitis, Crohn's and Colitis Canada	Symptoms are briefly stated. No mention of differential diagnosis, investigations and no discussion of medical and surgical treatment.
64	Inflammatory Bowel Disease (IBD), MUSC Health	Symptoms of inflammatory bowel disease are not clearly written. One would wonder, are "bowel sores" and "intestinal bleeding" symptoms? Differential diagnosis is not mentioned. The approach for diagnosing inflammatory bowel disease is not mentioned and the treatment of IBD is not explained.	As discussed earlier.
68	Crohn's Disease Symptoms and Warning Signs, SymptomsFind	Although symptoms of Crohn's disease are mentioned briefly, they are not explained. Mild, moderate and severe inflammatory bowel disease are stated but not explained. This should be explained in a simple language. Complications are mentioned but there was no mention how the disease is diagnosed, and what investigations are needed. Nothing is mentioned about nutritional changes, medical and surgical treatment of inflammatory bowel disease.	Patients are described to have mild ulcerative colitis when they have: -Fewer than four bowel motions (stools) per day. -No bleeding or small amounts of bleeding in their stools. -Normal erythrocyte sedimentation rate (ESR) -No fever, no anaemia and no increases in their heart rate, Patients are described to have moderate ulcerative colitis when they have: -More than four stools per day. -Mild elevation in ESR. Patients are described to have severe ulcerative colitis when they have: -More than six stools a day (loose stools). -Fever, rapid heartbeat, and anaemia. -Elevated ESR. The website may also mention changes that necessitate hospital admission and medical attention. Websites may provide key questions that patients may use when they review their treating doctors. Examples of these questions: •I wonder what's causing these symptoms? •What type of tests do I need? Do these tests require any special preparation? •What treatments are available, and which do you recommend? •Are there any medications that I should avoid? •Do I need to follow any dietary restrictions? •Are there any risks if I become pregnant? The MayoClinic website has listed a number of useful questions that patients can use.
12	Inflammatory Bowel Disease Center, cedars-Sinai.	Under symptoms of Crohn's disease, it is written, "The most common signs are pain in the stomach area (usually on the right side) and diarrhea", it is not clear what is meant by pain in the stomach area on the right side? Complications are provided but no signs are stated. No mention of differential diagnosis, no mention of investigations and possible findings. Nutritional changes, medical and surgical treatment are not explained.	The authors should differentiate between symptoms and signs. Scientific errors are noted in the website and common presenting symptoms should be stated. It may be useful to explain the symptoms under two main headings: symptoms in children, and symptoms in adults. Investigations needed to diagnose the disease should be discussed. Patients are usually interested to know more detail about these investigations. Information provided should answer questions such as -Name of the test -Why the test is need? -What is the test about? -Nature of the investigation (invasive vs non-invasive) -Are there special preparations needed prior to the test? -Any possible complications related to the investigation? -What can the results of the test tell the patient and the treating doctor? It is also important to state that IBD is an ongoing condition (chronic disease), so some of the tests may need to be repeated from time to time, or extra tests may be needed. These investigations may include: (1) Blood tests including full blood count, inflammatory markers tests including erythrocyte sedimentation rate (ESR), c-reactive protein (CRP), liver function tests, urea and electrolytes, and other biochemical tests, (2) stool tests including stool microscopy, stool culture and sensitivity, fecal markers such as fecal calprotectin, fecal lactoferrin, (3) Endoscopy including colonoscopy, sigmoidoscopy, proctoscopy, with biopsies for histological studies (4) radiological studies such as barium studies, CT scans, MRI scans and PET scans.

Readability level of websites

Table 2 summarizes the readability scores calculated by using two methods, the Flesch-Kincaid Grade Level Index and Coleman-Liau Readability Index. The minimum score for the Flesch-Kincaid Grade Level Index was 6.7 for the website Crohn's Disease Symptoms and Treatment, United States news Wellness, while the maximum score was 20.3 for the website Crohn's Disease, UPMC Life Changing Medicine. Out of the 84 websites, 28 received a mean of 6.7 to 10.9, forty-six received a mean of 11.0 to 14.5, and ten websites received a mean of 15.7 to 20.3. The overall mean score for the 84 websites was 11.9 ± 2.4.

For the Coleman-Liau Readability Index the minimum score was 9.0 for the website Crohn's Disease, Bupa, while the maximum score was 16 for the website Inflammatory Bowel Disease, Fact Sheet, Womenshealth. Out of the 84 websites, eleven received a mean score of 9.0 to 10.9, thirty-nine received a score of 11.0 to 12.7, and thirty-four received a score of 13.0 to 16.0. The overall mean score for the 84 websites was 12.6 ± 1.5.

Significant correlation was found between the Flesch-Kincaid Grade Level index scores and the Coleman-Liau Index scores (R² = 0.509, P < 0.001) (Figure 2).

Figure 2 Correlation between the scores of the two readability methods: Coleman-Liau Readability Index and Flesch-Kincaid Grade Level Index.

The agreement between the evaluators

Table 5 summarizes the inter-rater agreement between evaluators for the DISCERN instrument items. The overall Cohen kappa scores were in the range of 0.804-0.876.

Table 5 Summarizes the inter-rater agreement between evaluators calculated using Cohen kappa coefficient scores.

DISCERN items	Mean score (95%CI of the difference)			Reviewer variability (κrange)
	Evaluator 1	Evaluator 2	Evaluator3
1. Are the aims clear?	1.7 (1.5-2.0)	1.7 (1.5-2.0)	1.8 (1.5-2.0)	0.885-0.953
2. Does it achieve its aims?	1.4 (0.9-1.8)	1.5 (1.0-1.9)	1.5 (1.0-1.9)	0.790-0.904
3. Is it relevant?	3.8 (3.6-4.0)	3.8 (3.6-4.0)	3.8 (3.6-4.0)	0.792-0.887
4. Is it clear what sources of information were used to compile the publication (other than the author or producer)?	2.5 (2.2-2.9)	2.5 (2.2-2.8)	2.5 (2.2-2.8)	0.879-0.880
5. Is it clear when the information used or reported in the publication was produced?	2.7 (2.4-3.1)	2.7 (2.3-3.0)	2.7 (2.3-3.0)	0.793-0.875
6. Is it balanced and unbiased?	3.7 (3.5-4.0)	3.8 (3.6-4.0)	3.7 (3.5-3.9)	0.796-0.890
7. Does it provide details of additional sources of support and information?	2.2 (1.9-2.5)	2.2 (1.9-2.5)	2.2 (1.9-2.5)	0.792-0.827
8. Does it refer to areas of uncertainty?	3.2 (3.0-3.4)	3.1 (3.0-3.3)	3.2 (3.0-3.3)	0.764-0.832
9. Does it describe how each treatment works?	3.1 (2.8-3.4)	3.1 (2.8-3.4)	3.1 (2.8-3.4)	0.859-0.874
10. Does it describe the benefits of each treatment?	2.4 (2.2-2.7)	2.4 (2.2-2.7)	2.5 (2.2-2.8)	0.782-0.841
11. Does it describe the risks of each treatment?	2.3 (2.0-2.6)	2.3 (2.0-2.6)	2.3 (2.0-2.7)	0.772-0.902
12. Does it describe what would happen if no treatment is used?	1.5 (1.3-1.7)	1.5 (1.3-1.7)	1.5 (1.3-1.8)	0.870-0.923
13. Does it describe how the treatment choices affect overall quality of life?	2.0 (1.8-2.2)	2.0 (1.8-2.2)	2.1 (1.9-2.3)	0.859-0.906
14. Is it clear that there may be more than one possible treatment choice?	3.6 (3.4-3.9)	3.6 (3.4-3.9)	3.6 (3.4-3.8)	0.773-0.849
15. Does it provide support for shared decision-making?	2.7 (2.4-2.9)	2.8 (2.6-3.0)	2.8 (2.5-3.0)	0.767-0.854
16. Based on the answers to all of the above questions, rate the overall quality of the publication as a source of information about treatment choices?	2.9 (2.7-3.2)	3.0 (2.7-3.2)	2.9 (2.6-3.2)	0.900-0.959

DISCUSSION

Several studies pointed to continuous progress from paper to electronic and online-based patient education[29,30]. The aims of the study were to evaluate the quality and accuracy of information available on IBD websites and calculate the readability level using two methods. To maximize the yield of the search, we searched three search engines commonly used by the public seeking information related to healthcare. The study showed that the 84 websites identified were created by universities, affiliated hospitals and research centers, professional foundations and associations, commercial and pharmaceutical companies created, charities and volunteers, as well as non-university educational bodies (such as colleges, academies, councils, and WebMed). The involvement of universities, affiliated hospitals, and research centers is directed at health information exchange as well as public and patient education with the aim to improve the quality of care, engage the patient in the decision-making processes and the journey of treatment as well as enhance patient's awareness about the nature of their illness. Such educational approaches while having multiple impacts on the patients' healthcare; it can also help in reducing the costs of treatment[31]. The current move from paper-based to online health care education may be related to the progressive increases in the use of the Internet by the public and patients[32]. Furthermore, Morgan et al[33] showed that patients with genetic and chronic diseases have great interest in participating in clinical studies and a desire to understand information discussed during reviewing their healthcare provider. These patients may have more questions after they leave the doctor's clinic and usually tend to search the Internet for answers[33]. Compared to paper-based health education, the Internet appears to provide a wider range of answers and options. However, the quality of information provided and the readability level remain as areas of concern[8,34].

As per this study, the DISCERN and the HONcode scores varied. However, no significant differences in the DISCERN scores were found between the groups but when the groups were compared on the basis of the HONcode scores, the difference was significant. A weak correlation was found between the DISCERN scores and the HONcode scores (R² = 0.217). The results are consistent with the variability of the DISCERN scores of websites in each group and the fact that the two instruments are not measuring the same characteristics[25]. Interestingly, seven out of the top 10 websites on IBD scored higher on both the DISCERN and the HONcode scales. Looking into the readability levels of these seven websites, the readability using the Flesch-Kincaid Grade level was in the range 11 to 15, while for the Coleman-Liau Readability Index the range was 12 to 15. This indicates that even the top 7 websites had a readability level not adjusted to the public level.

Out of the 84 websites, only 17 displayed the HONcode certificate. A recent study found that only three websites out of 78 showed HONcode certificates[35]. Although the number of websites granted a HONcode certificate is small yet there is no correlation between the calculated HONcode scores and having a certificate on the website. Absence of the HONcode certificate from a website doesn't necessarily indicate poor quality of the website. This is because the process of issuing the HONcode is based on a voluntary application for the certificate. Therefore, it is possible that the owners/authority responsible for these websites did not apply for the HONcode certificate.

The readability scores were calculated by using two methods, the Flesch-Kincaid Grade Level Index and Coleman-Liau Readability Index. The moderate correlation between the Flesch-Kincaid Grade Level index scores and the Coleman-Liau Index scores is consistent with other work[20] and indicates that the results from the two calculations are consistent. The findings show that the majority of the studies had a readability level equivalent to year 11 and 12. However, the national reading grade level average has been estimated to be about the 6^th-grade[36] and the general agreement is that the reading level for patient information materials should not exceed this level and be no less than what a 4^th-grade is capable of reading[37]. With these findings in mind, there is a need for editing the content of most websites identified and adjusting the reading levels to meet the recommended reading levels for the public.

This study has a number of strengths; first, we searched three different search engines commonly used by the public seeking health-related information with the aim to maximize the yield of the search. Second, we used two instruments the DISCERN and the HONcode to measure the accuracy of contents. Both instruments have been widely used in assessing online health information material. Third, three evaluators independently conducted the evaluation and the inter-rater agreement among the assessors was within the accepted limits. Finally, the readability was measured by using two different methods. However, this study is not without limitations; the study is just a representation of websites identified at the time of the search. Only websites in the English language were included, and there is the possibility that there are other websites in other languages that match with our inclusion criteria and were not included. A multinational study may be needed to identify any differences if any and resolve gaps in this area. Therefore, despite all efforts and the plans considered, we may have missed some websites.

The results of this study may be of value to general practitioners, physicians, gastroenterologists, nurses, and allied health professionals, the public and medical students interested in online education material on IBD. The top 10 websites with the highest DISCERN and the HONcode scores identified from this study provide examples of educationally useful websites that can be recommended by treating physicians to their patients. However, their readability level was above the recommended level for the public and they may be suitable for educated patients only.

Future directions

This study highlights a number of future directions in research in the area of Internet-based patient education particularly patients with IBD. These can be summarized as follows: First, planning for creating online educational material for the public and patients with IBD necessitates more care for innovation, content accuracy and readability level to match the recommended needs of the public. Second, more work is needed to enhance the use of images, illustrations, and videos in improving the educational usefulness of websites on IBD and engage the patients and the public using such online resources. The use of these educational tools should aim at explaining difficult concepts, and enhancing understanding of the message given. As shown from this study the use of these educational tools was deficient in most websites. Third, future research should aim at assessing the impact of using Internet education and health literacy in patients with IBD and whether such resources have made impacts on number of hospital admissions, costs associated with poor health literacy, effective health education techniques, and how poor health literacy influences management outcome in these patients.

In conclusion, health literacy about IBD and the use of Internet as a medium for education appears to be increasing. Universities, research centers, commercial and pharmaceutical companies, professional foundations and associations were the major contributors to online resources written for the public and patients. Several deficiencies in the content were observed and most websites failed to meet the recommendations set by the National Institute of Health and American Medical Association that patients resources should be written about the 6^th-grade level. Effective use of diagrams, illustrations, videos, and tables to explain difficult concepts should be encouraged. Revising the websites and resolving the gap between the readability of written health information and the literacy skills of the public will improve the purpose of these websites and make them a useful healthcare resource to patients with IBD.

ACKNOWLEDGMENTS

The authors would like to thank Dr. Sarah Azer of Box Hill hospital for her kind review of the manuscript.

COMMENTS

Background

Patients with inflammatory bowel disease (IBD), as it is the case with other chronic diseases, seek information about the nature of their disease. The increasing use of the Internet embraces the significance of online resources educating patients and the public.

Research frontiers

With the abundance of information there is concern about the quality, accuracy and readability level of information available on the web, thus it might be useful to assess the quality of these resources, identify specific deficiencies and examine whether these websites meet the recommendations of national bodies.

Innovations and breakthrough

The goal of this paper is to use comprehensive analysis to assess the quality of websites, accuracy of content and readability levels of websites on IBD dedicated to patients and the public.

Applications

The study highlights a number of future directions in the area of Internet-based patient education particularly patients with IBD and raises the need for improving such resources particularly in relation to specific areas identified in the study.

Terminology

Scientific accuracy and quality of content were evaluated using two standardised instruments widely used in research. The readability level was calculated on the bases of word length, sentence length and syllables.

Peer-review

This paper is an interesting evaluation of quality, accuracy, and readability of websites dedicated to the public. It is a novelty and represents a beginning point for judging and improving websites dedicated to IBD.