YouTube as a source of patient information on gallstone disease

Abstract

AIM: To investigate the quality of YouTube videos on gallstone disease and to assess viewer response according to quality.

METHODS: A YouTube search was performed on September 18, 2013, using the keywords ‘‘gallbladder disease’’, ‘‘gallstone disease’’, and ‘‘gallstone treatment’’. Three researchers assessed the source, length, number of views, number of likes, and days since upload. The upload source was categorised as physician or hospital (PH), medical website or TV channel, commercial website (CW), or civilian. A usefulness score was devised to assess video quality and to categorise the videos into ‘‘very useful’’, ‘‘useful’’, ‘‘slightly useful’’, or ‘‘not useful’’. Videos with misleading content were categorised as ‘‘misleading’’.

RESULTS: One hundred and thirty-one videos were analysed. Seventy-four videos (56.5%) were misleading, 36 (27.5%) were slightly useful, 15 (11.5%) were useful, three (2.3%) were very useful, and three (2.3%) were not useful. The number of mean likes (1.3 ± 1.5 vs 17.2 ± 38.0, P = 0.007) and number of views (756.3 ± 701.0 vs 8910.7 ± 17094.7, P = 0.001) were both significantly lower in the very useful group compared with the misleading group. All three very useful videos were PH videos. Among the 74 misleading videos, 64 (86.5%) were uploaded by a CW. There was no correlation between usefulness and the number of views, the number of likes, or the length. The “gallstone flush” was the method advocated most frequently by misleading videos (25.7%).

CONCLUSION: More than half of the YouTube videos on gallstone disease are misleading. Credible videos uploaded by medical professionals and filtering by the staff of YouTube appear to be necessary.

Key Words: YouTube; Gallstone disease; Gallstone; Gallbladder; Cholecystitis

Core tip: Many people now use the Internet for medical information. There have been many studies evaluating the available information on YouTube, which is one of the most popular sources of medical information. In this paper, we present the first report of an evaluation of YouTube videos on gallstone disease. More than half of the videos were misleading, and there was no correlation between video quality and the number of views or number of likes. Credible videos uploaded by medical professionals, and a filtering process appear to be necessary.

INTRODUCTION

In the past, most people sought medical information by consulting medical professionals; however, due to the development and spread of the Internet, many people now use online resources to access medical information[1]. One of the most popular sources of Internet-based medical information is YouTube (http://www.youtube.com). As opposed to search engines where a keyword brings up websites and images as well as videos, the search results of YouTube contain only video content. Unregistered users can watch videos, and registered users can upload an unlimited number of videos. The only limitation for viewers is that the videos considered offensive are available only to registered users 18 years or older[2]. Since Keelan et al. reported on the quality of YouTube videos regarding immunisation in 2007[3], many authors have analysed videos on topics such as prostate cancer, burns, and tonsillectomy[1,4-8]. As more medical professionals recognise the importance of YouTube as a source of medical information for the general public, the number of studies evaluating the quality of information found on YouTube is increasing[9].

Calculous gallbladder disease, or gallstone disease, is common. When asymptomatic gallstones are discovered, follow up is recommended, except in special circumstances such as porcelain gallbladder or large gallstones[10]. Symptomatic gallstone disease should be treated with cholecystectomy, preferably via laparoscopy[11]. The only medication known to dissolute gallstones in carefully selected patients is ursodeoxycholic acid[10]. When treated inappropriately, serious complications may occur. In this study, the authors aimed to evaluate the accuracy of medical information about gallstone disease available on YouTube. The second goal was to evaluate the response of the general population to the quality of videos. To the best of our knowledge, this is the first study to evaluate the quality of YouTube videos about gallstone disease.

MATERIALS AND METHODS

A YouTube search was performed on September 18, 2013. Keywords used in the search were gallbladder disease, gallstone disease, and gallstone treatment. One hundred videos were analysed for each keyword under the assumption that the user would not go beyond the first five pages of search results. The videos were sorted in order of relevance, which is the current YouTube default. Approval of the Institutional Review Board of the study institution was not required for this study. Three researchers (Lee JS, Seo HS, and an additional researcher, Kim KM) independently assessed the videos. All researchers had finished their respective residencies at tertiary centres and had sufficient experience in the diagnosis and management of calculous gallbladder disease. Characteristics such as name of video, source, length, number of views, number of ‘‘likes’’ and ‘‘dislikes’’, and days since upload were recorded. The upload source was categorised as physician or hospital (PH), medical website (MW) or TV channel, commercial website (CW), or civilian (CI). Videos with a primary content of ‘‘gallstone disease’’ were analysed. Videos not in English, videos with no audio, surgical videos, and videos aimed at professional medical personnel (such as medical school lectures) were excluded. A usefulness score was devised to assess the information in each video (Table 1). This score was used to categorise videos as very useful, useful, slightly useful, or not useful. Regardless of the usefulness score, videos with misleading content were categorised as ‘‘misleading’’ and were further categorised according to which treatment modality was advocated in the video. Disagreements between the researchers regarding the categorisation of a particular video were resolved by discussing the issue until a consensus was reached.

Table 1 Usefulness score criteria.

Score criteria
Cause
Symptoms
Diagnosis
Treatment
Recovery

Not mentioned: 0; Mentioned briefly: 1; Mentioned in detail: 2. Total score: not useful (0), slightly useful (1-3), useful (4-7), very useful (8-10).

Differences between groups were compared with ANOVA, and Tukey’s test was used for post hoc comparisons. Spearman’s rank coefficient was used to analyse correlations. A weighted kappa score was calculated pairwise to evaluate the interobserver variability. Statistical analyses were performed with SPSS version 18 (SPSS Inc., Chicago, IL).

RESULTS

One hundred videos were analysed for each of the three keywords (gallbladder disease, gallstone disease, and gallstone treatment), and 135 duplicates were excluded. Of the remaining 165 videos, 34 were excluded based on the aforementioned exclusion criteria. A total of 131 videos were analysed. The mean length of the videos was 257 s, and each video was viewed an average of 14620 times.

Video demographics according to usefulness are shown in Table 2. More than half of the videos were misleading (74, 56.5%), while 36 of the videos (27.5%) were slightly useful, 15 (11.5%) were useful, and only three (2.3%) were deemed very useful. Three videos (2.3%) were categorised as not useful. The number of mean likes in the very useful group was 1.3 ± 1.5, which was significantly lower than the misleading group (17.2 ± 38.0, P = 0.007). The number of mean views in the very useful group was 756.3 ± 701.0, which was also significantly lower than in the misleading group (8910.7 ± 17094.7, P = 0.001). There were no other significant differences between groups regarding either the number of likes or the number of views. All three very useful videos were uploaded by a PH source. Among the 74 misleading videos, 64 (86.5%) were uploaded by CW, eight (10.8%) were uploaded by CI, and two (2.7%) were uploaded by PH. A Spearman’s rank correlation analysis showed no correlation between the usefulness category and number of views (r = 0.065, P = 0.464), number of likes (r = -0.038, P = 0.663), or video length (r = -0.151, P = 0.086).

Table 2 Video demographics according to usefulness category.

Video demographics	Usefulness of information					Total	P value
	Very useful	Useful	Slightly useful	Not useful	Misleading
Videos, n (%)	3 (2.3)	15 (11.5)	36 (27.5)	3 (2.3)	74 (56.5)	131	-
Total length (h:min:s)	0:11:07	1:00:18	1:52:26	0:09:10	6:06:14	9:20:15	-
Mean length (h:min:s)	00:03:42 ± 00:01:17	00:04:05 ± 00:03:33	00:03:07 ± 00:03:20	00:03:03 ± 00:01:50	00:04:57 ± 00:05:10	00:04:17 ± 00:04:29	0.364
Mean ''likes'' (n)	1.3 ± 1.5	42.8 ± 145.8	10.6 ± 22.4	10.7 ± 18.5	17.2 ± 38.0	17.8 ± 57.7	0.0071
Mean ''dislikes'' (n)	0.3 ± 0.6	3.3 ± 11.6	0.9 ± 1.7	1.3 ± 2.3	2.9 ± 6.8	2.3 ± 6.5	0.158
Total views (n)	2269	1030071	205081	18476	659392	1915289	-
Mean views (n)	756.3 ± 701.0	68671.4 ± 248999.6	5696.7 ± 11330.7	6158.7 ± 10231.0	8910.7 ± 17094.7	14620.5 ± 85212.0	0.0011
Days since upload	518.7 ± 430.9	737.3 ± 480.4	706.2 ± 487.7	426.7 ± 331.9	518.3 ± 368.3	592.9 ± 423.7	0.126
Upload source, n (%)
Physician	3 (100)	3 (20.0)	12 (33.3)	1 (33.3)	2 (2.7)	21 (16.0)	-
Website	0	12 (80.0)	18 (50.0)	1 (33.3)	0	31 (23.7)	-
Commercial	0	0	0	0	64 (86.5)	64 (48.9)	-
Civilian	0	0	6 (16.7)	1 (33.3)	8 (10.8)	15 (11.5)	-

¹Comparison of very useful and misleading.

Video demographics according to the upload source are shown in Table 3. The highest number of videos was uploaded by a CW (64, 48.9%). Thirty-one videos (23.7%) were uploaded by MW, 21 videos (16.0%) were uploaded by PH, and 15 (11.5%) were uploaded by CI. The mean length of the CI videos was significantly longer (7.58 ± 5.09) compared with the PH videos (2.50 ± 1.31, P = 0.01) and the MW videos (2.43 ± 2.43, P = 0.009). There were no significant differences in the mean length between the other groups. There were no differences in the mean number of likes received or the mean number of views.

Table 3 Video demographics according to upload source.

Video demographics	Upload source				Total	P value
	Physician or hospital	Medical website or TV channel	Commercial website	Civilian
Videos, n (%)	21 (16.0)	31 (23.7)	64 (48.9)	15 (11.5)	131	-
Total length (h:min:s)	0:59:29	1:24:08	4:57:10	1:59:28	9:20:15	-
Mean length (h:min:s)	00:02:50 ± 00:01:31	00:02:43 ± 00:02:43	00:04:39 ± 00:05:06	00:07:58 ± 00:05:09	00:04:17 ± 00:04:29	0.0091, 0.0102
Mean ‘likes’ (n)	10.9 ± 25.4	25.4 ± 101.9	17.6 ±40.3	12.7 ±15.1	17.8 ± 57.7	0.254
Mean ‘dislikes’ (n)	0.5 ± 1.8	2.1 ± 8.1	3.0 ± 7.2	2.3 ± 2.7	2.3 ± 6.5	0.083
Total views (n)	74549	1162278	581839	96623	1915289	-
Mean views (n)	3550.0 ± 7036.5	37492.8 ± 173179.9	9091.2 ± 18010.5	6441.5 ± 8558.2	14620.5 ± 85212.0	0.168
Usefulness information n (%)
Very useful	3 (14.3)	0	0	0	3 (2.3)	-
Useful	3 (14.3)	12 (38.7)	0	0	15 (11.5)	-
Slightly useful	12 (57.1)	18 (58.1)	0	6 (40.0)	36 (27.5)	-
Not useful	1 (4.8)	1 (3.2)	0	1 (6.7)	3 (2.3)	-
Misleading	2 (9.5)	0	64 (100)	8 (53.3)	74 (56.5)	-

¹Civilian vs website;

²Civilian vs physician.

Table 4 shows various treatment methods advocated by misleading videos. The highest number of these videos advocated the ‘‘gallstone flush’’ (39, 25.7%). Twenty-four videos (32.4%) advocated medication that can dissolve gallstones, and six videos (8.1%) advocated herbal treatment.

Table 4 Methods advocated by misleading videos.

Misleading videos	n (%)
Gallstone flush	39 (52.7)
Medication	24 (32.4)
Herbal treatment	6 (8.1)
Other	5 (6.8)
Total	74 (100)

The interobserver variability was calculated as a weighted kappa score of 0.94 between Lee JL and Seo HS, 0.84 between Seo HS and Kim KM, and 0.80 between Lee JL and Kim KM.

DISCUSSION

This study evaluated the content quality of YouTube videos regarding gallstone disease. Out of 131 videos, 74 (56.5%) videos were misleading. This percentage is disturbingly high compared with that of previous studies evaluating video content in other fields[1,4,6]. Recent advances in imaging technologies have led to an increase in the diagnosis of asymptomatic gallstones. In contrast to appendicitis patients or hernia patients, patients with asymptomatic gallstones have more time to seek medical information about their condition. In turn, many people appear to be targeting these patients with the aim of profiting.

Notably, videos uploaded by commercial websites were even less credible than videos uploaded by civilians. It appears that commercial entities did not perform any research before uploading these videos. The high percentage of misleading videos reflects the abundance of commercial products advertising treatment of gallstones without surgery. The ‘‘gallstone flush’’ is by far the most popular method[12]. Many commercial websites advertise books that carry specific guidelines for the gallstone flush. In this method, a patient with gallstones drinks olive oil and lemon juice following a specific protocol. Three to five days later, the patient passes several ‘‘gallstones’’. These ‘‘stones’’ have been found to be simply the product of the mixture of oil and lemon juice[13]. There are also various medications that allegedly detoxify the gallbladder and remove the gallstones[14,15].

Not only are these videos misleading, they are potentially fatal. Even more concerning than these commercial websites are the two videos uploaded by physicians with misleading content. In one video, a medical doctor states that the treatment of choice for symptomatic gallstones is a low fat diet[16]. In the other video, a medical doctor claims that drinking herbal tea can dissolve gallstones[17]. These two videos may do even more harm than the commercial videos.

With these facts in mind, the most important issue is that the general population tends to view the misleading videos more than the credible videos. Biggs et al[1] suggest that this is because useful videos tended to be longer than misleading videos. In the present study, all of the videos were approximately the same length, yet the videos that were deemed very useful had significantly fewer views and likes than the misleading videos. In a study performed by Butler et al[4], the usefulness score and the number of views were only weakly correlated. The present study found no correlation between the usefulness and the number of views or number of likes.

Interestingly, while the weighted kappa score was 0.94 between the two researchers who received training at the same centre, the score was 0.84 and 0.80 between these two researchers and another researcher who had received training at a different centre. Although the range of kappa scores demonstrates significant interobserver agreement, it also demonstrates that the training of the researcher influenced their assessment of the videos.

This study has several limitations. First, a subjective score criteria was used to evaluate the videos, as there are as of yet no validated tools for assessing video data. Second, the study results may change according to the keywords used in the search. This study used ‘‘gallbladder disease’’, ‘‘gallstone disease’’, and ‘‘gallstone treatment’’ under the assumption that these are the keywords a layperson would choose rather than ‘‘cholecystitis’’ or ‘‘cholelithiasis’’. This may not always be the case. Third, the evaluated videos were sorted by relevance, which is the YouTube default. This relevance may have been affected by advertisements, and the results may be different when sorted with another standard. Lastly, these results demonstrate the quality of information at one point in time, and results may change with time as videos are added or removed.

In summary, more than half of the YouTube videos regarding gallstone disease are misleading and present a risk of harmful consequences. Credible videos with accurate information need to be uploaded by medical professionals and medical institutions. Active filtering by the managing staff of YouTube may also be necessary.

COMMENTS

Background

Due to the development and spread of the Internet, many people use the Internet as a source of medical information. One of the most popular of these sources is YouTube.

Research frontiers

As more medical professionals recognise the importance of YouTube as a source of medical information for the general public, studies have increasingly evaluated the quality of YouTube videos on a wide range of topics such as prostate cancer, burns, and tonsillectomy.

Innovations and breakthroughs

This was the first study to evaluate YouTube videos on gallstone disease. The results showed that more than half of the videos were misleading and that the quality of the videos did not correlate with the number of likes or number of views.

Applications

Gallstone disease is very common, and the public should be informed that widely advertised methods such as the “gallstone flush” are ineffective and may delay appropriate treatment. A method for filtering misleading information is necessary.

Terminology

When viewing a video on YouTube (http://www.youtube.com), the viewer can click on an icon showing a thumbs-up gesture. This is known as a ‘‘like’’ and generally shows that the viewer is satisfied with the content of the video. The number of times viewers have clicked ‘‘like’’, as well as the total number of views for each video is shown.

Peer review

The current study is a review of YouTube videos on the topic of gallbladder disease. The findings show that a significant portion of these videos are misleading to viewers and that low quality videos are more frequently distributed by commercial entities. The study provides an interesting topic with clinical significance due to an increasing number of online resources utilised by patients.