1
|
Kula B, Kula A, Bagcier F, Alyanak B. Artificial intelligence solutions for temporomandibular joint disorders: Contributions and future potential of ChatGPT. Korean J Orthod 2025; 55:131-141. [PMID: 40104855 PMCID: PMC11922634 DOI: 10.4041/kjod24.106] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2024] [Revised: 10/25/2024] [Accepted: 12/09/2024] [Indexed: 03/20/2025] Open
Abstract
Objective This study aimed to evaluate the reliability and usefulness of information generated by Chat Generative Pre-Trained Transformer (ChatGPT) on temporomandibular joint disorders (TMD). Methods We asked ChatGPT about the diseases specified in the TMD classification and scored the responses using Likert reliability and usefulness scales, the modified DISCERN (mDISCERN) scale, and the Global Quality Scale (GQS). Results The highest Likert scores for both reliability and usefulness were for masticatory muscle disorders (mean ± standard deviation [SD]: 6.0 ± 0), and the lowest scores were for inflammatory disorders of the temporomandibular joint (mean ± SD: 4.3 ± 0.6 for reliability, 4.0 ± 0 for usefulness). The median Likert reliability score indicates that the responses are highly reliable. The median Likert usefulness score was 5 (4-6), indicating that the responses were moderately useful. A comparative analysis was performed, and no statistically significant differences were found in any subject for either reliability or usefulness (P = 0.083-1.000). The median mDISCERN score was 4 (3-5) for the two raters. A statistically significant difference was observed in the mean mDISCERN scores between the two raters (P = 0.046). The GQS scores indicated a moderate to high quality (mean ± SD: 3.8 ± 0.8 for rater 1, 4.0 ± 0.5 for rater 2). No statistically significant correlation was found between mDISCERN and GQS scores (r = -0.006, P = 0.980). Conclusions Although ChatGPT-4 has significant potential, it can be used as an additional source of information regarding TMD for patients and clinicians.
Collapse
Affiliation(s)
- Betul Kula
- Department of Orthodontics, Istanbul Galata University, Istanbul, Türkiye
| | - Ahmet Kula
- Department of Prosthodontics, Uskudar University, Istanbul, Türkiye
| | - Fatih Bagcier
- Physical Medicine and Rehabilitation Clinic, Basaksehir Cam and Sakura City Hospital, Istanbul, Türkiye
| | - Bulent Alyanak
- Department of Physical Medicine and Rehabilitation, Golcuk Necati Celik State Hospital, Kocaeli, Türkiye
| |
Collapse
|
2
|
Dando CJ, Adam CE. Collecting initial accounts using ChatCharlie chatbot improves eyewitness memory in later investigative interviews. Sci Rep 2025; 15:9456. [PMID: 40108261 PMCID: PMC11923045 DOI: 10.1038/s41598-025-93281-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2024] [Accepted: 03/04/2025] [Indexed: 03/22/2025] Open
Abstract
Initial account interviews (IAi) offer eyewitnesses more immediate opportunities to answer a series of brief questions about their experiences prior to an in-depth, more formal investigative interview. An IAi is typically elicited in-person near/at the scene of a crime using broadly systematic questioning. Retrieval practice can improve subsequent recall in some contexts, but there is a dearth of research centred on the potential costs and benefits of a quick IAi. Furthermore, where an in-person IAi is impossible, no alternative quick provision exists. Given the systematic nature of the IAi protocol, we developed a conversational chatbot as a potential alternative. Using a mock-witness paradigm, we investigated the memory performance of adults from the general population during in-depth in-person interviews one week after having provided an IAi 10 min post event either (1) in person, (2) via the ChatCharlie chatbot, or (3) no IAi (control). IAi conditions leveraged significantly improved event recall during later investigative interviews versus the Control. Accounts were more accurate and complete, and more correct information was remembered without increased errors indicating the potential of digital agents for IAi purposes Findings concur with predictions from theoretical understanding of episodic memory consolidation and the empirical eyewitness literature regarding the benefits of practice in some contexts.
Collapse
Affiliation(s)
- Coral J Dando
- Department of Psychology, University of Westminster, 115 New Cavendish Street, London, W1W 6UW, UK.
| | | |
Collapse
|
3
|
Bhavsar D, Duffy L, Jo H, Lokker C, Haynes RB, Iorio A, Marusic A, Ng JY. Policies on artificial intelligence chatbots among academic publishers: a cross-sectional audit. Res Integr Peer Rev 2025; 10:1. [PMID: 40022253 PMCID: PMC11869395 DOI: 10.1186/s41073-025-00158-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2024] [Accepted: 02/10/2025] [Indexed: 03/03/2025] Open
Abstract
BACKGROUND Artificial intelligence (AI) chatbots are novel computer programs that can generate text or content in a natural language format. Academic publishers are adapting to the transformative role of AI chatbots in producing or facilitating scientific research. This study aimed to examine the policies established by scientific, technical, and medical academic publishers for defining and regulating the authors' responsible use of AI chatbots. METHODS This study performed a cross-sectional audit on the publicly available policies of 162 academic publishers, indexed as members of the International Association of the Scientific, Technical, and Medical Publishers (STM). Data extraction of publicly available policies on the webpages of all STM academic publishers was performed independently, in duplicate, with content analysis reviewed by a third contributor (September 2023-December 2023). Data was categorized into policy elements, such as 'proofreading' and 'image generation'. Counts and percentages of 'yes' (i.e., permitted), 'no', and 'no available information' (NAI) were established for each policy element. RESULTS A total of 56/162 (34.6%) STM academic publishers had a publicly available policy guiding the authors' use of AI chatbots. No policy allowed authorship for AI chatbots (or other AI tool). Most (49/56 or 87.5%) required specific disclosure of AI chatbot use. Four policies/publishers placed a complete ban on the use of AI chatbots by authors. CONCLUSIONS Only a third of STM academic publishers had publicly available policies as of December 2023. A re-examination of all STM members in 12-18 months may uncover evolving approaches toward AI chatbot use with more academic publishers having a policy.
Collapse
Affiliation(s)
- Daivat Bhavsar
- Department of Health Research Methods, Evidence, and Impact, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
| | - Laura Duffy
- Department of Health Research Methods, Evidence, and Impact, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
| | - Hamin Jo
- Department of Health Research Methods, Evidence, and Impact, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
| | - Cynthia Lokker
- Department of Health Research Methods, Evidence, and Impact, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
| | - R Brian Haynes
- Department of Health Research Methods, Evidence, and Impact, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
- Department of Medicine, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
| | - Alfonso Iorio
- Department of Health Research Methods, Evidence, and Impact, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
- Department of Medicine, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
| | - Ana Marusic
- Department of Research in Biomedicine and Health and Center for Evidence-Based Medicine, School of Medicine, University of Split, Split, Croatia
| | - Jeremy Y Ng
- Department of Health Research Methods, Evidence, and Impact, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada.
- Centre for Journalology, Ottawa Methods Centre, Ottawa Hospital Research Institute, The Ottawa Hospital, Ottawa, Ontario, Canada.
| |
Collapse
|
4
|
Wei Q, Pan S, Liu X, Hong M, Nong C, Zhang W. The integration of AI in nursing: addressing current applications, challenges, and future directions. Front Med (Lausanne) 2025; 12:1545420. [PMID: 40007584 PMCID: PMC11850350 DOI: 10.3389/fmed.2025.1545420] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2024] [Accepted: 01/13/2025] [Indexed: 02/27/2025] Open
Abstract
Artificial intelligence is increasingly influencing healthcare, providing transformative opportunities and challenges for nursing practice. This review critically evaluates the integration of AI in nursing, focusing on its current applications, limitations, and areas that require further investigation. A comprehensive analysis of recent studies highlights the use of AI in clinical decision support systems, patient monitoring, and nursing education. However, several barriers to successful implementation are identified, including technical constraints, ethical dilemmas, and the need for workforce adaptation. Significant gaps in the literature are also evident, such as the limited development of nursing-specific AI tools, insufficient long-term impact assessments, and the absence of comprehensive ethical frameworks tailored to nursing contexts. The potential of AI to reshape personalized care, advance robotics in nursing, and address global health challenges is explored in depth. This review integrates existing knowledge and identifies critical areas for future research, emphasizing the necessity of aligning AI advancements with the specific needs of nursing. Addressing these gaps is essential to fully harness AI's potential while reducing associated risks, ultimately enhancing nursing practice and improving patient outcomes.
Collapse
Affiliation(s)
- Qiuying Wei
- Anesthesia Surgery Center, The First Affiliated Hospital of Guangxi Medical University, Naning, Guangxi, China
| | - Songcheng Pan
- Anesthesia Surgery Center, The First Affiliated Hospital of Guangxi Medical University, Naning, Guangxi, China
- Guangdong Lingnan Nightingale Nursing Academy, Guangzhou, Guangdong, China
| | - Xiaoyu Liu
- Anesthesia Surgery Center, The First Affiliated Hospital of Guangxi Medical University, Naning, Guangxi, China
| | - Mei Hong
- Anesthesia Surgery Center, The First Affiliated Hospital of Guangxi Medical University, Naning, Guangxi, China
| | - Chunying Nong
- Anesthesia Surgery Center, The First Affiliated Hospital of Guangxi Medical University, Naning, Guangxi, China
| | - Weiqi Zhang
- Anesthesia Surgery Center, The First Affiliated Hospital of Guangxi Medical University, Naning, Guangxi, China
| |
Collapse
|
5
|
Wu X, Lam CS, Hui KH, Loong HHF, Zhou KR, Ngan CK, Cheung YT. Perceptions in 3.6 Million Web-Based Posts of Online Communities on the Use of Cancer Immunotherapy: Data Mining Using BERTopic. J Med Internet Res 2025; 27:e60948. [PMID: 39928933 PMCID: PMC11851037 DOI: 10.2196/60948] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2024] [Revised: 10/17/2024] [Accepted: 12/30/2024] [Indexed: 02/12/2025] Open
Abstract
BACKGROUND Immunotherapy has become a game changer in cancer treatment. The internet has been used by patients as a platform to share personal experiences and seek medical guidance. Despite the increased utilization of immunotherapy in clinical practice, few studies have investigated the perceptions about its use by analyzing social media data. OBJECTIVE This study aims to use BERTopic (a topic modeling technique that is an extension of the Bidirectional Encoder Representation from Transformers machine learning model) to explore the perceptions of online cancer communities regarding immunotherapy. METHODS A total of 4.9 million posts were extracted from Facebook, Twitter, Reddit, and 16 online cancer-related forums. The textual data were preprocessed by natural language processing. BERTopic modeling was performed to identify topics from the posts. The effectiveness of isolating topics from the posts was evaluated using 3 metrics: topic diversity, coherence, and quality. Sentiment analysis was performed to determine the polarity of each topic and categorize them as positive or negative. Based on the topics generated through topic modeling, thematic analysis was conducted to identify themes associated with immunotherapy. RESULTS After data cleaning, 3.6 million posts remained for modeling. The highest overall topic quality achieved by BERTopic was 70.47% (topic diversity: 87.86%; topic coherence: 80.21%). BERTopic generated 14 topics related to the perceptions of immunotherapy. The sentiment score of around 0.3 across the 14 topics suggested generally positive sentiments toward immunotherapy within the online communities. Six themes were identified, primarily covering (1) hopeful prospects offered by immunotherapy, (2) perceived effectiveness of immunotherapy, (3) complementary therapies or self-treatments, (4) financial and mental impact of undergoing immunotherapy, (5) impact on lifestyle and time schedules, and (6) side effects due to treatment. CONCLUSIONS This study provides an overview of the multifaceted considerations essential for the application of immunotherapy as a therapeutic intervention. The topics and themes identified can serve as supporting information to facilitate physician-patient communication and the decision-making process. Furthermore, this study also demonstrates the effectiveness of BERTopic in analyzing large amounts of data to identify perceptions underlying social media and online communities.
Collapse
Affiliation(s)
- Xingyue Wu
- School of Pharmacy, Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong SAR, China
| | - Chun Sing Lam
- School of Pharmacy, Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong SAR, China
| | - Ka Ho Hui
- School of Pharmacy, Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong SAR, China
| | - Herbert Ho-Fung Loong
- Department of Clinical Oncology, Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong SAR, China
| | - Keary Rui Zhou
- School of Pharmacy, Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong SAR, China
| | - Chun-Kit Ngan
- Data Science Program, Worcester Polytechnic Institute, Worcester, MA, United States
| | - Yin Ting Cheung
- School of Pharmacy, Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong SAR, China
| |
Collapse
|
6
|
Su B, Jones R, Chen K, Kostenko E, Schmid M, DeMaria AL, Villa A, Swarup M, Weida J, Tuuli MG. Chatbot for patient education for prenatal aneuploidy testing: A multicenter randomized controlled trial. PATIENT EDUCATION AND COUNSELING 2025; 131:108557. [PMID: 39642634 DOI: 10.1016/j.pec.2024.108557] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Revised: 11/03/2024] [Accepted: 11/14/2024] [Indexed: 12/09/2024]
Abstract
INTRODUCTION Digital tools could assist obstetric providers by delivering information given increasing options for fetal aneuploidy screening. PURPOSE To determine the impact of a chatbot for pre-test education and counseling in low-risk pregnancies. METHODS Two sites participated in this randomized controlled trial. Patients in the intervention group used a chatbot prior to the provider visit, while patients in the control group only received education by the provider. The primary outcome was change in patient knowledge scores after provider education. Analysis was by intention to treat. RESULTS Overall, 258 women participated (n = 130; intervention and n = 128; control). Knowledge gain was significantly higher among patients using the chatbot (mean increase in correct answers [out of 20]: +4.1 vs +1.9, p < 0.001). Both groups reported high satisfaction, with no statistically significant difference between intervention and control groups (mean patient satisfaction [1-10]: 8.2 vs 8.5 respectively, p = 0.35). Providers also reported high satisfaction scores with no significant difference between intervention and control groups (mean provider satisfaction [1 - 10]: 8.7 vs 8.4 respectively, p = 0.13). CONCLUSIONS Pre-test education via a chatbot can increase patient knowledge of prenatal testing choices, with high patient and provider satisfaction.
Collapse
Affiliation(s)
- Bowdoin Su
- Ariosa Diagnostics, Inc., Roche Diagnostics Solutions, San Jose, CA, USA.
| | - Renee Jones
- Ariosa Diagnostics, Inc., Roche Diagnostics Solutions, San Jose, CA, USA
| | - Kelly Chen
- Ariosa Diagnostics, Inc., Roche Diagnostics Solutions, San Jose, CA, USA
| | - Emilia Kostenko
- Ariosa Diagnostics, Inc., Roche Diagnostics Solutions, San Jose, CA, USA
| | - Maximilian Schmid
- Ariosa Diagnostics, Inc., Roche Diagnostics Solutions, San Jose, CA, USA
| | - Andrea L DeMaria
- Department of Public Health, Purdue University, West Lafayette, IN, USA
| | - Andrew Villa
- New Horizons Women's Care Branch of Arizona Ob/Gyn Affiliates, Chandler, AZ, USA
| | - Monte Swarup
- New Horizons Women's Care Branch of Arizona Ob/Gyn Affiliates, Chandler, AZ, USA
| | - Jennifer Weida
- Department of Obstetrics & Gynecology, Indiana University School of Medicine, Indianapolis, IN, USA
| | - Methodius G Tuuli
- Department of Obstetrics & Gynecology, Brown University School of Medicine, Providence, RI, USA
| |
Collapse
|
7
|
Silwal PR, Pirouzi M, Murphy R, Harwood M, Grey C, Squirrell D, Ramke J. Barriers and enablers of access to diabetes eye care in Auckland, New Zealand: a qualitative study. BMJ Open 2025; 15:e087650. [PMID: 39890153 PMCID: PMC11784328 DOI: 10.1136/bmjopen-2024-087650] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/15/2024] [Accepted: 12/06/2024] [Indexed: 02/03/2025] Open
Abstract
OBJECTIVE To explore the barriers and enablers to accessing diabetes eye care services among adults in Auckland. DESIGN This was a qualitative study that used semistructured interviews. We performed a thematic analysis and described the main barriers and enablers to accessing services using the Theoretical Domains Framework. SETTING The study took place in two of the three public funding and planning agencies that provide primary and secondary health services in Auckland, the largest city in Aotearoa New Zealand. PARTICIPANTS Thirty people with diabetes in Auckland who had experienced interrupted diabetes eye care, having missed at least one appointment or being discharged back to their general practitioner after missing several appointments. RESULTS We identified barriers and enablers experienced by our predominantly Pacific and Māori participants that aligned with 7 (of the 14) domains in the Theoretical Domains Framework. The most reported barriers were transport issues, lack of awareness regarding the importance of retinal screening, time constraints, limited and/or inflexible appointment times and competing family commitments. Enablers included positive interactions with healthcare providers and timely appointment notifications and reminders. CONCLUSIONS Diabetes eye services could be made more responsive by addressing systemic barriers such as service location and transport links, appointment availability and meaningful information to aid understanding.
Collapse
Affiliation(s)
- Pushkar Raj Silwal
- School of Optometry and Vision Science, The University of Auckland Faculty of Medical and Health Sciences, Auckland, Auckland, New Zealand
| | - Maryam Pirouzi
- School of Optometry and Vision Science, The University of Auckland Faculty of Medical and Health Sciences, Auckland, Auckland, New Zealand
- Department of General Practice and Primary Care, The University of Auckland Faculty of Medical and Health Sciences, Auckland, Auckland, New Zealand
| | - Rinki Murphy
- Department of Medicine, The University of Auckland Faculty of Medical and Health Sciences, Auckland, Auckland, New Zealand
- Auckland Diabetes Centre, Greenlane Clinical Centre, Auckland, New Zealand
- Specialist Weight Management Service, Te Mana Ki Tua, Counties Manukau Health, Auckland, New Zealand
| | - Matire Harwood
- Department of General Practice and Primary Care, The University of Auckland Faculty of Medical and Health Sciences, Auckland, Auckland, New Zealand
| | - Corina Grey
- Department of General Practice and Primary Care, The University of Auckland Faculty of Medical and Health Sciences, Auckland, Auckland, New Zealand
| | - David Squirrell
- Department of Ophthalmology, Greenlane Clinical Centre, Auckland, New Zealand
| | - Jacqueline Ramke
- School of Optometry and Vision Science, The University of Auckland Faculty of Medical and Health Sciences, Auckland, Auckland, New Zealand
- International Centre for Eye Health, London School of Hygiene & Tropical Medicine, London, UK
| |
Collapse
|
8
|
Mohamed Jasim K, Malathi A, Bhardwaj S, Aw ECX. A systematic review of AI-based chatbot usages in healthcare services. J Health Organ Manag 2025. [PMID: 39865955 DOI: 10.1108/jhom-12-2023-0376] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/28/2025]
Abstract
PURPOSE This systematic literature review aims to provide a comprehensive and structured synthesis of the existing knowledge about chatbots in healthcare from both a theoretical and methodological perspective. DESIGN/METHODOLOGY/APPROACH To this end, a systematic literature review was conducted with 89 articles selected through a SPAR-4-SLR systematic procedure. The document for this systematic review was collected from Scopus database. The VoSviewer software facilitates the analysis of keyword co-occurrence to form the fundamental structure of the subject field. FINDINGS In addition, this study proposes a future research agenda revolving around three main themes such as (1) telemedicine, (2) mental health and (3) medical information. ORIGINALITY/VALUE This study underscores the significance, implications and predictors of chatbot usage in healthcare services. It is concluded that adopting the proposed future direction and further research on chatbots in healthcare will help to refine chatbot systems to better meet the needs of patients.
Collapse
Affiliation(s)
- K Mohamed Jasim
- VIT Business School, Vellore Institute of Technology, Vellore, India
| | - A Malathi
- VIT Business School, Vellore Institute of Technology, Vellore, India
| | - Seema Bhardwaj
- Symbiosis Institute of Business Management, Nagpur, Symbiosis International (Deemed University), Pune, India
- Middlesex University, Dubai, United Arab Emirates
| | - Eugene Cheng-Xi Aw
- UCSI University Kuala Lumpur Campus, Kuala Lumpur, Malaysia
- Faculty of International Tourism and Management, City University of Macau, Macau, China
| |
Collapse
|
9
|
Liu S, Ma J, Sun M, Zhang C, Gao Y, Xu J. Mapping the Landscape of Digital Health Intervention Strategies: 25-Year Synthesis. J Med Internet Res 2025; 27:e59027. [PMID: 39804697 PMCID: PMC11773286 DOI: 10.2196/59027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2024] [Revised: 07/03/2024] [Accepted: 11/30/2024] [Indexed: 01/30/2025] Open
Abstract
BACKGROUND Digital health interventions have emerged as promising tools to promote health behavior change and improve health outcomes. However, a comprehensive synthesis of strategies contributing to these interventions is lacking. OBJECTIVE This study aims to (1) identify and categorize the strategies used in digital health interventions over the past 25 years; (2) explore the differences and changes in these strategies across time periods, countries, populations, delivery methods, and senders; and (3) serve as a valuable reference for future researchers and practitioners to improve the effectiveness of digital health interventions. METHODS This study followed a systematic review approach, complemented by close reading and text coding. A comprehensive search for published English academic papers from PubMed, Web of Science, and Scopus was conducted. The search employed a combination of digital health and intervention-related terms, along with database-specific subject headings and filters. The time span covered 25 years, from January 1, 1999, to March 10, 2024. Sample papers were selected based on study design, intervention details, and strategies. The strategies were identified and categorized based on the principles of Behavior Change Techniques and Behavior Strategies. RESULTS A total of 885 papers involving 954,847 participants met the eligibility criteria. We identified 173 unique strategies used in digital health interventions, categorized into 19 themes. The 3 most frequently used strategies in the sample papers were "guide" (n=492, 55.6%), "monitor" (n=490, 55.4%), and "communication" (n=392, 44.3%). The number of strategies employed in each paper ranged from 1 to 32. Most interventions targeted clients (n=844, 95.4%) and were carried out in hospitals (n=268, 30.3%). High-income countries demonstrated a substantially higher number and diversity of identified strategies than low- and middle-income countries, and the number of studies targeting the public (n=647, 73.1%) far exceeded those focusing on vulnerable groups (n=238, 26.9%). CONCLUSIONS Digital health interventions and strategies have undergone considerable development over the past 25 years. They have evolved from simple approaches to sophisticated, personalized techniques and are trending toward multifaceted interventions, leveraging advanced technologies for real-time monitoring and feedback. Future studies should focus on rigorous evaluations, long-term effectiveness, and tailored approaches for diverse populations, and more attention should be given to vulnerable groups.
Collapse
Affiliation(s)
- Shiyu Liu
- School of Public Health, Xi'an Jiaotong University, Xi'an, China
| | - Jingru Ma
- School of Public Health, Xi'an Jiaotong University, Xi'an, China
| | - Meichen Sun
- School of Public Health, Xi'an Jiaotong University, Xi'an, China
| | - Chao Zhang
- School of Pharmacy, Shanghai University of Traditional Chinese Medicine, Shanghai, China
| | - Yujing Gao
- School of Journalism and Cultural Communication, Zhongnan University of Economics and Law, Wuhan, China
| | - Jinghong Xu
- School of Journalism and Communication, Beijing Normal University, Beijing, China
- The International College, Krirk University, Bangkok, Thailand
| |
Collapse
|
10
|
Ng JY, Maduranayagam SG, Suthakar N, Li A, Lokker C, Iorio A, Haynes RB, Moher D. Attitudes and perceptions of medical researchers towards the use of artificial intelligence chatbots in the scientific process: an international cross-sectional survey. Lancet Digit Health 2025; 7:e94-e102. [PMID: 39550312 DOI: 10.1016/s2589-7500(24)00202-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2024] [Revised: 07/16/2024] [Accepted: 09/12/2024] [Indexed: 11/18/2024]
Abstract
Chatbots are artificial intelligence (AI) programs designed to simulate conversations with humans that present opportunities and challenges in scientific research. Despite growing clarity from publishing organisations on the use of AI chatbots, researchers' perceptions remain less understood. In this international cross-sectional survey, we aimed to assess researchers' attitudes, familiarity, perceived benefits, and limitations related to AI chatbots. Our online survey was open from July 9 to Aug 11, 2023, with 61 560 corresponding authors identified from 122 323 articles indexed in PubMed. 2452 (4·0%) provided responses and 2165 (94·5%) of 2292 who met eligibility criteria completed the survey. 1161 (54·0%) of 2149 respondents were male and 959 (44·6%) were female. 1294 (60·5%) of 2138 respondents were familiar with AI chatbots, and 945 (44·5%) of 2125 had previously used AI chatbots in research. Only 244 (11·4%) of 2137 reported institutional training on AI tools, and 211 (9·9%) of 2131 noted institutional policies on AI chatbot use. Despite mixed opinions on the benefits, 1428 (69·7%) of 2048 expressed interest in further training. Although many valued AI chatbots for reducing administrative workload (1299 [66·9%] of 1941), there was insufficient understanding of the decision making process (1484 [77·2%] of 1923). Overall, this study highlights substantial interest in AI chatbots among researchers, but also points to the need for more formal training and clarity on their use.
Collapse
Affiliation(s)
- Jeremy Y Ng
- Centre for Journalology, Methods Centre, Ottawa Hospital Research Institute, Ottawa, ON, Canada.
| | - Sharleen G Maduranayagam
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
| | - Nirekah Suthakar
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
| | - Amy Li
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
| | - Cynthia Lokker
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
| | - Alfonso Iorio
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada; Department of Medicine, McMaster University, Hamilton, ON, Canada
| | - R Brian Haynes
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
| | - David Moher
- Centre for Journalology, Methods Centre, Ottawa Hospital Research Institute, Ottawa, ON, Canada; School of Epidemiology, Public Health, and Preventive Medicine, University of Ottawa, Ottawa, ON, Canada
| |
Collapse
|
11
|
Lee J, Byun HK, Kim YT, Shin J, Kim YB. A Study on Breast Cancer Patient Care Using Chatbot and Video Education for Radiation Therapy: A Randomized Controlled Trial. Int J Radiat Oncol Biol Phys 2024:S0360-3016(24)03732-5. [PMID: 39732344 DOI: 10.1016/j.ijrobp.2024.12.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2024] [Revised: 11/28/2024] [Accepted: 12/14/2024] [Indexed: 12/30/2024]
Abstract
PURPOSE This study aimed to evaluate the use of chatbot and video education to reduce anxiety in patients with breast cancer undergoing radiation therapy (RT). METHODS AND MATERIALS This randomized controlled trial included patients with breast cancer scheduled for RT after surgery at an outpatient department of radiation oncology in a cancer center, randomly assigned to 4 groups: (1) video + chatbot, (2) video + paper, (3) paper + chatbot, and (4) paper + paper. In each group, patients received information regarding the treatment process and were frequently asked questions using the designated tool. Patient anxiety was evaluated using the Amsterdam preoperative anxiety and information scale (APAIS), state-trait anxiety inventory (STAI), and linear analog scale assessment (LASA) at 3 points: (1) initial outpatient visit (T0), (2) before the RT course (T1), and (3) after the RT course (T2). The primary endpoint was APAIS, and the secondaries were STAI and LASA. A mixed-model repeated-measures ANOVA was conducted with time as a within-group factor and treatment conditions as a between-group factor. RESULTS The final analysis included 145 patients. No significant interaction was observed between groups and time for the APAIS, STAI, or LASA. Although unplanned, analyses were conducted using the age of 50 years as the cutoff, based on a previous systematic review of digital literacy in the medical field. A trend toward reduced APAIS was found among patients aged ≤50 years who used the chatbot (per-protocol subgroup). In the video + chatbot group, the APAIS score decreased from 3.06 (T0) to 1.88 (T2); in contrast, in the paper + paper group, it decreased from 2.42 (T0) to 2.06 (T2). In contrast, no significant interaction was observed in the APAIS of per-protocol patients aged ≥50 years. CONCLUSIONS Overall, no significant differences were found in the effectiveness of different types of educational media in reducing patients' anxiety. However, for young patients who actively use video or chatbot resources, education through digital media may meaningfully reduce their anxiety during the RT process.
Collapse
Affiliation(s)
- Junbok Lee
- Institute for Innovation in Digital Healthcare, Yonsei University, Seoul, Korea; Department of Human Systems Medicine, Seoul National University College of Medicine, Seoul, Korea
| | - Hwa Kyung Byun
- Department of Radiation Oncology, Yonsei University College of Medicine, Seoul, Korea
| | - Yong Tae Kim
- Department of Radiation Oncology, Yonsei University College of Medicine, Seoul, Korea
| | - Jaeyong Shin
- Department of Preventive Medicine and Public Health, Yonsei University College of Medicine, Seoul, Korea; Institute of Health Services Research, Yonsei University College of Medicine, Seoul, Korea.
| | - Yong Bae Kim
- Department of Radiation Oncology, Yonsei University College of Medicine, Seoul, Korea.
| |
Collapse
|
12
|
Strika Z, Petkovic K, Likic R, Batenburg R. Bridging healthcare gaps: a scoping review on the role of artificial intelligence, deep learning, and large language models in alleviating problems in medical deserts. Postgrad Med J 2024; 101:4-16. [PMID: 39323384 DOI: 10.1093/postmj/qgae122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2024] [Revised: 08/08/2024] [Accepted: 09/04/2024] [Indexed: 09/27/2024]
Abstract
"Medical deserts" are areas with low healthcare service levels, challenging the access, quality, and sustainability of care. This qualitative narrative review examines how artificial intelligence (AI), particularly large language models (LLMs), can address these challenges by integrating with e-Health and the Internet of Medical Things to enhance services in under-resourced areas. It explores AI-driven telehealth platforms that overcome language and cultural barriers, increasing accessibility. The utility of LLMs in providing diagnostic assistance where specialist deficits exist is highlighted, demonstrating AI's role in supplementing medical expertise and improving outcomes. Additionally, the development of AI chatbots offers preliminary medical advice, serving as initial contact points in remote areas. The review also discusses AI's role in enhancing medical education and training, supporting the professional development of healthcare workers in these regions. It assesses AI's strategic use in data analysis for effective resource allocation, identifying healthcare provision gaps. AI, especially LLMs, is seen as a promising solution for bridging healthcare gaps in "medical deserts," improving service accessibility, quality, and distribution. However, continued research and development are essential to fully realize AI's potential in addressing the challenges of medical deserts.
Collapse
Affiliation(s)
- Zdeslav Strika
- University of Zagreb School of Medicine, Salata 3, Zagreb 10000, Croatia
| | - Karlo Petkovic
- University of Zagreb School of Medicine, Salata 3, Zagreb 10000, Croatia
| | - Robert Likic
- University of Zagreb School of Medicine, Salata 3, Zagreb 10000, Croatia
- Department of Internal Medicine, Division of Clinical Pharmacology, Clinical Hospital Centre Zagreb, Kispaticeva 12, Zagreb 10000, Croatia
| | - Ronald Batenburg
- Netherlands Institute for Health Services Research (NIVEL), Otterstraat 118, Utrecht 3553, The Netherlands
- Department of Sociology, Radboud University, Thomas Van Aquinostraat 4, Nijmegen 6524, The Netherlands
| |
Collapse
|
13
|
Preiser C, Radionova N, Ög E, Koch R, Klemmt M, Müller R, Ranisch R, Joos S, Rieger MA. The Doctors, Their Patients, and the Symptom Checker App: Qualitative Interview Study With General Practitioners in Germany. JMIR Hum Factors 2024; 11:e57360. [PMID: 39556813 PMCID: PMC11612597 DOI: 10.2196/57360] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2024] [Revised: 08/01/2024] [Accepted: 08/22/2024] [Indexed: 11/20/2024] Open
Abstract
BACKGROUND Symptom checkers are designed for laypeople and promise to provide a preliminary diagnosis, a sense of urgency, and a suggested course of action. OBJECTIVE We used the international symptom checker app (SCA) Ada App as an example to answer the following question: How do general practitioners (GPs) experience the SCA in relation to the macro, meso, and micro level of their daily work, and how does this interact with work-related psychosocial resources and demands? METHODS We conducted 8 semistructured interviews with GPs in Germany between December 2020 and February 2022. We analyzed the data using the integrative basic method, an interpretative-reconstructive method, to identify core themes and modes of thematization. RESULTS Although most GPs in this study were open to digitization in health care and their practice, only one was familiar with the SCA. GPs considered the SCA as part of the "unorganized stage" of patients' searching about their conditions. Some preferred it to popular search engines. They considered it relevant to their work as soon as the SCA would influence patients' decisions to see a doctor. Some wanted to see the results of the SCA in advance in order to decide on the patient's next steps. GPs described the diagnostic process as guided by shared decision-making, with the GP taking the lead and the patient deciding. They saw diagnosis as an act of making sense of data, which the SCA would not be able to do, despite the huge amounts of data. CONCLUSIONS GPs took a techno-pragmatic view of SCA. They operate in a health care system of increasing scarcity. They saw the SCA as a potential work-related resource if it helped them to reduce administrative tasks and unnecessary patient contacts. The SCA was seen as a potential work-related demand if it increased workload, for example, if it increased patients' anxiety, was too risk-averse, or made patients more insistent on their own opinions.
Collapse
Affiliation(s)
- Christine Preiser
- Institute of Occupational and Social Medicine and Health Services Research, University Hospital Tübingen, Tübingen, Germany
| | - Natalia Radionova
- Institute of Occupational and Social Medicine and Health Services Research, University Hospital Tübingen, Tübingen, Germany
| | - Eylem Ög
- Institute of Occupational and Social Medicine and Health Services Research, University Hospital Tübingen, Tübingen, Germany
| | - Roland Koch
- Institute for General Practice and Interprofessional Care, University Hospital Tübingen, Tübingen, Germany
| | - Malte Klemmt
- Institute for General Practice and Palliative Care, Hannover Medical School, Hannover, Germany
| | - Regina Müller
- Institute of Philosophy, University Bremen, Bremen, Germany
| | - Robert Ranisch
- Faculty of Health Sciences Brandenburg, University of Potsdam, Potsdam, Germany
| | - Stefanie Joos
- Institute for General Practice and Interprofessional Care, University Hospital Tübingen, Tübingen, Germany
| | - Monika A Rieger
- Institute of Occupational and Social Medicine and Health Services Research, University Hospital Tübingen, Tübingen, Germany
| |
Collapse
|
14
|
Johnvictor AC, Poonkodi M, Prem Sankar N, VS T. TinyML-Based Lightweight AI Healthcare Mobile Chatbot Deployment. J Multidiscip Healthc 2024; 17:5091-5104. [PMID: 39539515 PMCID: PMC11559246 DOI: 10.2147/jmdh.s483247] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2024] [Accepted: 10/03/2024] [Indexed: 11/16/2024] Open
Abstract
Introduction In healthcare applications, AI-driven innovations are set to revolutionise patient interactions and care, with the aim of improving patient satisfaction. Recent advancements in Artificial Intelligence have significantly affected nursing, assistive management, medical diagnoses, and other critical medical procedures. Purpose Many artificial intelligence (AI) solutions operate online, posing potential risks to patient data security. To address these security concerns and ensure swift operation, this study has developed a chatbot tailored for hospital environments, running on a local server, and utilising TinyML for processing patient data. Patients and Methods Edge computing technology enables secure on-site data processing. The implementation includes patient identification using a Histogram of Gradient (HOG)-based classification, followed by basic patient care tasks, such as temperature measurement and demographic recording. Results The classification accuracy of patient detection was 95.8%. An autonomous temperature-sensing unit equipped with a medical-grade infrared temperature scanner detected and recorded patient temperature. Following the temperature assessment, the tinyML-powered chatbot engaged patients in a series of questions customised by doctors to train the model for diagnostic scenarios. Patients' responses, recorded as "yes" or "no", are stored and printed in their case sheet. The accuracy of the TinyML model is 95.3% and the on-device processing time is 217 ms. The implemented TinyML model uses only 8.8Kb RAM and 50.3Kb Flash memory, with a latency of only 4 ms. Conclusion Each patient was assigned a unique ID, and their data were securely stored for further consultation and diagnosis via hospital management. This research demonstrates faster patient data recording and increased security compared to existing AI-based healthcare solutions, as all processes occur within the local host.
Collapse
Affiliation(s)
| | - M Poonkodi
- School of Computer Science and Engineering, Vellore Institute of Technology, Chennai, Tamil Nadu, India
| | - N Prem Sankar
- School of Computer Science and Engineering, Vellore Institute of Technology, Chennai, Tamil Nadu, India
| | - Thinesh VS
- Arista Networks Pvt Ltd, Bangalore, India
| |
Collapse
|
15
|
Ilan Y. The Co-Piloting Model for Using Artificial Intelligence Systems in Medicine: Implementing the Constrained-Disorder-Principle-Based Second-Generation System. Bioengineering (Basel) 2024; 11:1111. [PMID: 39593770 PMCID: PMC11592301 DOI: 10.3390/bioengineering11111111] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2024] [Revised: 10/23/2024] [Accepted: 11/01/2024] [Indexed: 11/28/2024] Open
Abstract
The development of artificial intelligence (AI) and machine learning (ML)-based systems in medicine is growing, and these systems are being used for disease diagnosis, drug development, and treatment personalization. Some of these systems are designed to perform activities that demand human cognitive function. However, use of these systems in routine care by patients and caregivers lags behind expectations. This paper reviews several challenges that healthcare systems face and the obstacles of integrating digital systems into routine care. This paper focuses on integrating digital systems with human physicians. It describes second-generation AI systems designed to move closer to biology and reduce complexity, augmenting but not replacing physicians to improve patient outcomes. The constrained disorder principle (CDP) defines complex biological systems by their degree of regulated variability. This paper describes the CDP-based second-generation AI platform, which is the basis for the Digital Pill that is humanizing AI by moving closer to human biology via using the inherent variability of biological systems for improving outcomes. This system augments physicians, assisting them in decision-making to improve patients' responses and adherence but not replacing healthcare providers. It restores the efficacy of chronic drugs and improves adherence while generating data-driven therapeutic regimens. While AI can substitute for many medical activities, it is unlikely to replace human physicians. Human doctors will continue serving patients with capabilities augmented by AI. The described co-piloting model better reflects biological pathways and provides assistance to physicians for better care.
Collapse
Affiliation(s)
- Yaron Ilan
- Department of Medicine, Hadassah Medical Center, Faculty of Medicine, Hebrew University, Jerusalem 9112001, Israel
| |
Collapse
|
16
|
Chen A, Qilleri A, Foster T, Rao AS, Gopalakrishnan S, Niezgoda J, Oropallo A. Generative Artificial Intelligence: Applications in Scientific Writing and Data Analysis in Wound Healing Research. Adv Skin Wound Care 2024; 37:601-607. [PMID: 39792511 DOI: 10.1097/asw.0000000000000226] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2025]
Abstract
ABSTRACT Generative artificial intelligence (AI) models are a new technological development with vast research use cases among medical subspecialties. These powerful large language models offer a wide range of possibilities in wound care, from personalized patient support to optimized treatment plans and improved scientific writing. They can also assist in efficiently navigating the literature and selecting and summarizing articles, enabling researchers to focus on impactful studies relevant to wound care management and enhancing response quality through prompt-learning iterations. For nonnative English-speaking medical practitioners and authors, generative AI may aid in grammar and vocabulary selection. Although reports have suggested limitations of the conversational agent on medical translation pertaining to the precise interpretation of medical context, when used with verified resources, this language model can breach language barriers and promote practice-changing advancements in global wound care. Further, AI-powered chatbots can enable continuous monitoring of wound healing progress and real-time insights into treatment responses through frequent, readily available remote patient follow-ups.However, implementing AI in wound care research requires careful consideration of potential limitations, especially in accurately translating complex medical terms and workflows. Ethical considerations are vital to ensure reliable and credible wound care research when using AI technologies. Although ChatGPT shows promise for transforming wound care management, the authors warn against overreliance on the technology. Considering the potential limitations and risks, proper validation and oversight are essential to unlock its true potential while ensuring patient safety and the effectiveness of wound care treatments.
Collapse
Affiliation(s)
- Adrian Chen
- At the Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, United States, Adrian Chen, BS, Aleksandra Qilleri, BS, and Timothy Foster, BS, are Medical Students. Amit S. Rao, MD, is Project Manager, Department of Surgery, Wound Care Division, Northwell Wound Healing Center and Hyperbarics, Northwell Health, Hempstead. Sandeep Gopalakrishnan, PhD, MAPWCA, is Associate Professor and Director, Wound Healing and Tissue Repair Analytics Laboratory, School of Nursing, College of Health Professions, University of Wisconsin-Milwaukee. Jeffrey Niezgoda, MD, MAPWCA, is Founder and President Emeritus, AZH Wound Care and Hyperbaric Oxygen Therapy Center, Milwaukee, and President and Chief Medical Officer, WebCME, Greendale, Wisconsin. Alisha Oropallo, MD, is Professor of Surgery, Donald and Barbara Zucker School of Medicine and The Feinstein Institutes for Medical Research, Manhasset New York; Director, Comprehensive Wound Healing Center, Northwell Health; and Program Director, Wound and Burn Fellowship program, Northwell Health
| | | | | | | | | | | | | |
Collapse
|
17
|
Baumgärtner K, Byczkowski M, Schmid T, Muschko M, Woessner P, Gerlach A, Bonekamp D, Schlemmer HP, Hohenfellner M, Görtz M. Effectiveness of the Medical Chatbot PROSCA to Inform Patients About Prostate Cancer: Results of a Randomized Controlled Trial. EUR UROL SUPPL 2024; 69:80-88. [PMID: 39329071 PMCID: PMC11424957 DOI: 10.1016/j.euros.2024.08.022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/30/2024] [Indexed: 09/28/2024] Open
Abstract
Background and objective Artificial intelligence (AI)-powered conversational agents are increasingly finding application in health care, as these can provide patient education at any time. However, their effectiveness in medical settings remains largely unexplored. This study aimed to assess the impact of the chatbot "PROState cancer Conversational Agent" (PROSCA), which was trained to provide validated support from diagnostic tests to treatment options for men facing prostate cancer (PC) diagnosis. Methods The chatbot PROSCA, developed by urologists at Heidelberg University Hospital and SAP SE, was evaluated through a randomized controlled trial (RCT). Patients were assigned to either the chatbot group, receiving additional access to PROSCA alongside standard information by urologists, or the control group (1:1), receiving standard information. A total of 112 men were included, of whom 103 gave feedback at study completion. Key findings and limitations Over time, patients' information needs decreased significantly more in the chatbot group than in the control group (p = 0.035). In the chatbot group, 43/54 men (79.6%) used PROSCA, and all of them found it easy to use. Of the men, 71.4% agreed that the chatbot improved their informedness about PC and 90.7% would like to use PROSCA again. Limitations are study sample size, single-center design, and specific clinical application. Conclusions and clinical implications With the introduction of the PROSCA chatbot, we created and evaluated an innovative, evidence-based AI health information tool as an additional source of information for PC. Our RCT results showed significant benefits of the chatbot in reducing patients' information needs and enhancing their understanding of PC. This easy-to-use AI tool provides accurate, timely, and accessible support, demonstrating its value in the PC diagnosis process. Future steps include further customization of the chatbot's responses and integration with the existing health care systems to maximize its impact on patient outcomes. Patient summary This study evaluated an artificial intelligence-powered chatbot-PROSCA, a digital tool designed to support men facing prostate cancer diagnosis by providing validated information from diagnosis to treatment. Results showed that patients who used the chatbot as an additional tool felt better informed than those who received standard information from urologists. The majority of users appreciated the ease of use of the chatbot and expressed a desire to use it again; this suggests that PROSCA could be a valuable resource to improve patient understanding in prostate cancer diagnosis.
Collapse
Affiliation(s)
- Kilian Baumgärtner
- Medical Faculty, Ruprecht-Karls University of Heidelberg, Heidelberg, Germany
| | | | | | | | | | | | - David Bonekamp
- Department of Radiology, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | | | | | - Magdalena Görtz
- Department of Urology, Heidelberg University Hospital, Heidelberg, Germany
- Junior Clinical Cooperation Unit ‘Multiparametric Methods for Early Detection of Prostate Cancer’, German Cancer Research Center (DKFZ), Heidelberg, Germany
| |
Collapse
|
18
|
Liu XQ, Wang X, Zhang HR. Large multimodal models assist in psychiatry disorders prevention and diagnosis of students. World J Psychiatry 2024; 14:1415-1421. [DOI: 10.5498/wjp.v14.i10.1415] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/10/2024] [Revised: 09/03/2024] [Accepted: 09/25/2024] [Indexed: 10/17/2024] Open
Abstract
Students are considered one of the groups most affected by psychological problems. Given the highly dangerous nature of mental illnesses and the increasingly serious state of global mental health, it is imperative for us to explore new methods and approaches concerning the prevention and treatment of mental illnesses. Large multimodal models (LMMs), as the most advanced artificial intelligence models (i.e. ChatGPT-4), have brought new hope to the accurate prevention, diagnosis, and treatment of psychiatric disorders. The assistance of these models in the promotion of mental health is critical, as the latter necessitates a strong foundation of medical knowledge and professional skills, emotional support, stigma mitigation, the encouragement of more honest patient self-disclosure, reduced health care costs, improved medical efficiency, and greater mental health service coverage. However, these models must address challenges related to health, safety, hallucinations, and ethics simultaneously. In the future, we should address these challenges by developing relevant usage manuals, accountability rules, and legal regulations; implementing a human-centered approach; and intelligently upgrading LMMs through the deep optimization of such models, their algorithms, and other means. This effort will thus substantially contribute not only to the maintenance of students’ health but also to the achievement of global sustainable development goals.
Collapse
Affiliation(s)
- Xin-Qiao Liu
- School of Education, Tianjin University, Tianjin 300350, China
| | - Xin Wang
- School of Education, Tianjin University, Tianjin 300350, China
| | - Hui-Rui Zhang
- Faculty of Education, The Open University of China, Beijing 100039, China
| |
Collapse
|
19
|
Hesso I, Zacharias L, Kayyali R, Charalambous A, Lavdaniti M, Stalika E, Ajami T, Acampa W, Boban J, Nabhani-Gebara S. Artificial Intelligence for Optimizing Cancer Imaging: User Experience Study. JMIR Cancer 2024; 10:e52639. [PMID: 39388693 PMCID: PMC11502975 DOI: 10.2196/52639] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Revised: 02/23/2024] [Accepted: 06/28/2024] [Indexed: 10/12/2024] Open
Abstract
BACKGROUND The need for increased clinical efficacy and efficiency has been the main force in developing artificial intelligence (AI) tools in medical imaging. The INCISIVE project is a European Union-funded initiative aiming to revolutionize cancer imaging methods using AI technology. It seeks to address limitations in imaging techniques by developing an AI-based toolbox that improves accuracy, specificity, sensitivity, interpretability, and cost-effectiveness. OBJECTIVE To ensure the successful implementation of the INCISIVE AI service, a study was conducted to understand the needs, challenges, and expectations of health care professionals (HCPs) regarding the proposed toolbox and any potential implementation barriers. METHODS A mixed methods study consisting of 2 phases was conducted. Phase 1 involved user experience (UX) design workshops with users of the INCISIVE AI toolbox. Phase 2 involved a Delphi study conducted through a series of sequential questionnaires. To recruit, a purposive sampling strategy based on the project's consortium network was used. In total, 16 HCPs from Serbia, Italy, Greece, Cyprus, Spain, and the United Kingdom participated in the UX design workshops and 12 completed the Delphi study. Descriptive statistics were performed using SPSS (IBM Corp), enabling the calculation of mean rank scores of the Delphi study's lists. The qualitative data collected via the UX design workshops was analyzed using NVivo (version 12; Lumivero) software. RESULTS The workshops facilitated brainstorming and identification of the INCISIVE AI toolbox's desired features and implementation barriers. Subsequently, the Delphi study was instrumental in ranking these features, showing a strong consensus among HCPs (W=0.741, P<.001). Additionally, this study also identified implementation barriers, revealing a strong consensus among HCPs (W=0.705, P<.001). Key findings indicated that the INCISIVE AI toolbox could assist in areas such as misdiagnosis, overdiagnosis, delays in diagnosis, detection of minor lesions, decision-making in disagreement, treatment allocation, disease prognosis, prediction, treatment response prediction, and care integration throughout the patient journey. Limited resources, lack of organizational and managerial support, and data entry variability were some of the identified barriers. HCPs also had an explicit interest in AI explainability, desiring feature relevance explanations or a combination of feature relevance and visual explanations within the toolbox. CONCLUSIONS The results provide a thorough examination of the INCISIVE AI toolbox's design elements as required by the end users and potential barriers to its implementation, thus guiding the design and implementation of the INCISIVE technology. The outcome offers information about the degree of AI explainability required of the INCISIVE AI toolbox across the three services: (1) initial diagnosis; (2) disease staging, differentiation, and characterization; and (3) treatment and follow-up indicated for the toolbox. By considering the perspective of end users, INCISIVE aims to develop a solution that effectively meets their needs and drives adoption.
Collapse
Affiliation(s)
- Iman Hesso
- Pharmacy Department, Faculty of Health, Science, Social Care and Education, Kingston University London, Kingston Upon Thames, United Kingdom
| | - Lithin Zacharias
- Pharmacy Department, Faculty of Health, Science, Social Care and Education, Kingston University London, Kingston Upon Thames, United Kingdom
| | - Reem Kayyali
- Pharmacy Department, Faculty of Health, Science, Social Care and Education, Kingston University London, Kingston Upon Thames, United Kingdom
| | | | - Maria Lavdaniti
- Department of Nursing, International Hellenic University, Thessaloniki, Greece
| | - Evangelia Stalika
- Department of Nursing, International Hellenic University, Thessaloniki, Greece
| | - Tarek Ajami
- Urology Department, Hospital Clinic de Barcelona, Barcelona, Spain
| | - Wanda Acampa
- Department of Advanced Biomedical Science, University of Naples Federico II, Naples, Italy
| | - Jasmina Boban
- Department of Radiology, Faculty of Medicine, University of Novi Sad, Novi Sad,
| | - Shereen Nabhani-Gebara
- Pharmacy Department, Faculty of Health, Science, Social Care and Education, Kingston University London, Kingston Upon Thames, United Kingdom
| |
Collapse
|
20
|
Wahab N, Forsyth RA. Experiences of patients with hard-to-heal wounds: insights from a pilot survey. J Wound Care 2024; 33:788-794. [PMID: 39388206 DOI: 10.12968/jowc.2024.0109] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/12/2024]
Abstract
OBJECTIVE To learn about the experiences of people who seek treatment for hard-to-heal wounds, we distributed a nationwide pilot survey, asking questions about the nature of their wound, how it shaped their daily lives, pathways to receiving care and experiences with treatment. The long-term objective is to quantify the journey of patients with hard-to-heal wounds to identify ideal intervention points that will lead to the best outcomes. This article summarises the findings, implications, limitations and suggestions for future research. METHOD Qualitative data were self-reported from patients with hard-to-heal wounds (open for ≥4 weeks) in a pilot chatbot survey, (Wound Expert Survey (WES)) provided online in the US on Meta platforms (Facebook and Instagram) between 2021 and 2022. RESULTS The US national pilot survey attracted responses from 780 patients, 27 of whom provided a video testimonial. Some 57% of patients delayed treatment because they believed their wound would heal on its own, and only 4% saw a wound care specialist. Respondents reported the cost of care as the most frequent reason for not following all of a doctor's treatment recommendations. Queries regarding quality of life (QoL) revealed that more than half (65%) said they have negative thoughts associated with their wound at least every few days. Some 19% of respondents said their wound had an odour and, of them, 34% said odour had a major or severe negative impact on their self-confidence. Economically, nearly one-quarter of respondents said having a wound led to a drop in their total household income and 17% said their wound led to a change in their employment status. CONCLUSION A national pilot survey of patients with hard-to-heal wounds revealed that many delay seeking professional assistance and only a small minority see a wound care specialist. Experiencing an ulcer, even for a few months, can have significant negative effects on a patient's QoL. Patients frequently had negative thoughts associated with their wound, and odour compounded these negative effects, leading to major or severe negative impacts on self-confidence. Households experienced a decline in income, due to both the direct reduction or loss of patient employment and the additional time spent by family members assisting in patient recovery. Thus, a variety of factors contribute to poor outcomes for patients with hard-to-heal wounds. To validate and extend these preliminary results, future surveys of patients with hard-to-heal wounds should focus on additional reasons patients do not seek professional help sooner. To improve health outcomes and QoL, assessment of patient socioeconomic variables should occur whenever wound closure stalls.
Collapse
Affiliation(s)
- Naz Wahab
- Wound Care Experts, NV, US
- HCA Mountain View Hospital, NV, US
- Roseman University College of Medicine, NV, US
- Common Spirit Dignity Hospitals, NV, US
| | - R Allyn Forsyth
- MIMEDX Group Inc., GA, US
- Department of Biology, San Diego State University, CA, US
| |
Collapse
|
21
|
Finch L, Broach V, Feinberg J, Al-Niaimi A, Abu-Rustum NR, Zhou Q, Iasonos A, Chi DS. ChatGPT compared to national guidelines for management of ovarian cancer: Did ChatGPT get it right? - A Memorial Sloan Kettering Cancer Center Team Ovary study. Gynecol Oncol 2024; 189:75-79. [PMID: 39042956 PMCID: PMC11402584 DOI: 10.1016/j.ygyno.2024.07.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/04/2024] [Revised: 07/08/2024] [Accepted: 07/15/2024] [Indexed: 07/25/2024]
Abstract
OBJECTIVES We evaluated the performance of a chatbot compared to the National Comprehensive Cancer Network (NCCN) Guidelines for the management of ovarian cancer. METHODS Using NCCN Guidelines, we generated 10 questions and answers regarding management of ovarian cancer at a single point in time. Questions were thematically divided into risk factors, surgical management, medical management, and surveillance. We asked ChatGPT (GPT-4) to provide responses without prompting (unprompted GPT) and with prompt engineering (prompted GPT). Responses were blinded and evaluated for accuracy and completeness by 5 gynecologic oncologists. A score of 0 was defined as inaccurate, 1 as accurate and incomplete, and 2 as accurate and complete. Evaluations were compared among NCCN, unprompted GPT, and prompted GPT answers. RESULTS Overall, 48% of responses from NCCN, 64% from unprompted GPT, and 66% from prompted GPT were accurate and complete. The percentage of accurate but incomplete responses was higher for NCCN vs GPT-4. The percentage of accurate and complete scores for questions regarding risk factors, surgical management, and surveillance was higher for GPT-4 vs NCCN; however, for questions regarding medical management, the percentage was lower for GPT-4 vs NCCN. Overall, 14% of responses from unprompted GPT, 12% from prompted GPT, and 10% from NCCN were inaccurate. CONCLUSIONS GPT-4 provided accurate and complete responses at a single point in time to a limited set of questions regarding ovarian cancer, with best performance in areas of risk factors, surgical management, and surveillance. Occasional inaccuracies, however, should limit unsupervised use of chatbots at this time.
Collapse
Affiliation(s)
- Lindsey Finch
- Gynecology Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Vance Broach
- Gynecology Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USA; Department of Obstetrics and Gynecology, Weill Cornell Medical College, New York, NY, USA
| | - Jacqueline Feinberg
- Gynecology Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USA; Department of Obstetrics and Gynecology, Weill Cornell Medical College, New York, NY, USA
| | - Ahmed Al-Niaimi
- Gynecology Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USA; Department of Obstetrics and Gynecology, Weill Cornell Medical College, New York, NY, USA
| | - Nadeem R Abu-Rustum
- Gynecology Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USA; Department of Obstetrics and Gynecology, Weill Cornell Medical College, New York, NY, USA
| | - Qin Zhou
- Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Alexia Iasonos
- Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Dennis S Chi
- Gynecology Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USA; Department of Obstetrics and Gynecology, Weill Cornell Medical College, New York, NY, USA.
| |
Collapse
|
22
|
MacNeill AL, MacNeill L, Luke A, Doucet S. Health Professionals' Views on the Use of Conversational Agents for Health Care: Qualitative Descriptive Study. J Med Internet Res 2024; 26:e49387. [PMID: 39320936 PMCID: PMC11464950 DOI: 10.2196/49387] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Revised: 03/01/2024] [Accepted: 06/01/2024] [Indexed: 09/26/2024] Open
Abstract
BACKGROUND In recent years, there has been an increase in the use of conversational agents for health promotion and service delivery. To date, health professionals' views on the use of this technology have received limited attention in the literature. OBJECTIVE The purpose of this study was to gain a better understanding of how health professionals view the use of conversational agents for health care. METHODS Physicians, nurses, and regulated mental health professionals were recruited using various web-based methods. Participants were interviewed individually using the Zoom (Zoom Video Communications, Inc) videoconferencing platform. Interview questions focused on the potential benefits and risks of using conversational agents for health care, as well as the best way to integrate conversational agents into the health care system. Interviews were transcribed verbatim and uploaded to NVivo (version 12; QSR International, Inc) for thematic analysis. RESULTS A total of 24 health professionals participated in the study (19 women, 5 men; mean age 42.75, SD 10.71 years). Participants said that the use of conversational agents for health care could have certain benefits, such as greater access to care for patients or clients and workload support for health professionals. They also discussed potential drawbacks, such as an added burden on health professionals (eg, program familiarization) and the limited capabilities of these programs. Participants said that conversational agents could be used for routine or basic tasks, such as screening and assessment, providing information and education, and supporting individuals between appointments. They also said that health professionals should have some oversight in terms of the development and implementation of these programs. CONCLUSIONS The results of this study provide insight into health professionals' views on the use of conversational agents for health care, particularly in terms of the benefits and drawbacks of these programs and how they should be integrated into the health care system. These collective findings offer useful information and guidance to stakeholders who have an interest in the development and implementation of this technology.
Collapse
Affiliation(s)
- A Luke MacNeill
- Centre for Research in Integrated Care, University of New Brunswick, Saint John, NB, Canada
- Department of Nursing and Health Sciences, University of New Brunswick, Saint John, NB, Canada
| | - Lillian MacNeill
- Centre for Research in Integrated Care, University of New Brunswick, Saint John, NB, Canada
- Department of Nursing and Health Sciences, University of New Brunswick, Saint John, NB, Canada
| | - Alison Luke
- Centre for Research in Integrated Care, University of New Brunswick, Saint John, NB, Canada
- Department of Nursing and Health Sciences, University of New Brunswick, Saint John, NB, Canada
| | - Shelley Doucet
- Centre for Research in Integrated Care, University of New Brunswick, Saint John, NB, Canada
- Department of Nursing and Health Sciences, University of New Brunswick, Saint John, NB, Canada
| |
Collapse
|
23
|
Goumas G, Dardavesis TI, Syrigos K, Syrigos N, Simou E. Chatbots in Cancer Applications, Advantages and Disadvantages: All that Glitters Is Not Gold. J Pers Med 2024; 14:877. [PMID: 39202068 PMCID: PMC11355580 DOI: 10.3390/jpm14080877] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2024] [Revised: 08/12/2024] [Accepted: 08/14/2024] [Indexed: 09/03/2024] Open
Abstract
The emergence of digitalization and artificial intelligence has had a profound impact on society, especially in the field of medicine. Digital health is now a reality, with an increasing number of people using chatbots for prognostic or diagnostic purposes, therapeutic planning, and monitoring, as well as for nutritional and mental health support. Initially designed for various purposes, chatbots have demonstrated significant advantages in the medical field, as indicated by multiple sources. However, there are conflicting views in the current literature, with some sources highlighting their drawbacks and limitations, particularly in their use in oncology. This state-of-the-art review article seeks to present both the benefits and the drawbacks of chatbots in the context of medicine and cancer, while also addressing the challenges in their implementation, offering expert insights on the subject.
Collapse
Affiliation(s)
- Georgios Goumas
- Department of Public Health Policy, School of Public Health, University of West Attica, 115 21 Athens, Greece;
| | - Theodoros I. Dardavesis
- Laboratory of Hygiene, Social & Preventive Medicine and Medical Statistics, School of Medicine, Faculty of Health Sciences, Aristotle University of Thessaloniki, 541 24 Thessaloniki, Greece;
| | - Konstantinos Syrigos
- Oncology Unit, 3rd Department of Medicine, “Sotiria” Hospital for Diseases of the Chest, National and Kapodistrian University of Athens, 115 27 Athens, Greece; (K.S.); (N.S.)
| | - Nikolaos Syrigos
- Oncology Unit, 3rd Department of Medicine, “Sotiria” Hospital for Diseases of the Chest, National and Kapodistrian University of Athens, 115 27 Athens, Greece; (K.S.); (N.S.)
- Dana-Farber Cancer Institute, Boston, MA 02215, USA
| | - Effie Simou
- Department of Public Health Policy, School of Public Health, University of West Attica, 115 21 Athens, Greece;
| |
Collapse
|
24
|
He L, Basar E, Krahmer E, Wiers R, Antheunis M. Effectiveness and User Experience of a Smoking Cessation Chatbot: Mixed Methods Study Comparing Motivational Interviewing and Confrontational Counseling. J Med Internet Res 2024; 26:e53134. [PMID: 39106097 PMCID: PMC11336496 DOI: 10.2196/53134] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2023] [Revised: 04/18/2024] [Accepted: 05/02/2024] [Indexed: 08/07/2024] Open
Abstract
BACKGROUND Cigarette smoking poses a major public health risk. Chatbots may serve as an accessible and useful tool to promote cessation due to their high accessibility and potential in facilitating long-term personalized interactions. To increase effectiveness and acceptability, there remains a need to identify and evaluate counseling strategies for these chatbots, an aspect that has not been comprehensively addressed in previous research. OBJECTIVE This study aims to identify effective counseling strategies for such chatbots to support smoking cessation. In addition, we sought to gain insights into smokers' expectations of and experiences with the chatbot. METHODS This mixed methods study incorporated a web-based experiment and semistructured interviews. Smokers (N=229) interacted with either a motivational interviewing (MI)-style (n=112, 48.9%) or a confrontational counseling-style (n=117, 51.1%) chatbot. Both cessation-related (ie, intention to quit and self-efficacy) and user experience-related outcomes (ie, engagement, therapeutic alliance, perceived empathy, and interaction satisfaction) were assessed. Semistructured interviews were conducted with 16 participants, 8 (50%) from each condition, and data were analyzed using thematic analysis. RESULTS Results from a multivariate ANOVA showed that participants had a significantly higher overall rating for the MI (vs confrontational counseling) chatbot. Follow-up discriminant analysis revealed that the better perception of the MI chatbot was mostly explained by the user experience-related outcomes, with cessation-related outcomes playing a lesser role. Exploratory analyses indicated that smokers in both conditions reported increased intention to quit and self-efficacy after the chatbot interaction. Interview findings illustrated several constructs (eg, affective attitude and engagement) explaining people's previous expectations and timely and retrospective experience with the chatbot. CONCLUSIONS The results confirmed that chatbots are a promising tool in motivating smoking cessation and the use of MI can improve user experience. We did not find extra support for MI to motivate cessation and have discussed possible reasons. Smokers expressed both relational and instrumental needs in the quitting process. Implications for future research and practice are discussed.
Collapse
Affiliation(s)
- Linwei He
- Department of Communication and Cognition, Tilburg School of Humanities and Digital Sciences, Tilburg University, Tilburg, Netherlands
| | - Erkan Basar
- Behavioral Science Institute, Radboud University, Nijmegen, Netherlands
| | - Emiel Krahmer
- Department of Communication and Cognition, Tilburg School of Humanities and Digital Sciences, Tilburg University, Tilburg, Netherlands
| | - Reinout Wiers
- Addiction Development and Psychopathology (ADAPT)-lab, Department of Psychology and Centre for Urban Mental Health, University of Amsterdam, Amsterdam, Netherlands
| | - Marjolijn Antheunis
- Department of Communication and Cognition, Tilburg School of Humanities and Digital Sciences, Tilburg University, Tilburg, Netherlands
| |
Collapse
|
25
|
Laymouna M, Ma Y, Lessard D, Schuster T, Engler K, Lebouché B. Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review. J Med Internet Res 2024; 26:e56930. [PMID: 39042446 PMCID: PMC11303905 DOI: 10.2196/56930] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2024] [Revised: 04/07/2024] [Accepted: 04/12/2024] [Indexed: 07/24/2024] Open
Abstract
BACKGROUND Chatbots, or conversational agents, have emerged as significant tools in health care, driven by advancements in artificial intelligence and digital technology. These programs are designed to simulate human conversations, addressing various health care needs. However, no comprehensive synthesis of health care chatbots' roles, users, benefits, and limitations is available to inform future research and application in the field. OBJECTIVE This review aims to describe health care chatbots' characteristics, focusing on their diverse roles in the health care pathway, user groups, benefits, and limitations. METHODS A rapid review of published literature from 2017 to 2023 was performed with a search strategy developed in collaboration with a health sciences librarian and implemented in the MEDLINE and Embase databases. Primary research studies reporting on chatbot roles or benefits in health care were included. Two reviewers dual-screened the search results. Extracted data on chatbot roles, users, benefits, and limitations were subjected to content analysis. RESULTS The review categorized chatbot roles into 2 themes: delivery of remote health services, including patient support, care management, education, skills building, and health behavior promotion, and provision of administrative assistance to health care providers. User groups spanned across patients with chronic conditions as well as patients with cancer; individuals focused on lifestyle improvements; and various demographic groups such as women, families, and older adults. Professionals and students in health care also emerged as significant users, alongside groups seeking mental health support, behavioral change, and educational enhancement. The benefits of health care chatbots were also classified into 2 themes: improvement of health care quality and efficiency and cost-effectiveness in health care delivery. The identified limitations encompassed ethical challenges, medicolegal and safety concerns, technical difficulties, user experience issues, and societal and economic impacts. CONCLUSIONS Health care chatbots offer a wide spectrum of applications, potentially impacting various aspects of health care. While they are promising tools for improving health care efficiency and quality, their integration into the health care system must be approached with consideration of their limitations to ensure optimal, safe, and equitable use.
Collapse
Affiliation(s)
- Moustafa Laymouna
- Department of Family Medicine, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada
- Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, QC, Canada
- Infectious Diseases and Immunity in Global Health Program, Research Institute of McGill University Health Centre, Montreal, QC, Canada
| | - Yuanchao Ma
- Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, QC, Canada
- Infectious Diseases and Immunity in Global Health Program, Research Institute of McGill University Health Centre, Montreal, QC, Canada
- Chronic and Viral Illness Service, Division of Infectious Disease, Department of Medicine, McGill University Health Centre, Montreal, QC, Canada
- Department of Biomedical Engineering, Polytechnique Montréal, Montreal, QC, Canada
| | - David Lessard
- Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, QC, Canada
- Infectious Diseases and Immunity in Global Health Program, Research Institute of McGill University Health Centre, Montreal, QC, Canada
- Chronic and Viral Illness Service, Division of Infectious Disease, Department of Medicine, McGill University Health Centre, Montreal, QC, Canada
| | - Tibor Schuster
- Department of Family Medicine, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada
| | - Kim Engler
- Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, QC, Canada
- Infectious Diseases and Immunity in Global Health Program, Research Institute of McGill University Health Centre, Montreal, QC, Canada
- Chronic and Viral Illness Service, Division of Infectious Disease, Department of Medicine, McGill University Health Centre, Montreal, QC, Canada
| | - Bertrand Lebouché
- Department of Family Medicine, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada
- Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montreal, QC, Canada
- Infectious Diseases and Immunity in Global Health Program, Research Institute of McGill University Health Centre, Montreal, QC, Canada
- Chronic and Viral Illness Service, Division of Infectious Disease, Department of Medicine, McGill University Health Centre, Montreal, QC, Canada
| |
Collapse
|
26
|
Lin SJ, Sun CY, Chen DN, Kang YN, Lai NM, Chen KH, Chen C. Perioperative application of chatbots: a systematic review and meta-analysis. BMJ Health Care Inform 2024; 31:e100985. [PMID: 39032946 PMCID: PMC11261686 DOI: 10.1136/bmjhci-2023-100985] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2023] [Accepted: 04/02/2024] [Indexed: 07/23/2024] Open
Abstract
BACKGROUND AND OBJECTIVES Patient-clinician communication and shared decision-making face challenges in the perioperative period. Chatbots have emerged as valuable support tools in perioperative care. A simultaneous and complete comparison of overall benefits and harm of chatbot application is conducted. MATERIALS MEDLINE, EMBASE and the Cochrane Library were systematically searched for studies published before May 2023 on the benefits and harm of chatbots used in the perioperative period. The major outcomes assessed were patient satisfaction and knowledge acquisition. Untransformed proportion (PR) with a 95% CI was used for the analysis of continuous data. Risk of bias was assessed using the Cochrane Risk of Bias assessment tool version 2 and the Methodological Index for Non-Randomised Studies. RESULTS Eight trials comprising 1073 adults from four countries were included. Most interventions (n = 5, 62.5%) targeted perioperative care in orthopaedics. Most interventions use rule-based chatbots (n = 7, 87.5%). This meta-analysis found that the majority of the participants were satisfied with the use of chatbots (mean proportion=0.73; 95% CI: 0.62 to 0.85), and agreed that they gained knowledge in their perioperative period (mean proportion=0.80; 95% CI: 0.74 to 0.87). CONCLUSION This review demonstrates that perioperative chatbots are well received by the majority of patients with no reports of harm to-date. Chatbots may be considered as an aid in perioperative communication between patients and clinicians and shared decision-making. These findings may be used to guide the healthcare providers, policymakers and researchers for enhancing perioperative care.
Collapse
Affiliation(s)
- Shih-Jung Lin
- School of Medicine, College of Medicine, Taipei Medical University, Taipei, Taiwan
| | - Chin-Yu Sun
- Department of Computer Science and Information Engineering, National Taipei University of Technology, Taipei, Taiwan
| | - Dan-Ni Chen
- Department of Computer Science and Information Engineering, National Taipei University of Technology, Taipei, Taiwan
- Executive Master of Business Administration Program, College of Business, University of Texas at Arlington, Arlington, Texas, USA
| | - Yi-No Kang
- Cochrane Taiwan, Taipei Medical University, Taipei, Taiwan
- Research Center of Big Data and Meta-Analysis, Wan Fang Hospital, Taipei Medical University, Taipei, Taiwan
- Institute of Health Policy and Management, College of Public Health, National Taiwan University, Taipei, Taiwan
- Evidence-Based Medicine Center, Wan Fang Hospital, Taipei Medical University, Taipei, Taiwan
| | - Nai Ming Lai
- Digital Health and Innovation Impact Lab, Taylor's University, Subang Jaya, Malaysia
- School of Medicine, Faculty of Health and Medical Sciences, Taylor's University, Subang Jaya, Malaysia
| | - Kee-Hsin Chen
- Cochrane Taiwan, Taipei Medical University, Taipei, Taiwan
- Post-Baccalaureate Program in Nursing, College of Nursing, Taipei Medical University, Taipei, Taiwan
- Research Center in Nursing Clinical Practice, Wan Fang Hospital, Taipei Medical University, Taipei, Taiwan
- Department of Nursing, Wan Fang Hospital, Taipei Medical University, Taipei, Taiwan
- Evidence-Based Knowledge Translation Center, Wan Fang Hospital, Taipei Medical University, Taipei, Taiwan
- Visiting Associate Professor, School of Medicine, Faculty of Health and Medical Sciences, Taylor's University, Selangor 47500, Malaysia
| | - Chiehfeng Chen
- Cochrane Taiwan, Taipei Medical University, Taipei, Taiwan
- Evidence-Based Medicine Center, Wan Fang Hospital, Taipei Medical University, Taipei, Taiwan
- Department of Public Health, School of Medicine, College of Medicine, Taipei Medical University, Taipei, Taiwan
- Division of Plastic Surgery, Department of Surgery, Wan Fang Hospital, Taipei Medical University, Taipei, Taiwan
| |
Collapse
|
27
|
Rothlisberger PN. AI-Powered Patient-Centered Care: A Call to Action for Innovation. J Healthc Manag 2024; 69:255-266. [PMID: 38976786 DOI: 10.1097/jhm-d-24-00102] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/10/2024]
Abstract
SUMMARY The influential report Crossing the Quality Chasm: A New Health System for the 21st Century established six core objectives to enhance healthcare quality. It highlighted the necessity for healthcare to encompass safety, effectiveness, a patient-centered approach, timeliness, efficiency, and equity. This essay focuses on one of these six core objectives: a patient-centered approach. Healthcare leaders actively seek solutions to improve and ensure the delivery of high-quality care. The imperative to provide quality healthcare underscores the need for artificial intelligence (AI) to become an essential component in a patient-centered approach rather than merely an optional advantage. Despite the expansion of AI, there is a lack of understanding of how AI can improve patient-centered care. This essay examines the fundamental aspects of patient-centered care, as outlined by the Picker Institute, while also exploring the prospective role of AI in advancing the core principles of patient-centered care and proposing frameworks for applying AI in healthcare.
Collapse
Affiliation(s)
- Paige N Rothlisberger
- Department of Public and Allied Health, Bowling Green State University, Bowling Green, Ohio
| |
Collapse
|
28
|
Zhu L, Rong Y, McGee LA, Rwigema JCM, Patel SH. Testing and Validation of a Custom Retrained Large Language Model for the Supportive Care of HN Patients with External Knowledge Base. Cancers (Basel) 2024; 16:2311. [PMID: 39001375 PMCID: PMC11240646 DOI: 10.3390/cancers16132311] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2024] [Revised: 06/17/2024] [Accepted: 06/18/2024] [Indexed: 07/16/2024] Open
Abstract
PURPOSE This study aimed to develop a retrained large language model (LLM) tailored to the needs of HN cancer patients treated with radiotherapy, with emphasis on symptom management and survivorship care. METHODS A comprehensive external database was curated for training ChatGPT-4, integrating expert-identified consensus guidelines on supportive care for HN patients and correspondences from physicians and nurses within our institution's electronic medical records for 90 HN patients. The performance of our model was evaluated using 20 patient post-treatment inquiries that were then assessed by three Board certified radiation oncologists (RadOncs). The rating of the model was assessed on a scale of 1 (strongly disagree) to 5 (strongly agree) based on accuracy, clarity of response, completeness s, and relevance. RESULTS The average scores for the 20 tested questions were 4.25 for accuracy, 4.35 for clarity, 4.22 for completeness, and 4.32 for relevance, on a 5-point scale. Overall, 91.67% (220 out of 240) of assessments received scores of 3 or higher, and 83.33% (200 out of 240) received scores of 4 or higher. CONCLUSION The custom-trained model demonstrates high accuracy in providing support to HN patients offering evidence-based information and guidance on their symptom management and survivorship care.
Collapse
Affiliation(s)
| | - Yi Rong
- Correspondence: (Y.R.); (S.H.P.)
| | | | | | - Samir H. Patel
- Department of Radiation Oncology, Mayo Clinic, Phoenix, AZ 85054, USA; (L.Z.); (L.A.M.); (J.-C.M.R.)
| |
Collapse
|
29
|
Collins L, Nicholson N, Lidbetter N, Smithson D, Baker P. Implementation of Anxiety UK's Ask Anxia Chatbot Service: Lessons Learned. JMIR Hum Factors 2024; 11:e53897. [PMID: 38885016 PMCID: PMC11217701 DOI: 10.2196/53897] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2023] [Accepted: 05/05/2024] [Indexed: 06/18/2024] Open
Abstract
Chatbots are increasingly being applied in the context of health care, providing access to services when there are constraints on human resources. Simple, rule-based chatbots are suited to high-volume, repetitive tasks and can therefore be used effectively in providing users with important health information. In this Viewpoint paper, we report on the implementation of a chatbot service called Ask Anxia as part of a wider provision of information and support services offered by the UK national charity, Anxiety UK. We reflect on the changes made to the chatbot over the course of approximately 18 months as the Anxiety UK team monitored its performance and responded to recurrent themes in user queries by developing further information and services. We demonstrate how corpus linguistics can contribute to the evaluation of user queries and the optimization of responses. On the basis of these observations of how Anxiety UK has developed its own chatbot service, we offer recommendations for organizations looking to add automated conversational interfaces to their services.
Collapse
Affiliation(s)
- Luke Collins
- Linguistics and English Language, Lancaster University, Lancaster, United Kingdom
| | | | | | | | - Paul Baker
- Linguistics and English Language, Lancaster University, Lancaster, United Kingdom
| |
Collapse
|
30
|
Rokhshad R, Zhang P, Mohammad-Rahimi H, Pitchika V, Entezari N, Schwendicke F. Accuracy and consistency of chatbots versus clinicians for answering pediatric dentistry questions: A pilot study. J Dent 2024; 144:104938. [PMID: 38499280 DOI: 10.1016/j.jdent.2024.104938] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2023] [Revised: 03/06/2024] [Accepted: 03/11/2024] [Indexed: 03/20/2024] Open
Abstract
OBJECTIVES Artificial Intelligence has applications such as Large Language Models (LLMs), which simulate human-like conversations. The potential of LLMs in healthcare is not fully evaluated. This pilot study assessed the accuracy and consistency of chatbots and clinicians in answering common questions in pediatric dentistry. METHODS Two expert pediatric dentists developed thirty true or false questions involving different aspects of pediatric dentistry. Publicly accessible chatbots (Google Bard, ChatGPT4, ChatGPT 3.5, Llama, Sage, Claude 2 100k, Claude-instant, Claude-instant-100k, and Google Palm) were employed to answer the questions (3 independent new conversations). Three groups of clinicians (general dentists, pediatric specialists, and students; n = 20/group) also answered. Responses were graded by two pediatric dentistry faculty members, along with a third independent pediatric dentist. Resulting accuracies (percentage of correct responses) were compared using analysis of variance (ANOVA), and post-hoc pairwise group comparisons were corrected using Tukey's HSD method. ACronbach's alpha was calculated to determine consistency. RESULTS Pediatric dentists were significantly more accurate (mean±SD 96.67 %± 4.3 %) than other clinicians and chatbots (p < 0.001). General dentists (88.0 % ± 6.1 %) also demonstrated significantly higher accuracy than chatbots (p < 0.001), followed by students (80.8 %±6.9 %). ChatGPT showed the highest accuracy (78 %±3 %) among chatbots. All chatbots except ChatGPT3.5 showed acceptable consistency (Cronbach alpha>0.7). CLINICAL SIGNIFICANCE Based on this pilot study, chatbots may be valuable adjuncts for educational purposes and for distributing information to patients. However, they are not yet ready to serve as substitutes for human clinicians in diagnostic decision-making. CONCLUSION In this pilot study, chatbots showed lower accuracy than dentists. Chatbots may not yet be recommended for clinical pediatric dentistry.
Collapse
Affiliation(s)
- Rata Rokhshad
- Department of Pediatric Dentistry, University of Alabama at Birmingham, Birmingham, AL, USA.
| | - Ping Zhang
- Department of Pediatric Dentistry, University of Alabama at Birmingham, Birmingham, AL, USA
| | - Hossein Mohammad-Rahimi
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Berlin, Germany
| | - Vinay Pitchika
- Department of Conservative Dentistry and Periodontology, LMU Klinikum Munich, Germany
| | - Niloufar Entezari
- Department of pediatric dentistry, School of Dentistry, Qom University of Medical Sciences, Qom, Iran
| | - Falk Schwendicke
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Berlin, Germany; Department of Conservative Dentistry and Periodontology, LMU Klinikum Munich, Germany
| |
Collapse
|
31
|
Marisei M, Crocetto F, Sicignano E, Pagano G, Napolitano L. Doctor patient relationship in AI era: trying to decipher the problem. J Basic Clin Physiol Pharmacol 2024; 35:99-100. [PMID: 38830187 DOI: 10.1515/jbcpp-2024-0075] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/05/2024]
Affiliation(s)
- Mariagrazia Marisei
- Department of Advanced Biomedical Sciences, 9307 University of Naples Federico II , Naples, Italy
| | - Felice Crocetto
- Department of Neurosciences, Science of Reproduction and Odontostomatology, 165474 University of Naples Federico II , Naples, Italy
| | - Enrico Sicignano
- Department of Neurosciences, Science of Reproduction and Odontostomatology, 165474 University of Naples Federico II , Naples, Italy
| | - Giovanni Pagano
- Department of Neurosciences, Science of Reproduction and Odontostomatology, 165474 University of Naples Federico II , Naples, Italy
| | | |
Collapse
|
32
|
Zou X, Na Y, Lai K, Liu G. Unpacking public resistance to health Chatbots: a parallel mediation analysis. Front Psychol 2024; 15:1276968. [PMID: 38659671 PMCID: PMC11041026 DOI: 10.3389/fpsyg.2024.1276968] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2023] [Accepted: 04/01/2024] [Indexed: 04/26/2024] Open
Abstract
Introduction Despite the numerous potential benefits of health chatbots for personal health management, a substantial proportion of people oppose the use of such software applications. Building on the innovation resistance theory (IRT) and the prototype willingness model (PWM), this study investigated the functional barriers, psychological barriers, and negative prototype perception antecedents of individuals' resistance to health chatbots, as well as the rational and irrational psychological mechanisms underlying their linkages. Methods Data from 398 participants were used to construct a partial least squares structural equation model (PLS-SEM). Results Resistance intention mediated the relationship between functional barriers, psychological barriers, and resistance behavioral tendency, respectively. Furthermore, The relationship between negative prototype perceptions and resistance behavioral tendency was mediated by resistance intention and resistance willingness. Moreover, negative prototype perceptions were a more effective predictor of resistance behavioral tendency through resistance willingness than functional and psychological barriers. Discussion By investigating the role of irrational factors in health chatbot resistance, this study expands the scope of the IRT to explain the psychological mechanisms underlying individuals' resistance to health chatbots. Interventions to address people's resistance to health chatbots are discussed.
Collapse
Affiliation(s)
- Xiqian Zou
- School of Journalism and Communication, Tsinghua University, Beijing, China
| | - Yuxiang Na
- School of Journalism and Communication, Jinan University, Guangzhou, Guangdong, China
| | - Kaisheng Lai
- School of Journalism and Communication, Jinan University, Guangzhou, Guangdong, China
| | - Guan Liu
- Center for Computational Communication Studies, Jinan University, Guangzhou, Guangdong, China
| |
Collapse
|
33
|
Hegde N, Vardhan M, Nathani D, Rosenzweig E, Speed C, Karthikesalingam A, Seneviratne M. Infusing behavior science into large language models for activity coaching. PLOS DIGITAL HEALTH 2024; 3:e0000431. [PMID: 38564502 PMCID: PMC10986996 DOI: 10.1371/journal.pdig.0000431] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/15/2023] [Accepted: 12/14/2023] [Indexed: 04/04/2024]
Abstract
Large language models (LLMs) have shown promise for task-oriented dialogue across a range of domains. The use of LLMs in health and fitness coaching is under-explored. Behavior science frameworks such as COM-B, which conceptualizes behavior change in terms of capability (C), Opportunity (O) and Motivation (M), can be used to architect coaching interventions in a way that promotes sustained change. Here we aim to incorporate behavior science principles into an LLM using two knowledge infusion techniques: coach message priming (where exemplar coach responses are provided as context to the LLM), and dialogue re-ranking (where the COM-B category of the LLM output is matched to the inferred user need). Simulated conversations were conducted between the primed or unprimed LLM and a member of the research team, and then evaluated by 8 human raters. Ratings for the primed conversations were significantly higher in terms of empathy and actionability. The same raters also compared a single response generated by the unprimed, primed and re-ranked models, finding a significant uplift in actionability and empathy from the re-ranking technique. This is a proof of concept of how behavior science frameworks can be infused into automated conversational agents for a more principled coaching experience.
Collapse
|
34
|
Høj S, Thomsen SF, Meteran H, Sigsgaard T, Meteran H. Artificial intelligence and allergic rhinitis: does ChatGPT increase or impair the knowledge? J Public Health (Oxf) 2024; 46:123-126. [PMID: 37968109 DOI: 10.1093/pubmed/fdad219] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2023] [Revised: 09/14/2023] [Accepted: 10/06/2023] [Indexed: 11/17/2023] Open
Abstract
BACKGROUND Optimal management of allergic rhinitis requires patient education with easy access to accurate information. However, previous online platforms have provided misleading information. The demand for online medical information continues to grow, especially with the introduction of advanced chatbots like ChatGPT. METHODS This study aimed to evaluate the quality of information provided by ChatGPT regarding allergic rhinitis. A Likert scale was used to assess the accuracy of responses, ranging from 1 to 5. Four authors independently rated the responses from a healthcare professional's perspective. RESULTS A total of 20 questions covering various aspects of allergic rhinitis were asked. Among the answers, eight received a score of 5 (no inaccuracies), five received a score of 4 (minor non-harmful inaccuracies), six received a score of 3 (potentially misinterpretable inaccuracies) and one answer had a score of 2 (minor potentially harmful inaccuracies). CONCLUSIONS The variability in accuracy scores highlights the need for caution when relying solely on chatbots like ChatGPT for medical advice. Patients should consult qualified healthcare professionals and use online sources as a supplement. While ChatGPT has advantages in medical information delivery, its use should be approached with caution. ChatGPT can be useful for patient education but cannot replace healthcare professionals.
Collapse
Affiliation(s)
- Simon Høj
- Steno Diabetes Center Copenhagen, Copenhagen University Hospital, Herlev 2730, Denmark
- Department of Dermatology, Venereology, and Wound Healing Centre, Copenhagen University Hospital-Bispebjerg, Copenhagen 2400, Denmark
- Department of Public Health, Environment, Occupation, and Health, Aarhus University, Aarhus 8000, Denmark
| | - Simon F Thomsen
- Department of Dermatology, Venereology, and Wound Healing Centre, Copenhagen University Hospital-Bispebjerg, Copenhagen 2400, Denmark
- Department of Biomedical Sciences, University of Copenhagen, Copenhagen 2200, Denmark
| | - Hanieh Meteran
- Department of Internal Medicine, Section of Endocrinology, Copenhagen University Hospital-Hvidovre, Hvidovre 2650, Denmark
| | - Torben Sigsgaard
- Department of Public Health, Environment, Occupation, and Health, Aarhus University, Aarhus 8000, Denmark
| | - Howraman Meteran
- Department of Public Health, Environment, Occupation, and Health, Aarhus University, Aarhus 8000, Denmark
- Department of Internal Medicine, Respiratory Medicine Section, Copenhagen University Hospital-Hvidovre, Hvidovre 2650, Denmark
- Department of Respiratory Medicine, Zealand University Hospital Roskilde-Næstved, Næstved 4700, Denmark
| |
Collapse
|
35
|
Ding H, Simmich J, Vaezipour A, Andrews N, Russell T. Evaluation framework for conversational agents with artificial intelligence in health interventions: a systematic scoping review. J Am Med Inform Assoc 2024; 31:746-761. [PMID: 38070173 PMCID: PMC10873847 DOI: 10.1093/jamia/ocad222] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2023] [Revised: 11/04/2023] [Accepted: 11/24/2023] [Indexed: 02/18/2024] Open
Abstract
OBJECTIVES Conversational agents (CAs) with emerging artificial intelligence present new opportunities to assist in health interventions but are difficult to evaluate, deterring their applications in the real world. We aimed to synthesize existing evidence and knowledge and outline an evaluation framework for CA interventions. MATERIALS AND METHODS We conducted a systematic scoping review to investigate designs and outcome measures used in the studies that evaluated CAs for health interventions. We then nested the results into an overarching digital health framework proposed by the World Health Organization (WHO). RESULTS The review included 81 studies evaluating CAs in experimental (n = 59), observational (n = 15) trials, and other research designs (n = 7). Most studies (n = 72, 89%) were published in the past 5 years. The proposed CA-evaluation framework includes 4 evaluation stages: (1) feasibility/usability, (2) efficacy, (3) effectiveness, and (4) implementation, aligning with WHO's stepwise evaluation strategy. Across these stages, this article presents the essential evidence of different study designs (n = 8), sample sizes, and main evaluation categories (n = 7) with subcategories (n = 40). The main evaluation categories included (1) functionality, (2) safety and information quality, (3) user experience, (4) clinical and health outcomes, (5) costs and cost benefits, (6) usage, adherence, and uptake, and (7) user characteristics for implementation research. Furthermore, the framework highlighted the essential evaluation areas (potential primary outcomes) and gaps across the evaluation stages. DISCUSSION AND CONCLUSION This review presents a new framework with practical design details to support the evaluation of CA interventions in healthcare research. PROTOCOL REGISTRATION The Open Science Framework (https://osf.io/9hq2v) on March 22, 2021.
Collapse
Affiliation(s)
- Hang Ding
- RECOVER Injury Research Centre, Faculty of Health and Behavioural Sciences, The University of Queensland, Brisbane, QLD, Australia
- STARS Education and Research Alliance, Surgical Treatment and Rehabilitation Service (STARS), The University of Queensland and Metro North Health, Brisbane, QLD, Australia
| | - Joshua Simmich
- RECOVER Injury Research Centre, Faculty of Health and Behavioural Sciences, The University of Queensland, Brisbane, QLD, Australia
- STARS Education and Research Alliance, Surgical Treatment and Rehabilitation Service (STARS), The University of Queensland and Metro North Health, Brisbane, QLD, Australia
| | - Atiyeh Vaezipour
- RECOVER Injury Research Centre, Faculty of Health and Behavioural Sciences, The University of Queensland, Brisbane, QLD, Australia
- STARS Education and Research Alliance, Surgical Treatment and Rehabilitation Service (STARS), The University of Queensland and Metro North Health, Brisbane, QLD, Australia
| | - Nicole Andrews
- RECOVER Injury Research Centre, Faculty of Health and Behavioural Sciences, The University of Queensland, Brisbane, QLD, Australia
- STARS Education and Research Alliance, Surgical Treatment and Rehabilitation Service (STARS), The University of Queensland and Metro North Health, Brisbane, QLD, Australia
- The Tess Cramond Pain and Research Centre, Metro North Hospital and Health Service, Brisbane, QLD, Australia
- The Occupational Therapy Department, The Royal Brisbane and Women’s Hospital, Metro North Hospital and Health Service, Brisbane, QLD, Australia
| | - Trevor Russell
- RECOVER Injury Research Centre, Faculty of Health and Behavioural Sciences, The University of Queensland, Brisbane, QLD, Australia
- STARS Education and Research Alliance, Surgical Treatment and Rehabilitation Service (STARS), The University of Queensland and Metro North Health, Brisbane, QLD, Australia
| |
Collapse
|
36
|
Ni Z, Peng ML, Balakrishnan V, Tee V, Azwa I, Saifi R, Nelson LE, Vlahov D, Altice FL. Implementation of Chatbot Technology in Health Care: Protocol for a Bibliometric Analysis. JMIR Res Protoc 2024; 13:e54349. [PMID: 38228575 PMCID: PMC10905346 DOI: 10.2196/54349] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2023] [Revised: 12/07/2023] [Accepted: 01/16/2024] [Indexed: 01/18/2024] Open
Abstract
BACKGROUND Chatbots have the potential to increase people's access to quality health care. However, the implementation of chatbot technology in the health care system is unclear due to the scarce analysis of publications on the adoption of chatbot in health and medical settings. OBJECTIVE This paper presents a protocol of a bibliometric analysis aimed at offering the public insights into the current state and emerging trends in research related to the use of chatbot technology for promoting health. METHODS In this bibliometric analysis, we will select published papers from the databases of CINAHL, IEEE Xplore, PubMed, Scopus, and Web of Science that pertain to chatbot technology and its applications in health care. Our search strategy includes keywords such as "chatbot," "virtual agent," "virtual assistant," "conversational agent," "conversational AI," "interactive agent," "health," and "healthcare." Five researchers who are AI engineers and clinicians will independently review the titles and abstracts of selected papers to determine their eligibility for a full-text review. The corresponding author (ZN) will serve as a mediator to address any discrepancies and disputes among the 5 reviewers. Our analysis will encompass various publication patterns of chatbot research, including the number of annual publications, their geographic or institutional distribution, and the number of annual grants supporting chatbot research, and further summarize the methodologies used in the development of health-related chatbots, along with their features and applications in health care settings. Software tool VOSViewer (version 1.6.19; Leiden University) will be used to construct and visualize bibliometric networks. RESULTS The preparation for the bibliometric analysis began on December 3, 2021, when the research team started the process of familiarizing themselves with the software tools that may be used in this analysis, VOSViewer and CiteSpace, during which they consulted 3 librarians at the Yale University regarding search terms and tentative results. Tentative searches on the aforementioned databases yielded a total of 2340 papers. The official search phase started on July 27, 2023. Our goal is to complete the screening of papers and the analysis by February 15, 2024. CONCLUSIONS Artificial intelligence chatbots, such as ChatGPT (OpenAI Inc), have sparked numerous discussions within the health care industry regarding their impact on human health. Chatbot technology holds substantial promise for advancing health care systems worldwide. However, developing a sophisticated chatbot capable of precise interaction with health care consumers, delivering personalized care, and providing accurate health-related information and knowledge remain considerable challenges. This bibliometric analysis seeks to fill the knowledge gap in the existing literature on health-related chatbots, entailing their applications, the software used in their development, and their preferred functionalities among users. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) PRR1-10.2196/54349.
Collapse
Affiliation(s)
- Zhao Ni
- School of Nursing, Yale University, Orange, CT, United States
- Center for Interdisciplinary Research on AIDS, Yale University, New Haven, CT, United States
| | - Mary L Peng
- Department of Global Health and Social Medicine, Harvard Medical School, Harvard University, Boston, MA, United States
| | - Vimala Balakrishnan
- Department of Information Systems, Faculty of Computer Science and Information Technology, Unversity of Malaya, Kuala Lumpur, Malaysia
| | - Vincent Tee
- Centre of Excellence for Research in AIDS, Faculty of Medicine, University of Malaya, Kuala Lumpur, Malaysia
| | - Iskandar Azwa
- Centre of Excellence for Research in AIDS, Faculty of Medicine, University of Malaya, Kuala Lumpur, Malaysia
- Infectious Disease Unit, Faculty of Medicine, University of Malaya, Kuala Lumpur, Malaysia
| | - Rumana Saifi
- Centre of Excellence for Research in AIDS, Faculty of Medicine, University of Malaya, Kuala Lumpur, Malaysia
| | - LaRon E Nelson
- School of Nursing, Yale University, Orange, CT, United States
- Center for Interdisciplinary Research on AIDS, Yale University, New Haven, CT, United States
| | - David Vlahov
- School of Nursing, Yale University, Orange, CT, United States
- Center for Interdisciplinary Research on AIDS, Yale University, New Haven, CT, United States
| | - Frederick L Altice
- Center for Interdisciplinary Research on AIDS, Yale University, New Haven, CT, United States
- Centre of Excellence for Research in AIDS, Faculty of Medicine, University of Malaya, Kuala Lumpur, Malaysia
- Section of Infectious Disease, Department of Internal Medicine, Yale School of Medicine, New Haven, CT, United States
- Division of Epidemiology of Microbial Diseases, Yale School of Public Health, New Haven, CT, United States
| |
Collapse
|
37
|
Wang L, Bi W, Zhao S, Ma Y, Lv L, Meng C, Fu J, Lv H. Investigating the Impact of Prompt Engineering on the Performance of Large Language Models for Standardizing Obstetric Diagnosis Text: Comparative Study. JMIR Form Res 2024; 8:e53216. [PMID: 38329787 PMCID: PMC10884897 DOI: 10.2196/53216] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Revised: 12/25/2023] [Accepted: 01/11/2024] [Indexed: 02/09/2024] Open
Abstract
BACKGROUND The accumulation of vast electronic medical records (EMRs) through medical informatization creates significant research value, particularly in obstetrics. Diagnostic standardization across different health care institutions and regions is vital for medical data analysis. Large language models (LLMs) have been extensively used for various medical tasks. Prompt engineering is key to use LLMs effectively. OBJECTIVE This study aims to evaluate and compare the performance of LLMs with various prompt engineering techniques on the task of standardizing obstetric diagnostic terminology using real-world obstetric data. METHODS The paper describes a 4-step approach used for mapping diagnoses in electronic medical records to the International Classification of Diseases, 10th revision, observation domain. First, similarity measures were used for mapping the diagnoses. Second, candidate mapping terms were collected based on similarity scores above a threshold, to be used as the training data set. For generating optimal mapping terms, we used two LLMs (ChatGLM2 and Qwen-14B-Chat [QWEN]) for zero-shot learning in step 3. Finally, a performance comparison was conducted by using 3 pretrained bidirectional encoder representations from transformers (BERTs), including BERT, whole word masking BERT, and momentum contrastive learning with BERT (MC-BERT), for unsupervised optimal mapping term generation in the fourth step. RESULTS LLMs and BERT demonstrated comparable performance at their respective optimal levels. LLMs showed clear advantages in terms of performance and efficiency in unsupervised settings. Interestingly, the performance of the LLMs varied significantly across different prompt engineering setups. For instance, when applying the self-consistency approach in QWEN, the F1-score improved by 5%, with precision increasing by 7.9%, outperforming the zero-shot method. Likewise, ChatGLM2 delivered similar rates of accurately generated responses. During the analysis, the BERT series served as a comparative model with comparable results. Among the 3 models, MC-BERT demonstrated the highest level of performance. However, the differences among the versions of BERT in this study were relatively insignificant. CONCLUSIONS After applying LLMs to standardize diagnoses and designing 4 different prompts, we compared the results to those generated by the BERT model. Our findings indicate that QWEN prompts largely outperformed the other prompts, with precision comparable to that of the BERT model. These results demonstrate the potential of unsupervised approaches in improving the efficiency of aligning diagnostic terms in daily research and uncovering hidden information values in patient data.
Collapse
Affiliation(s)
| | | | | | - Yinyao Ma
- The People's Hospital of Guangxi Zhuang Autonomous Region, Guangxi, China
| | | | | | | | | |
Collapse
|
38
|
Yin R, Neyens DM. Examining how information presentation methods and a chatbot impact the use and effectiveness of electronic health record patient portals: An exploratory study. PATIENT EDUCATION AND COUNSELING 2024; 119:108055. [PMID: 37976665 DOI: 10.1016/j.pec.2023.108055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/07/2023] [Revised: 11/02/2023] [Accepted: 11/03/2023] [Indexed: 11/19/2023]
Abstract
OBJECTIVES Examining information presentation strategies that may facilitate patient education through patient portals is important for effective health education. METHODS A randomized exploratory study evaluated information presentation (text or videos) and a chatbot in patient education and examined several performance and outcome variables (e.g., search duration, Decisional Conflict Scale, and eye-tracking measures), along with a simple descriptive qualitative content analysis of the transcript of chatbot. RESULTS Of the 92 participants, those within the text conditions (n = 46, p < 0.001), had chatbot experiences (B =-74.85, p = 0.046), knew someone with IBD (B =-98.66, p = 0.039), and preferred to engage in medical decision-making (B =102.32, p = 0.006) were more efficient in information-searching. Participants with videos spent longer in information-searching (mean=666.5 (SD=171.6) VS 480.3 (SD=159.5) seconds, p < 0.001) but felt more informed (mean score=18.8 (SD=17.6) VS 27.4 (SD=18.9), p = 0.027). The participants' average eye fixation duration with videos was significantly higher (mean= 473.8 ms, SD=52.9, p < 0.001). CONCLUSIONS Participants in video conditions were less efficient but more effective in information seeking. Exploring the trade-offs between efficiency and effectiveness for user interface designs is important to appropriately deliver education within patient portals. PRACTICE IMPLICATIONS This study suggests that user interface designs and chatbots impact health information's efficiency and effectiveness.
Collapse
Affiliation(s)
- Rong Yin
- Department of Industrial Engineering, Pittsburg Institute, Sichuan University, Chengdu, Sichuan, China.
| | - David M Neyens
- Department of Industrial Engineering, Department of Bioengineering, Clemson University, Clemson, SC, USA
| |
Collapse
|
39
|
Reis F, Lenz C. Performance of Artificial Intelligence (AI)-Powered Chatbots in the Assessment of Medical Case Reports: Qualitative Insights From Simulated Scenarios. Cureus 2024; 16:e53899. [PMID: 38465163 PMCID: PMC10925004 DOI: 10.7759/cureus.53899] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/08/2024] [Indexed: 03/12/2024] Open
Abstract
Introduction With the expanding awareness and use of AI-powered chatbots, it seems possible that an increasing number of people could use them to assess and evaluate their medical symptoms. If chatbots are used for this purpose, that have not previously undergone a thorough medical evaluation for this specific use, various risks might arise. The aim of this study is to analyze and compare the performance of popular chatbots in differentiating between severe and less critical medical symptoms described from a patient's perspective and to examine the variations in substantive medical assessment accuracy and empathetic communication style among the chatbots' responses. Materials and methods Our study compared three different AI-supported chatbots - OpenAI's ChatGPT 3.5, Microsoft's Bing Chat, and Inflection's Pi AI. Three exemplary case reports for medical emergencies as well as three cases without an urgent reason for an emergency medical admission were constructed and analyzed. Each case report was accompanied by identical questions concerning the most likely suspected diagnosis and the urgency of an immediate medical evaluation. The respective answers of the chatbots were qualitatively compared with each other regarding the medical accuracy of the differential diagnoses mentioned and the conclusions drawn, as well as regarding patient-oriented and empathetic language. Results All examined chatbots were capable of providing medically plausible and probable diagnoses and classifying situations as acute or less critical. However, their responses varied slightly in the level of their urgency assessment. Clear differences could be seen in the level of detail of the differential diagnoses, the overall length of the answers, and how the chatbot dealt with the challenge of being confronted with medical issues. All given answers were comparable in terms of empathy level and comprehensibility. Conclusion Even AI chatbots that are not designed for medical applications already offer substantial guidance in assessing typical medical emergency indications but should always be provided with a disclaimer. In responding to medical queries, characteristic differences emerge among chatbots in the extent and style of their respective answers. Given the lack of medical supervision of many established chatbots, subsequent studies, and experiences are essential to clarify whether a more extensive use of these chatbots for medical concerns will have a positive impact on healthcare or rather pose major medical risks.
Collapse
Affiliation(s)
- Florian Reis
- Medical Affairs, Pfizer Pharma GmbH, Berlin, DEU
| | | |
Collapse
|
40
|
Marey A, Saad AM, Killeen BD, Gomez C, Tregubova M, Unberath M, Umair M. Generative Artificial Intelligence: Enhancing Patient Education in Cardiovascular Imaging. BJR Open 2024; 6:tzae018. [PMID: 39086557 PMCID: PMC11290812 DOI: 10.1093/bjro/tzae018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2024] [Revised: 04/18/2024] [Accepted: 07/10/2024] [Indexed: 08/02/2024] Open
Abstract
Cardiovascular disease (CVD) is a major cause of mortality worldwide, especially in resource-limited countries with limited access to healthcare resources. Early detection and accurate imaging are vital for managing CVD, emphasizing the significance of patient education. Generative artificial intelligence (AI), including algorithms to synthesize text, speech, images, and combinations thereof given a specific scenario or prompt, offers promising solutions for enhancing patient education. By combining vision and language models, generative AI enables personalized multimedia content generation through natural language interactions, benefiting patient education in cardiovascular imaging. Simulations, chat-based interactions, and voice-based interfaces can enhance accessibility, especially in resource-limited settings. Despite its potential benefits, implementing generative AI in resource-limited countries faces challenges like data quality, infrastructure limitations, and ethical considerations. Addressing these issues is crucial for successful adoption. Ethical challenges related to data privacy and accuracy must also be overcome to ensure better patient understanding, treatment adherence, and improved healthcare outcomes. Continued research, innovation, and collaboration in generative AI have the potential to revolutionize patient education. This can empower patients to make informed decisions about their cardiovascular health, ultimately improving healthcare outcomes in resource-limited settings.
Collapse
Affiliation(s)
- Ahmed Marey
- Alexandria University Faculty of Medicine, Alexandria, 21521, Egypt
| | | | | | - Catalina Gomez
- Department of Computer Science, Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, 21218, United States
| | - Mariia Tregubova
- Department of Radiology, Amosov National Institute of Cardiovascular Surgery, Kyiv, 02000, Ukraine
| | - Mathias Unberath
- Department of Computer Science, Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, 21218, United States
| | - Muhammad Umair
- Russell H. Morgan Department of Radiology and Radiological Sciences, The Johns Hopkins Hospital, Baltimore, MD, 21205, United States
| |
Collapse
|
41
|
Talyshinskii A, Naik N, Hameed BMZ, Juliebø-Jones P, Somani BK. Potential of AI-Driven Chatbots in Urology: Revolutionizing Patient Care Through Artificial Intelligence. Curr Urol Rep 2024; 25:9-18. [PMID: 37723300 PMCID: PMC10787686 DOI: 10.1007/s11934-023-01184-3] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/05/2023] [Indexed: 09/20/2023]
Abstract
PURPOSE OF REVIEW Artificial intelligence (AI) chatbots have emerged as a potential tool to transform urology by improving patient care and physician efficiency. With an emphasis on their potential advantages and drawbacks, this literature review offers a thorough assessment of the state of AI-driven chatbots in urology today. RECENT FINDINGS The capacity of AI-driven chatbots in urology to give patients individualized and timely medical advice is one of its key advantages. Chatbots can help patients prioritize their symptoms and give advice on the best course of treatment. By automating administrative duties and offering clinical decision support, chatbots can also help healthcare providers. Before chatbots are widely used in urology, there are a few issues that need to be resolved. The precision of chatbot diagnoses and recommendations might be impacted by technical constraints like system errors and flaws. Additionally, issues regarding the security and privacy of patient data must be resolved, and chatbots must adhere to all applicable laws. Important issues that must be addressed include accuracy and dependability because any mistakes or inaccuracies could seriously harm patients. The final obstacle is resistance from patients and healthcare professionals who are hesitant to use new technology or who value in-person encounters. AI-driven chatbots have the potential to significantly improve urology care and efficiency. However, it is essential to thoroughly test and ensure the accuracy of chatbots, address privacy and security concerns, and design user-friendly chatbots that can integrate into existing workflows. By exploring various scenarios and examining the current literature, this review provides an analysis of the prospects and limitations of implementing chatbots in urology.
Collapse
Affiliation(s)
- Ali Talyshinskii
- Department of Urology, Astana Medical University, Astana, Kazakhstan
| | - Nithesh Naik
- Department of Mechanical and Industrial Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, Karnataka, 576104, India
| | - B M Zeeshan Hameed
- Department of Urology, Father Muller Medical College, Mangalore, Karnataka, India
| | - Patrick Juliebø-Jones
- Department of Urology, Haukeland University Hospital, Bergen, Norway.
- Department of Clinical Medicine, University of Bergen, Bergen, Norway.
| | | |
Collapse
|
42
|
Cook D, Peters D, Moradbakhti L, Su T, Da Re M, Schuller BW, Quint J, Wong E, Calvo RA. A text-based conversational agent for asthma support: Mixed-methods feasibility study. Digit Health 2024; 10:20552076241258276. [PMID: 38894942 PMCID: PMC11185032 DOI: 10.1177/20552076241258276] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2024] [Accepted: 05/13/2024] [Indexed: 06/21/2024] Open
Abstract
Objective Millions of people in the UK have asthma, yet 70% do not access basic care, leading to the largest number of asthma-related deaths in Europe. Chatbots may extend the reach of asthma support and provide a bridge to traditional healthcare. This study evaluates 'Brisa', a chatbot designed to improve asthma patients' self-assessment and self-management. Methods We recruited 150 adults with an asthma diagnosis to test our chatbot. Participants were recruited over three waves through social media and a research recruitment platform. Eligible participants had access to 'Brisa' via a WhatsApp or website version for 28 days and completed entry and exit questionnaires to evaluate user experience and asthma control. Weekly symptom tracking, user interaction metrics, satisfaction measures, and qualitative feedback were utilised to evaluate the chatbot's usability and potential effectiveness, focusing on changes in asthma control and self-reported behavioural improvements. Results 74% of participants engaged with 'Brisa' at least once. High task completion rates were observed: asthma attack risk assessment (86%), voice recording submission (83%) and asthma control tracking (95.5%). Post use, an 8% improvement in asthma control was reported. User satisfaction surveys indicated positive feedback on helpfulness (80%), privacy (87%), trustworthiness (80%) and functionality (84%) but highlighted a need for improved conversational depth and personalisation. Conclusions The study indicates that chatbots are effective for asthma support, demonstrated by the high usage of features like risk assessment and control tracking, as well as a statistically significant improvement in asthma control. However, lower satisfaction in conversational flexibility highlights rising expectations for chatbot fluency, influenced by advanced models like ChatGPT. Future health-focused chatbots must balance conversational capability with accuracy and safety to maintain engagement and effectiveness.
Collapse
Affiliation(s)
- Darren Cook
- Dyson School of Design Engineering, Imperial College London, London, UK
| | - Dorian Peters
- Dyson School of Design Engineering, Imperial College London, London, UK
| | - Laura Moradbakhti
- Dyson School of Design Engineering, Imperial College London, London, UK
| | - Ting Su
- Dyson School of Design Engineering, Imperial College London, London, UK
| | - Marco Da Re
- Dyson School of Design Engineering, Imperial College London, London, UK
| | - Bjorn W. Schuller
- Dyson School of Design Engineering, Imperial College London, London, UK
| | | | - Ernie Wong
- Imperial College Healthcare NHS Trust, London, UK
| | - Rafael A. Calvo
- Dyson School of Design Engineering, Imperial College London, London, UK
| |
Collapse
|
43
|
Yang Y, Liu S, Lei P, Huang Z, Liu L, Tan Y. Assessing usability of intelligent guidance chatbots in Chinese hospitals: Cross-sectional study. Digit Health 2024; 10:20552076241260504. [PMID: 38854920 PMCID: PMC11159538 DOI: 10.1177/20552076241260504] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/23/2024] [Indexed: 06/11/2024] Open
Abstract
Objective This study aimed to assessing usability of intelligent guidance chatbots (IGCs) in Chinese hospitals. Methods A cross-sectional study based on expert survey was conducted between August to December 2023. The survey assessed the usability of chatbots in 590 Chinese hospitals. One-way ANOVA was used to analyze the impact of the number of functions, human-like characteristics, number of outpatients, and staff size on the usability of the IGCs. Results The results indicate that there are 273 (46.27%) hospitals scoring above 45 points. In terms of function development, 581(98.47%) hospitals have set the number of functions between 1 and 5. Besides, 350 hospitals have excellent function implementation, accounting for 59.32%. In terms of the IGC's human-like characteristic, 220 hospitals have both an avatar and a nickname. Results of One-way ANOVA show that, the number of functions(F = 202.667, P < 0.001), human-like characteristics(F = 372.29, P < 0.001), staff size(F = 9.846, P < 0.001), and the number of outpatients(F = 5.709, P = 0.004) have significant impact on the usability of hospital IGCs. Conclusions This study found that the differences in the usability of hospital IGCs at various levels of the number of functions, human-like characteristics, number of outpatients, and staff size. These findings provide insights for deploying hospital IGCs and can inform improvements in patient's experience and adoption of chatbots.
Collapse
Affiliation(s)
- Yanni Yang
- School of Literature and Media, China Three Gorges University, Yichang, Hubei, China
| | - Siyang Liu
- School of Literature and Media, China Three Gorges University, Yichang, Hubei, China
| | - Ping Lei
- Department of Orthopedics, Zhijiang Hospital of Traditional Chinese Medicine, Zhijiang, Hubei, China
| | - Zhengwei Huang
- College of Economics & Management, China Three Gorges University, Yichang, Hubei, China
| | - Lu Liu
- College of Electrical Engineering & New Energy, China Three Gorges University, Yichang, Hubei, China
| | - Yiting Tan
- School of Literature and Media, China Three Gorges University, Yichang, Hubei, China
| |
Collapse
|
44
|
Singh A, Schooley B, Patel N. Effects of User-Reported Risk Factors and Follow-Up Care Activities on Satisfaction With a COVID-19 Chatbot: Cross-Sectional Study. JMIR Mhealth Uhealth 2023; 11:e43105. [PMID: 38096007 PMCID: PMC10727483 DOI: 10.2196/43105] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Revised: 06/19/2023] [Accepted: 11/03/2023] [Indexed: 12/18/2023] Open
Abstract
BACKGROUND The COVID-19 pandemic influenced many to consider methods to reduce human contact and ease the burden placed on health care workers. Conversational agents or chatbots are a set of technologies that may aid with these challenges. They may provide useful interactions for users, potentially reducing the health care worker burden while increasing user satisfaction. Research aims to understand these potential impacts of chatbots and conversational recommender systems and their associated design features. OBJECTIVE The objective of this study was to evaluate user perceptions of the helpfulness of an artificial intelligence chatbot that was offered free to the public in response to COVID-19. The chatbot engaged patients and provided educational information and the opportunity to report symptoms, understand personal risks, and receive referrals for care. METHODS A cross-sectional study design was used to analyze 82,222 chats collected from patients in South Carolina seeking services from the Prisma Health system. Chi-square tests and multinomial logistic regression analyses were conducted to assess the relationship between reported risk factors and perceived chat helpfulness using chats started between April 24, 2020, and April 21, 2022. RESULTS A total of 82,222 chat series were started with at least one question or response on record; 53,805 symptom checker questions with at least one COVID-19-related activity series were completed, with 5191 individuals clicking further to receive a virtual video visit and 2215 clicking further to make an appointment with a local physician. Patients who were aged >65 years (P<.001), reported comorbidities (P<.001), had been in contact with a person with COVID-19 in the last 14 days (P<.001), and responded to symptom checker questions that placed them at a higher risk of COVID-19 (P<.001) were 1.8 times more likely to report the chat as helpful than those who reported lower risk factors. Users who engaged with the chatbot to conduct a series of activities were more likely to find the chat helpful (P<.001), including seeking COVID-19 information (3.97-4.07 times), in-person appointments (2.46-1.99 times), telehealth appointments with a nearby provider (2.48-1.9 times), or vaccination (2.9-3.85 times) compared with those who did not perform any of these activities. CONCLUSIONS Chatbots that are designed to target high-risk user groups and provide relevant actionable items may be perceived as a helpful approach to early contact with the health system for assessing communicable disease symptoms and follow-up care options at home before virtual or in-person contact with health care providers. The results identified and validated significant design factors for conversational recommender systems, including triangulating a high-risk target user population and providing relevant actionable items for users to choose from as part of user engagement.
Collapse
Affiliation(s)
- Akanksha Singh
- Department of Health Services Administration, School of Health Professions, University of Alabama at Birmingham, Birmingham, AL, United States
| | - Benjamin Schooley
- IT & Cybersecurity, Department of Electrical and Computer Engineering, Brigham Young University, Provo, UT, United States
| | - Nitin Patel
- Hackensack Meridian Health, Hackensack, NJ, United States
| |
Collapse
|
45
|
Zalake M. Doctors' perceptions of using their digital twins in patient care. Sci Rep 2023; 13:21693. [PMID: 38066016 PMCID: PMC10709415 DOI: 10.1038/s41598-023-48747-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Accepted: 11/29/2023] [Indexed: 12/18/2023] Open
Abstract
Recent Artificial Intelligence (AI) advancements have facilitated tools capable of generating digital twins of real human faces and voices for interactive communication. In this research, we explore utilizing Digital Twins of Doctors (DTDs) in healthcare because using a doctor's identity can provide benefits like enhancing the credibility of the health information delivered using computers. DTDs are computer-controlled AI-generated digital replicas of doctors that closely resemble their characteristics. However, there exist limitations, including the social implications of using a doctor's identity, potential negative impacts on doctor-patient communication, and liability concerns. To ensure a comprehensive understanding of DTD usage in healthcare before widespread adoption, systematic research is essential. As a step towards this direction, in this qualitative research, we report findings from 13 semi-structured interviews with doctors. Our findings indicate that doctors believe DTDs offer benefits by saving doctors' time through the efficient delivery of repetitive information and personalizing patient care. Moreover, while using a doctor's identity can enhance credibility, it also raises concerns about using a doctor's identity to spread potential misinformation. These findings contribute by informing future researchers about doctors' perspectives on utilizing DTDs in healthcare, guiding the development of effective implementation strategies for responsible DTD integration into healthcare.
Collapse
Affiliation(s)
- Mohan Zalake
- Biomedical and Health Information Sciences, University of Illinois Chicago, Chicago, 60601, USA.
| |
Collapse
|
46
|
Beavers J, Schell RF, VanCleave H, Dillon RC, Simmons A, Chen H, Chen Q, Anders S, Weinger MB, Nelson SD. Evaluation of inpatient medication guidance from an artificial intelligence chatbot. Am J Health Syst Pharm 2023; 80:1822-1829. [PMID: 37611187 DOI: 10.1093/ajhp/zxad193] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2023] [Indexed: 08/25/2023] Open
Abstract
PURPOSE To analyze the clinical completeness, correctness, usefulness, and safety of chatbot and medication database responses to everyday inpatient medication-use questions. METHODS We evaluated the responses from an artificial intelligence chatbot, a medication database, and clinical pharmacists to 200 real-world medication-use questions. Answer quality was rated by a blinded group of pharmacists, providers, and nurses. Chatbot and medication database responses were deemed "acceptable" if the mean reviewer rating was within 3 points of the mean rating for pharmacists' answers. We used descriptive statistics for reviewer ratings and Kendall's coefficient to evaluate interrater agreement. RESULTS The medication database generated responses to 194 (97%) questions, with 88% considered acceptable for clinical correctness, 76% considered acceptable for completeness, 83% considered acceptable for safety, and 81% considered acceptable for usefulness compared to pharmacists' answers. The chatbot responded to only 160 (80%) questions, with 85% considered acceptable for clinical correctness, 65% considered acceptable for completeness, 71% considered acceptable for safety, and 68% considered acceptable for usefulness. CONCLUSION Traditional search methods using a drug database provide more clinically correct, complete, safe, and useful answers than a chatbot. When the chatbot generated a response, the clinical correctness was similar to that of a drug database; however, it was not rated as favorably for clinical completeness, safety, or usefulness. Our results highlight the need for ongoing training and continued improvements to artificial intelligence chatbots for them to be incorporated reliably into the clinical workflow. With continued improvement in chatbot functionality, chatbots could be a useful pharmacist adjunct, providing healthcare providers with quick and reliable answers to medication-use questions.
Collapse
Affiliation(s)
- Jennifer Beavers
- Department of Pharmaceutical Services, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Ryan F Schell
- Department of Pharmaceutical Services, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Halden VanCleave
- Department of Pharmaceutical Services, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Ryan C Dillon
- Department of Pharmaceutical Services, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Austin Simmons
- Quality Department, Drug Diversion Support, Lifepoint Health, Brentwood, TN, USA
| | - Huiding Chen
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Qingxia Chen
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Shilo Anders
- Departments of Anesthesiology and Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Matthew B Weinger
- Departments of Anesthesiology, Biomedical Informatics, and Medical Education, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Scott D Nelson
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA
| |
Collapse
|
47
|
Moldt JA, Festl-Wietek T, Madany Mamlouk A, Nieselt K, Fuhl W, Herrmann-Werner A. Chatbots for future docs: exploring medical students' attitudes and knowledge towards artificial intelligence and medical chatbots. MEDICAL EDUCATION ONLINE 2023; 28:2182659. [PMID: 36855245 PMCID: PMC9979998 DOI: 10.1080/10872981.2023.2182659] [Citation(s) in RCA: 18] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/15/2022] [Revised: 02/06/2023] [Accepted: 02/16/2023] [Indexed: 06/18/2023]
Abstract
Artificial intelligence (AI) in medicine and digital assistance systems such as chatbots will play an increasingly important role in future doctor - patient communication. To benefit from the potential of this technical innovation and ensure optimal patient care, future physicians should be equipped with the appropriate skills. Accordingly, a suitable place for the management and adaptation of digital assistance systems must be found in the medical education curriculum. To determine the existing levels of knowledge of medical students about AI chatbots in particular in the healthcare setting, this study surveyed medical students of the University of Luebeck and the University Hospital of Tuebingen. Using standardized quantitative questionnaires and qualitative analysis of group discussions, the attitudes of medical students toward AI and chatbots in medicine were investigated. From this, relevant requirements for the future integration of AI into the medical curriculum could be identified. The aim was to establish a basic understanding of the opportunities, limitations, and risks, as well as potential areas of application of the technology. The participants (N = 12) were able to develop an understanding of how AI and chatbots will affect their future daily work. Although basic attitudes toward the use of AI were positive, the students also expressed concerns. There were high levels of agreement regarding the use of AI in administrative settings (83.3%) and research with health-related data (91.7%). However, participants expressed concerns that data protection may be insufficiently guaranteed (33.3%) and that they might be increasingly monitored at work in the future (58.3%). The evaluations indicated that future physicians want to engage more intensively with AI in medicine. In view of future developments, AI and data competencies should be taught in a structured way during the medical curriculum and integrated into curricular teaching.
Collapse
Affiliation(s)
| | | | - Amir Madany Mamlouk
- Institute for Neuro- and Bioinformatics, University of Luebeck, Luebeck, Germany
| | - Kay Nieselt
- Institute for Bioinformatics and Medical Informatics, University of Tuebingen, Germany
| | - Wolfgang Fuhl
- Institute for Bioinformatics and Medical Informatics, University of Tuebingen, Germany
| | - Anne Herrmann-Werner
- University of Tuebingen, Tuebingen, Germany
- Department of Internal Medicine VI/Psychosomatic Medicine and Psychotherapy, University Hospital Tuebingen, Tuebingen, Germany
| |
Collapse
|
48
|
De Silva L, Baysari M, Keep M, Kench P, Clarke J. Patient initiated radiology requests: proof of wellness through images. Aust J Prim Health 2023; 29:670-678. [PMID: 37614071 DOI: 10.1071/py22247] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2022] [Accepted: 07/24/2023] [Indexed: 08/25/2023]
Abstract
BACKGROUND Traditionally, general practitioners (GPs) have initiated the need for, and ordered, radiological tests. With the emergence of consumer-centred care, patients have started to request scans from doctors on their own initiative. Consumeristic health care has shifted the patient-doctor dyadic relationship, with GPs trending towards accommodating patients' requests. METHODS A mixed method analysis was conducted using a survey instrument with open ended questions and concurrent interviews to explore participants' responses from their requests for radiological studies from GPs. Themes emerging from both qualitative and quantitative methodologies were mapped onto the Andersen Newman Model (ANM). RESULTS Data were analysed for 'predisposing,' 'need' and 'enabling' elements of the ANM model and were correspondingly mapped to patient's requests for radiological referrals according to the elements of the ANM. Participants expressed anxiety about their health, were confident in the types of radiological scans they desired and typically indicated the need for evidence of good health. Their desire for such requested scans was often enabled through prior exposure to health information and the experience of specific symptoms. Requests came with the expectation of validation, and if these requests were denied, participants indicated that they would seek another doctor who would oblige. CONCLUSIONS In our modest study of Australian patients, participants were well informed about their health. Exposure to information seems to create a sense of anxiousness prior to visiting the doctor. Individuals sought visual proof of wellness through imaging, and doctors in return often accommodated patient requests for radiological studies to appease patients' needs and to maintain workflow.
Collapse
Affiliation(s)
- Lizzie De Silva
- Discipline of Medical Imaging Science, Sydney School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Susan Wakil Health Building, Camperdown Campus, Western Avenue, Camperdown, NSW 2006, Australia
| | - Melissa Baysari
- Biomedical Informatics and Digital Health, Faculty of Medicine and Health, Charles Perkins Centre D17, The University of Sydney, Sydney, NSW 2006, Australia
| | - Melanie Keep
- Sydney School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Susan Wakil Health Building D18, Camperdown, NSW 2006, Australia
| | - Peter Kench
- Discipline of Medical Imaging Science, Sydney School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Susan Wakil Health Building, Camperdown Campus, Western Avenue, Camperdown, NSW 2006, Australia
| | - Jillian Clarke
- Discipline of Medical Imaging Science, Sydney School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Susan Wakil Health Building, Camperdown Campus, Western Avenue, Camperdown, NSW 2006, Australia
| |
Collapse
|
49
|
Morato JEM, do Nascimento JWA, Roque GDSL, de Souza RR, Santos ICRV. Development, Validation, and Usability of the Chatbot ESTOMABOT to Promote Self-care of People With Intestinal Ostomy. Comput Inform Nurs 2023; 41:1037-1045. [PMID: 37725781 DOI: 10.1097/cin.0000000000001075] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/21/2023]
Abstract
This study aimed to describe the process of construction, validation, and usability of the chatbot ESTOMABOT to assist in the self-care of patients with intestinal ostomies. Methodological research was conducted in three phases: construction, validation, and usability. The first stage corresponded to the elaboration of a script through a literature review, and the second stage corresponded to face and content validation through a panel of enterostomal therapy nurses. In the third phase, the usability of ESTOMABOT was assessed with the participation of surgical clinic nurses, patients with intestinal elimination ostomies, and information technology professionals, using the System Usability Scale. The ESTOMABOT content reached excellent criteria of adequacy, with percentages of agreement equal to or greater than 90%, which were considered adequate, relevant, and representative. The evaluation of the content validity of the script using the scale content validity index/average proportion method reached a result above 0.90, and the Fleiss κ was excellent ( P < .05). The overall usability score of the chatbot was 81.5, demonstrating excellent usability. The script, developed and incorporated into the ESTOMABOT prototype, achieved satisfactory content validity. The usability of the chatbot was considered to be good, thereby increasing the credibility of the instrument.
Collapse
Affiliation(s)
- Jéssica Emanuela Mendes Morato
- Author Affiliations: Nursing Department, University of Pernambuco (Dr Morato and Dr Santos); Informatics Center, Federal University of Pernambuco (Dr do Nascimento and Dr Roque); and Catholic University of Pernambuco (Dr de Souza), Recife, Brazil
| | | | | | | | | |
Collapse
|
50
|
Vo V, Chen G, Aquino YSJ, Carter SM, Do QN, Woode ME. Multi-stakeholder preferences for the use of artificial intelligence in healthcare: A systematic review and thematic analysis. Soc Sci Med 2023; 338:116357. [PMID: 37949020 DOI: 10.1016/j.socscimed.2023.116357] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2023] [Revised: 09/04/2023] [Accepted: 10/24/2023] [Indexed: 11/12/2023]
Abstract
INTRODUCTION Despite the proliferation of Artificial Intelligence (AI) technology over the last decade, clinician, patient, and public perceptions of its use in healthcare raise a number of ethical, legal and social questions. We systematically review the literature on attitudes towards the use of AI in healthcare from patients, the general public and health professionals' perspectives to understand these issues from multiple perspectives. METHODOLOGY A search for original research articles using qualitative, quantitative, and mixed methods published between 1 Jan 2001 to 24 Aug 2021 was conducted on six bibliographic databases. Data were extracted and classified into different themes representing views on: (i) knowledge and familiarity of AI, (ii) AI benefits, risks, and challenges, (iii) AI acceptability, (iv) AI development, (v) AI implementation, (vi) AI regulations, and (vii) Human - AI relationship. RESULTS The final search identified 7,490 different records of which 105 publications were selected based on predefined inclusion/exclusion criteria. While the majority of patients, the general public and health professionals generally had a positive attitude towards the use of AI in healthcare, all groups indicated some perceived risks and challenges. Commonly perceived risks included data privacy; reduced professional autonomy; algorithmic bias; healthcare inequities; and greater burnout to acquire AI-related skills. While patients had mixed opinions on whether healthcare workers suffer from job loss due to the use of AI, health professionals strongly indicated that AI would not be able to completely replace them in their professions. Both groups shared similar doubts about AI's ability to deliver empathic care. The need for AI validation, transparency, explainability, and patient and clinical involvement in the development of AI was emphasised. To help successfully implement AI in health care, most participants envisioned that an investment in training and education campaigns was necessary, especially for health professionals. Lack of familiarity, lack of trust, and regulatory uncertainties were identified as factors hindering AI implementation. Regarding AI regulations, key themes included data access and data privacy. While the general public and patients exhibited a willingness to share anonymised data for AI development, there remained concerns about sharing data with insurance or technology companies. One key domain under this theme was the question of who should be held accountable in the case of adverse events arising from using AI. CONCLUSIONS While overall positivity persists in attitudes and preferences toward AI use in healthcare, some prevalent problems require more attention. There is a need to go beyond addressing algorithm-related issues to look at the translation of legislation and guidelines into practice to ensure fairness, accountability, transparency, and ethics in AI.
Collapse
Affiliation(s)
- Vinh Vo
- Centre for Health Economics, Monash University, Australia.
| | - Gang Chen
- Centre for Health Economics, Monash University, Australia
| | - Yves Saint James Aquino
- Australian Centre for Health Engagement, Evidence and Values, School of Health and Soceity, University of Wollongong, Australia
| | - Stacy M Carter
- Australian Centre for Health Engagement, Evidence and Values, School of Health and Soceity, University of Wollongong, Australia
| | - Quynh Nga Do
- Department of Economics, Monash University, Australia
| | - Maame Esi Woode
- Centre for Health Economics, Monash University, Australia; Monash Data Futures Research Institute, Australia
| |
Collapse
|