1
|
Haak F, Müller PC, Kollmar O, Billeter AT, Lavanchy JL, Wiencierz A, Müller-Stich BP, von Strauss Und Torney M. Digital standardization in liver surgery through a surgical workflow management system: A pilot randomized controlled trial. Langenbecks Arch Surg 2025; 410:96. [PMID: 40069334 PMCID: PMC11897067 DOI: 10.1007/s00423-025-03634-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2024] [Accepted: 02/02/2025] [Indexed: 03/15/2025]
Abstract
INTRODUCTION Surgical process models (SPM) are simplified representations of operations and their visualization by surgical workflow management systems (SWMS), and offer a solution to enhance communication and workflow. METHODS A 1:1 randomized controlled trial was conducted. A SPM consisting of six surgical steps was defined to represent the surgical procedure. The primary outcome, termed "deviation" measured the difference between actual and planned surgery duration. Secondary outcomes included stress levels of the operating team and complications. Analyses employed Welch t-tests and linear regression models. RESULTS 18 procedures were performed with a SWMS and 18 without. The deviation showed no significant difference between the intervention and control group. Stress levels (TLX score) of the team remained largely unaffected. Duration of operation steps defined by SPM allows a classification of all liver procedures into three phases: The Start Phase (low IQR of operation time), the Main Phase (high IQR of operation time) and the End Phase (low IQR of operation time). CONCLUSION This study presents a novel SPM for open liver resections visualized by a SWMS. No significant reduction of deviations from planned operation time was observed with system use. Stress levels of the operation team were not influenced by the SWMS.
Collapse
Affiliation(s)
- Fabian Haak
- Clarunis, Department of Visceral Surgery, University Digestive Health Care Center, St. Clara Hospital and University Hospital Basel, Petersgraben 4, 4031, Basel, Switzerland.
- Department of Visceral, Transplant, Thoracic and Vascular Surgery, Division of Hepatobiliary Surgery and Visceral Transplant Surgery, University Hospital Leipzig, Leipzig , Germany.
| | - Philip C Müller
- Clarunis, Department of Visceral Surgery, University Digestive Health Care Center, St. Clara Hospital and University Hospital Basel, Petersgraben 4, 4031, Basel, Switzerland
| | - Otto Kollmar
- Department of General, Visceral, Vascular and Thoracic Surgery, Kantonsspital Baselland, Liestal, Switzerland
| | - Adrian T Billeter
- Clarunis, Department of Visceral Surgery, University Digestive Health Care Center, St. Clara Hospital and University Hospital Basel, Petersgraben 4, 4031, Basel, Switzerland
| | - Joël L Lavanchy
- Clarunis, Department of Visceral Surgery, University Digestive Health Care Center, St. Clara Hospital and University Hospital Basel, Petersgraben 4, 4031, Basel, Switzerland
- Department of Biomedical Engineering, University of Basel, Allschwil, Switzerland
| | - Andrea Wiencierz
- Department of Clinical Research, University of Basel, University Hospital, Basel, Switzerland
| | - Beat Peter Müller-Stich
- Clarunis, Department of Visceral Surgery, University Digestive Health Care Center, St. Clara Hospital and University Hospital Basel, Petersgraben 4, 4031, Basel, Switzerland
| | - Marco von Strauss Und Torney
- Clarunis, Department of Visceral Surgery, University Digestive Health Care Center, St. Clara Hospital and University Hospital Basel, Petersgraben 4, 4031, Basel, Switzerland
- Department of Clinical Research, University of Basel, University Hospital, Basel, Switzerland
- St. Clara Research Ltd, Basel, Switzerland
| |
Collapse
|
2
|
Trevena W, Zhong X, Lal A, Rovati L, Cubro E, Dong Y, Schulte P, Gajic O. Model-driven engineering for digital twins: a graph model-based patient simulation application. Front Physiol 2024; 15:1424931. [PMID: 39189027 PMCID: PMC11345177 DOI: 10.3389/fphys.2024.1424931] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2024] [Accepted: 07/19/2024] [Indexed: 08/28/2024] Open
Abstract
INTRODUCTION Digital twins of patients are virtual models that can create a digital patient replica to test clinical interventions in silico without exposing real patients to risk. With the increasing availability of electronic health records and sensor-derived patient data, digital twins offer significant potential for applications in the healthcare sector. METHODS This article presents a scalable full-stack architecture for a patient simulation application driven by graph-based models. This patient simulation application enables medical practitioners and trainees to simulate the trajectory of critically ill patients with sepsis. Directed acyclic graphs are utilized to model the complex underlying causal pathways that focus on the physiological interactions and medication effects relevant to the first 6 h of critical illness. To realize the sepsis patient simulation at scale, we propose an application architecture with three core components, a cross-platform frontend application that clinicians and trainees use to run the simulation, a simulation engine hosted in the cloud on a serverless function that performs all of the computations, and a graph database that hosts the graph model utilized by the simulation engine to determine the progression of each simulation. RESULTS A short case study is presented to demonstrate the viability of the proposed simulation architecture. DISCUSSION The proposed patient simulation application could help train future generations of healthcare professionals and could be used to facilitate clinicians' bedside decision-making.
Collapse
Affiliation(s)
- William Trevena
- Department of Industrial and Systems Engineering, University of Florida, Gainesville, FL, United States
| | - Xiang Zhong
- Department of Industrial and Systems Engineering, University of Florida, Gainesville, FL, United States
| | - Amos Lal
- Mayo Clinic, Rochester, MN, United States
| | | | - Edin Cubro
- Mayo Clinic, Rochester, MN, United States
| | - Yue Dong
- Mayo Clinic, Rochester, MN, United States
| | | | | |
Collapse
|
3
|
Strechen I, Herasevich S, Barwise A, Garcia-Mendez J, Rovati L, Pickering B, Diedrich D, Herasevich V. Centralized Multipatient Dashboards' Impact on Intensive Care Unit Clinician Performance and Satisfaction: A Systematic Review. Appl Clin Inform 2024; 15:414-427. [PMID: 38574763 PMCID: PMC11136527 DOI: 10.1055/a-2299-7643] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2024] [Accepted: 04/03/2024] [Indexed: 04/06/2024] Open
Abstract
BACKGROUND Intensive care unit (ICU) clinicians encounter frequent challenges with managing vast amounts of fragmented data while caring for multiple critically ill patients simultaneously. This may lead to increased provider cognitive load that may jeopardize patient safety. OBJECTIVES This systematic review assesses the impact of centralized multipatient dashboards on ICU clinician performance, perceptions regarding the use of these tools, and patient outcomes. METHODS A literature search was conducted on February 9, 2023, using the EBSCO CINAHL, Cochrane Central Register of Controlled Trials, Embase, IEEE Xplore, MEDLINE, Scopus, and Web of Science Core Collection databases. Eligible studies that included ICU clinicians as participants and tested the effect of dashboards designed for use by multiple users to manage multiple patients on user performance and/or satisfaction compared with the standard practice. We narratively synthesized eligible studies following the SWiM (Synthesis Without Meta-analysis) guidelines. Studies were grouped based on dashboard type and outcomes assessed. RESULTS The search yielded a total of 2,407 studies. Five studies met inclusion criteria and were included. Among these, three studies evaluated interactive displays in the ICU, one study assessed two dashboards in the pediatric ICU (PICU), and one study examined centralized monitor in the PICU. Most studies reported several positive outcomes, including reductions in data gathering time before rounds, a decrease in misrepresentations during multidisciplinary rounds, improved daily documentation compliance, faster decision-making, and user satisfaction. One study did not report any significant association. CONCLUSION The multipatient dashboards were associated with improved ICU clinician performance and were positively perceived in most of the included studies. The risk of bias was high, and the certainty of evidence was very low, due to inconsistencies, imprecision, indirectness in the outcome measure, and methodological limitations. Designing and evaluating multipatient tools using robust research methodologies is an important focus for future research.
Collapse
Affiliation(s)
- Inna Strechen
- Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, Minnesota, United States
| | - Svetlana Herasevich
- Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, Minnesota, United States
| | - Amelia Barwise
- Division of Pulmonary and Critical Care Medicine, Mayo Clinic, Rochester, Minnesota, United States
| | - Juan Garcia-Mendez
- Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, Minnesota, United States
| | - Lucrezia Rovati
- Division of Pulmonary and Critical Care Medicine, Mayo Clinic, Rochester, Minnesota, United States
- Department of Emergency Medicine, University of Milano-Bicocca, School of Medicine and Surgery, Milan, Italy
| | - Brian Pickering
- Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, Minnesota, United States
| | - Daniel Diedrich
- Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, Minnesota, United States
| | - Vitaly Herasevich
- Department of Anesthesiology and Perioperative Medicine, Division of Critical Care, Mayo Clinic, Rochester, Minnesota, United States
| |
Collapse
|
4
|
Zhang S, Cui W, Wu Y, Ji M. Description of an individualised delirium intervention in intensive care units for critically ill patients delivered by an artificial intelligence-assisted system: using the TIDieR checklist. J Res Nurs 2024; 29:112-124. [PMID: 39070574 PMCID: PMC11271677 DOI: 10.1177/17449871231219124] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/30/2024] Open
Abstract
Background Delirium is a preventable and reversible complication for intensive care unit (ICU) patients, which can be linked to negative outcomes. Early intervention to cope with the risk factors of delirium is necessary. Yet no specific description of the Artificial Intelligence Assisted Prevention and Management for Delirium (AI-AntiDelirium) following the Template for Intervention Description and Replication (TIDieR) checklist was reported. This is the first study to describe a detailed process for the development of an evidence-based delirium intervention. Aims To describe an individualised delirium intervention which is delivered by an artificial intelligence-assisted system in the ICU for critically ill patients. Methods and results The TIDieR checklist improved the description of ICU delirium interventions, including several key features for improved implementation of the intervention. This descriptive research describes the AI-assisted ICU delirium interventions for improving cognitive load and adherence of nurses and reducing ICU delirium incidence. Following the TIDieR checklist, we standardised the flow chart of ICU delirium assessment tools; formed an evaluation sheet of ICU delirium risk factors; and translated the evidence-based ABCDEF bundle intervention into practice. Therefore, nurses and researchers would benefit from replicating the interventions for clinical use or experimental research. Conclusions The TIDieR checklist provided a systematic approach for reporting the complex ICU delirium interventions delivered in a clinical interventional trial, which contributes to the nursing practice policy for the standardisation of interventions.
Collapse
Affiliation(s)
- Shan Zhang
- Associate Professor, School of Nursing, Capital Medical University, China
| | - Wei Cui
- Registered Nurse, School of Nursing, Capital Medical University, China
| | - Ying Wu
- Professor, School of Nursing, Capital Medical University, China
| | - Meihua Ji
- Associate Professor, School of Nursing, Capital Medical University, China
- Associate Professor, Advanced Innovation Center for Human Brain Protection, Capital Medical University, China
| |
Collapse
|
5
|
Zhang S, Cui W, Ding S, Li X, Zhang XW, Wu Y. A cluster-randomized controlled trial of a nurse-led artificial intelligence assisted prevention and management for delirium (AI-AntiDelirium) on delirium in intensive care unit: Study protocol. PLoS One 2024; 19:e0298793. [PMID: 38422003 PMCID: PMC10903907 DOI: 10.1371/journal.pone.0298793] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Accepted: 01/22/2024] [Indexed: 03/02/2024] Open
Abstract
BACKGROUND Delirium is a common complication among intensive care unit (ICU) patients that is linked to negative clinical outcomes. However, adherence to the Clinical Practice Guidelines for the Prevention and Management of Pain, Agitation/Sedation, Delirium, Immobility, and Sleep Disruption in Adult Patients in the ICU (PADIS guidelines), which recommend the use of the ABCDEF bundle, is sub-optimal in routine clinical care. To address this issue, AI-AntiDelirium, a nurse-led artificial intelligence-assisted prevention and management tool for delirium, was developed by our research team. Our pilot study yielded positive findings regarding the use of AI-AntiDelirium in preventing patient ICU delirium and improving activities of daily living and increasing intervention adherence by health care staff. METHODS The proposed large-scale pragmatic, open-label, parallel-group, cluster randomized controlled study will assess the impact of AI-AntiDelirium on the incidence of ICU delirium and delirium-related outcomes. Six ICUs in two tertiary hospitals in China will be randomized in a 1:1 ratio to an AI-AntiDelirium or a PADIS guidelines group. A target sample size of 1,452 ICU patients aged 50 years and older treated in the ICU for at least 24 hours will be included. The primary outcome evaluated will be the incidence of ICU delirium and the secondary outcomes will be the duration of ICU delirium, length of ICU and hospital stay, ICU and in-hospital mortality rates, patient cognitive function, patient activities of daily living, and ICU nurse adherence to the ABCDEF bundle. DISCUSSION If this large-scale trial provides evidence of the effectiveness of AI-AntiDelirium, an artificial intelligence-assisted system tool, in decreasing the incidence of ICU delirium, length of ICU and hospital stay, ICU and in-hospital mortality rates, patient cognitive function, and patient activities of daily living while increasing ICU nurse adherence to the ABCDEF bundle, it will have a profound impact on the management of ICU delirium in both research and clinical practice. CLINICAL TRIAL REGISTRATION ChiCTR1900023711 (Chinese Clinical Trial Registry).
Collapse
Affiliation(s)
- Shan Zhang
- School of Nursing, Capital Medical University, Beijing, China
| | - Wei Cui
- School of Nursing, Capital Medical University, Beijing, China
| | - Shu Ding
- School of Nursing, Capital Medical University, Beijing, China
- Cardiology Department, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China
| | - Xiangyu Li
- School of Nursing, Capital Medical University, Beijing, China
| | - Xi-Wei Zhang
- Nursing Department, Anzhen Hospital, Capital Medical University, Beijing, China
| | - Ying Wu
- School of Nursing, Capital Medical University, Beijing, China
| |
Collapse
|
6
|
Rovati L, Gary PJ, Cubro E, Dong Y, Kilickaya O, Schulte PJ, Zhong X, Wörster M, Kelm DJ, Gajic O, Niven AS, Lal A. Development and usability testing of a patient digital twin for critical care education: a mixed methods study. Front Med (Lausanne) 2024; 10:1336897. [PMID: 38274456 PMCID: PMC10808677 DOI: 10.3389/fmed.2023.1336897] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Accepted: 12/26/2023] [Indexed: 01/27/2024] Open
Abstract
BACKGROUND Digital twins are computerized patient replicas that allow clinical interventions testing in silico to minimize preventable patient harm. Our group has developed a novel application software utilizing a digital twin patient model based on electronic health record (EHR) variables to simulate clinical trajectories during the initial 6 h of critical illness. This study aimed to assess the usability, workload, and acceptance of the digital twin application as an educational tool in critical care. METHODS A mixed methods study was conducted during seven user testing sessions of the digital twin application with thirty-five first-year internal medicine residents. Qualitative data were collected using a think-aloud and semi-structured interview format, while quantitative measurements included the System Usability Scale (SUS), NASA Task Load Index (NASA-TLX), and a short survey. RESULTS Median SUS scores and NASA-TLX were 70 (IQR 62.5-82.5) and 29.2 (IQR 22.5-34.2), consistent with good software usability and low to moderate workload, respectively. Residents expressed interest in using the digital twin application for ICU rotations and identified five themes for software improvement: clinical fidelity, interface organization, learning experience, serious gaming, and implementation strategies. CONCLUSION A digital twin application based on EHR clinical variables showed good usability and high acceptance for critical care education.
Collapse
Affiliation(s)
- Lucrezia Rovati
- Department of Medicine, Division of Pulmonary and Critical Care Medicine, Mayo Clinic, Rochester, MN, United States
- School of Medicine and Surgery, University of Milano-Bicocca, Milan, Italy
| | - Phillip J. Gary
- Department of Medicine, Division of Pulmonary and Critical Care Medicine, Mayo Clinic, Rochester, MN, United States
| | - Edin Cubro
- Department of Information Technology, Mayo Clinic, Rochester, MN, United States
| | - Yue Dong
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, MN, United States
| | - Oguz Kilickaya
- Department of Medicine, Division of Pulmonary and Critical Care Medicine, Mayo Clinic, Rochester, MN, United States
| | - Phillip J. Schulte
- Department of Quantitative Health Sciences, Division of Clinical Trials and Biostatistics, Mayo Clinic, Rochester, MN, United States
| | - Xiang Zhong
- Department of Industrial and Systems Engineering, University of Florida, Gainesville, FL, United States
| | - Malin Wörster
- Center for Anesthesiology and Intensive Care Medicine, Department of Anesthesiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Diana J. Kelm
- Department of Medicine, Division of Pulmonary and Critical Care Medicine, Mayo Clinic, Rochester, MN, United States
| | - Ognjen Gajic
- Department of Medicine, Division of Pulmonary and Critical Care Medicine, Mayo Clinic, Rochester, MN, United States
| | - Alexander S. Niven
- Department of Medicine, Division of Pulmonary and Critical Care Medicine, Mayo Clinic, Rochester, MN, United States
| | - Amos Lal
- Department of Medicine, Division of Pulmonary and Critical Care Medicine, Mayo Clinic, Rochester, MN, United States
| |
Collapse
|
7
|
Yang S, Galvagno S, Badjatia N, Stein D, Teeter W, Scalea T, Shackelford S, Fang R, Miller C, Hu P. A Novel Continuous Real-Time Vital Signs Viewer for Intensive Care Units: Design and Evaluation Study. JMIR Hum Factors 2024; 11:e46030. [PMID: 38180791 PMCID: PMC10799282 DOI: 10.2196/46030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Revised: 11/03/2023] [Accepted: 11/20/2023] [Indexed: 01/06/2024] Open
Abstract
BACKGROUND Clinicians working in intensive care units (ICUs) are immersed in a cacophony of alarms and a relentless onslaught of data. Within this frenetic environment, clinicians make high-stakes decisions using many data sources and are often oversaturated with information of varying quality. Traditional bedside monitors only depict static vital signs data, and these data are not easily viewable remotely. Clinicians must rely on separate nursing charts-handwritten or electric-to review physiological patterns, including signs of potential clinical deterioration. An automated physiological data viewer has been developed to provide at-a-glance summaries and to assist with prioritizing care for multiple patients who are critically ill. OBJECTIVE This study aims to evaluate a novel vital signs viewer system in a level 1 trauma center by subjectively assessing the viewer's utility in a high-volume ICU setting. METHODS ICU attendings were surveyed during morning rounds. Physicians were asked to conduct rounds normally, using data reported from nurse charts and briefs from fellows to inform their clinical decisions. After the physician finished their assessment and plan for the patient, they were asked to complete a questionnaire. Following completion of the questionnaire, the viewer was presented to ICU physicians on a tablet personal computer that displayed the patient's physiologic data (ie, shock index, blood pressure, heart rate, temperature, respiratory rate, and pulse oximetry), summarized for up to 72 hours. After examining the viewer, ICU physicians completed a postview questionnaire. In both questionnaires, the physicians were asked questions regarding the patient's stability, status, and need for a higher or lower level of care. A hierarchical clustering analysis was used to group participating ICU physicians and assess their general reception of the viewer. RESULTS A total of 908 anonymous surveys were collected from 28 ICU physicians from February 2015 to June 2017. Regarding physicians' perception of whether the viewer enhanced the ability to assess multiple patients in the ICU, 5% (45/908) strongly agreed, 56.6% (514/908) agreed, 35.3% (321/908) were neutral, 2.9% (26/908) disagreed, and 0.2% (2/908) strongly disagreed. CONCLUSIONS Morning rounds in a trauma center ICU are conducted in a busy environment with many data sources. This study demonstrates that organized physiologic data and visual assessment can improve situation awareness, assist clinicians with recognizing changes in patient status, and prioritize care.
Collapse
Affiliation(s)
- Shiming Yang
- Department of Anesthesiology, University of Maryland School of Medicine, Baltimore, MD, United States
| | - Samuel Galvagno
- Department of Anesthesiology, University of Maryland School of Medicine, Baltimore, MD, United States
| | - Neeraj Badjatia
- Department of Neurology, University of Maryland School of Medicine, Baltimore, MD, United States
| | - Deborah Stein
- Department of Surgery, University of Maryland School of Medicine, Baltimore, MD, United States
| | - William Teeter
- Emergency Medicine, University of Maryland School of Medicine, Baltimore, MD, United States
| | - Thomas Scalea
- Department of Surgery, University of Maryland School of Medicine, Baltimore, MD, United States
| | - Stacy Shackelford
- United States Air Force Academy, Colorado Springs, CO, United States
| | - Raymond Fang
- Department of Surgery, Johns Hopkins University School of Medicine, Baltimore, MD, United States
| | - Catriona Miller
- Department of Surgery, University of Maryland School of Medicine, Baltimore, MD, United States
| | - Peter Hu
- Department of Anesthesiology, University of Maryland School of Medicine, Baltimore, MD, United States
| |
Collapse
|
8
|
Gasciauskaite G, Lunkiewicz J, Schweiger G, Budowski AD, Henckert D, Roche TR, Bergauer L, Meybohm P, Hottenrott S, Zacharowski K, Raimann FJ, Rivas E, López-Baamonde M, Ganter MT, Schmidt T, Nöthiger CB, Tscholl DW, Akbas S. User Perceptions of Visual Blood: An International Mixed Methods Study on Novel Blood Gas Analysis Visualization. Diagnostics (Basel) 2023; 13:3103. [PMID: 37835847 PMCID: PMC10572252 DOI: 10.3390/diagnostics13193103] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Revised: 09/22/2023] [Accepted: 09/26/2023] [Indexed: 10/15/2023] Open
Abstract
Blood gas analysis plays a central role in modern medicine. Advances in technology have expanded the range of available parameters and increased the complexity of their interpretation. By applying user-centered design principles, it is possible to reduce the cognitive load associated with interpreting blood gas analysis. In this international, multicenter study, we explored anesthesiologists' perspectives on Visual Blood, a novel visualization technique for presenting blood gas analysis results. We conducted interviews with participants following two computer-based simulation studies, the first utilizing virtual reality (VR) (50 participants) and the second without VR (70 participants). Employing the template approach, we identified key themes in the interview responses and formulated six statements, which were rated using Likert scales from 1 (strongly disagree) to 5 (strongly agree) in an online questionnaire. The most frequently mentioned theme was the positive usability features of Visual Blood. The online survey revealed that participants found Visual Blood to be an intuitive method for interpreting blood gas analysis (median 4, interquartile range (IQR) 4-4, p < 0.001). Participants noted that minimal training was required to effectively learn how to interpret Visual Blood (median 4, IQR 4-4, p < 0.001). However, adjustments are necessary to reduce visual overload (median 4, IQR 2-4, p < 0.001). Overall, Visual Blood received a favorable response. The strengths and weaknesses derived from these data will help optimize future versions of Visual Blood to improve the presentation of blood gas analysis results.
Collapse
Affiliation(s)
- Greta Gasciauskaite
- Institute of Anesthesiology, University and University Hospital Zurich, Raemistrasse 100, 8091 Zurich, Switzerland
| | - Justyna Lunkiewicz
- Institute of Anesthesiology, University and University Hospital Zurich, Raemistrasse 100, 8091 Zurich, Switzerland
| | - Giovanna Schweiger
- Institute of Anesthesiology, University and University Hospital Zurich, Raemistrasse 100, 8091 Zurich, Switzerland
| | - Alexandra D. Budowski
- Institute of Anesthesiology, University and University Hospital Zurich, Raemistrasse 100, 8091 Zurich, Switzerland
| | - David Henckert
- Institute of Anesthesiology, University and University Hospital Zurich, Raemistrasse 100, 8091 Zurich, Switzerland
| | - Tadzio R. Roche
- Institute of Anesthesiology, University and University Hospital Zurich, Raemistrasse 100, 8091 Zurich, Switzerland
| | - Lisa Bergauer
- Institute of Anesthesiology, University and University Hospital Zurich, Raemistrasse 100, 8091 Zurich, Switzerland
| | - Patrick Meybohm
- Department of Anaesthesiology, Intensive Care, Emergency and Pain Medicine, University Hospital Wuerzburg, University of Wuerzburg, 97080 Wuerzburg, Germany
| | - Sebastian Hottenrott
- Department of Anaesthesiology, Intensive Care, Emergency and Pain Medicine, University Hospital Wuerzburg, University of Wuerzburg, 97080 Wuerzburg, Germany
| | - Kai Zacharowski
- Department of Anaesthesiology, Intensive Care Medicine and Pain Therapy, University Hospital Frankfurt, Goethe University Frankfurt, 60323 Frankfurt, Germany
| | - Florian Jürgen Raimann
- Department of Anaesthesiology, Intensive Care Medicine and Pain Therapy, University Hospital Frankfurt, Goethe University Frankfurt, 60323 Frankfurt, Germany
| | - Eva Rivas
- Department of Anaesthesiology, Intensive Care Medicine and Pain Therapy, Hospital Clinic of Barcelona, University of Barcelona, 08036 Barcelona, Spain
| | - Manuel López-Baamonde
- Department of Anaesthesiology, Intensive Care Medicine and Pain Therapy, Hospital Clinic of Barcelona, University of Barcelona, 08036 Barcelona, Spain
| | - Michael Thomas Ganter
- Institute of Anaesthesiology and Critical Care Medicine, Clinic Hirslanden Zurich, 8032 Zurich, Switzerland
| | - Tanja Schmidt
- Institute of Anaesthesiology and Critical Care Medicine, Clinic Hirslanden Zurich, 8032 Zurich, Switzerland
| | - Christoph B. Nöthiger
- Institute of Anesthesiology, University and University Hospital Zurich, Raemistrasse 100, 8091 Zurich, Switzerland
| | - David W. Tscholl
- Institute of Anesthesiology, University and University Hospital Zurich, Raemistrasse 100, 8091 Zurich, Switzerland
| | - Samira Akbas
- Institute of Anesthesiology, University and University Hospital Zurich, Raemistrasse 100, 8091 Zurich, Switzerland
| |
Collapse
|
9
|
Herasevich S, Pinevich Y, Lipatov K, Barwise AK, Lindroth HL, LeMahieu AM, Dong Y, Herasevich V, Pickering BW. Evaluation of Digital Health Strategy to Support Clinician-Led Critically Ill Patient Population Management: A Randomized Crossover Study. Crit Care Explor 2023; 5:e0909. [PMID: 37151891 PMCID: PMC10158897 DOI: 10.1097/cce.0000000000000909] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/09/2023] Open
Abstract
To investigate whether a novel acute care multipatient viewer (AMP), created with an understanding of clinician information and process requirements, could reduce time to clinical decision-making among clinicians caring for populations of acutely ill patients compared with a widely used commercial electronic medical record (EMR). DESIGN Single center randomized crossover study. SETTING Quaternary care academic hospital. SUBJECTS Attending and in-training critical care physicians, and advanced practice providers. INTERVENTIONS AMP. MEASUREMENTS AND MAIN RESULTS We compared ICU clinician performance in structured clinical task completion using two electronic environments-the standard commercial EMR (Epic) versus the novel AMP in addition to Epic. Twenty subjects (10 pairs of clinicians) participated in the study. During the study session, each participant completed the tasks on two ICUs (7-10 beds each) and eight individual patients. The adjusted time for assessment of the entire ICU and the adjusted total time to task completion were significantly lower using AMP versus standard commercial EMR (-6.11; 95% CI, -7.91 to -4.30 min and -5.38; 95% CI, -7.56 to -3.20 min, respectively; p < 0.001). The adjusted time for assessment of individual patients was similar using both the EMR and AMP (0.73; 95% CI, -0.09 to 1.54 min; p = 0.078). AMP was associated with a significantly lower adjusted task load (National Aeronautics and Space Administration-Task Load Index) among clinicians performing the task versus the standard EMR (22.6; 95% CI, -32.7 to -12.4 points; p < 0.001). There was no statistically significant difference in adjusted total errors when comparing the two environments (0.68; 95% CI, 0.36-1.30; p = 0.078). CONCLUSIONS When compared with the standard EMR, AMP significantly reduced time to assessment of an entire ICU, total time to clinical task completion, and clinician task load. Additional research is needed to assess the clinicians' performance while using AMP in the live ICU setting.
Collapse
Affiliation(s)
- Svetlana Herasevich
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, MN
| | - Yuliya Pinevich
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, MN
- Department of Anesthesiology, Republican Clinical Medical Center, Minsk, Belarus
| | - Kirill Lipatov
- Division of Pulmonary and Critical Care Medicine, Mayo Clinic Health Systems, Eau Claire, WI
| | - Amelia K Barwise
- Division of Pulmonary and Critical Care Medicine, Mayo Clinic, Rochester, MN
- Bioethics Research Program, Mayo Clinic, Rochester, MN
| | - Heidi L Lindroth
- Department of Nursing, Mayo Clinic, Rochester, MN
- Center for Health Innovation and Implementation Science, Center for Aging Research, School of Medicine, Indiana University, Indianapolis, IN
| | | | - Yue Dong
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, MN
| | - Vitaly Herasevich
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, MN
| | - Brian W Pickering
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, MN
| |
Collapse
|
10
|
Abstract
BACKGROUND AND OBJECTIVE Electronic health records (EHRs) have become ubiquitous in medicine and continue to grow in informational content. Little has been documented regarding patient safety from the resultant information overload. The objective of this literature review is to better understand how information overload in EHR affects patient safety. METHODS A literature search was performed using the Transparent Reporting of Systematic Reviews and Meta-Analyses standards for literature review. PubMed and Web of Science were searched and articles selected that were relevant to EHR information overload based on keywords. RESULTS The literature search yielded 28 articles meeting the criteria for the study. Information overload was found to increase physician cognitive load and error rates in clinical simulations. Overabundance of clinically irrelevant information, poor data display, and excessive alerting were consistently identified as issues that may lead to information overload. CONCLUSIONS Information overload in EHRs may result in higher error rates and negatively impact patient safety. Further studies are necessary to define the role of EHR in adverse patient safety events and to determine methods to mitigate these errors. Changes focused on the usability of EHR should be considered with the end user (physician) in mind. Federal agencies have a role to play in encouraging faster adoption of improved EHR interfaces.
Collapse
|
11
|
Abstract
Clinical informatics can support quality improvement and patient safety in the pediatric intensive care unit (PICU) in several ways including data extraction, analysis, and decision support enabled by electronic health records (EHRs), and databases and registries. Clinical decision support (CDS), embedded in EHRs, now an integral part of the workflow in the PICU, includes several tools and is increasingly leveraging artificial intelligence (AI). Understanding the opportunities and challenges can improve the engagement of clinicians with the design, validation, and implementation of CDS, improve satisfaction with CDS, and improve patient safety, care quality, and value.
Collapse
|
12
|
Privitera MR. Promoting Clinician Well-Being and Patient Safety Using Human Factors Science: Reducing Unnecessary Occupational Stress. Health (London) 2022. [DOI: 10.4236/health.2022.1412095] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/05/2023]
|
13
|
Hendriks M, Willemsen MC, Sartor F, Hoonhout J. Respecting Human Autonomy in Critical Care Clinical Decision Support. FRONTIERS IN COMPUTER SCIENCE 2021. [DOI: 10.3389/fcomp.2021.690576] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Abstract
Clinical Decision Support (CDS) aims at helping physicians optimize their decisions. However, as each patient is unique in their characteristics and preferences, it is difficult to define the optimal outcome. Human physicians should retain autonomy over their decisions, to ensure that tradeoffs are made in a way that fits the unique patient. We tend to consider autonomy in the sense of not influencing decision-making. However, as CDS aims to improve decision-making, its very aim is to influence decision-making. We advocate for an alternative notion of autonomy as enabling the physician to make decisions in accordance with their professional goals and values and the goals and values of the patient. This perspective retains the role of autonomy as a gatekeeper for safeguarding other human values, while letting go of the idea that CDS should not influence the physician in any way. Rather than trying to refrain from incorporating human values into CDS, we should instead aim for a value-aware CDS that actively supports the physician in considering tradeoffs in human values. We suggest a conversational AI approach to enable the CDS to become value-aware and the use of story structures to help the user integrate facts and data-driven learnings provided by the CDS with their own value judgements in a natural way.
Collapse
|
14
|
Beaubien L, Conrad C, Music J, Toze S. Evaluating Simplified Web Interfaces of Risk Models for Clinical Use: Pilot Survey Study. JMIR Form Res 2021; 5:e22110. [PMID: 34269692 PMCID: PMC8325085 DOI: 10.2196/22110] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2020] [Revised: 02/14/2021] [Accepted: 05/31/2021] [Indexed: 11/13/2022] Open
Abstract
BACKGROUND In this pilot study, we investigated sociotechnical factors that affect intention to use a simplified web model to support clinical decision making. OBJECTIVE We investigated factors that are known to affect technology adoption using the unified theory of acceptance and use of technology (UTAUT2) model. The goal was to pilot and test a tool to better support complex clinical assessments. METHODS Based on the results of a previously published work, we developed a web-based mobile user interface, WebModel, to allow users to work with regression equations and their predictions to evaluate the impact of various characteristics or treatments on key outcomes (eg, survival time) for chronic obstructive pulmonary disease. The WebModel provides a way to combat information overload and more easily compare treatment options. It limits the number of web forms presented to a user to between 1 and 20, rather than the dozens of detailed calculations typically required. The WebModel uses responsive design and can be used on multiple devices. To test the WebModel, we designed a questionnaire to probe the efficacy of the WebModel and assess the usability and usefulness of the system. The study was live for one month, and participants had access to it over that time. The questionnaire was administered online, and data from 674 clinical users who had access to the WebModel were captured. SPSS and R were used for statistical analysis. RESULTS The regression model developed from UTAUT2 constructs was a fit. Specifically, five of the seven factors were significant positive coefficients in the regression: performance expectancy (β=.2730; t=7.994; P<.001), effort expectancy (β=.1473; t=3.870; P=.001), facilitating conditions (β=.1644; t=3.849; P<.001), hedonic motivation (β=.2321; t=3.991; P<.001), and habit (β=.2943; t=12.732). Social influence was not a significant factor, while price value had a significant negative influence on intention to use the WebModel. CONCLUSIONS Our results indicate that multiple influences impact positive response to the system, many of which relate to the efficiency of the interface to provide clear information. Although we found that the price value was a negative factor, it is possible this was due to the removal of health workers from purchasing decisions. Given that this was a pilot test, and that the system was not used in a clinical setting, we could not examine factors related to actual workflow, patient safety, or social influence. This study shows that the concept of a simplified WebModel could be effective and efficient in reducing information overload in complex clinical decision making. We recommend further study to test this in a clinical setting and gather qualitative data from users regarding the value of the tool in practice.
Collapse
Affiliation(s)
- Louis Beaubien
- Rowe School of Business, Faculty of Management, Dalhousie University, Halifax, NS, Canada
| | - Colin Conrad
- School of Information Management, Faculty of Management, Dalhousie University, Halifax, NS, Canada
| | - Janet Music
- School of Information Management, Faculty of Management, Dalhousie University, Halifax, NS, Canada
| | - Sandra Toze
- School of Information Management, Faculty of Management, Dalhousie University, Halifax, NS, Canada
| |
Collapse
|
15
|
Pamplin J, Nemeth CP, Serio-Melvin ML, Murray SJ, Rule GT, Veinott ES, Veazey SR, Hamilton AJ, Fenrich CA, Laufersweiler DE, Salinas J. Improving Clinician Decisions and Communication in Critical Care Using Novel Information Technology. Mil Med 2021; 185:e254-e261. [PMID: 31271437 DOI: 10.1093/milmed/usz151] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2019] [Revised: 04/27/2019] [Accepted: 06/03/2019] [Indexed: 11/14/2022] Open
Abstract
INTRODUCTION The electronic medical record (EMR) is presumed to support clinician decisions by documenting and retrieving patient information. Research shows that the EMR variably affects patient care and clinical decision making. The way information is presented likely has a significant impact on this variability. Well-designed representations of salient information can make a task easier by integrating information in useful patterns that clinicians use to make improved clinical judgments and decisions. Using Cognitive Systems Engineering methods, our research team developed a novel health information technology (NHIT) that interfaces with the EMR to display salient clinical information and enabled communication with a dedicated text-messaging feature. The software allows clinicians to customize displays according to their role and information needs. Here we present results of usability and validation assessments of the NHIT. MATERIALS AND METHODS Our subjects were physicians, nurses, respiratory therapists, and physician trainees. Two arms of this study were conducted, a usability assessment and then a validation assessment. The usability assessment was a computer-based simulation using deceased patient data. After a brief five-minute orientation, the usability assessment measured individual clinician performance of typical tasks in two clinical scenarios using the NHIT. The clinical scenarios included patient admission to the unit and patient readiness for surgery. We evaluated clinician perspective about the NHIT after completing tasks using 7-point Likert scale surveys. In the usability assessment, the primary outcome was participant perceptions about the system's ease of use compared to the legacy system.A subsequent cross-over, validation assessment compared performance of two clinical teams during simulated care scenarios: one using only the legacy IT system and one using the NHIT in addition to the legacy IT system. We oriented both teams to the NHIT during a 1-hour session on the night before the first scenario. Scenarios were conducted using high-fidelity simulation in a real burn intensive care unit room. We used observations, task completion times, semi-structured interviews, and surveys to compare user decisions and perceptions about their performance. The primary outcome for the validation assessment was time to reach accurate (correct) decision points. RESULTS During the usability assessment, clinicians were able to complete all tasks requested. Clinicians reported the NHIT was easier to use and the novel information display allowed for easier data interpretation compared to subject recollection of the legacy EMR.In the validation assessment, a more junior team of clinicians using the NHIT arrived at accurate diagnoses and decision points at similar times as a more experienced team. Both teams noted improved communication between team members when using the NHIT and overall rated the NHIT as easier to use than the legacy EMR, especially with respect to finding information. CONCLUSIONS The primary findings of these assessments are that clinicians found the NHIT easy to use despite minimal training and experience and that it did not degrade clinician efficiency or decision-making accuracy. These findings are in contrast to common user experiences when introduced to new EMRs in clinical practice.
Collapse
Affiliation(s)
- Jeremy Pamplin
- Telemedicine and Advanced Technology Research Center, Ft. Detrick, MD 21702.,Uniformed Services University of the Health Sciences, Bethesda, MD 20814
| | | | | | - Sarah J Murray
- U.S. Army Institute of Surgical Research, San Antonio, TX 78234
| | - Gregory T Rule
- Applied Research Associates, Inc., San Antonio, TX 78232
| | | | - Sena R Veazey
- U.S. Army Institute of Surgical Research, San Antonio, TX 78234
| | | | - Craig A Fenrich
- U.S. Army Institute of Surgical Research, San Antonio, TX 78234
| | | | - Jose Salinas
- U.S. Army Institute of Surgical Research, San Antonio, TX 78234
| |
Collapse
|
16
|
Khairat S, Coleman C, Ottmar P, Bice T, Koppel R, Carson SS. Physicians' gender and their use of electronic health records: findings from a mixed-methods usability study. J Am Med Inform Assoc 2021; 26:1505-1514. [PMID: 31504578 DOI: 10.1093/jamia/ocz126] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2019] [Revised: 06/05/2019] [Accepted: 07/01/2019] [Indexed: 11/13/2022] Open
Abstract
OBJECTIVE Physician burnout associated with EHRs is a major concern in health care. A comprehensive assessment of differences among physicians in the areas of EHR performance, efficiency, and satisfaction has not been conducted. The study sought to study relationships among physicians' performance, efficiency, perceived workload, satisfaction, and usability in using the electronic health record (EHR) with comparisons by age, gender, professional role, and years of experience with the EHR. MATERIALS AND METHODS Mixed-methods assessments of the medical intensivists' EHR use and perceptions. Using simulated cases, we employed standardized scales, performance measures, and extensive interviews. NASA Task Load Index (TLX), System Usability Scale (SUS), and Questionnaire on User Interface Satisfaction surveys were deployed. RESULTS The study enrolled 25 intensive care unit (ICU) physicians (11 residents, 9 fellows, 5 attendings); 12 (48%) were men, with a mean age of 33 (range, 28-55) years and a mean of 4 (interquartile range, 2.0-5.5) years of Epic experience. Overall task performance scores were similar for men (90% ± 9.3%) and women (92% ± 4.4%), with no statistically significant differences (P = .374). However, female physicians demonstrated higher efficiency in completion time (difference = 7.1 minutes; P = .207) and mouse clicks (difference = 54; P = .13). Overall, men reported significantly higher perceived EHR workload stress compared with women (difference = 17.5; P < .001). Men reported significantly higher levels of frustration with the EHR compared with women (difference = 33.15; P < .001). Women reported significantly higher satisfaction with the ease of use of the EHR interface than men (difference = 0.66; P =.03). The women's perceived overall usability of the EHR is marginally higher than that of the men (difference = 10.31; P =.06). CONCLUSIONS Among ICU physicians, we measured significant gender-based differences in perceived EHR workload stress, satisfaction, and usability-corresponding to objective patterns in EHR efficiency. Understanding the reasons for these differences may help reduce burnout and guide improvements to physician performance, efficiency, and satisfaction with EHR use. DESIGN Mixed-methods assessments of the medical intensivists' EHR use and perceptions. Using simulated cases, we employed standardized scales, performance measures, and extensive interviews.
Collapse
Affiliation(s)
- Saif Khairat
- Carolina Health Informatics Program, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA.,School of Nursing, University of North Carolina at Chapel Hill, North Carolina, USA
| | - Cameron Coleman
- Carolina Health Informatics Program, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA.,Preventive Medicine Residency Program, Department of Family Medicine, University of North Carolina School of Medicine, Chapel Hill, North Carolina, USA
| | - Paige Ottmar
- Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, USA
| | - Thomas Bice
- Pulmonary Diseases and Critical Care Medicine, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| | - Ross Koppel
- Sociology Department and Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania, Philadelphia, Pennsylvania, USA.,Department of Biomedical Informatics, University at Buffalo, Buffalo, New York, USA
| | - Shannon S Carson
- Pulmonary Diseases and Critical Care Medicine, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
| |
Collapse
|
17
|
Lasko TA, Owens DA, Fabbri D, Wanderer JP, Genkins JZ, Novak LL. User-Centered Clinical Display Design Issues for Inpatient Providers. Appl Clin Inform 2020; 11:700-709. [PMID: 33086396 DOI: 10.1055/s-0040-1716746] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022] Open
Abstract
BACKGROUND Suboptimal information display in electronic health records (EHRs) is a notorious pain point for users. Designing an effective display is difficult, due in part to the complex and varied nature of clinical practice. OBJECTIVE This article aims to understand the goals, constraints, frustrations, and mental models of inpatient medical providers when accessing EHR data, to better inform the display of clinical information. METHODS A multidisciplinary ethnographic study of inpatient medical providers. RESULTS Our participants' primary goal was usually to assemble a clinical picture around a given question, under the constraints of time pressure and incomplete information. To do so, they tend to use a mental model of multiple layers of abstraction when thinking of patients and disease; they prefer immediate pattern recognition strategies for answering clinical questions, with breadth-first or depth-first search strategies used subsequently if needed; and they are sensitive to data relevance, completeness, and reliability when reading a record. CONCLUSION These results conflict with the ubiquitous display design practice of separating data by type (test results, medications, notes, etc.), a mismatch that is known to encumber efficient mental processing by increasing both navigation burden and memory demands on users. A popular and obvious solution is to select or filter the data to display exactly what is presumed to be relevant to the clinical question, but this solution is both brittle and mistrusted by users. A less brittle approach that is more aligned with our users' mental model could use abstraction to summarize details instead of filtering to hide data. An abstraction-based approach could allow clinicians to more easily assemble a clinical picture, to use immediate pattern recognition strategies, and to adjust the level of displayed detail to their particular needs. It could also help the user notice unanticipated patterns and to fluidly shift attention as understanding evolves.
Collapse
Affiliation(s)
- Thomas A Lasko
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, United States
| | - David A Owens
- Owen Graduate School of Management, Vanderbilt University, Nashville, Tennessee, United States
| | - Daniel Fabbri
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, United States.,Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, Tennessee, United States
| | - Jonathan P Wanderer
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, United States.,Department of Anesthesiology, Vanderbilt University Medical Center, Nashville, Tennessee, United States
| | - Julian Z Genkins
- Department of Medicine, University of California, San Francisco, San Francisco, California, United States
| | - Laurie L Novak
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, United States
| |
Collapse
|
18
|
Calzoni L, Clermont G, Cooper GF, Visweswaran S, Hochheiser H. Graphical Presentations of Clinical Data in a Learning Electronic Medical Record. Appl Clin Inform 2020; 11:680-691. [PMID: 33058103 PMCID: PMC7560537 DOI: 10.1055/s-0040-1709707] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/01/2023] Open
Abstract
BACKGROUND Complex electronic medical records (EMRs) presenting large amounts of data create risks of cognitive overload. We are designing a Learning EMR (LEMR) system that utilizes models of intensive care unit (ICU) physicians' data access patterns to identify and then highlight the most relevant data for each patient. OBJECTIVES We used insights from literature and feedback from potential users to inform the design of an EMR display capable of highlighting relevant information. METHODS We used a review of relevant literature to guide the design of preliminary paper prototypes of the LEMR user interface. We observed five ICU physicians using their current EMR systems in preparation for morning rounds. Participants were interviewed and asked to explain their interactions and challenges with the EMR systems. Findings informed the revision of our prototypes. Finally, we conducted a focus group with five ICU physicians to elicit feedback on our designs and to generate ideas for our final prototypes using participatory design methods. RESULTS Participating physicians expressed support for the LEMR system. Identified design requirements included the display of data essential for every patient together with diagnosis-specific data and new or significantly changed information. Respondents expressed preferences for fishbones to organize labs, mouseovers to access additional details, and unobtrusive alerts minimizing color-coding. To address the concern about possible physician overreliance on highlighting, participants suggested that non-highlighted data should remain accessible. Study findings led to revised prototypes, which will inform the development of a functional user interface. CONCLUSION In the feedback we received, physicians supported pursuing the concept of a LEMR system. By introducing novel ways to support physicians' cognitive abilities, such a system has the potential to enhance physician EMR use and lead to better patient outcomes. Future plans include laboratory studies of both the utility of the proposed designs on decision-making, and the possible impact of any automation bias.
Collapse
Affiliation(s)
- Luca Calzoni
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
| | - Gilles Clermont
- Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
| | - Gregory F Cooper
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
| | - Harry Hochheiser
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
| |
Collapse
|
19
|
Fuller TE, Garabedian PM, Lemonias DP, Joyce E, Schnipper JL, Harry EM, Bates DW, Dalal AK, Benneyan JC. Assessing the cognitive and work load of an inpatient safety dashboard in the context of opioid management. APPLIED ERGONOMICS 2020; 85:103047. [PMID: 32174343 DOI: 10.1016/j.apergo.2020.103047] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/18/2018] [Revised: 12/19/2019] [Accepted: 01/09/2020] [Indexed: 06/10/2023]
Abstract
For health information technology to realize its potential to improve flow, care, and patient safety, applications should be intuitive to use and burden neutral for frontline clinicians. We assessed the impact of a patient safety dashboard on clinician cognitive and work load within a simulated information-seeking task for safe inpatient opioid medication management. Compared to use of an electronic health record for the same task, the dashboard was associated with significantly reduced time on task, mouse clicks, and mouse movement (each p < 0.001), with no significant increases in cognitive load nor task inaccuracy. Cognitive burden was higher for users with less experience, possibly partly attributable to usability issues identified during this study. Findings underscore the importance of assessing the usability, cognitive, and work load analysis during the design and implementation of health information technology applications.
Collapse
Affiliation(s)
- Theresa E Fuller
- Healthcare Systems Engineering Institute, Northeastern University, Boston, MA, USA; Brigham and Women's Hospital, Boston, MA, USA
| | | | - Demetri P Lemonias
- Healthcare Systems Engineering Institute, Northeastern University, Boston, MA, USA
| | - Erin Joyce
- Healthcare Systems Engineering Institute, Northeastern University, Boston, MA, USA
| | - Jeffrey L Schnipper
- Brigham and Women's Hospital, Boston, MA, USA; Harvard Medical School, Boston, MA, USA
| | - Elizabeth M Harry
- Brigham and Women's Hospital, Boston, MA, USA; Harvard Medical School, Boston, MA, USA
| | - David W Bates
- Brigham and Women's Hospital, Boston, MA, USA; Partners Healthcare, Incorporated, Boston, MA, USA; Harvard Medical School, Boston, MA, USA
| | - Anuj K Dalal
- Brigham and Women's Hospital, Boston, MA, USA; Harvard Medical School, Boston, MA, USA
| | - James C Benneyan
- Healthcare Systems Engineering Institute, Northeastern University, Boston, MA, USA; College of Engineering, Northeastern University, Boston, MA, USA.
| |
Collapse
|
20
|
King AJ, Cooper GF, Clermont G, Hochheiser H, Hauskrecht M, Sittig DF, Visweswaran S. Using machine learning to selectively highlight patient information. J Biomed Inform 2019; 100:103327. [PMID: 31676461 PMCID: PMC6932869 DOI: 10.1016/j.jbi.2019.103327] [Citation(s) in RCA: 20] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2019] [Revised: 08/20/2019] [Accepted: 10/28/2019] [Indexed: 02/05/2023]
Abstract
BACKGROUND Electronic medical record (EMR) systems need functionality that decreases cognitive overload by drawing the clinician's attention to the right data, at the right time. We developed a Learning EMR (LEMR) system that learns statistical models of clinician information-seeking behavior and applies those models to direct the display of data in future patients. We evaluated the performance of the system in identifying relevant patient data in intensive care unit (ICU) patient cases. METHODS To capture information-seeking behavior, we enlisted critical care medicine physicians who reviewed a set of patient cases and selected data items relevant to the task of presenting at morning rounds. Using patient EMR data as predictors, we built machine learning models to predict their relevancy. We prospectively evaluated the predictions of a set of high performing models. RESULTS On an independent evaluation data set, 25 models achieved precision of 0.52, 95% CI [0.49, 0.54] and recall of 0.77, 95% CI [0.75, 0.80] in identifying relevant patient data items. For data items missed by the system, the reviewers rated the effect of not seeing those data from no impact to minor impact on patient care in about 82% of the cases. CONCLUSION Data-driven approaches for adaptively displaying data in EMR systems, like the LEMR system, show promise in using information-seeking behavior of clinicians to identify and highlight relevant patient data.
Collapse
Affiliation(s)
- Andrew J King
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA; Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, PA, USA
| | - Gregory F Cooper
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA; Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA
| | - Gilles Clermont
- Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, PA, USA
| | - Harry Hochheiser
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA; Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA
| | - Milos Hauskrecht
- Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA; Department of Computer Science, University of Pittsburgh, Pittsburgh, PA, USA
| | - Dean F Sittig
- Department of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA; Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA.
| |
Collapse
|
21
|
Reese T, Segall N, Nesbitt P, Del Fiol G, Waller R, Macpherson BC, Tonna JE, Wright MC. Patient information organization in the intensive care setting: expert knowledge elicitation with card sorting methods. J Am Med Inform Assoc 2019; 25:1026-1035. [PMID: 30060091 DOI: 10.1093/jamia/ocy045] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2017] [Accepted: 04/11/2018] [Indexed: 11/13/2022] Open
Abstract
Introduction Many electronic health records fail to support information uptake because they impose low-level information organization tasks on users. Clinical concept-oriented views have shown information processing improvements, but the specifics of this organization for critical care are unclear. Objective To determine high-level cognitive processes and patient information organization schema in critical care. Methods We conducted an open card sort of 29 patient data elements and a modified Delphi card sort of 65 patient data elements. Study participants were 39 clinicians with varied critical care training and experience. We analyzed the open sort with a hierarchical cluster analysis (HCA) and factor analysis (FA). The Delphi sort was split into three initiating groups that resulted in three unique solutions. We compared results between open sort analyses (HCA and FA), between card sorting exercises (open and Delphi), and across the Delphi solutions. Results Between the HCA and FA, we observed common constructs including cardiovascular and hemodynamics, infectious disease, medications, neurology, patient overview, respiratory, and vital signs. The more comprehensive Delphi sort solutions also included gastrointestinal, renal, and imaging constructs. Conclusions We identified primarily system-based groupings (e.g., cardiovascular, respiratory). Source-based (e.g., medications, laboratory) groups became apparent when participants were asked to sort a longer list of concepts. These results suggest a hybrid approach to information organization, which may combine systems, source, or problem-based groupings, best supports clinicians' mental models. These results can contribute to the design of information displays to better support clinicians' access and interpretation of information for critical care decisions.
Collapse
Affiliation(s)
- Thomas Reese
- Department of Biomedical Informatics, University of Utah School of Medicine, Salt Lake City, UT, USA
| | - Noa Segall
- Department of Anesthesiology, Duke University Medical Center, Durham, NC, USA
| | - Paige Nesbitt
- Trinity Health and Saint Alphonsus Regional Medical Center, Boise, ID, USA
| | - Guilherme Del Fiol
- Department of Biomedical Informatics, University of Utah School of Medicine, Salt Lake City, UT, USA
| | - Rosalie Waller
- Department of Biomedical Informatics, University of Utah School of Medicine, Salt Lake City, UT, USA
| | | | - Joseph E Tonna
- Department of Surgery, University of Utah School of Medicine, Salt Lake City, UT, USA
| | - Melanie C Wright
- Trinity Health and Saint Alphonsus Regional Medical Center, Boise, ID, USA
| |
Collapse
|
22
|
Wright MC, Borbolla D, Waller RG, Del Fiol G, Reese T, Nesbitt P, Segall N. Critical care information display approaches and design frameworks: A systematic review and meta-analysis. J Biomed Inform 2019; 3:100041. [PMID: 31423485 PMCID: PMC6696941 DOI: 10.1016/j.yjbinx.2019.100041] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2018] [Revised: 06/10/2019] [Accepted: 06/16/2019] [Indexed: 11/06/2022]
Abstract
OBJECTIVE To systematically review original user evaluations of patient information displays relevant to critical care and understand the impact of design frameworks and information presentation approaches on decision-making, efficiency, workload, and preferences of clinicians. METHODS We included studies that evaluated information displays designed to support real-time care decisions in critical care or anesthesiology using simulated tasks. We searched PubMed and IEEExplore from 1/1/1990 to 6/30/2018. The search strategy was developed iteratively with calibration against known references. Inclusion screening was completed independently by two authors. Extraction of display features, design processes, and evaluation method was completed by one and verified by a second author. RESULTS Fifty-six manuscripts evaluating 32 critical care and 22 anesthesia displays were included. Primary outcome metrics included clinician accuracy and efficiency in recognizing, diagnosing, and treating problems. Implementing user-centered design (UCD) processes, especially iterative evaluation and redesign, resulted in positive impact in outcomes such as accuracy and efficiency. Innovative display approaches that led to improved human-system performance in critical care included: (1) improving the integration and organization of information, (2) improving the representation of trend information, and (3) implementing graphical approaches to make relationships between data visible. CONCLUSION Our review affirms the value of key principles of UCD. Improved information presentation can facilitate faster information interpretation and more accurate diagnoses and treatment. Improvements to information organization and support for rapid interpretation of time-based relationships between related quantitative data is warranted. Designers and developers are encouraged to involve users in formal iterative design and evaluation activities in the design of electronic health records (EHRs), clinical informatics applications, and clinical devices.
Collapse
Affiliation(s)
- Melanie C. Wright
- Trinity Health, Livonia, MI, USA
- Saint Alphonsus Regional Medical Center, Boise, ID, USA
| | - Damian Borbolla
- Biomedical Informatics, University of Utah, Salt Lake City, UT, USA
| | | | | | - Thomas Reese
- Biomedical Informatics, University of Utah, Salt Lake City, UT, USA
| | - Paige Nesbitt
- Saint Alphonsus Regional Medical Center, Boise, ID, USA
| | - Noa Segall
- Anesthesiology, Duke University, Durham, NC, USA
| |
Collapse
|
23
|
Lin YL, Tomasi J, Guerguerian AM, Trbovich P. Technology-mediated macrocognition: Investigating how physicians, nurses, and respiratory therapists make critical decisions. J Crit Care 2019; 53:132-141. [PMID: 31228764 DOI: 10.1016/j.jcrc.2019.06.002] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2018] [Revised: 01/09/2019] [Accepted: 06/03/2019] [Indexed: 10/26/2022]
Abstract
PURPOSE Although intensive care clinicians are expected to make data-driven critical decisions using the technologies available to them, the effect of those technologies on decision-making are not well understood. Using the macrocognitive framework, we studied critical decision-making and technology use to understand how different specialists within teams make decisions and guide the development of decision-making support technologies. MATERIALS AND METHODS The Critical Decision Method was used to understand the macrocognitive processes used during critical decision-making of twelve critical care clinicians. Deductive (based on the macrocognition framework) and inductive coding were used to analyze the macrocognitive processes, their interrelationships, and their relation to technologies. RESULTS Over 60% of critical decision-making macrocognition was devoted to Sensemaking, Anticipation, and Communication. The most technology-mediated process was Sensemaking. Of particular note, physicians and respiratory therapists extracted information for their own use, while nurses extracted information to communicate to others. Physicians switched between ten macrocognitive processes, whereas nurses and respiratory therapists switched between five processes. CONCLUSIONS This exploratory study provides much needed details about the different ways in which specialists use technologies to support decision-making tasks, particularly those involving sensemaking, which are essential to the design and development of decision-support technologies.
Collapse
Affiliation(s)
- Ying Ling Lin
- Institute of Biomaterials and Biomedical Engineering, Faculty of Engineering, University of Toronto, Toronto, Canada; Interdepartmental Division of Critical Care Medicine, Hospital for Sick Children, Toronto, Canada
| | - Jessica Tomasi
- Institute of Biomaterials and Biomedical Engineering, Faculty of Engineering, University of Toronto, Toronto, Canada
| | - Anne-Marie Guerguerian
- Institute of Biomaterials and Biomedical Engineering, Faculty of Engineering, University of Toronto, Toronto, Canada; Interdepartmental Division of Critical Care Medicine, Hospital for Sick Children, Toronto, Canada; Faculty of Medicine, University of Toronto, Toronto, Canada; Neuroscience and Mental Health Research, Hospital for Sick Children, Toronto, Canada
| | - Patricia Trbovich
- Institute of Biomaterials and Biomedical Engineering, Faculty of Engineering, University of Toronto, Toronto, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada; Research and Innovation, North York General Hospital, Toronto, Canada.
| |
Collapse
|
24
|
Lin YL, Trbovich P, Kolodzey L, Nickel C, Guerguerian AM. Association of Data Integration Technologies With Intensive Care Clinician Performance: A Systematic Review and Meta-analysis. JAMA Netw Open 2019; 2:e194392. [PMID: 31125104 PMCID: PMC6632132 DOI: 10.1001/jamanetworkopen.2019.4392] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/02/2023] Open
Abstract
IMPORTANCE Sources of data in the intensive care setting are increasing exponentially, but the benefits of displaying multiparametric, high-frequency data are unknown. Decision making may not benefit from this technology if clinicians remain cognitively overburdened by poorly designed data integration and visualization technologies (DIVTs). OBJECTIVE To systematically review and summarize the published evidence on the association of user-centered DIVTs with intensive care clinician performance. DATA SOURCES MEDLINE, Embase, Cochrane Central Register of Controlled Trials, PsycINFO, and Web of Science were searched in May 2014 and January 2018. STUDY SELECTION Studies had 3 requirements: (1) the study tested a viable DIVT, (2) participants involved were intensive care clinicians, and (3) the study reported quantitative results associated with decision making in an intensive care setting. DATA EXTRACTION AND SYNTHESIS Of 252 records screened, 20 studies, published from 2004 to 2016, were included. The human factors framework to assess health technologies was applied to measure study completeness, and the Quality Assessment Instrument was used to assess the quality of the studies. PRISMA guidelines were adapted to conduct the systematic review and meta-analysis. MAIN OUTCOMES AND MEASURES Study completeness and quality; clinician performance; physical, mental, and temporal demand; effort; frustration; time to decision; and decision accuracy. RESULTS Of the 20 included studies, 16 were experimental studies with 410 intensive care clinician participants and 4 were survey-based studies with 1511 respondents. Scores for study completeness ranged from 27 to 43, with a maximum score of 47, and scores for study quality ranged from 46 to 79, with a maximum score of 90. Of 20 studies, DIVTs were evaluated in clinical settings in 2 studies (10%); time to decision was measured in 14 studies (70%); and decision accuracy was measured in 11 studies (55%). Measures of cognitive workload pooled in the meta-analysis suggested that any DIVT was an improvement over paper-based data in terms of self-reported performance, mental and temporal demand, and effort. With a maximum score of 22, median (IQR) mental demand scores for electronic display were 10 (7-13), tabular display scores were 8 (6.0-11.5), and novel visualization scores were 8 (6-12), compared with 17 (14-19) for paper. The median (IQR) temporal demand scores were also lower for all electronic visualizations compared with paper, with scores of 8 (6-11) for electronic display, 7 (6-11) for tabular and bar displays, 7 (5-11) for novel visualizations, and 16 (14.3-19.0) for paper. The median (IQR) performance scores improved for all electronic visualizations compared with paper (lower score indicates better self-reported performance), with scores of 6 (3-11) for electronic displays, 6 (4-11) for tabular and bar displays, 6 (4-11) for novel visualizations, and 14 (11-16) for paper. Frustration and physical demand domains of cognitive workload did not change, and differences between electronic displays were not significant. CONCLUSIONS AND RELEVANCE This review suggests that DIVTs are associated with increased integration and consistency of data. Much work remains to identify which visualizations effectively reduce cognitive workload to enhance decision making based on intensive care data. Standardizing human factors testing by developing a repository of open access benchmarked test protocols, using a set of outcome measures, scenarios, and data sets, may accelerate the design and selection of the most appropriate DIVT.
Collapse
Affiliation(s)
- Ying Ling Lin
- Institute of Biomaterials and Biomedical Engineering, Faculty of Engineering, University of Toronto, Toronto, Ontario, Canada
- Department of Critical Care Medicine, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Patricia Trbovich
- Institute of Biomaterials and Biomedical Engineering, Faculty of Engineering, University of Toronto, Toronto, Ontario, Canada
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
- Badeau Family Research Chair in Patient Safety and Quality Improvement, North York General Hospital, University of Toronto, Toronto, Ontario, Canada
| | - Lauren Kolodzey
- Institute of Biomaterials and Biomedical Engineering, Faculty of Engineering, University of Toronto, Toronto, Ontario, Canada
| | - Cheri Nickel
- Hospital Library and Archives, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Anne-Marie Guerguerian
- Institute of Biomaterials and Biomedical Engineering, Faculty of Engineering, University of Toronto, Toronto, Ontario, Canada
- Department of Critical Care Medicine, The Hospital for Sick Children, Toronto, Ontario, Canada
- Interdepartmental Division of Critical Care Medicine, University of Toronto, Toronto, Ontario, Canada
- Neurosciences and Mental Health Program, Research Institute, The Hospital for Sick Children, Toronto, Ontario, Canada
| |
Collapse
|
25
|
Khairat S, Coleman C, Newlin T, Rand V, Ottmar P, Bice T, Carson SS. A mixed-methods evaluation framework for electronic health records usability studies. J Biomed Inform 2019; 94:103175. [PMID: 30981897 DOI: 10.1016/j.jbi.2019.103175] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2018] [Revised: 03/08/2019] [Accepted: 04/07/2019] [Indexed: 10/27/2022]
Abstract
BACKGROUND Poor EHR design adds further challenges, especially in the areas of order entry and information visualization, with a net effect of increased rates of incidents, accidents, and mortality in ICU settings. OBJECTIVE The purpose of this study was to propose a novel, mixed-methods framework to understand EHR-related information overload by identifying and characterizing areas of suboptimal usability and clinician frustration within a vendor-based, provider-facing EHR interface. METHODS A mixed-methods, live observational usability study was conducted at a single, large, tertiary academic medical center in the Southeastern US utilizing a commercial, vendor based EHR. Physicians were asked to complete usability patient cases, provide responses to three surveys, and participant in a semi-structured interview. RESULTS Of the 25 enrolled ICU physician participants, there were 5(20%) attending physicians, 9 (36%) fellows, and 11 (44%) residents; 52% of participants were females. On average, residents were the quickest in completing the tasks while attending physician took the longest to complete the same task. Poor usability, complex interface screens, and difficulty to navigate the EHR significantly correlated with high frustration levels. Significant association were found between the occurrence of error messages and temporal demand such that more error messages resulted in longer completion time (p = .03). DISCUSSION Physicians remain frustrated with the EHR due to difficulty in finding patient information. EHR usability remains a critical challenge in healthcare, with implications for medical errors, patient safety, and clinician burnout. There is a need for scientific findings on current information needs and ways to improve EHR-related information overload.
Collapse
Affiliation(s)
- Saif Khairat
- Carolina Health Informatics Program and School of Nursing, University of North Carolina at Chapel Hill, NC, USA.
| | - Cameron Coleman
- Carolina Health Informatics Program, University of North Carolina at Chapel Hill, NC, USA
| | - Thomas Newlin
- Carolina Health Informatics Program, University of North Carolina at Chapel Hill, NC, USA
| | - Victoria Rand
- Carolina Health Informatics Program, University of North Carolina at Chapel Hill, NC, USA
| | - Paige Ottmar
- Gilling's School of Global Public Health, University of North Carolina at Chapel Hill, NC, USA
| | - Thomas Bice
- Pulmonary Diseases and Critical Care Medicine, University of North Carolina at Chapel Hill, NC, USA
| | - Shannon S Carson
- Pulmonary Diseases and Critical Care Medicine, University of North Carolina at Chapel Hill, NC, USA
| |
Collapse
|
26
|
Privitera MR. Human Factor Based Leadership: Critical Leadership Tools to Reduce Burnout and Latent Error in a Time of Accelerating Change. Health (London) 2019. [DOI: 10.4236/health.2019.119095] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
27
|
King AJ, Cooper GF, Hochheiser H, Clermont G, Hauskrecht M, Visweswaran S. Using Machine Learning to Predict the Information Seeking Behavior of Clinicians Using an Electronic Medical Record System. AMIA ... ANNUAL SYMPOSIUM PROCEEDINGS. AMIA SYMPOSIUM 2018; 2018:673-682. [PMID: 30815109 PMCID: PMC6371238] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Poor electronic medical record (EMR) usability is detrimental to both clinicians and patients. A better EMR would provide concise, context sensitive patient data, but doing so entails the difficult task of knowing which data are relevant. To determine the relevance of patient data in different contexts, we collect and model the information seeking behavior of clinicians using a learning EMR (LEMR) system. Sufficient data were collected to train predictive models for 80 different targets (e.g., glucose level, heparin administration) and 27 of them had AUROC values of greater than 0.7. These results are encouraging considering the high variation in information seeking behavior (intraclass correlation 0.40). We plan to apply these models to a new set of patient cases and adapt the LEMR interface to highlight relevant patient data, and thus provide concise, context sensitive data.
Collapse
|
28
|
|
29
|
Franke S, Rockstroh M, Hofer M, Neumuth T. The intelligent OR: design and validation of a context-aware surgical working environment. Int J Comput Assist Radiol Surg 2018; 13:1301-1308. [DOI: 10.1007/s11548-018-1791-x] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2018] [Accepted: 05/09/2018] [Indexed: 11/28/2022]
|
30
|
Hulyalkar M, Gleich SJ, Kashyap R, Barwise A, Kaur H, Dong Y, Fan L, Murthy S, Arteaga GM, Tripathi S. Design and α-testing of an electronic rounding tool (CERTAINp) to improve process of care in pediatric intensive care unit. J Clin Monit Comput 2017; 31:1313-1320. [PMID: 27757740 DOI: 10.1007/s10877-016-9946-1] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2016] [Accepted: 10/11/2016] [Indexed: 12/18/2022]
Abstract
Increasing process complexity in the pediatric intensive care unit (PICU) can lead to information overload resulting in missing pertinent information and potential errors during morning rounds. An efficient model using a novel electronic rounding tool was designed as part of a broader critical care decision support system-checklist for early recognition and treatment of acute illness and injury in pediatrics (CERTAINp). We aimed to evaluate its impact on improving the process of care during rounding. Prospective pre- and post-interventional data included: team performance baseline assessment, patient safety discussion, guideline adherence, rounding time, and a survey of Residents' and Nurses' perception using a Likert scale. Attending physicians were blinded to the components of the assessment. A total of 113 pre-intervention and 114 post-intervention roundings were recorded by direct observation. Pre-intervention (108) and post-intervention staff surveys (80) were obtained. Adherence to standard of care guidelines improved to >97 % in all data points, with maximum increase seen in discussions of ulcer prophylaxis, bowel protocol, DVT prophylaxis, skin care, glucose control and head of bed elevation (2-28 % pre-vs. 100 % for all post-intervention, p < 0.01). Significant improvement was noticed in spontaneous breathing trials, sedation breaks and need for devices (45-57 % pre- vs. 100 % for all post-intervention, p < 0.01). Rounding time (mean ± SD) increased by 2 min/patient (8.0 ± 5.8 min pre-intervention vs. 9.9 ± 5.7 min post-intervention, p = 0.002). Staff reported improved perception of all aspects of rounding. Utilization of the CERTAINp rounding tool led to perfect compliance to the discussion of best practice guidelines; had minimal impact on rounding time and improved PICU staff satisfaction.
Collapse
Affiliation(s)
- Manasi Hulyalkar
- Department of Pediatric and Adolescent Medicine, Mayo Clinic, Rochester, MN, USA
| | - Stephen J Gleich
- Department of Pediatric and Adolescent Medicine, Mayo Clinic, Rochester, MN, USA
| | - Rahul Kashyap
- METRIC-Multidisciplinary Epidemiology and Translational Research in Intensive Care, Mayo Clinic, Rochester, MN, USA
| | - Amelia Barwise
- METRIC-Multidisciplinary Epidemiology and Translational Research in Intensive Care, Mayo Clinic, Rochester, MN, USA
| | - Harsheen Kaur
- Department of Pediatric and Adolescent Medicine, Mayo Clinic, Rochester, MN, USA
| | - Yue Dong
- METRIC-Multidisciplinary Epidemiology and Translational Research in Intensive Care, Mayo Clinic, Rochester, MN, USA
| | - Lei Fan
- METRIC-Multidisciplinary Epidemiology and Translational Research in Intensive Care, Mayo Clinic, Rochester, MN, USA
| | - Srinivas Murthy
- Division of Critical Care, Department of Pediatrics, University of British Columbia, Vancouver, Canada
| | - Grace M Arteaga
- Department of Pediatric and Adolescent Medicine, Mayo Clinic, Rochester, MN, USA
| | - Sandeep Tripathi
- Department of Pediatric and Adolescent Medicine, Mayo Clinic, Rochester, MN, USA.
- Department of Pediatrics, University of Illinois College of Medicine, Peoria, IL, USA.
| |
Collapse
|
31
|
Lin YL, Guerguerian AM, Tomasi J, Laussen P, Trbovich P. "Usability of data integration and visualization software for multidisciplinary pediatric intensive care: a human factors approach to assessing technology". BMC Med Inform Decis Mak 2017; 17:122. [PMID: 28806954 PMCID: PMC5557066 DOI: 10.1186/s12911-017-0520-7] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2017] [Accepted: 08/04/2017] [Indexed: 11/17/2022] Open
Abstract
Background Intensive care clinicians use several sources of data in order to inform decision-making. We set out to evaluate a new interactive data integration platform called T3™ made available for pediatric intensive care. Three primary functions are supported: tracking of physiologic signals, displaying trajectory, and triggering decisions, by highlighting data or estimating risk of patient instability. We designed a human factors study to identify interface usability issues, to measure ease of use, and to describe interface features that may enable or hinder clinical tasks. Methods Twenty-two participants, consisting of bedside intensive care physicians, nurses, and respiratory therapists, tested the T3™ interface in a simulation laboratory setting. Twenty tasks were performed with a true-to-setting, fully functional, prototype, populated with physiological and therapeutic intervention patient data. Primary data visualization was time series and secondary visualizations were: 1) shading out-of-target values, 2) mini-trends with exaggerated maxima and minima (sparklines), and 3) bar graph of a 16-parameter indicator. Task completion was video recorded and assessed using a use error rating scale. Usability issues were classified in the context of task and type of clinician. A severity rating scale was used to rate potential clinical impact of usability issues. Results Time series supported tracking a single parameter but partially supported determining patient trajectory using multiple parameters. Visual pattern overload was observed with multiple parameter data streams. Automated data processing using shading and sparklines was often ignored but the 16-parameter data reduction algorithm, displayed as a persistent bar graph, was visually intuitive. However, by selecting or automatically processing data, triggering aids distorted the raw data that clinicians use regularly. Consequently, clinicians could not rely on new data representations because they did not know how they were established or derived. Conclusions Usability issues, observed through contextual use, provided directions for tangible design improvements of data integration software that may lessen use errors and promote safe use. Data-driven decision making can benefit from iterative interface redesign involving clinician-users in simulated environments. This study is a first step in understanding how software can support clinicians’ decision making with integrated continuous monitoring data. Importantly, testing of similar platforms by all the different disciplines who may become clinician users is a fundamental step necessary to understand the impact on clinical outcomes of decision aids. Electronic supplementary material The online version of this article (doi:10.1186/s12911-017-0520-7) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Ying Ling Lin
- Institute of Biomaterials and Biomedical Engineering, University of Toronto, Rosebrugh Building (RS), 164 College Street, Room 407, Toronto, ON, M5S 3G9, Canada.,Department of Critical Care Medicine, The Hospital for Sick Children, Canada, 555 University Ave., 2nd Floor, Atrium - Room 2830A, Toronto, ON, M5G 1X8, Canada
| | - Anne-Marie Guerguerian
- Institute of Biomaterials and Biomedical Engineering, University of Toronto, Rosebrugh Building (RS), 164 College Street, Room 407, Toronto, ON, M5S 3G9, Canada.,Department of Critical Care Medicine, The Hospital for Sick Children, Canada, 555 University Ave., 2nd Floor, Atrium - Room 2830A, Toronto, ON, M5G 1X8, Canada.,Neurosciences and Mental Health Research, The Hospital for Sick Children Research Institute, Peter Gilgan Centre for Research & Learning, 686 Bay Street, Toronto, ON, M5G 0A4, Canada
| | - Jessica Tomasi
- Institute of Biomaterials and Biomedical Engineering, University of Toronto, Rosebrugh Building (RS), 164 College Street, Room 407, Toronto, ON, M5S 3G9, Canada
| | - Peter Laussen
- Department of Critical Care Medicine, The Hospital for Sick Children, Canada, 555 University Ave., 2nd Floor, Atrium - Room 2830A, Toronto, ON, M5G 1X8, Canada
| | - Patricia Trbovich
- Institute of Biomaterials and Biomedical Engineering, University of Toronto, Rosebrugh Building (RS), 164 College Street, Room 407, Toronto, ON, M5S 3G9, Canada. .,Institute of Health Policy, Management and Evaluation, University of Toronto, 155 College St., Suite 425, Toronto, ON, M5T 3M6, Canada. .,Research and Innovation, North York General Hospital, 4001 Leslie Street, Toronto, ON, M2K 1E1, Canada.
| |
Collapse
|
32
|
Abstract
Due to the rapidly evolving medical, technological, and technical possibilities, surgical procedures are becoming more and more complex. On the one hand, this offers an increasing number of advantages for patients, such as enhanced patient safety, minimal invasive interventions, and less medical malpractices. On the other hand, it also heightens pressure on surgeons and other clinical staff and has brought about a new policy in hospitals, which must rely on a great number of economic, social, psychological, qualitative, practical, and technological resources. As a result, medical disciplines, such as surgery, are slowly merging with technical disciplines. However, this synergy is not yet fully matured. The current information and communication technology in hospitals cannot manage the clinical and operational sequence adequately. The consequences are breaches in the surgical workflow, extensions in procedure times, and media disruptions. Furthermore, the data accrued in operating rooms (ORs) by surgeons and systems are not sufficiently implemented. A flood of information, “big data”, is available from information systems. That might be deployed in the context of Medicine 4.0 to facilitate the surgical treatment. However, it is unused due to infrastructure breaches or communication errors. Surgical process models (SPMs) alleviate these problems. They can be defined as simplified, formal, or semiformal representations of a network of surgery-related activities, reflecting a predefined subset of interest. They can employ different means of generation, languages, and data acquisition strategies. They can represent surgical interventions with high resolution, offering qualifiable and quantifiable information on the course of the intervention on the level of single, minute, surgical work-steps. The basic idea is to gather information concerning the surgical intervention and its activities, such as performance time, surgical instrument used, trajectories, movements, or intervention phases. These data can be gathered by means of workflow recordings. These recordings are abstracted to represent an individual surgical process as a model and are an essential requirement to enable Medicine 4.0 in the OR. Further abstraction can be generated by merging individual process models to form generic SPMs to increase the validity for a larger number of patients. Furthermore, these models can be applied in a wide variety of use-cases. In this regard, the term “modeling” can be used to support either one or more of the following tasks: “to describe”, “to understand”, “to explain”, to optimize”, “to learn”, “to teach”, or “to automate”. Possible use-cases are requirements analyses, evaluating surgical assist systems, generating surgeon-specific training-recommendation, creating workflow management systems for ORs, and comparing different surgical strategies. The presented chapter will give an introduction into this challenging topic, presenting different methods to generate SPMs from the workflow in the OR, as well as various use-cases, and state-of-the-art research in this field. Although many examples in the article are given according to SPMs that were computed based on observations, the same approaches can be easily applied to SPMs that were measured automatically and mined from big data.
Collapse
Affiliation(s)
- Thomas Neumuth
- Innovation Center Computer Assisted Surgery (ICCAS), Universität Leipzig, Leipzig, Germany
| |
Collapse
|
33
|
Upadhyaya SG, Murphree DH, Ngufor CG, Knight AM, Cronk DJ, Cima RR, Curry TB, Pathak J, Carter RE, Kor DJ. Automated Diabetes Case Identification Using Electronic Health Record Data at a Tertiary Care Facility. Mayo Clin Proc Innov Qual Outcomes 2017; 1:100-110. [PMID: 30225406 PMCID: PMC6135013 DOI: 10.1016/j.mayocpiqo.2017.04.005] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/05/2022] Open
Abstract
Objective To develop and validate a phenotyping algorithm for the identification of patients with type 1 and type 2 diabetes mellitus (DM) preoperatively using routinely available clinical data from electronic health records. Patients and Methods We used first-order logic rules (if-then-else rules) to imply the presence or absence of DM types 1 and 2. The “if” clause of each rule is a conjunction of logical and, or predicates that provides evidence toward or against the presence of DM. The rule includes International Classification of Diseases, Ninth Revision, Clinical Modification diagnostic codes, outpatient prescription information, laboratory values, and positive annotation of DM in patients’ clinical notes. This study was conducted from March 2, 2015, through February 10, 2016. The performance of our rule-based approach and similar approaches proposed by other institutions was evaluated with a reference standard created by an expert reviewer and implemented for routine clinical care at an academic medical center. Results A total of 4208 surgical patients (mean age, 52 years; males, 48%) were analyzed to develop the phenotyping algorithm. Expert review identified 685 patients (16.28% of the full cohort) as having DM. Our proposed method identified 684 patients (16.25%) as having DM. The algorithm performed well—99.70% sensitivity, 99.97% specificity—and compared favorably with previous approaches. Conclusion Among patients undergoing surgery, determination of DM can be made with high accuracy using simple, computationally efficient rules. Knowledge of patients’ DM status before surgery may alter physicians’ care plan and reduce postsurgical complications. Nevertheless, future efforts are necessary to determine the effect of first-order logic rules on clinical processes and patient outcomes.
Collapse
Key Words
- CCW, Chronic Condition Data Warehouse
- DDC, Durham Diabetes Coalition
- DM, diabetes mellitus
- EHR, electronic health record
- HbA1c of NYC, Hemoglobin A1c of New York City
- HbA1c, hemoglobin A1c
- ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification
- MICS, Mayo Integrated Clinical Systems
- NLP, natural language processing
- SUPREME-DM, Surveillance, Prevention, and Management of Diabetes Mellitus
- T1DM, type 1 diabetes mellitus
- T2DM, type 2 diabetes mellitus
- eMERGE, Electronic Medical Records and Genomics
Collapse
Affiliation(s)
| | | | - Che G Ngufor
- Department of Health Sciences Research, Mayo Clinic, Rochester, MN
| | - Alison M Knight
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, MN
| | - Daniel J Cronk
- Department of Information Technology, Mayo Clinic, Rochester, MN
| | - Robert R Cima
- Division of Colon and Rectal Surgery, Mayo Clinic, Rochester, MN.,Robert D. and Patricia E. Kern Center for Science of Health Care Delivery, Mayo Clinic, Rochester, MN
| | - Timothy B Curry
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, MN.,Department of Physiology and Biomedical Engineering, Mayo Clinic, Rochester, MN
| | | | - Rickey E Carter
- Department of Health Sciences Research, Mayo Clinic, Rochester, MN
| | - Daryl J Kor
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, MN
| |
Collapse
|
34
|
Fedosov V, Dziadzko M, Dearani JA, Brown DR, Pickering BW, Herasevich V. Decision Support Tool to Improve Glucose Control Compliance After Cardiac Surgery. AACN Adv Crit Care 2017; 27:274-282. [PMID: 27959310 DOI: 10.4037/aacnacc2016634] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/01/2022]
Abstract
Hyperglycemia control is associated with improved outcomes in patients undergoing cardiac surgery. The Surgical Care Improvement Project metric (SCIP-inf-4) was introduced as a performance measure in surgical patients and included hyperglycemia control. Compliance with the SCIP-inf-4 metric remains suboptimal. A novel real-time decision support tool (DST) with guaranteed feedback that is based on the existing electronic medical record system was developed at a tertiary academic center. Implementation of the DST increased the compliance rate with the SCIP-inf-4 from 87.3% to 96.5%. Changes in tested clinical outcomes were not observed with improved metric compliance. This new framework can serve as a backbone for development of quality control processes for other metrics. Further and, ideally, multicenter studies are required to test if implementation of electronic DSTs will translate into improved resource utilization and outcomes for patients.
Collapse
Affiliation(s)
- Vitali Fedosov
- Vitali Fedosov is Research Fellow, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Mikhail Dziadzko is Research Fellow, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Joseph A. Dearani is Professor of Surgery, Department of Surgery, Mayo Clinic, Rochester, Minnesota. Daniel R. Brown is Professor of Anesthesiology, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Brian W. Pickering is Assistant Professor of Anesthesiology, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Vitaly Herasevich is Associate Professor of Anesthesiology and Medicine, Department of Anesthesiology, Mayo Clinic, 200 First St SW, Rochester, MN, 55905
| | - Mikhail Dziadzko
- Vitali Fedosov is Research Fellow, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Mikhail Dziadzko is Research Fellow, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Joseph A. Dearani is Professor of Surgery, Department of Surgery, Mayo Clinic, Rochester, Minnesota. Daniel R. Brown is Professor of Anesthesiology, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Brian W. Pickering is Assistant Professor of Anesthesiology, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Vitaly Herasevich is Associate Professor of Anesthesiology and Medicine, Department of Anesthesiology, Mayo Clinic, 200 First St SW, Rochester, MN, 55905
| | - Joseph A Dearani
- Vitali Fedosov is Research Fellow, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Mikhail Dziadzko is Research Fellow, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Joseph A. Dearani is Professor of Surgery, Department of Surgery, Mayo Clinic, Rochester, Minnesota. Daniel R. Brown is Professor of Anesthesiology, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Brian W. Pickering is Assistant Professor of Anesthesiology, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Vitaly Herasevich is Associate Professor of Anesthesiology and Medicine, Department of Anesthesiology, Mayo Clinic, 200 First St SW, Rochester, MN, 55905
| | - Daniel R Brown
- Vitali Fedosov is Research Fellow, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Mikhail Dziadzko is Research Fellow, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Joseph A. Dearani is Professor of Surgery, Department of Surgery, Mayo Clinic, Rochester, Minnesota. Daniel R. Brown is Professor of Anesthesiology, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Brian W. Pickering is Assistant Professor of Anesthesiology, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Vitaly Herasevich is Associate Professor of Anesthesiology and Medicine, Department of Anesthesiology, Mayo Clinic, 200 First St SW, Rochester, MN, 55905
| | - Brian W Pickering
- Vitali Fedosov is Research Fellow, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Mikhail Dziadzko is Research Fellow, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Joseph A. Dearani is Professor of Surgery, Department of Surgery, Mayo Clinic, Rochester, Minnesota. Daniel R. Brown is Professor of Anesthesiology, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Brian W. Pickering is Assistant Professor of Anesthesiology, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Vitaly Herasevich is Associate Professor of Anesthesiology and Medicine, Department of Anesthesiology, Mayo Clinic, 200 First St SW, Rochester, MN, 55905
| | - Vitaly Herasevich
- Vitali Fedosov is Research Fellow, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Mikhail Dziadzko is Research Fellow, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Joseph A. Dearani is Professor of Surgery, Department of Surgery, Mayo Clinic, Rochester, Minnesota. Daniel R. Brown is Professor of Anesthesiology, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Brian W. Pickering is Assistant Professor of Anesthesiology, Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota. Vitaly Herasevich is Associate Professor of Anesthesiology and Medicine, Department of Anesthesiology, Mayo Clinic, 200 First St SW, Rochester, MN, 55905
| |
Collapse
|
35
|
Harrison AM, Thongprayoon C, Aakre CA, Jeng JY, Dziadzko MA, Gajic O, Pickering BW, Herasevich V. Comparison of methods of alert acknowledgement by critical care clinicians in the ICU setting. PeerJ 2017; 5:e3083. [PMID: 28316887 PMCID: PMC5354075 DOI: 10.7717/peerj.3083] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2016] [Accepted: 02/12/2017] [Indexed: 12/29/2022] Open
Abstract
BACKGROUND Electronic Health Record (EHR)-based sepsis alert systems have failed to demonstrate improvements in clinically meaningful endpoints. However, the effect of implementation barriers on the success of new sepsis alert systems is rarely explored. OBJECTIVE To test the hypothesis time to severe sepsis alert acknowledgement by critical care clinicians in the ICU setting would be reduced using an EHR-based alert acknowledgement system compared to a text paging-based system. STUDY DESIGN In one arm of this simulation study, real alerts for patients in the medical ICU were delivered to critical care clinicians through the EHR. In the other arm, simulated alerts were delivered through text paging. The primary outcome was time to alert acknowledgement. The secondary outcomes were a structured, mixed quantitative/qualitative survey and informal group interview. RESULTS The alert acknowledgement rate from the severe sepsis alert system was 3% (N = 148) and 51% (N = 156) from simulated severe sepsis alerts through traditional text paging. Time to alert acknowledgement from the severe sepsis alert system was median 274 min (N = 5) and median 2 min (N = 80) from text paging. The response rate from the EHR-based alert system was insufficient to compare primary measures. However, secondary measures revealed important barriers. CONCLUSION Alert fatigue, interruption, human error, and information overload are barriers to alert and simulation studies in the ICU setting.
Collapse
Affiliation(s)
- Andrew M. Harrison
- Medical Scientist Training Program, Mayo Clinic, Rochester, MN, United States of America
| | - Charat Thongprayoon
- Department of Anesthesiology, Mayo Clinic, Rochester, MN, United States of America
| | - Christopher A. Aakre
- Department of Internal Medicine, Mayo Clinic, Rochester, MN, United States of America
| | - Jack Y. Jeng
- Mayo Medical School, Mayo Clinic, Rochester, MN, United States of America
| | - Mikhail A. Dziadzko
- Department of Anesthesiology, Mayo Clinic, Rochester, MN, United States of America
| | - Ognjen Gajic
- Division of Pulmonology and Critical Care Medicine, Mayo Clinic, Rochester, MN, United States of America
| | - Brian W. Pickering
- Department of Anesthesiology, Mayo Clinic, Rochester, MN, United States of America
| | - Vitaly Herasevich
- Department of Anesthesiology, Mayo Clinic, Rochester, MN, United States of America
| |
Collapse
|
36
|
Kogan A, Pennington KM, Vallabhajosyula S, Dziadzko M, Bennett CE, Jensen JB, Gajic O, O'Horo JC. Reliability and Validity of the Checklist for Early Recognition and Treatment of Acute Illness and Injury as a Charting Tool in the Medical Intensive Care Unit. Indian J Crit Care Med 2017; 21:746-750. [PMID: 29279635 PMCID: PMC5699002 DOI: 10.4103/ijccm.ijccm_209_17] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022] Open
Abstract
Background: Resuscitation of critically ill patients is complex and potentially prone to diagnostic errors and therapeutic harm. The Checklist for early recognition and treatment of acute illness and injury (CERTAIN) is an electronic tool that aims to provide decision-support, charting, and prompting for standardization. This study sought to evaluate the validity and reliability of CERTAIN in a real-time Intensive Care Unit (ICU). Materials and Methods: This was a prospective pilot study in the medical ICU of a tertiary care medical center. A total of thirty patient encounters over 2 months period were charted independently by two CERTAIN investigators. The inter-observer recordings and comparison to the electronic medical records (EMR) were used to evaluate reliability and validity, respectively. The primary outcome was reliability and validity measured using Cohen's Kappa statistic. Secondary outcomes included time to completion, user satisfaction, and learning curve. Results: A total of 30 patients with a median age of 59 (42–78) years and median acute physiology and chronic health evaluation III score of 38 (23–50) were included in this study. Inter-observer agreement was very good (κ = 0.79) in this study and agreement between CERTAIN and the EMR was good (κ = 0.5). CERTAIN charting was completed in real-time that was 121 (92–150) min before completion of EMR charting. The subjective learning curve was 3.5 patients without differences in providers with different levels of training. Conclusions: CERTAIN provides a reliable and valid method to evaluate resuscitation events in real time. CERTAIN provided the ability to complete data in real-time.
Collapse
Affiliation(s)
- Alexander Kogan
- Department of Internal Medicine, Division of Pulmonary and Critical Care Medicine, Mayo Clinic, MN, USA.,Research Faculty, Multidisciplinary Epidemiology and Translational Research in Intensive Care Laboratory, MN, USA.,Center for Clinical and Translational Science, Mayo Clinic Graduate School of Biomedical Sciences, Mayo Clinic, MN, USA
| | - Kelly M Pennington
- Research Faculty, Multidisciplinary Epidemiology and Translational Research in Intensive Care Laboratory, MN, USA.,Department of Internal Medicine, Mayo Clinic, MN, USA
| | - Saraschandra Vallabhajosyula
- Department of Internal Medicine, Division of Pulmonary and Critical Care Medicine, Mayo Clinic, MN, USA.,Research Faculty, Multidisciplinary Epidemiology and Translational Research in Intensive Care Laboratory, MN, USA.,Center for Clinical and Translational Science, Mayo Clinic Graduate School of Biomedical Sciences, Mayo Clinic, MN, USA
| | - Mikhail Dziadzko
- Department of Anesthesiology, Division of Critical Care Anesthesiology, Mayo Clinic, MN, USA
| | - Courtney E Bennett
- Department of Internal Medicine, Division of Pulmonary and Critical Care Medicine, Mayo Clinic, MN, USA.,Research Faculty, Multidisciplinary Epidemiology and Translational Research in Intensive Care Laboratory, MN, USA.,Center for Clinical and Translational Science, Mayo Clinic Graduate School of Biomedical Sciences, Mayo Clinic, MN, USA
| | - Jeffrey B Jensen
- Department of Anesthesiology, Division of Critical Care Anesthesiology, Mayo Clinic, MN, USA
| | - Ognjen Gajic
- Department of Internal Medicine, Division of Pulmonary and Critical Care Medicine, Mayo Clinic, MN, USA.,Research Faculty, Multidisciplinary Epidemiology and Translational Research in Intensive Care Laboratory, MN, USA
| | - John C O'Horo
- Department of Internal Medicine, Division of Pulmonary and Critical Care Medicine, Mayo Clinic, MN, USA.,Research Faculty, Multidisciplinary Epidemiology and Translational Research in Intensive Care Laboratory, MN, USA.,Department of Internal Medicine, Division of Infectious Diseases, Mayo Clinic, MN, USA.,Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Mayo Clinic Rochester, MN, USA
| |
Collapse
|
37
|
Dziadzko MA, Gajic O, Pickering BW, Herasevich V. Clinical calculators in hospital medicine: Availability, classification, and needs. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2016; 133:1-6. [PMID: 27393794 DOI: 10.1016/j.cmpb.2016.05.006] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/06/2015] [Revised: 03/08/2016] [Accepted: 05/17/2016] [Indexed: 06/06/2023]
Abstract
OBJECTIVE Clinical calculators are widely used in modern clinical practice, but are not generally applied to electronic health record (EHR) systems. Important barriers to the application of these clinical calculators into existing EHR systems include the need for real-time calculation, human-calculator interaction, and data source requirements. The objective of this study was to identify, classify, and evaluate the use of available clinical calculators for clinicians in the hospital setting. METHODS Dedicated online resources with medical calculators and providers of aggregated medical information were queried for readily available clinical calculators. Calculators were mapped by clinical categories, mechanism of calculation, and the goal of calculation. Online statistics from selected Internet resources and clinician opinion were used to assess the use of clinical calculators. RESULTS One hundred seventy-six readily available calculators in 4 categories, 6 primary specialties, and 40 subspecialties were identified. The goals of calculation included prediction, severity, risk estimation, diagnostic, and decision-making aid. A combination of summation logic with cutoffs or rules was the most frequent mechanism of computation. Combined results, online resources, statistics, and clinician opinion identified 13 most utilized calculators. CONCLUSION Although not an exhaustive list, a total of 176 validated calculators were identified, classified, and evaluated for usefulness. Most of these calculators are used for adult patients in the critical care or internal medicine settings. Thirteen of 176 clinical calculators were determined to be useful in our institution. All of these calculators have an interface for manual input.
Collapse
Affiliation(s)
| | - Ognjen Gajic
- Division of Pulmonology and Critical Care Medicine, Mayo Clinic, Rochester, MN, USA
| | | | | |
Collapse
|
38
|
Friedman ML, McBride ME. Changes in cognitive function after pediatric intensive care unit rounds: a prospective study. ACTA ACUST UNITED AC 2016. [PMID: 29536896 DOI: 10.1515/dx-2016-0018] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
BACKGROUND Mental fatigue is impaired cognitive function induced by engaging in cognitively demanding activities. Pediatric intensive care unit (PICU) rounds are demanding and may be a cause of impaired cognitive functioning. The purpose of this study is to evaluate if PICU rounds induce poorer performance on cognitive tasks after rounds compared to before rounds and assess the feasibility of cognitive testing in the PICU. METHODS This was a prospective study of residents in the PICU. Participants were evaluated before and after rounds on a single day, consisting of two tests of cognitive function that are sensitive to mental fatigue, the cognitive estimation test (CET) and the repeatable episodic memory test (REMT). RESULTS Thirty residents participated. The mean length of rounds was 191 min (SD 33.8 min), the mean number of patients rounded on by the team was 14.9 (SD 2.3) and the median patients presented by the participant was two (range 0-6). The average number of words recalled on the REMT was significantly lower after rounds compared to before (29.6 vs. 31.2, p < 0.05). There were significantly more falsely recalled words after rounds (1.3 vs. 0.7, p=0.02). There was a correlation between worsening performance and later time of testing in the 4-week PICU rotation (r=0.42, p < 0.02). There were no differences in performance on the CET. CONCLUSIONS PICU rounds induced impairments on cognitive testing but the effect size is small and not consistent across tests. There is an increased susceptibility to impaired cognition induced by rounds over the course of a rotation, this finding merits further investigation.
Collapse
Affiliation(s)
- Matthew L Friedman
- 1Section of Pediatric Critical Care, Riley Hospital for Children and Indiana University School of Medicine - Pediatrics, 705 Riley Hospital Drive, Indianapolis, IN 46202, United States of America
| | - Mary E McBride
- 2Division of Pediatric Cardiology, Ann and Robert H. Lurie Children's Hospital of Chicago and Northwestern University Feinberg School of Medicine, Chicago, IL, United States of America
| |
Collapse
|
39
|
Hoskote SS, Racedo Africano CJ, Braun AB, O'Horo JC, Sevilla Berrios RA, Loftsgard TO, Bryant KM, Iyer VN, Smischney NJ. Improving the Quality of Handoffs in Patient Care Between Critical Care Providers in the Intensive Care Unit. Am J Med Qual 2016; 32:376-383. [PMID: 27329489 DOI: 10.1177/1062860616654758] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
With the ever-increasing adoption of shift models for intensive care unit (ICU) staffing, improving shift-to-shift handoffs represents an important step in reducing medical errors. The authors developed an electronic handoff tool integrated within the existing electronic medical record to improve handoffs in an adult ICU. First, stakeholder (staff intensivists, fellows, and nurse practitioners/physician assistants) input was sought to define what elements they perceived as being essential to a quality handoff. The principal outcome measure of handoff accuracy was the concordance between data transmitted by the outgoing team and data received by the incoming team (termed as agreement). Based on stakeholder input, the authors developed the handoff tool and provided regular education on its use. Handoffs were observed before and after implementation of the tool. There was an increase in the level of agreement for tasks and other important data points handed off without an increase in the time required to complete the handoff.
Collapse
|
40
|
Aakre CA, Chaudhry R, Pickering BW, Herasevich V. Information Needs Assessment for a Medicine Ward-Focused Rounding Dashboard. J Med Syst 2016; 40:183. [PMID: 27307266 DOI: 10.1007/s10916-016-0542-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2016] [Accepted: 06/08/2016] [Indexed: 12/22/2022]
Abstract
To identify the routine information needs of inpatient clinicians on the general wards for the development of an electronic dashboard. Survey of internal medicine and subspecialty clinicians from March 2014-July 2014 at Saint Marys Hospital in Rochester, Minnesota. An information needs assessment was generated from all unique data elements extracted from all handoff and rounding tools used by clinicians in our ICUs and general wards. An electronic survey was distributed to 104 inpatient medical providers. 89 unique data elements were identified from currently utilized handoff and rounding instruments. All data elements were present in our multipurpose ICU-based dashboard. 42 of 104 (40 %) surveys were returned. Data elements important (50/89, 56 %) and unimportant (24/89, 27 %) for routine use were identified. No significant differences in data element ranking were observed between supervisory and nonsupervisory roles. The routine information needs of general ward clinicians are a subset of data elements used routinely by ICU clinicians. Our findings suggest an electronic dashboard could be adapted from the critical care setting to the general wards with minimal modification.
Collapse
Affiliation(s)
- Christopher A Aakre
- Division of General Internal Medicine, Department of Internal Medicine, Mayo Clinic, 200 First St SW, Rochester, MN, USA.
| | - Rajeev Chaudhry
- Division of Primary Care Internal Medicine, Department of Internal Medicine, Mayo Clinic, Rochester, MN, USA
| | | | - Vitaly Herasevich
- Department of Anesthesiology, Mayo Clinic, Rochester, MN, USA
- Multidisciplinary Epidemiology and Translation Research in Intensive Care (METRIC), Mayo Clinic, Rochester, MN, USA
| |
Collapse
|
41
|
Cahan A, Cimino JJ. Visual assessment of the similarity between a patient and trial population: Is This Clinical Trial Applicable to My Patient? Appl Clin Inform 2016; 7:477-88. [PMID: 27437055 DOI: 10.4338/aci-2015-12-ra-0178] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2015] [Accepted: 03/23/2016] [Indexed: 01/16/2023] Open
Abstract
BACKGROUND A critical consideration when applying the results of a clinical trial to a particular patient is the degree of similarity of the patient to the trial population. However, similarity assessment rarely is practical in the clinical setting. Here, we explore means to support similarity assessment by clinicians. METHODS A scale chart was developed to represent the distribution of reported clinical and demographic characteristics of clinical trial participant populations. Constructed for an individual patient, the scale chart shows the patient's similarity to the study populations in a graphical manner. A pilot test case was conducted using case vignettes assessed by clinicians. Two pairs of clinical trials were used, each addressing a similar clinical question. Scale charts were manually constructed for each simulated patient. Clinicians were asked to estimate the degree of similarity of each patient to the populations of a pair of trials. Assessors relied on either the scale chart, a summary table (aligning characteristics of 2 trial populations), or original trial reports. Assessment time and between-assessor agreement were compared. Population characteristics considered important by assessors were recorded. RESULTS Six assessors evaluated 6 cases each. Using a visual scale chart, agreement between physicians was higher and the time required for similarity assessment was comparable. CONCLUSION We suggest that further research is warranted to explore visual tools facilitating the choice of the most applicable clinical trial to a specific patient. Automating patient and trial population characteristics extraction is key to support this effort.
Collapse
Affiliation(s)
- Amos Cahan
- IBM T.J. Watson Research Center, Yorktown Heights, NY; National Library of Medicine, Bethesda, MD; Informatics Institute
| | - James J Cimino
- University of Alabama at Birmingham, Birmingham, AL; National Institutes of Health Clinical Center, Bethesda, MD
| |
Collapse
|
42
|
Dziadzko MA, Thongprayoon C, Ahmed A, Tiong IC, Li M, Brown DR, Pickering BW, Herasevich V. Automatic quality improvement reports in the intensive care unit: One step closer toward meaningful use. World J Crit Care Med 2016; 5:165-170. [PMID: 27152259 PMCID: PMC4848159 DOI: 10.5492/wjccm.v5.i2.165] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/31/2015] [Revised: 10/27/2015] [Accepted: 12/18/2015] [Indexed: 02/06/2023] Open
Abstract
AIM: To examine the feasibility and validity of electronic generation of quality metrics in the intensive care unit (ICU).
METHODS: This minimal risk observational study was performed at an academic tertiary hospital. The Critical Care Independent Multidisciplinary Program at Mayo Clinic identified and defined 11 key quality metrics. These metrics were automatically calculated using ICU DataMart, a near-real time copy of all ICU electronic medical record (EMR) data. The automatic report was compared with data from a comprehensive EMR review by a trained investigator. Data was collected for 93 randomly selected patients admitted to the ICU during April 2012 (10% of admitted adult population). This study was approved by the Mayo Clinic Institution Review Board.
RESULTS: All types of variables needed for metric calculations were found to be available for manual and electronic abstraction, except information for availability of free beds for patient-specific time-frames. There was 100% agreement between electronic and manual data abstraction for ICU admission source, admission service, and discharge disposition. The agreement between electronic and manual data abstraction of the time of ICU admission and discharge were 99% and 89%. The time of hospital admission and discharge were similar for both the electronically and manually abstracted datasets. The specificity of the electronically-generated report was 93% and 94% for invasive and non-invasive ventilation use in the ICU. One false-positive result for each type of ventilation was present. The specificity for ICU and in-hospital mortality was 100%. Sensitivity was 100% for all metrics.
CONCLUSION: Our study demonstrates excellent accuracy of electronically-generated key ICU quality metrics. This validates the feasibility of automatic metric generation.
Collapse
|
43
|
Hafeez B, Paolicchi J, Pon S, Howell JD, Grinspan ZM. Feasibility of Automatic Extraction of Electronic Health Data to Evaluate a Status Epilepticus Clinical Protocol. J Child Neurol 2016; 31:709-16. [PMID: 26518205 DOI: 10.1177/0883073815613564] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/15/2015] [Accepted: 09/29/2015] [Indexed: 11/17/2022]
Abstract
Status epilepticus is a common neurologic emergency in children. Pediatric medical centers often develop protocols to standardize care. Widespread adoption of electronic health records by hospitals affords the opportunity for clinicians to rapidly, and electronically evaluate protocol adherence. We reviewed the clinical data of a small sample of 7 children with status epilepticus, in order to (1) qualitatively determine the feasibility of automated data extraction and (2) demonstrate a timeline-style visualization of each patient's first 24 hours of care. Qualitatively, our observations indicate that most clinical data are well labeled in structured fields within the electronic health record, though some important information, particularly electroencephalography (EEG) data, may require manual abstraction. We conclude that a visualization that clarifies a patient's clinical course can be automatically created using the patient's electronic clinical data, supplemented with some manually abstracted data. Future work could use this timeline to evaluate adherence to status epilepticus clinical protocols.
Collapse
Affiliation(s)
- Baria Hafeez
- Department of Healthcare Policy & Research, Weill Cornell Medicine, New York, NY, USA
| | - Juliann Paolicchi
- Komansky Center for Children's Health, Weill Cornell Medicine, New York, NY, USA NewYork-Presbyterian Hospital, New York, NY, USA
| | - Steven Pon
- Division of Pediatric Critical Care, Weill Cornell Medicine, New York, NY, USA Komansky Center for Children's Health, Weill Cornell Medicine, New York, NY, USA NewYork-Presbyterian Hospital, New York, NY, USA
| | - Joy D Howell
- Division of Pediatric Critical Care, Weill Cornell Medicine, New York, NY, USA Komansky Center for Children's Health, Weill Cornell Medicine, New York, NY, USA NewYork-Presbyterian Hospital, New York, NY, USA
| | - Zachary M Grinspan
- Department of Healthcare Policy & Research, Weill Cornell Medicine, New York, NY, USA Division of Pediatric Critical Care, Weill Cornell Medicine, New York, NY, USA Komansky Center for Children's Health, Weill Cornell Medicine, New York, NY, USA NewYork-Presbyterian Hospital, New York, NY, USA
| |
Collapse
|
44
|
Abstract
In medicine, providers strive to produce quality outcomes and work to continually improve those outcomes. Whether it is reducing cost, decreasing length of stay, mitigating nosocomial infections, or improving survival, there are a myriad of complex factors that contribute to each outcome. One of the greatest challenges to outcome improvement is in pediatric intensive care units, which tend to host the sickest, most complex, smallest, and frailest of pediatric patients. This article highlights some studies and advances in informatics that have influenced intensive care unit outcomes.
Collapse
|
45
|
Harrison AM, Gajic O, Pickering BW, Herasevich V. Development and Implementation of Sepsis Alert Systems. Clin Chest Med 2016; 37:219-29. [PMID: 27229639 DOI: 10.1016/j.ccm.2016.01.004] [Citation(s) in RCA: 29] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
Abstract
Development and implementation of sepsis alert systems is challenging, particularly outside the monitored intensive care unit (ICU) setting. Barriers to wider use of sepsis alerts include evolving clinical definitions of sepsis, information overload, and alert fatigue, due to suboptimal alert performance. Outside the ICU, barriers include differences in health care delivery models, charting behaviors, and availability of electronic data. Current evidence does not support routine use of sepsis alert systems in clinical practice. Continuous improvement in the afferent and efferent aspects will help translate theoretic advantages into measurable patient benefit.
Collapse
Affiliation(s)
- Andrew M Harrison
- Medical Scientist Training Program, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | - Ognjen Gajic
- Division of Pulmonology and Critical Care Medicine, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | - Brian W Pickering
- Department of Anesthesiology, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA
| | - Vitaly Herasevich
- Department of Anesthesiology, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA.
| |
Collapse
|
46
|
Dziadzko MA, Herasevich V, Sen A, Pickering BW, Knight AMA, Moreno Franco P. User perception and experience of the introduction of a novel critical care patient viewer in the ICU setting. Int J Med Inform 2016; 88:86-91. [PMID: 26878767 DOI: 10.1016/j.ijmedinf.2016.01.011] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2015] [Revised: 01/22/2016] [Accepted: 01/26/2016] [Indexed: 10/22/2022]
Abstract
OBJECTIVE Failure to rapidly identify high-value information due to inappropriate output may alter user acceptance and satisfaction. The information needs for different intensive care unit (ICU) providers are not the same. This can obstruct successful implementation of electronic medical record (EMR) systems. We evaluated the implementation experience and satisfaction of providers using a novel EMR interface-based on the information needs of ICU providers-in the context of an existing EMR system. METHODS This before-after study was performed in the ICU setting at two tertiary care hospitals from October 2013 through November 2014. Surveys were delivered to ICU providers before and after implementation of the novel EMR interface. Overall satisfaction and acceptance was reported for both interfaces. RESULTS A total of 246 before (existing EMR) and 115 after (existing EMR+novel EMR interface) surveys were analyzed. 14% of respondents were prescribers and 86% were non-prescribers. Non-prescribers were more satisfied with the existing EMR, whereas prescribers were more satisfied with the novel EMR interface. Both groups reported easier data gathering, routine tasks & rounding, and fostering of team work with the novel EMR interface. This interface was the primary tool for 18% of respondents after implementation and 73% of respondents intended to use it further. Non-prescribers reported an intention to use this novel interface as their primary tool for information gathering. CONCLUSION Compliance and acceptance of new system is not related to previous duration of work in ICU, but ameliorates with the length of EMR interface usage. Task-specific and role-specific considerations are necessary for design and successful implementation of a EMR interface. The difference in user workflows causes disparity of the way of EMR data usage.
Collapse
Affiliation(s)
| | - Vitaly Herasevich
- Department of Anesthesiology, Mayo Clinic, Rochester, MN, United States
| | - Ayan Sen
- Division of Pulmonary and Critical Care Medicine, Mayo Clinic, Scottsdale, AZ, United States
| | - Brian W Pickering
- Department of Anesthesiology, Mayo Clinic, Rochester, MN, United States
| | - Ann-Marie A Knight
- Division of Critical Care Medicine, Mayo Clinic, Jacksonville, FL, United States
| | - Pablo Moreno Franco
- Division of Critical Care Medicine, Mayo Clinic, Jacksonville, FL, United States; Department of Transplantation Medicine, Mayo Clinic, Jacksonville, FL, United States.
| |
Collapse
|
47
|
King AJ, Cooper GF, Hochheiser H, Clermont G, Visweswaran S. Development and Preliminary Evaluation of a Prototype of a Learning Electronic Medical Record System. AMIA ... ANNUAL SYMPOSIUM PROCEEDINGS. AMIA SYMPOSIUM 2015; 2015:1967-1975. [PMID: 26958296 PMCID: PMC4765593] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
Electronic medical records (EMRs) are capturing increasing amounts of data per patient. For clinicians to efficiently and accurately understand a patient's clinical state, better ways are needed to determine when and how to display EMR data. We built a prototype system that records how physicians view EMR data, which we used to train models that predict which EMR data will be relevant in a given patient. We call this approach a Learning EMR (LEMR). A physician used the prototype to review 59 intensive care unit (ICU) patient cases. We used the data-access patterns from these cases to train logistic regression models that, when evaluated, had AUROC values as high as 0.92 and that averaged 0.73, supporting that the approach is promising. A preliminary usability study identified advantages of the system and a few concerns about implementation. Overall, 3 of 4 ICU physicians were enthusiastic about features of the prototype.
Collapse
Affiliation(s)
- Andrew J King
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA
| | - Gregory F Cooper
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA; Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA
| | - Harry Hochheiser
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA; Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA
| | - Gilles Clermont
- Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, PA, USA
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA; Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA
| |
Collapse
|
48
|
Vukoja M, Kashyap R, Gavrilovic S, Dong Y, Kilickaya O, Gajic O. Checklist for early recognition and treatment of acute illness: International collaboration to improve critical care practice. World J Crit Care Med 2015; 4:55-61. [PMID: 25685723 PMCID: PMC4326764 DOI: 10.5492/wjccm.v4.i1.55] [Citation(s) in RCA: 25] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/23/2014] [Revised: 12/19/2014] [Accepted: 01/18/2015] [Indexed: 02/07/2023] Open
Abstract
Processes to ensure world-wide best-practice for critical care delivery are likely to minimize preventable death, disability and costly complications for any healthcare system's sickest patients, but no large-scale efforts have so far been undertaken towards these goals. The advances in medical informatics and human factors engineering have provided possibility for novel and user-friendly clinical decision support tools that can be applied in a complex and busy hospital setting. To facilitate timely and accurate best-practice delivery in critically ill patients international group of intensive care unit (ICU) physicians and researchers developed a simple decision support tool: Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN). The tool has been refined and tested in high fidelity simulated clinical environment and has been shown to improve performance of clinical providers faced with simulated emergencies. The aim of this international educational intervention is to implement CERTAIN into clinical practice in hospital settings with variable resources (included those in low income countries) and evaluate the impact of the tool on the care processes and patient outcomes. To accomplish our aims, CERTAIN will be uniformly available on either mobile or fixed computing devices (as well as a backup paper version) and applied in a standardized manner in the ICUs of diverse hospitals. To ensure the effectiveness of the proposed intervention, access to CERTAIN is coupled with structured training of bedside ICU providers.
Collapse
|
49
|
The implementation of clinician designed, human-centered electronic medical record viewer in the intensive care unit: a pilot step-wedge cluster randomized trial. Int J Med Inform 2015; 84:299-307. [PMID: 25683227 DOI: 10.1016/j.ijmedinf.2015.01.017] [Citation(s) in RCA: 67] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2014] [Revised: 12/08/2014] [Accepted: 01/22/2015] [Indexed: 11/23/2022]
Abstract
OBJECTIVES AWARE (Ambient Warning and Response Evaluation) is a novel electronic medical record (EMR) dashboard designed by clinicians to support bedside clinical information management in the ICU. AWARE sits on top of pre-existing, comprehensive EMR systems. The purpose of the study was to test the acceptance and impact of AWARE on data management in live clinical ICU settings. The primary outcome measure was observed efficiency of data utilization as determined by time spent in data gathering before morning rounds. DESIGN Step wedge cluster randomization trial. SETTING Four ICUs (surgical, medical, and mixed) at an academic referral center. SUBJECTS All members of the critical care team participating in morning ICU rounds. INTERVENTION Pilot implementation of a novel EMR interface with direct observation and survey. MEASUREMENTS AND MAIN RESULTS The study took place between April and July 2012. A total of 80 and 63 direct observations were made in the pre- and post-implementation study periods respectively. The time spent on pre-round data gathering per patient decreased from 12 (10-15) to 9 (7.3-11) min for pre- and post-implementation phases respectively (p=0.03). Compared to the existing EMR, information management (data presentation format, efficiency of data access) was reported to be better after AWARE implementation. AWARE made the task of gathering data for rounds significantly less difficult and mentally demanding. CONCLUSIONS The introduction of a novel, patient-centered EMR viewer for the ICU was associated with improved efficiency and ease of clinical data management compared to the standard EMR.
Collapse
|
50
|
Collinsworth AW, Masica AL, Priest EL, Berryman CD, Kouznetsova M, Glorioso O, Montgomery D. Modifying the electronic health record to facilitate the implementation and evaluation of a bundled care program for intensive care unit delirium. EGEMS 2014; 2:1121. [PMID: 25848599 PMCID: PMC4371482 DOI: 10.13063/2327-9214.1121] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
CONTEXT Electronic health records (EHRs) have been promoted as a key driver of improved patient care and outcomes and as an essential component of learning health systems. However, to date, many EHRs are not optimized to support delivery of quality and safety initiatives, particularly in Intensive Care Units (ICUs). Delirium is a common and severe problem for ICU patients that may be prevented or mitigated through the use of evidence-based care processes (daily awakening and breathing trials, formal delirium screening, and early mobility-collectively known as the "ABCDE bundle"). This case study describes how an integrated health care delivery system modified its inpatient EHR to accelerate the implementation and evaluation of ABCDE bundle deployment as a safety and quality initiative. CASE DESCRIPTION In order to facilitate uptake of the ABCDE bundle and measure delivery of the care processes within the bundle, we worked with clinical and technical experts to create structured data fields for documentation of bundle elements and to identify where these fields should be placed within the EHR to streamline staff workflow. We created an "ABCDE" tab in the existing patient viewer that allowed providers to easily identify which components of the bundle the patient had and had not received. We examined the percentage of ABCDE bundle elements captured in these structured data fields over time to track compliance with data entry procedures and to improve documentation of care processes. MAJOR THEMES Modifying the EHR to support ABCDE bundle deployment was a complex and time-consuming process. We found that it was critical to gain buy-in from senior leadership on the importance of the ABCDE bundle to secure information technology (IT) resources, understand the different workflows of members of multidisciplinary care teams, and obtain continuous feedback from staff on the EHR revisions during the development cycle. We also observed that it was essential to provide ongoing training to staff on proper use of the new EHR documentation fields. Lastly, timely reporting on ABCDE bundle performance may be essential to improved practice adoption and documentation of care processes. CONCLUSION The creation of learning health systems is contingent on an ability to modify EHRs to meet emerging care delivery and quality improvement needs. Although this study focuses on the prevention and mitigation of delirium in ICUs, our process for identifying key data elements and making modifications to the EHR, as well as the lessons learned from the IT components of this program, are generalizable to other health care settings and conditions.
Collapse
|