1
|
Rzepiński T. Subjectivity of pre-test probability value: controversies over the use of Bayes' Theorem in medical diagnosis. THEORETICAL MEDICINE AND BIOETHICS 2023; 44:301-324. [PMID: 36881191 PMCID: PMC10491529 DOI: 10.1007/s11017-023-09614-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 01/27/2023] [Indexed: 06/18/2023]
Abstract
This article discusses the use of Bayes' Theorem in medical diagnosis with a view to examining the epistemological problems of interpreting the concept of pre-test probability value. It is generally maintained that pre-test probability values are determined subjectively. Accordingly, this paper investigates three main philosophical interpretations of probability (the "classic" one, based on the principle of non-sufficient reason, the frequentist one, and the personalistic one). This study argues that using Bayes' Theorem in medical diagnosis does not require accepting the radical personalistic interpretation. It will be shown that what distinguishes radical and moderate personalist interpretations is the criterion of conditional inter-subjectivity which applies only to the moderate account of personalist interpretation.
Collapse
Affiliation(s)
- Tomasz Rzepiński
- Department of Logic and Methodology of Science, Institute of Philosophy, University of A. Mickiewicz, ul. Szamarzewskiego 89c, 60-569, Poznan, Poland.
- Department of Biology and Environmental Protection, The Clinical Hospital of Christ Transfiguration, University of Medical Sciences, ul. Długa, Poznan, Poland.
| |
Collapse
|
2
|
Will DD, Whiting TL. Animal protection reporting requirements of Canadian veterinarians: Example case. THE CANADIAN VETERINARY JOURNAL = LA REVUE VETERINAIRE CANADIENNE 2022; 63:301-306. [PMID: 35237019 PMCID: PMC8842380] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Affiliation(s)
- Dennis D Will
- 223 Thain Way, Saskatoon, Saskatchewan S7K 6T4 (Will); 191 Lawndale Avenue, Winnipeg, Manitoba R2H 1T4 (Whiting)
| | - Terry L Whiting
- 223 Thain Way, Saskatoon, Saskatchewan S7K 6T4 (Will); 191 Lawndale Avenue, Winnipeg, Manitoba R2H 1T4 (Whiting)
| |
Collapse
|
3
|
Gkiouras K, Grammatikopoulou MG, Theodoridis X, Pagkalidou E, Chatzikyriakou E, Apostolidou AG, Rigopoulou EI, Sakkas LI, Bogdanos DP. Diagnostic and clinical significance of antigen-specific pancreatic antibodies in inflammatory bowel diseases: A meta-analysis. World J Gastroenterol 2020; 26:246-265. [PMID: 31988587 PMCID: PMC6962435 DOI: 10.3748/wjg.v26.i2.246] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/01/2019] [Revised: 12/19/2019] [Accepted: 01/02/2020] [Indexed: 02/06/2023] Open
Abstract
BACKGROUND Non-invasive criteria are needed for Crohn's disease (CD) diagnosis, with several biomarkers being tested. Results of individual diagnostic test accuracy studies assessing the diagnostic value of pancreatic autoantibodies-to-glycoprotein-2 (anti-GP2) tests for the diagnosis of CD appear promising. AIM To systematically review and meta-analyze evidence on the diagnostic accuracy of anti-GP2 tests in patients with suspected/confirmed CD. METHODS An electronic search was conducted on PubMed, Cochrane-CENTRAL and grey literature (CRD42019125947). The structured research question in PICPTR format was "Population" including patients with symptoms akin to CD, the "Index test" being anti-GP2 testing, the "Comparator" involved standard CD diagnosis, the "Purpose of test" being diagnostic, "Target disorder" was CD, and the "Reference standard" included standard clinical, radiological, endoscopical, and histological CD diagnostic criteria. Quality was assessed with the Quality Assessment of Diagnostic Accuracy Studies-2 tool and hierarchical models were employed to synthesize the data. RESULTS Out of 722 studies retrieved, 15 were meta-analyzed. Thirteen studies had industry-related conflicts-of-interest, and most included healthy donors as controls (spectrum bias). For the combination of IgA and/or IgG anti-GP2 test, the summary sensitivity was 20% (95% confidence interval: 10%-29%) at a median specificity of 97%. If the test was applied in 10000 suspected patients, 9669 would be true negatives and in 26, the diagnosis would be missed. In this hypothetical cohort, the anti-GP2 would fail to produce a diagnosis for 81.3% of the positive cases. Low summary points of sensitivity and high specificity were estimated for the IgG or IgA anti-GP2 test. Analogous results were observed when the analyses were restricted using specific cut-offs, or when ulcerative colitis patients were used as comparators. CONCLUSION Anti-GP2 tests demonstrate low sensitivity and high specificity. These results indicate that caution is required before relying on its diagnostic value. Additionally, the need for improving the methodology of diagnostic test accuracy studies is evident.
Collapse
Affiliation(s)
- Konstantinos Gkiouras
- Department of Rheumatology and Clinical Immunology, Faculty of Medicine, School of Health Sciences, University of Thessaly, Biopolis, Larissa GR41110, Greece
- Faculty of Medicine, School of Health Sciences, Aristotle University of Thessaloniki, University Campus, Thessaloniki GR54124, Greece
| | - Maria G Grammatikopoulou
- Department of Rheumatology and Clinical Immunology, Faculty of Medicine, School of Health Sciences, University of Thessaly, Biopolis, Larissa GR41110, Greece
- Faculty of Medicine, School of Health Sciences, Aristotle University of Thessaloniki, University Campus, Thessaloniki GR54124, Greece
- Department of Nutritional Sciences and Dietetics, School of Health Sciences, International Hellenic University, Sindos Campus, Thessaloniki GR57400, Greece
| | - Xenophon Theodoridis
- Department of Rheumatology and Clinical Immunology, Faculty of Medicine, School of Health Sciences, University of Thessaly, Biopolis, Larissa GR41110, Greece
- Faculty of Medicine, School of Health Sciences, Aristotle University of Thessaloniki, University Campus, Thessaloniki GR54124, Greece
| | - Eirini Pagkalidou
- Laboratory of Hygiene, Social and Preventive Medicine and Medical Statistics, Faculty of Medicine, School of Health Sciences, Aristotle University of Thessaloniki, University Campus, Thessaloniki GR54124, Greece
| | - Evangelia Chatzikyriakou
- Faculty of Medicine, School of Health Sciences, Aristotle University of Thessaloniki, University Campus, Thessaloniki GR54124, Greece
- Laboratory of Clinical Neurophysiology, AHEPA University Hospital, Faculty of Medicine, Aristotle University of Thessaloniki, University Campus, Thessaloniki GR54124, Greece
| | - Anna G Apostolidou
- Department of Nutritional Sciences and Dietetics, School of Health Sciences, International Hellenic University, Sindos Campus, Thessaloniki GR57400, Greece
| | - Eirini I Rigopoulou
- Department of Medicine and Research Laboratory of Internal Medicine, University Hospital of Larissa, Biopolis, Larissa GR41110, Greece
| | - Lazaros I Sakkas
- Department of Rheumatology and Clinical Immunology, Faculty of Medicine, School of Health Sciences, University of Thessaly, Biopolis, Larissa GR41110, Greece
| | - Dimitrios Petrou Bogdanos
- Department of Rheumatology and Clinical Immunology, Faculty of Medicine, School of Health Sciences, University of Thessaly, Biopolis, Larissa GR41110, Greece
- Division of Transplantation, Immunology and Mucosal Biology, MRC Centre for Transplantation, King's College London Medical School, London GR41110, United Kingdom
| |
Collapse
|
4
|
Monteiro S, Norman G, Sherbino J. The 3 faces of clinical reasoning: Epistemological explorations of disparate error reduction strategies. J Eval Clin Pract 2018. [PMID: 29532584 DOI: 10.1111/jep.12907] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/12/2023]
Abstract
There is general consensus that clinical reasoning involves 2 stages: a rapid stage where 1 or more diagnostic hypotheses are advanced and a slower stage where these hypotheses are tested or confirmed. The rapid hypothesis generation stage is considered inaccessible for analysis or observation. Consequently, recent research on clinical reasoning has focused specifically on improving the accuracy of the slower, hypothesis confirmation stage. Three perspectives have developed in this line of research, and each proposes different error reduction strategies for clinical reasoning. This paper considers these 3 perspectives and examines the underlying assumptions. Additionally, this paper reviews the evidence, or lack of, behind each class of error reduction strategies. The first perspective takes an epidemiological stance, appealing to the benefits of incorporating population data and evidence-based medicine in every day clinical reasoning. The second builds on the heuristic and bias research programme, appealing to a special class of dual process reasoning models that theorizes a rapid error prone cognitive process for problem solving with a slower more logical cognitive process capable of correcting those errors. Finally, the third perspective borrows from an exemplar model of categorization that explicitly relates clinical knowledge and experience to diagnostic accuracy.
Collapse
Affiliation(s)
- Sandra Monteiro
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada.,McMaster (Faculty of Health Sciences Program) Education Research, Innovation and Theory, McMaster University, Hamilton, Ontario, Canada
| | - Geoff Norman
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada.,McMaster (Faculty of Health Sciences Program) Education Research, Innovation and Theory, McMaster University, Hamilton, Ontario, Canada
| | - Jonathan Sherbino
- McMaster (Faculty of Health Sciences Program) Education Research, Innovation and Theory, McMaster University, Hamilton, Ontario, Canada.,Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| |
Collapse
|
5
|
Wilkerson GB, Denegar CR. A Growing Consensus for Change in Interpretation of Clinical Research Evidence. J Athl Train 2018; 53:320-326. [PMID: 29624454 PMCID: PMC5894384 DOI: 10.4085/1062-6050-8-17] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
CONTEXT The paradigm of evidence-based practice (EBP) is well established among the health care professions, but perspectives on the best methods for acquiring, analyzing, appraising, and using research evidence are evolving. BACKGROUND The EBP paradigm has shifted away from a hierarchy of research-evidence quality to recognize that multiple research methods can yield evidence to guide clinicians and patients through a decision-making process. Whereas the "frequentist" approach to data interpretation through hypothesis testing has been the dominant analytical method used by and taught to athletic training students and scholars, this approach is not optimal for integrating evidence into routine clinical practice. Moreover, the dichotomy of rejecting, or failing to reject, a null hypothesis is inconsistent with the Bayesian-like clinical decision-making process that skilled health care providers intuitively use. We propose that data derived from multiple research methods can be best interpreted by reporting a credible lower limit that represents the smallest treatment effect at a specified level of certainty, which should be judged in relation to the smallest effect considered to be clinically meaningful. Such an approach can provide a quantifiable estimate of certainty that an individual patient needs follow-up attention to prevent an adverse outcome or that a meaningful level of therapeutic benefit will be derived from a given intervention. CONCLUSIONS The practice of athletic training will be influenced by the evolution of the EBP paradigm. Contemporary practice will require clinicians to expand their critical-appraisal skills to effectively integrate the results derived from clinical research into the care of individual patients. Proper interpretation of a credible lower limit value for a magnitude ratio has the potential to increase the likelihood of favorable patient outcomes, thereby advancing the practice of evidence-based athletic training.
Collapse
Affiliation(s)
- Gary B. Wilkerson
- Graduate Athletic Training Education Program, University of Tennessee at Chattanooga
| | | |
Collapse
|
6
|
Bianchi MT. Bayes' theorem and the rule of 100: a commentary on 'Validity of administrative data for identification of obstructive sleep apnea'. J Sleep Res 2017; 26:401. [PMID: 28425149 DOI: 10.1111/jsr.12534] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Affiliation(s)
- Matt T Bianchi
- Neurology Department, Massachusetts General Hospital, Boston, MA, USA.,Division of Sleep Medicine, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
7
|
Olmos A, Bertolini E, Ruiz-García AB, Martínez C, Peiró R, Vidal E. Modeling the Accuracy of Three Detection Methods of Grapevine leafroll-associated virus 3 During the Dormant Period Using a Bayesian Approach. PHYTOPATHOLOGY 2016; 106:510-518. [PMID: 26780435 DOI: 10.1094/phyto-10-15-0246-r] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
Grapevine leafroll-associated virus 3 (GLRaV-3) has a worldwide distribution and is the most economically important virus that causes grapevine leafroll disease. Reliable, sensitive, and specific methods are required for the detection of the pathogen in order to assure the production of healthy plant material and control of the disease. Although different serological and nucleic acid-based methods have been developed for the detection of GLRaV-3, diagnostic parameters have not been established, and there is no gold standard method. Therefore, the main aim of this work was to determine the sensitivity, specificity, and likelihood ratios of three commonly used methods, including one serological test (double-antibody sandwich enzyme-linked immunosorbent assay [DAS-ELISA]) and two nucleic acid-based techniques (spot and conventional real-time reverse transcription-polymerase chain reaction [RT-PCR]). Latent class models using a Bayesian approach have been applied to determine diagnostic test parameters and to facilitate decision-making regarding diagnostic test selection. Statistical analysis has been based on the results of a total of 281 samples, which were collected during the dormant period from three different populations. The best-fit model out of the 49 implemented models revealed that DAS-ELISA was the most specific method (value = 0.99) and provided the highest degree of confidence in positive results. Conversely, conventional real-time RT-PCR was the most sensitive method (value = 0.98) and produced the highest degree of confidence in negative results. Furthermore, the estimation of likelihood ratios showed that in populations with low GLRaV-3 prevalence the most appropriate method could be DAS-ELISA, while conventional real-time RT-PCR could be the most appropriate method in medium or high prevalence populations. Combining both techniques significantly increases detection accuracy. The flexibility and power of Bayesian latent class models open new possibilities for the evaluation of diagnostic tests for plant viruses.
Collapse
Affiliation(s)
- Antonio Olmos
- First, third, fourth, and sixth authors: Centro de Protección Vegetal y Biotecnología, Instituto Valenciano de Investigaciones Agrarias (IVIA), Ctra. Moncada-Náquera km 4.5, 46113 Moncada, Valencia, Spain; second author: Departamento de Fitossanidade, Faculdade de Agronomia, Universidade Federal do Rio Grande do Sul (UFRGS), Avenida Bento Gonçalves 7712, 91450-000 Porto Alegre, RS, Brazil; and fifth author: Instituto de Conservación y Mejora de la Agrodiversidad Valenciana, Universitat Politècnica de Valencia (COMAV-UPV), 46022 Valencia, Spain
| | - Edson Bertolini
- First, third, fourth, and sixth authors: Centro de Protección Vegetal y Biotecnología, Instituto Valenciano de Investigaciones Agrarias (IVIA), Ctra. Moncada-Náquera km 4.5, 46113 Moncada, Valencia, Spain; second author: Departamento de Fitossanidade, Faculdade de Agronomia, Universidade Federal do Rio Grande do Sul (UFRGS), Avenida Bento Gonçalves 7712, 91450-000 Porto Alegre, RS, Brazil; and fifth author: Instituto de Conservación y Mejora de la Agrodiversidad Valenciana, Universitat Politècnica de Valencia (COMAV-UPV), 46022 Valencia, Spain
| | - Ana B Ruiz-García
- First, third, fourth, and sixth authors: Centro de Protección Vegetal y Biotecnología, Instituto Valenciano de Investigaciones Agrarias (IVIA), Ctra. Moncada-Náquera km 4.5, 46113 Moncada, Valencia, Spain; second author: Departamento de Fitossanidade, Faculdade de Agronomia, Universidade Federal do Rio Grande do Sul (UFRGS), Avenida Bento Gonçalves 7712, 91450-000 Porto Alegre, RS, Brazil; and fifth author: Instituto de Conservación y Mejora de la Agrodiversidad Valenciana, Universitat Politècnica de Valencia (COMAV-UPV), 46022 Valencia, Spain
| | - Carmen Martínez
- First, third, fourth, and sixth authors: Centro de Protección Vegetal y Biotecnología, Instituto Valenciano de Investigaciones Agrarias (IVIA), Ctra. Moncada-Náquera km 4.5, 46113 Moncada, Valencia, Spain; second author: Departamento de Fitossanidade, Faculdade de Agronomia, Universidade Federal do Rio Grande do Sul (UFRGS), Avenida Bento Gonçalves 7712, 91450-000 Porto Alegre, RS, Brazil; and fifth author: Instituto de Conservación y Mejora de la Agrodiversidad Valenciana, Universitat Politècnica de Valencia (COMAV-UPV), 46022 Valencia, Spain
| | - Rosa Peiró
- First, third, fourth, and sixth authors: Centro de Protección Vegetal y Biotecnología, Instituto Valenciano de Investigaciones Agrarias (IVIA), Ctra. Moncada-Náquera km 4.5, 46113 Moncada, Valencia, Spain; second author: Departamento de Fitossanidade, Faculdade de Agronomia, Universidade Federal do Rio Grande do Sul (UFRGS), Avenida Bento Gonçalves 7712, 91450-000 Porto Alegre, RS, Brazil; and fifth author: Instituto de Conservación y Mejora de la Agrodiversidad Valenciana, Universitat Politècnica de Valencia (COMAV-UPV), 46022 Valencia, Spain
| | - Eduardo Vidal
- First, third, fourth, and sixth authors: Centro de Protección Vegetal y Biotecnología, Instituto Valenciano de Investigaciones Agrarias (IVIA), Ctra. Moncada-Náquera km 4.5, 46113 Moncada, Valencia, Spain; second author: Departamento de Fitossanidade, Faculdade de Agronomia, Universidade Federal do Rio Grande do Sul (UFRGS), Avenida Bento Gonçalves 7712, 91450-000 Porto Alegre, RS, Brazil; and fifth author: Instituto de Conservación y Mejora de la Agrodiversidad Valenciana, Universitat Politècnica de Valencia (COMAV-UPV), 46022 Valencia, Spain
| |
Collapse
|
8
|
Bianchi MT. Consumer Sleep Apps: When it Comes to the Big Picture, it's All About the Frame. J Clin Sleep Med 2015; 11:695-6. [PMID: 26094923 DOI: 10.5664/jcsm.4834] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2015] [Accepted: 05/29/2015] [Indexed: 11/13/2022]
Affiliation(s)
- Matt T Bianchi
- Neurology Department, Sleep Division, Massachusetts General Hospital, Boston MA
| |
Collapse
|
9
|
Newman-Toker DE, McDonald KM, Meltzer DO. How much diagnostic safety can we afford, and how should we decide? A health economics perspective. BMJ Qual Saf 2013; 22 Suppl 2:ii11-ii20. [PMID: 24048914 PMCID: PMC3786645 DOI: 10.1136/bmjqs-2012-001616] [Citation(s) in RCA: 84] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2013] [Revised: 08/07/2013] [Accepted: 08/08/2013] [Indexed: 12/31/2022]
Affiliation(s)
- David E Newman-Toker
- Department of Neurology,Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
| | - Kathryn M McDonald
- Center for Primary Care and Outcomes Research/Center for Health Policy, Stanford University, Stanford, California, USA
- School of Public Health, University of California, Berkeley, California, USA
| | - David O Meltzer
- Department of Medicine, Center for Health and the Social Sciences, University of Chicago, Chicago, Illinois, USA
| |
Collapse
|
10
|
|
11
|
Gambino B. Setting Criterion Thresholds for Estimating Prevalence: What is Being Validated? J Gambl Stud 2013; 30:577-607. [DOI: 10.1007/s10899-013-9380-y] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
|
12
|
Austin LC, Reventlow S, Sandøe P, Brodersen J. The structure of medical decisions: uncertainty, probability and risk in five common choice situations. HEALTH, RISK & SOCIETY 2013. [DOI: 10.1080/13698575.2012.746286] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
|
13
|
Westover MB, Eiseman NA, Cash SS, Bianchi MT. Information theoretic quantification of diagnostic uncertainty. Open Med Inform J 2012; 6:36-50. [PMID: 23304251 PMCID: PMC3537080 DOI: 10.2174/1874431101206010036] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2012] [Revised: 08/19/2012] [Accepted: 08/21/2012] [Indexed: 11/22/2022] Open
Abstract
Diagnostic test interpretation remains a challenge in clinical practice. Most physicians receive training in the use of Bayes' rule, which specifies how the sensitivity and specificity of a test for a given disease combine with the pre-test probability to quantify the change in disease probability incurred by a new test result. However, multiple studies demonstrate physicians' deficiencies in probabilistic reasoning, especially with unexpected test results. Information theory, a branch of probability theory dealing explicitly with the quantification of uncertainty, has been proposed as an alternative framework for diagnostic test interpretation, but is even less familiar to physicians. We have previously addressed one key challenge in the practical application of Bayes theorem: the handling of uncertainty in the critical first step of estimating the pre-test probability of disease. This essay aims to present the essential concepts of information theory to physicians in an accessible manner, and to extend previous work regarding uncertainty in pre-test probability estimation by placing this type of uncertainty within a principled information theoretic framework. We address several obstacles hindering physicians' application of information theoretic concepts to diagnostic test interpretation. These include issues of terminology (mathematical meanings of certain information theoretic terms differ from clinical or common parlance) as well as the underlying mathematical assumptions. Finally, we illustrate how, in information theoretic terms, one can understand the effect on diagnostic uncertainty of considering ranges instead of simple point estimates of pre-test probability.
Collapse
Affiliation(s)
- M Brandon Westover
- Neurology Department, Massachusetts General Hospital, Wang 720, Boston, MA 02114, USA
| | | | | | | |
Collapse
|
14
|
Alemayehu D, Zou KH. Applications of ROC analysis in medical research: recent developments and future directions. Acad Radiol 2012; 19:1457-64. [PMID: 23122565 DOI: 10.1016/j.acra.2012.09.006] [Citation(s) in RCA: 40] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2012] [Revised: 09/17/2012] [Accepted: 09/18/2012] [Indexed: 12/14/2022]
Abstract
With the growing focus on comparative effectiveness research and personalized medicine, receiver-operating characteristic analysis can continue to play an important role in health care decision making. Specific applications of receiver-operating characteristic analysis include predictive model assessment and validation, biomarker diagnostics, responder analysis in patient-reported outcomes, and comparison of alternative treatment options. The authors present a survey of the potential applications of the method and briefly review several relevant extensions. Given the level of attention paid to biomarker validation, personalized medicine and comparative effectiveness research, it is highly likely that the receiver-operating characteristic analysis will remain an important visual and analytic tool for medical research and evidence-based medicine in the foreseeable future.
Collapse
|
15
|
Srinivasan P, Westover MB, Bianchi MT. Propagation of uncertainty in Bayesian diagnostic test interpretation. South Med J 2012; 105:452-9. [PMID: 22948322 DOI: 10.1097/smj.0b013e3182621a2c] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
OBJECTIVES Bayesian interpretation of diagnostic test results usually involves point estimates of the pretest probability and the likelihood ratio corresponding to the test result; however, it may be more appropriate in clinical situations to consider instead a range of possible values to express uncertainty in the estimates of these parameters. We thus sought to demonstrate how uncertainty in sensitivity, specificity, and disease pretest probability can be accommodated in Bayesian interpretation of diagnostic testing. METHODS We investigated three questions: How does uncertainty in the likelihood ratio propagate to the posttest probability range, assuming a point estimate of pretest probability? How does uncertainty in the sensitivity and specificity of a test affect uncertainty in the likelihood ratio? How does uncertainty propagate when present in both the pretest probability and the likelihood ratio? RESULTS Propagation of likelihood ratio uncertainty depends on the pretest probability and is more prominent for unexpected test results. Uncertainty in sensitivity and specificity propagates into the calculation of likelihood ratio prominently as these parameters approach 100%; even modest errors of ± 10% caused dramatic propagation. Combining errors of ± 20% in the pretest probability and in the likelihood ratio exhibited modest propagation to posttest probability, suggesting a realistic target range for clinical estimations. CONCLUSIONS The results provide a framework for incorporating ranges of uncertainty into Bayesian reasoning. Although point estimates simplify the implementation of Bayesian reasoning, it is important to recognize the implications of error propagation when ranges are considered in this multistep process.
Collapse
Affiliation(s)
- Preethi Srinivasan
- Bioinformatics Department, Northeastern University, Boston, Massachusetts, USA
| | | | | |
Collapse
|
16
|
Abstract
A survey of doctors working in two large NHS hospitals identified over 120 laboratory tests, imaging investigations and investigational procedures that they considered not to be overused. A common suggestion in this survey was that more training was required. And, this prompted the development of a list of core principles for high-quality, high-value testing. The list can be used as a framework for training and as a reference source. The core principles are: (1) Base testing practices on the best available evidence. (2) Apply the evidence on test performance with careful judgement. (3) Test efficiently. (4) Consider the value (and affordability) of a test before requesting it. (5) Be aware of the downsides and drivers of overdiagnosis. (6) Confront uncertainties. (7) Be patient-centred in your approach. (8) Consider ethical issues. (9) Be aware of normal cognitive limitations and biases when testing. (10) Follow the ‘knowledge journey’ when teaching and learning these core principles.
Collapse
Affiliation(s)
- Michael Power
- Pharmacy Department, Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, UK.
| | | | | |
Collapse
|
17
|
Severo M, Gaio AR, Lourenço P, Alvelos M, Gonçalves A, Lunet N, Bettencourt P, Azevedo A. Diagnostic value of patterns of symptoms and signs of heart failure: application of latent class analysis with concomitant variables in a cross-sectional study. BMJ Open 2012; 2:bmjopen-2012-001510. [PMID: 23148342 PMCID: PMC3532992 DOI: 10.1136/bmjopen-2012-001510] [Citation(s) in RCA: 11] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/13/2023] Open
Abstract
OBJECTIVE The diagnosis of heart failure (HF) requires a compatible clinical syndrome and demonstration of cardiac dysfunction by imaging or functional tests. Since individual symptoms and signs are generally unreliable and have limited value for diagnosing HF, the authors aimed to identify patterns of symptoms and signs, based on findings routinely collected in current clinical practice, and to evaluate their diagnostic value, taking into account the a priori likelihood of HF. DESIGN Cross-sectional evaluation. PARTICIPANTS 1115 community participants aged ≥45 years from Porto, Portugal, in 2006-2008. MAIN OUTCOMES MEASURES Patterns were identified by latent class analysis, using concomitant variables to predict class membership. Patterns used 11 symptoms/signs, covering dimensions of congestion and hypoperfusion. Sex, age, education, obesity, diabetes and history of myocardial infarction or HF were included as concomitants. RESULTS Bayesian information criteria supported a solution with three patterns: 10.1% of participants followed a pattern with symptoms of troubled breathing and signs of congestion (pattern 1), 27.8% a pattern characterised mainly by signs of congestion (pattern 2) and 62.1% were essentially asymptomatic (pattern 3); model fit was best when including concomitant variables. The likelihood ratio of patterns 1, 2 and 3 for left ventricular systolic dysfunction was 3.4, 1.1 and 0.6, and for left ventricular diastolic dysfunction 3.5, 1.4 and 0.5, respectively. CONCLUSIONS The use of concomitant variables can improve the diagnostic value of the symptoms and signs patterns and, consequently, improve the usefulness of the symptoms and signs for diagnosis and as an outcome measure. The potential for application in other settings of complex diagnoses is very high. These models were shown to be useful to standardise and quantify the probabilistic reasoning in clinical diagnosis, upon which decisions of further investigation and even treatment need to be made.
Collapse
Affiliation(s)
- Milton Severo
- Department of Clinical Epidemiology, Predictive Medicine and Public Health, University of Porto Medical School, Porto, Portugal
- Institute of Public Health of the University of Porto, Porto, Portugal
| | - Ana Rita Gaio
- Department of Mathematics, University of Porto Science School, Porto, Portugal
- Mathematics Center, University of Porto, Porto, Portugal
| | - Patrícia Lourenço
- Department of Internal Medicine, Heart Failure Clinic, Hospital São João, Porto, Portugal
| | - Margarida Alvelos
- Department of Internal Medicine, Heart Failure Clinic, Hospital São João, Porto, Portugal
| | | | - Nuno Lunet
- Department of Clinical Epidemiology, Predictive Medicine and Public Health, University of Porto Medical School, Porto, Portugal
- Institute of Public Health of the University of Porto, Porto, Portugal
| | - Paulo Bettencourt
- Department of Internal Medicine, Heart Failure Clinic, Hospital São João, Porto, Portugal
| | - Ana Azevedo
- Department of Clinical Epidemiology, Predictive Medicine and Public Health, University of Porto Medical School, Porto, Portugal
- Institute of Public Health of the University of Porto, Porto, Portugal
- Department of Internal Medicine, Heart Failure Clinic, Hospital São João, Porto, Portugal
| |
Collapse
|
18
|
Vidal E, Yokomi RK, Moreno A, Bertolini E, Cambra M. Calculation of diagnostic parameters of advanced serological and molecular tissue-print methods for detection of Citrus tristeza virus: a model for other plant pathogens. PHYTOPATHOLOGY 2012; 102:114-121. [PMID: 21879789 DOI: 10.1094/phyto-05-11-0139] [Citation(s) in RCA: 12] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/31/2023]
Abstract
Citrus tristeza virus (CTV) is one of the most important virus diseases that affect citrus. Control of CTV is achieved by grafting selected virus-free citrus scions onto CTV-tolerant or -resistant rootstocks. Quarantine and certification programs are essential for avoiding the entry and propagation of severe strains of CTV. Citrus nurseries in Spain and central California (United States) maintain zero-tolerance policies for CTV that require sensitive, specific, and reliable pathogen-detection methods. Tissue-print (TP) real-time reverse-transcriptase polymerase chain reaction (RT-PCR) assay was compared with the validated TP enzyme-linked immunosorbent assay (ELISA), using the CTV-specific monoclonal antibodies 3DF1 and 3CA5, for CTV detection. In total, 1,395 samples from healthy and CTV-infected nursery and mature tree plants were analyzed with both methods. The total agreement between both detection methods was substantial (Cohen's kappa index of 0.77 ± 0.03). The diagnostic parameters of each technique (i.e., the sensitivity, specificity, and likelihood ratios) were evaluated in a second test involving 658 Citrus macrophylla nursery plants. Mexican lime indexing was used to evaluate samples with discrepant results in the analysis. For TP-ELISA, a sensitivity of 0.8015, a specificity of 0.9963, and a positive and negative likelihood ratio of 216.42 and 0.199, respectively, were estimated. For TP real-time RT-PCR, a sensitivity of 0.9820, a specificity of 0.8519, and a positive and negative likelihood ratio of 6.63 and 0.021, respectively, were estimated. These diagnostic parameters show that TP real-time RT-PCR was the most sensitive technique, whereas TP-ELISA showed the highest specificity, validating the use of the molecular technique for routine CTV-detection purposes. In addition, our results show that the combination of both techniques can accurately substitute for the conventional biological Mexican lime index for the detection of CTV. The calculation of diagnostic parameters is discussed, as a necessary tool, to validate detection or diagnostic methods in plant pathology. Furthermore, assessment of the post-test probability of disease after a diagnostic result and CTV prevalence allows selection of the best method for accurate and reliable diagnosis.
Collapse
Affiliation(s)
- E Vidal
- Instituto Valemciano de Investigaciones Agrarias, Valencia, Spain
| | | | | | | | | |
Collapse
|
19
|
Agoritsas T, Courvoisier DS, Combescure C, Deom M, Perneger TV. Does prevalence matter to physicians in estimating post-test probability of disease? A randomized trial. J Gen Intern Med 2011; 26:373-8. [PMID: 21053091 PMCID: PMC3055966 DOI: 10.1007/s11606-010-1540-5] [Citation(s) in RCA: 35] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/13/2010] [Revised: 07/26/2010] [Accepted: 10/01/2010] [Indexed: 11/30/2022]
Abstract
BACKGROUND The probability of a disease following a diagnostic test depends on the sensitivity and specificity of the test, but also on the prevalence of the disease in the population of interest (or pre-test probability). How physicians use this information is not well known. OBJECTIVE To assess whether physicians correctly estimate post-test probability according to various levels of prevalence and explore this skill across respondent groups. DESIGN Randomized trial. PARTICIPANTS Population-based sample of 1,361 physicians of all clinical specialties. INTERVENTION We described a scenario of a highly accurate screening test (sensitivity 99% and specificity 99%) in which we randomly manipulated the prevalence of the disease (1%, 2%, 10%, 25%, 95%, or no information). MAIN MEASURES We asked physicians to estimate the probability of disease following a positive test (categorized as <60%, 60-79%, 80-94%, 95-99.9%, and >99.9%). Each answer was correct for a different version of the scenario, and no answer was possible in the "no information" scenario. We estimated the proportion of physicians proficient in assessing post-test probability as the proportion of correct answers beyond the distribution of answers attributable to guessing. KEY RESULTS Most respondents in each of the six groups (67%-82%) selected a post-test probability of 95-99.9%, regardless of the prevalence of disease and even when no information on prevalence was provided. This answer was correct only for a prevalence of 25%. We estimated that 9.1% (95% CI 6.0-14.0) of respondents knew how to assess correctly the post-test probability. This proportion did not vary with clinical experience or practice setting. CONCLUSIONS Most physicians do not take into account the prevalence of disease when interpreting a positive test result. This may cause unnecessary testing and diagnostic errors.
Collapse
Affiliation(s)
- Thomas Agoritsas
- Division of Clinical Epidemiology, University Hospitals of Geneva, Gabrielle Perret-Gentil 6, 1211, Geneva 14, Switzerland.
| | | | | | | | | |
Collapse
|
20
|
Bianchi MT, Alexander BM, Cash SS. Incorporating Uncertainty Into Medical Decision Making: An Approach to Unexpected Test Results. Med Decis Making 2009; 29:116-24. [DOI: 10.1177/0272989x08323620] [Citation(s) in RCA: 25] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
The utility of diagnostic tests derives from the ability to translate the population concepts of sensitivity and specificity into information that will be useful for the individual patient: the predictive value of the result. As the array of available diagnostic testing broadens, there is a temptation to de-emphasize history and physical findings and defer to the objective rigor of technology. However, diagnostic test interpretation is not always straightforward. One significant barrier to routine use of probability-based test interpretation is the uncertainty inherent in pretest probability estimation, the critical first step of Bayesian reasoning. The context in which this uncertainty presents the greatest challenge is when test results oppose clinical judgment. It is this situation when decision support would be most helpful. The authors propose a simple graphical approach that incorporates uncertainty in pretest probability and has specific application to the interpretation of unexpected results. This method quantitatively demonstrates how uncertainty in disease probability may be amplified when test results are unexpected (opposing clinical judgment), even for tests with high sensitivity and specificity. The authors provide a simple nomogram for determining whether an unexpected test result suggests that one should ``switch diagnostic sides.'' This graphical framework overcomes the limitation of pretest probability uncertainty in Bayesian analysis and guides decision making when it is most challenging: interpretation of unexpected test results.
Collapse
Affiliation(s)
- Matt T. Bianchi
- Partners Neurology, Massachusetts General Hospital and Brigham and Women's Hospital, Wang Ambulatory Center, Boston, Massachusetts,
| | - Brian M. Alexander
- Harvard Radiation Oncology Program, Massachusetts General Hospital, Brigham and Women's Hospital, and Beth Israel Medical Center, Boston, Massachusetts
| | - Sydney S. Cash
- Partners Neurology, Massachusetts General Hospital and Brigham and Women's Hospital, Wang Ambulatory Center, Boston, Massachusetts
| |
Collapse
|
21
|
Zegarra Montes LR, Sanchez Mejia AA, Loza Munarriz CA, Gutierrez EC. Semen and urine culture in the diagnosis of chronic bacterial prostatitis. Int Braz J Urol 2008; 34:30-7, discussion 38-40. [DOI: 10.1590/s1677-55382008000100006] [Citation(s) in RCA: 22] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/20/2007] [Indexed: 11/22/2022] Open
|
22
|
The prevalence and prognosis of unrecognized myocardial infarction and silent myocardial ischemia in patients undergoing major vascular surgery. Coron Artery Dis 2007; 18:571-6. [PMID: 17925612 DOI: 10.1097/mca.0b013e3282f08e86] [Citation(s) in RCA: 23] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
23
|
|
24
|
Scott IA, Greenberg PB, Poole PJ. Cautionary tales in the clinical interpretation of studies of diagnostic tests. Intern Med J 2007; 38:120-9. [PMID: 17645501 DOI: 10.1111/j.1445-5994.2007.01436.x] [Citation(s) in RCA: 34] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
The use of investigational tests in making a diagnosis is a core activity of physicians and one that requires an understanding of the accuracy and usefulness of specific tests in discriminating between several diagnostic possibilities. Studies of diagnostic tests are frequently methodologically flawed and their results are often not well understood or applied in clinical practice. This article defines the performance characteristics of diagnostic tests, describes several commonly encountered deficiencies in study design which may invalidate reports of new diagnostic tests, and explains a Bayesian approach to interpreting test results in terms of disease probability.
Collapse
Affiliation(s)
- I A Scott
- Department of Internal Medicine and Clinical Epidemiology, Princess Alexandra Hospital, University of Queensland, Brisbane, Queensland, Australia.
| | | | | |
Collapse
|
25
|
Rutten ALB. Bayesian homeopathy: talking normal again. HOMEOPATHY 2007; 96:120-4. [PMID: 17437940 DOI: 10.1016/j.homp.2007.03.004] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2006] [Revised: 02/02/2007] [Accepted: 03/05/2007] [Indexed: 10/23/2022]
Abstract
Homeopathy has a communication problem: important homeopathic concepts are not understood by conventional colleagues. Homeopathic terminology seems to be comprehensible only after practical experience of homeopathy. The main problem lies in different handling of diagnosis. In conventional medicine diagnosis is the starting point for randomised controlled trials to determine the effect of treatment. In homeopathy diagnosis is combined with other symptoms and personal traits of the patient to guide treatment and predict response. Broadening our scope to include diagnostic as well as treatment research opens the possibility of multi factorial reasoning. Adopting Bayesian methodology opens the possibility of investigating homeopathy in everyday practice and of describing some aspects of homeopathy in conventional terms.
Collapse
Affiliation(s)
- A L B Rutten
- Commissie Methode en Validering VHAN (Dutch Association of Homeopathic Physicians), The Netherlands.
| |
Collapse
|
26
|
|
27
|
Hall GH. Evidence based diagnosis: terminology is unsatisfactory. BMJ 2006; 333:549. [PMID: 16960215 PMCID: PMC1562505 DOI: 10.1136/bmj.333.7567.549-b] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
|
28
|
Llewelyn H. Evidence based diagnosis: we may need to be open to new ideas. BMJ 2006; 333:549. [PMID: 16960217 PMCID: PMC1562482 DOI: 10.1136/bmj.333.7567.549] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
|
29
|
Kumar R. Evidence based diagnosis: multiple tests with multiple responses are important. BMJ 2006; 333:549. [PMID: 16960216 PMCID: PMC1562504 DOI: 10.1136/bmj.333.7567.549-a] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
|
30
|
Jolobe OM. Evidence based diagnosis: prior probability saves money, time, and possibly lives. BMJ 2006; 333:549-50. [PMID: 16960214 PMCID: PMC1562506 DOI: 10.1136/bmj.333.7567.549-c] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
|
31
|
Haslam DA. Evidence based diagnosis: what about the patients? BMJ 2006; 333:550. [PMID: 16960220 PMCID: PMC1562483 DOI: 10.1136/bmj.333.7567.550] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
|