Critical appraisal of a diagnostic paper Shakila Thangaratinam Professor of Maternal and Perinatal Health Women’s Health Research Unit R & D Director of Women’s Health Barts Health NHS Trust Relation of diagnosis and therapy in clinical practice Healthy and diseased population Diagnosis Diseased population Therapy Clinical outcome A hierarchical model for evaluation of tests • Assessment of the reliability and other technical aspects of a test • Assessment of the diagnostic accuracy • Assessment of the diagnostic effectiveness and cost effectiveness Fryback et al. Med Decis Making 1991;11:88-94 How does a diagnostic test help? A test is useful if it changes our ability to predict whether the patient has the disease or not • The likelihood of disease before the test should be altered by the test result • The likelihood of disease after the test should be different compared to the prevalence of disease in the population Value of a test Likelihood of disease before test TEST RESULT Prevalence of disease + (e.g. 7-11% of pregnant women have preterm delivery ) Likelihood of disease after test Should raise the likelihood to ~100% — Should lower the likelihood to ~ 0% How to do systematic reviews Formulate clear clinical questions from our knowledge needs identified in patient encounters Search the literature to identify relevant articles Critically appraise the evidence for its validity and usefulness Synthesis the evidence Implement useful findings in clinical practice Critical Appraisal of the Medical Literature • Are the results of the study valid? • What are the results? • Will the results help in caring for patients? Oxman et al. JAMA 1993;270:2093-5 A test accuracy study Study Sample Test Test positive Test Negative Gold standard Disease present Gold standard Disease absent Disease present Disease absent Are the results valid? • Was there an independent “blind” comparison with a reference standard? • Did the patient sample include an appropriate spectrum of patients to whom the test will be applied in clinical practice? • Did the results of the test influence the decision to perform the reference standard? Are the results valid? Study Sample Appropriate spectrum of disease Consecuitive or random sample Prospectively recruited Test Test Negative Test positive Gold standard Gold standard Independent “Blind” Verification of all test positive cases Independent “Blind” Verification of all test negative cases Disease present Disease absent Disease present Disease absent Hierarchy of evidence for test accuracy studies 1 An independent, blind comparison with reference standard among an appropriate population of consecutive patients. 2 An independent, blind comparison with reference standard among an appropriate population of non-consecutive patients or confined to a narrow population of study patients. 3 An independent, blind comparison among an appropriate population of patients, but reference standard not applied to all study patients. 4 Non-blind comparison or reference standard not applied independently 5 Expert opinion with no explicit critical appraisal, based on physiology, bench research or first principles Critical Appraisal of the Medical Literature • Are the results of the study valid? • What are the results? • Will the results help in caring for patients? Oxman et al. JAMA 1993;270:2093-5 What are the results? What is the “benefit” or “added value” of test as a diagnostic aid? Can we measure this added benefit? • Are clinically meaningful measures of diagnostic accuracy provided? • Are the data necessary to calculate the diagnostic accuracy measures provided? Consequences of testing Study Sample Test Test positive Gold standard Disease present (True positive) Test Negative Gold standard Disease absent (False postive) Disease present (False negative) Disease absent (True negative) Measures of test accuracy T e s t Positive Negative Disease Present Absent TP FP FN TN • Sensitivity and Specificity • Predictive values • Likelihood ratios • Diagnostic odds ratio Sensitivity Sensitivity Sensitivity is the proportion of those people who really have the disease (TP+FN) who are correctly identified as such (TP) • Sensitivity = TP/(TP+FN) Disease Positive Present Absent TP FP FN TN Test Negative Sensitivity Sensitivity is the proportion of those people who really have the disease (TP+FN) who are correctly identified as such (TP) • Sensitivity = TP/(TP+FN) Disease Positive Present Absent TP FP FN TN Test Negative Specificity Specificity Specificity is the proportion of those people who really do not have the disease (TN+FP) who are correctly identified as such (TN) • Specificity = TN/(TN+FP) Disease Positive Present Absent TP FP FN TN Test Negative Specificity Specificity is the proportion of those people who really do not have the disease (TN+FP) who are correctly identified as such (TN) • Specificity = TN/(TN+FP) Disease Positive Present Absent TP FP FN TN Test Negative Predictive Values PPV Positive Predictive Value is the proportion of the people who test positive (TP+FP) who truly have the disease (TP) • Positive predictive value = TP/(TP+FP) Disease Positive Present Absent TP FP FN TN Test Negative PPV Positive Predictive Value is the proportion of the people who test positive (TP+FP) who truly have the disease (TP) • Positive predictive value = TP/(TP+FP) Disease Positive Present Absent TP FP FN TN Test Negative NPV Negative Predictive Value is the proportion of the people who test negative (TN+FN) who truly do not have the disease (TN) Negative predictive value = TN/(TN+FN) Disease Positive Present Absent TP FP FN TN Test Negative NPV Negative Predictive Value is the proportion of the people who test negative (TN+FN) who truly do not have the disease (TN) Negative predictive value = TN/(TN+FN) Disease Positive Present Absent TP FP FN TN Test Negative Problems • Sensitivity and specificity are characteristics of the test • Predictive values are dependent on the prevalence of the disease • Our population is often quite different from the study population The systematic review process Formulate research question Further selection of primary studies using inclusion criteria Extract data Design search strategy Search bibliographic databases Retrieve papers Identify possible papers from titles/abstracts Quality appraisal Synthesis Formulate research / policy conclusions
© Copyright 2024 Paperzz
