Welcome

Getting Evidence-Based Medicine Off The Shelf And Into Your Hands
A Selected, Annotated Bibliography To Help You Get Started

W. Scott Richardson, M.D.
University of Texas Health Sciences Center at San Antonio

Mark C. Wilson, M.D., M.P.H.
Wake Forest University School of Medicine

 

What Is Evidence-Based Medicine And Why Do We Need It?

1. Evidence-Based Medicine Working Group. Evidence-based medicine: a new approach to teaching the practice of medicine. JAMA 1992; 268: 2420 – 2425.
[The early clarion call. Be careful who you give this to.]

2. Rosenberg W, Donald A. Evidence-based medicine: an approach to clinical problem-solving. Br Med J 1995; 310: 1122 – 1126.

3. Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS. Evidence-Based Medicine: What it is and what it isn’t [Editorial]. Br Med J 1996; 312: 71–72.
[This editorial copies onto 2 sides of a page, making a ready handout.]

4. Haynes RB, Sackett DL, Gray JAM, Cook DJ, Guyatt GH. Transferring evidence from research into practice: 1. The role of clinical care research evidence in clinical decisions [Editorial]. ACP J Club 1996 (Nov/Dec); 125: A14 - A16.

5. Haynes RB, Sackett DL, Gray JAM, Cook DJ, Guyatt. Transferring evidence from research into practice: 2. Getting the evidence straight [Editorial]. ACP J Club 1997 (Jan/Feb); 126: A14 - A16.

6. Gray JAM, Haynes RB, Sackett DL, Cook DJ, Guyatt GH. Transferring evidence from research into practice: 3. Developing evidence-based clinical policy [Editorial]. ACP J Club 1997 (Mar/Apr); 126: A14 – A16.

7. Haynes RB, Sackett DL, Guyatt GH, Cook DJ, Gray JAM. Transferring evidence from research into practice: 4. Overcoming barriers to application [Editorial]. ACP J Club 1997 (May/Jun); 126: A14 – A15.
[This series of 4 editorials expands the notions of EBM and adds perspective. The first one has a nifty diagram for use in gentle persuasion about EBM.]

8. Sackett DL, Richardson WS, Rosenberg WMC, Haynes RB, eds. Evidence-Based Medicine: How To Practice And Teach EBM, New York, Churchill Livingstone, 1997.
[Just out … could be good. If you lose your book’s cards or want to order more copies, contact Scott Richardson.]

Resources in these next sections are arranged by some important skills to practice EBM

How To Search For The Best Available Evidence

1. Snowball R. Using the clinical question to teach search strategy: fostering transferable conceptual skills in user education by active learning. Health Libraries Review 1997; 14: 167– 172.  

2. McKibbon KA, Walker-Dilks CJ. Beyond ACP J Club: How to harness MEDLINE for therapy problems [Editorial]. ACP J Club 1994 Jul/Aug; 121: A10 – A12.  

3. McKibbon KA, Walker-Dilks CJ. Beyond ACP J Club: how to harness MEDLINE for diagnostic problems [Editorial]. ACP J Club 1994 Sep/Oct; 121: A10 – A12. 

4. Walker-Dilks CJ, McKibbon KA, Haynes RB. Beyond ACP J Club: How to harness MEDLINE for etiology problems [Editorial]. ACP J Club 1994 Nov/Dec; 121: A10 – A11.  

5. McKibbon KA, Walker-Dilks CJ, Haynes RB, Wilczynski NL. Beyond ACP Journal Club: How to harness MEDLINE for prognosis problems [Editorial]. ACP J Club 1995 Jul/Aug; 123: A12 – A14. 

6. McKibbon KA, Walker-Dilks CJ, Wilczynski NL, Haynes RB. Beyond ACP Journal Club: How to harness MEDLINE for review articles [Editorial]. ACP J Club 1996 May/Jun; 124: A12 – A13.
[These 6 articles can help you get ‘off the shelf’ with your searching; starting with planning your search and carrying through with tools for finding 5 types of articles.] 

7. Lowe HJ, Barnett GO. Understanding and using the Medical Subject Heading (MeSH) vocabulary to perform literature searches. JAMA 1994; 271: 1103 – 1108. 

8. Haynes RB, Wilczynski NL, McKibbon KA, Walker-Dilks CJ, Sinclair JC. Developing optimal search strategies for detecting clinically sound studies in MEDLINE. J Am Med Inform Assoc 1994; 1: 447 – 458.

How To Understand, Critically Appraise, And Apply The Evidence

A. Therapy and Prevention 

  1. Guyatt GH, Sackett DL, Cook DJ, for the Evidence–Based Medicine Working Group. Users’ guides to the medical literature: II. How to use an article about therapy or prevention: A. Are the results of the study valid? JAMA 1993; 270: 2598-2601.  
  2. Guyatt GH, Sackett DL, Cook DJ, for the Evidence–Based Medicine Working Group. Users’ guides to the medical literature: II. How to use an article about therapy or prevention: B. What are the results and will they help me in caring for my patients? JAMA 1994; 271: 59-63.  
  3. Dans AL, Dans LF, Guyatt GH, Richardson WS, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature: XIV. How to decide on the applicability of clinical trial results to your patient. JAMA 1998; 279: 545 – 549.
    [These 3 will get you off the shelf for understanding, appraising and using RCTs.]

  4. Guyatt GH, Cook DJ, Jaeschke R. How should clinicians use the results of randomized trials? [Editorial] ACP J Club 1995 Jan/Feb; 122: A12 – A13.  
  5. Guyatt GH, Jaeschke R, Cook DJ. Applying the findings of clinical trials to individual patients [Editorial]. ACP J Club 1995 Mar/Apr; 122: A12 – A13.
    [These 2 emphasize the judgments we make when applying results to patients.]

  6. Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med 1988; 318: 1728 – 1733.  
    [The initial description of the NNT. It includes method of specifying the particular time span over which results are expected.]

  7. Cook RJ, Sackett DL. The number needed to treat: a clinically useful measure of treatment effect. Br Med J 1995; 310: 452 – 454.
    [Recaps the simple arithmetic, and adds a method for adjusting NNT depending on patient’s baseline risks.]

  8. Chatellier G, Zapletal E, Lemaitre D, Menard J, Degoulet P. The number needed to treat: a clinically useful nomogram in its proper context. Br Med J 1996; 312: 426 – 429.
    [The cautions are important and the nomogram is nifty – it’s on the pocket cards in the Sackett et al EBM book.]

  9. Glasziou PP, Irwig LM. An evidence-based approach to individualising treatment. Br Med J 1995; 311: 1356 – 1359. 
    [Includes a nice diagram about how the patient’s baseline risk influences the expected benefit of treatment.]

  10. Sinclair JC, Bracken MB. Clinically useful measures of effect in binary analyses of randomized trials. J Clin Epidemiol 1994; 47: 881 – 889.
    [For when you want more on ‘the numbers’.]

B. Diagnostic Tests 

  1. Jaeschke R, Guyatt GH, Sackett DL, for the Evidence–Based Medicine Working Group. Users’ guides to the medical literature: III. How to use an article about a diagnostic test: A. Are the results of the study valid? JAMA 1994; 271: 389-391. 
  2. Jaeschke R, Guyatt GH, Sackett DL, for the Evidence–Based Medicine Working Group. Users’ guides to the medical literature: III. How to use an article about a diagnostic test: B. What are the results and will they help me in caring for my patients? JAMA 1994; 271: 703-707.
    [These 2 emphasize how to understand, appraise and use articles about tests.]

  3. Richardson WS. Evidence-based diagnosis: more is needed [EBM Note]. Evidence-Based Medicine 1997 May/Jun; 2: 70 – 71.
    [What it might take to help clinical diagnosis become more evidence-based.]

 C. Harm and Etiology 

  1. Levine MS, Walter SS, Lee HN, Haines T, Holbrook A, Moyer V, for the Evidence–Based Medicine Working Group. Users’ guides to the medical literature: IV. How to use an article about harm. JAMA 1994; 271; 1615-1619.
    [Emphasizes how to understand, appraise and use evidence about the etiology of a specific harm to patients, whether from ‘natural’ or iatrogenic exposures.] 

D. Prognosis

  1. Laupacis A, Wells G, Richardson WS, Tugwell P, for the Evidence–Based Medicine Working Group. Users’ guides to the medical literature: V. How to use an article about prognosis. JAMA 1994; 272: 234 – 237.
    [Emphasizes how to understand, appraise and use information about possible outcomes of disease, and individual prognostic factors, but not clinical prediction rules.]

  2. Christakis NA, Sachs GA. The role of prognosis in clinical decision making. J Gen Intern Med 1996; 11: 422 – 425.
    [A nice answer to the question "how do we use prognostic information?"]

  E. Clinical Prediction Rules

  1. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules: a review and suggested modifications of methodological standards. JAMA 1997; 277: 488 – 494. 
  2. Braitman LE, Davidoff F. Predicting clinical states in individual patients. Ann Intern Med 1996; 125: 406 – 412.
    [Not in the Users’ Guides format, but both present useful frameworks for understanding the issues related to clinical prediction rules and their use in practice.]

  3. McGinn T, Randolph A, Richardson WS, Sackett DL. Clinical prediction guides [Editorial]. ACP J Club 1998 Jan-Feb; 128: A14 – A15.
    [Explains the new policy of ACP J Club & EBM journals regarding these articles, illustrated with a worked example.]

F. Systematic Reviews

  1. Oxman AD, Cook DJ, Guyatt GH, for the Evidence–Based Medicine Working Group. Users’ guides to the medical literature: VI. How to use an overview. JAMA 1994; 272: 1367-1371.
    [Emphasizes how to understand, appraise and use systematic reviews.]

  2. Sackett DL, Deeks JJ, Altman DG. Down with odds ratios! [EBM Note] Evidence-Based Medicine 1996 Sep-Oct; 1: 164 – 166.
    [Explains why, and why not, to use odds ratios to express results of reviews.]

  3. Jadad AR, Cook DJ, Browman GP. A guide to interpreting discordant systematic reviews. Can Med Assoc J 1997; 156: 1411 – 1416.
    [Not only do individual trials sometimes disagree, but so too do systematic reviews. This piece helps you understand how that happens and what you can do in the meantime.]

When you’re ready for more depth on systematic reviews, consider using the series on them that has appeared in Annals of Internal Medicine, citations below. 

  1. Mulrow CD, Cook DJ, Davidoff F. Systematic reviews: critical links in the great chain of evidence [Editorial]. Ann Intern Med 1997; 126: 389 – 391.  
  2. Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med 1997; 126: 376 – 380.
  3. Hunt DL, McKibbon KA. Locating and appraising systematic reviews. Ann Intern Med 1997; 126: 532 – 538.
  4. McQuay HM, Moore RA. Using numerical results from systematic reviews in clinical practice. Ann Intern Med 1997; 126: 712 – 720. 
  5. Badget RG, O’Keefe MO, Henderson MC. Using systematic reviews in clinical education. Ann Intern Med 1997; 126: 886 – 891.
  6. Bero LA, Jadad AR. How consumers and policymakers can use systematic reviews for decision making. Ann Intern Med 1997; 127: 37 – 42. 
  7. Cook DJ, Greengold NL, Ellrodt AG, Weingarten SR. The relation between systematic reviews and practice guidelines. Ann Intern Med 1997; 127: 210 – 216. 
  8. Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med 1997; 127: 380 – 387.
  9. Meade MO, Richardson WS. Selecting and appraising studies for systematic review. Ann Intern Med 1997; 127: 531 – 537. 
  10. Lau J, Ioannidis JPA, Schmid CH. Quantitative synthesis in systematic reviews. Ann Intern Med 1997; 127: 820 – 826. 
  11. Mulrow CD, Langhorne P, Grimshaw J. Integrating heterogeneous pieces of evidence in systematic reviews. Ann Intern Med 1997; 127: 989 – 995.

G. Clinical Decision Analysis 

  1. Richardson WS, Detsky AS, for the Evidence–Based Medicine Working Group. Users’ guides to the medical literature: VII. How to use a clinical decision analysis: A. Are the results of the study valid? JAMA 1995; 273: 1292 – 1295. 
  2. Richardson WS, Detsky AS, for the Evidence–Based Medicine Working Group. Users’ guides to the medical literature: VII. How to use a clinical decision analysis: B. What are the results and will they help me in caring for my patients? JAMA 1995; 273: 1610 – 1613.
    [These emphasize how to understand, appraise and use CDA, not how to perform it.]

 H. Economic Analysis

  1. Drummond MF, Richardson WS, O’Brien BJ, Levine MR, Heyland DK, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature: XIII. How to use an article on economic analysis of clinical practice: A. Are the results valid? JAMA 1997; 277: 1552 – 1557. 
  2. O’Brien BJ, Heyland DK, Richardson WS, Levine MR, Drummond MF, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature: XIII. How to use an article on economic analysis of clinical practice: B. What are the results and will they help me in caring for my patients? JAMA 1997; 277: 1802 – 1806.
    [Again, the emphasis is on understanding, appraising and using, not doing.] 

I. Clinical Practice Guidelines

  1. Hayward RS, Wilson MC, Tunis S, Bass EB, Guyatt GH, for the Evidence–Based Medicine Working Group. Users’ guides to the medical literature: VIII. How to use clinical practice guidelines A. Are the results valid? JAMA 1995; 274: 570–574.
  2. Wilson MC, Hayward RSA, Tunis S, Bass EB, Guyatt GH, for the Evidence–Based Medicine Working Group. Users’ guides to the medical literature: VIII. How to use clinical practice guidelines B. What are the recommendations and will they help me in caring for my patients? JAMA 1995; 274: 1630 – 1632.
    [These 2 emphasize how to understand, appraise and use guidelines, not how to make them or keep them up to date.]

  3. Guyatt GH, Sackett DL, Sinclair JC, Hayward RSA, Cook DJ, Cook RJ, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature: IX. A method for grading health care recommendations. JAMA 1995; 274: 1800 – 1804.
    [This lays out a grading scheme for recommendations from guidelines.]

  4. Ellrodt AG, Cook DJ, Lee J, Cho M, Hunt DL, Weingarten SR. Evidence-based disease management. JAMA 1997; 278: 1687 – 1692.
    [This illustrates one way of how CPGs and other tools can be put to use.]

  J. Health Services Research 

  1. Naylor CD, Guyatt GH, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature: X. How to use an article reporting variations in the outcomes of health services. JAMA 1996; 275: 554 – 558.
    [Much of ‘outcomes research’ compares therapies without an experimental design; this article shows you how to understand, appraise and decide whether, and if so how, to use such evidence. Notice how crucial the issues of prognosis are to understanding these studies.]

K. Utilization Review 

  1. Naylor CD, Guyatt GH, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature: XI. How to use an article about a clinical utilization review. JAMA 1996; 275: 1435 – 1439.
    [Another piece that emphasizes how to understand, appraise and use nonexperimental evidence about health care utilization, where prognosis is again key.]

L. Health-Related Quality of Life

  1. Guyatt GH, Naylor CD, Juniper E, Heyland DK, Jaeschke R, Cook DJ, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature: XII. How to use articles about health-related quality of life. JAMA 1997; 277: 1232 – 1237.
    [Emphasizes how to understand, appraise and use articles that use measures of HRQL in reporting the results. Since this situation comes up most often in studies of therapy, use these guides in combination with the guides on therapy.]

Stay Tuned … A Few More Installments in the Users’ Guides Series Still to Come

Other Resources You May Find Useful

Evidence-Based Care (CMAJ) Series:  
[This series of 5 articles will help you get ‘off the shelf’ in a practical, organized way. They discuss how to organize your clinical work to provide evidence-based care in the real world. Read these before deciding whether or not to resuscitate your journal club again.]

  1. Oxman AD, MacDonald PJ, and the Evidence-Based Care Resource Group. Evidence-based care: 1. Setting priorities: How important is this problem? Can Med Assoc J 1994; 150: 1249 – 1254.
  2. Oxman AD, Feightner JW, and the Evidence-Based Care Resource Group. Evidence-based care: 2. Setting guidelines: How should we manage this problem? Can Med Assoc J 1994; 150: 1417 – 1423. 
  3. Hutchison BG, Feightner JW, Lusk S, and the Evidence-Based Care Resource Group. Evidence-based care: 3. Measuring performance: How are we managing this problem? Can Med Assoc J 1994; 150: 1575 – 1579.
  4. Oxman AD, Davis DA, and the Evidence-Based Care Resource Group. Evidence-based care: 4. Improving performance: How can we improve the way we manage this problem? Can Med Assoc J 1994; 150: 1793 – 1796.
  5. Oxman AD, and the Evidence-Based Care Resource Group. Evidence-based care: 5. Lifelong learning: How can we learn to be more effective? Can Med Assoc J 1994; 150: 1971 – 1973.

Evidence-Based Critical Care Medicine Series: 
[This series introduces critical appraisal to the critical care audience, and after the 2 introductory pieces, walks readers through applying the Users’ Guides to actual examples. Thus, these articles will include some more ‘worked examples’, using clinical topics that may appeal to some of your colleagues and learners.]

  1. Cook DJ, Sibbald WJ, Vincent JL, Cerra FB for the Evidence-Based Medicine in Critical Care Group. Evidence-based critical care medicine: What is it and what can it do for us? Crit Care Med 1996; 24: 334–337.
  2. Cook DJ, Meade MO, Fink MP, for the Evidence-Based Medicine in Critical Care Group. How to keep up with the critical care literature and avoid being buried alive. Crit Care Med 1996; 24: 1757 – 1768.
    [This article presents 15 tips for (attempting to) keep up to date with any literature, not just critical care.]

  3. Cook DJ, Hebert PC, Heyland DK, Guyatt GH, Brun-Buisson C, Marshall JC, Russell J, Vincent JL, Sprung CL, Rutledge F. How to use an article on therapy or prevention: pneumonia prevention using subglottic secretion drainage. Crit Care Med 1997; 25: 1502 – 1513. 
  4. Jaeschke RZ, Meade MO, Guyatt GH, Keenan SP, Cook DJ. How to use diagnotic test articles in the intensive care unit: diagnosing weanability using f/Vt. Crit Care Med 1997; 25: 1514 – 1521. 
  5. Meade MO, Cook DJ, Kernerman P, Bernard G. How to use articles about harm: The relationship between high tidal volumes, ventilating pressures, and ventilator-induced lung injury. Crit Care Med 1997; 25: 1915 – 1922.
    [More installments to follow]

Evidence-Based Emergency Medicine Series: 

  1. Waeckerle JF, Cordell WH, Wyer P, Osborn HH. Evidence-based emergency medicine: integrating research into practice. Ann Emerg Med 1997 (Nov); 30: 626 – 628. 
  2. Guyatt GH. Evidence-based emergency medicine. Ann Emerg Med 1997; 30: 675 – 676.
    [These 2 introduce a new series in this journal; just as with critical care, these articles should help bring the topic alive for your emergency medicine colleagues.]

What If They Ask Me About Biostatistics or Clinical Epidemiology?

Basic Statistics for Clinicians (CMAJ) Series:
[These 4 articles cover some statistical fundamentals in relatively painless language. Consider having these at hand for when the inevitable questions arise.]

  1. Guyatt GH, Jaeschke R, Heddle N, Cook DJ, Shannon H, Walter S . Basic statistics for clinicians. I. Introduction and hypothesis testing. Can Med Assoc J 1995; 152: 27–32. 
  2. Guyatt GH, Jaeschke R, Heddle N, Cook DJ, Shannon H, Walter S. Basic statistics for clinicians. II. Interpreting study results: confidence intervals. Can Med Assoc J 1995; 152: 169–173. 
  3. Jaeschke R, Guyatt GH, Shannon H, Walter S, Cook DJ, Heddle N. Basic statistics for clinicians. III. Assessing the effects of treatment: Measures of association. Can Med Assoc J 1995; 152: 351–357. 
  4. Guyatt GH, Walter S, Shannon H, Cook DJ, Jaeschke R, Heddle N. Basic statistics for clinicians. IV. Correlation and regression. Can Med Assoc J 1995; 152: 497–504.

Other Resources on Biostatistics and Clinical Epidemiology

  1. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology: A Basic Science for Clinicians, 2/e, Little, Brown & Co., 1991. 
  2. Fletcher RH, Fletcher SW, Wagner EH. Clinical Epidemiology: The Essentials, 3/e, Baltimore, Williams & Wilkins, 1996.
    [Both these books provide good depth and additional references. They provide complementary perspectives on the field of clinical epidemiology, and include many useful teaching illustrations.]

  3. Atkins CD. A Clinician’s View of Statistics. J Gen Intern Med 1997; 12: 500-504. 
  4. Atkins CD. Translating Statistics for Use in the Clinic. J Gen Intern Med1997; 12: 626-628.
    [Watch for this ongoing series in the Journal of General Internal Medicine that is entitled "From Research to Practice". These 2 installments use practical clinical examples to illustrate rational interpretation of data from clinical research]  

  5. Norman GR, Streiner DL. PDQ Statistics , 2/e, St. Louis, Mosby, 1997. 
  6. Streiner DL, Norman GR. PDQ Epidemiology , 2/e, St. Louis, Mosby, 1996.
    [Both are compact and focused on how to understand, not how to do clinical research. Norman has a wonderful, wry sense of irreverent humor as manifest in his use of ‘C.R.A.P. Detectors’ (Convoluted Reasoning or Anti-Intellectual Pomposity Detectors) at the end of chapters to illustrate real examples from research where the unsuspecting clinician could be hoodwinked into accepting faulty conclusions.]

  7. Lang TA, Secic M. How To Report Statistics In Medicine Philadelphia, ACP, 1996.
    [Less compact, with broad range and new topics, e.g. about meta-analysis.]

  8. Norman GR, Streiner DL. Biostatistics: The Bare Essentials St. Louis, Mosby, 1994. 
  9. Altman DG. Practical Statistics for Medical Research London, Chapman & Hall, 1991.
    [These 2 are more ‘traditional’ in teaching about how to do statistics.]

  10. Friedland DJ, Go A, Davoren JB, Shlipak MG, Bent SW, Subak LL, Mendelson T, eds. Evidence-Based Medicine: A Framework For Clinical Practice, Appleton-Lange, 1998.
    [This book grew out of a successful EBM faculty development program as part of a successful EBM-oriented residency program. Coming soon to a bookstore near you …]

What If Folks Ask For Advice On Keeping Up To Date?

 

  1. Cook DJ, Meade MO, Fink MP, for the Evidence-Based Critical Care Medicine Group. How to keep up with the critical care literature and avoid being buried alive. Crit Care Med 1996; 24: 1757 – 1768.
    [Presents 15-tip plan useful for keeping up in any field, not just critical care.]

  2. Fletcher RH, Fletcher SW. Evidence-based approach to the medical literature. J Gen Intern Med 1997 (Apr); 12 [Suppl. 2]: S5 – S14. 
  3. Larson EB. How can clinicians incorporate research advances into practice? J Gen Intern Med 1997 (Apr); 12 [Suppl. 2]: S20 – S24.
    [These 2 articles present ideas about organizing yourself to stay up to date. They are in a supplement to J Gen Intern Med that addresses many issues of importance to APDIM members.]

Has Anyone Written More On How To Teach Critical Appraisal And EBM?

[Arranged by year of publication, from oldest to newest; within years, arranged alphabetically by last name of first author. As you read these, keep in mind you can learn by both imitation and evitation.]

  1. Cullinan TR. Teaching concepts in epidemiology. J Epidemiol Community Health 1980; 34: 201 – 203. 
  2. Gehlbach SH, Bobula JA, Dickinson JC. Teaching residents to read the literature. J Med Educ 1980; 55: 362 – 365. 
  3. Mulvihill MN, Wallman G, Blum S. A seven year retrospective view of a course in epidemiology and biostatistics. J Med Educ 1980; 55: 457 – 459.
  4. Inui TS. Critical reading seminars for medical residents: report of a teaching technique. Med Care 1981; 29: 122 – 124.
  5. Woods JR, Winkel CE. Journal club format emphasizing techniques of critical reading. J Med Educ 1982; 52: 799 – 801. 
  6. Cuddy PG, Elenbaas JK, Coit KJ. The effectiveness of a slide tape program on literature evaluation. J Biocommun 1984; 11: 2 – 4. 
  7. Heller RF, Peach H. Evaluation of a new course to teach the principles and clinical applications of epidemiology to medical students. Int J Epidemiol 1984; 13: 533 – 537.
  8. Linzer ML, Mercando A, Hupart KH. Role of a medical journal club in residency training. J Med Educ 1986; 61: 471 – 473.
  9. Radack KL, Valanis B. Teaching critical appraisal and application of medical literature to clinical problem solving. J Med Educ 1986; 61: 329 – 331.
  10. Riegelman RK. Effects of teaching first year medical students skills to read medical literature. J Med Educ 1986; 61: 454 – 460. 
  11. Bennett KJ, Sackett DL, Haynes RB, Neufeld VR, Tugwell P, Roberts RS. A controlled trial of teaching critical appraisal of the clinical literature to medical students. JAMA 1987; 257: 2451 – 2454. 
    [Shows that critical appraisal is a "use it or lose it" skill, among other things.]

  12. Knapp RG, Miller MC. Clinical relevance: an issue in biostatistical training of medical students. Med Educ 1987; 21: 32 – 37. 
  13. Linzer ML, DeLong ER, Hupart KH. A comparison of two formats for teaching critical reading skills in a medical journal club. J Med Educ 1987; 62: 690 – 692.
  14. Linzer ML, Brown JT, Frazier LM, DeLong ER, Siegel WC. Impact of a medical journal club on house staff reading habits, knowledge and critical appraisal: a randomized controlled trial. JAMA 1988; 260: 2537 – 2541. 
  15. Kitchens J, Pfeiffer MP. Teaching residents to read the medical literature: a controlled trial of curriculum in critical appraisal/clinical epidemiology. J Gen Intern Med 1989; 4: 385 – 387. 
  16. Romm FJ, Dignan M, Hermann JM. Teaching clinical epidemiology: a controlled trial of two methods. Am J Prev Med 1989; 1: 50 – 51. 
  17. Linzer ML. Critical appraisal: more work to be done. J Gen Intern Med 1990; 5: 457 – 459. 
  18. Seeling CB. Affecting residents’ literature reading attitudes, behaviors and knowledge through a journal club intervention. J Gen Intern Med 1991; 6: 330 – 334. 
  19. O’Sullivan PS, Jeremiah J, Wartman SA. Learner centered journal club teaches residents more about assessing the medical literature [Abstract]. Clin Res 1991; 39: 620A. 
  20. Frasca MA, Dorsch JL, Aldag JC, Christiansen RG. A multidisciplinary approach to information management and critical appraisal instruction: a controlled study. Bull Med Library Assoc 1992; 80: 23 – 27. 
  21. Audet N, Gagnon R, Ladouceur R, Marcil M. L’enseignement de l’analyse critique des publications scientifiques medicales est-il efficace? Revision des etudes et de leur qualite methodologique. Can Med Assoc J 1993; 148: 945 – 952. 
    [A systematic review of what worked ‘til then; sorry we’ve left off the accents.]

  22. Shin JH, Haynes RB, Johnston ME. Effect of problem-based, self-directed undergraduate education on life-long learning. Can Med Assoc J 1993; 148: 969 – 976.
    [Compares students from 2 schools, one teaching critical appraisal, one not.]

  23. Landry FJ, Pangaro L, Kroenke K, Lucey C, Herbers J. A controlled trial of a seminar to improve medical students attitudes toward, knowledge about and use of the medical literature. J Gen Intern Med 1994; 9: 436 – 439. 
  24. Sidorov J. How are internal medicine residency journal clubs organized and what makes them successful? Arch Intern Med 1995; 155: 1193 – 1197.
    [Describes journal clubs, whether or not they’re evidence-based.]

  25. Bordley DR, Fagan M, Thiege D. Evidence-based medicine: a powerful educational tool for clerkship education. Am J Med 1997; 102: 427 – 432. 
  26. Burls AJE. An evaluation of the impact of half-day workshops teaching critical appraisal skills. Oxford (UK): Institute of Health Sciences; 1997.
  27. Green ML, Ellis PJ. Impact of an evidence-based medicine curriculum based on adult learning theory. J Gen Intern Med 1997; 12: 742 – 750.
    [This study includes teaching the ‘whole package’, including coaching on questions, on searching, on critical appraisal and use of evidence.]

  28. Reilly B, Lemon M. Evidence-based morning report: a popular new format in a teaching hospital. Am J Med 1997; 103: 419 – 426.
    [Describes the application of EBM in a forum other than journal club.]

  29. Taylor RS, Reeves B, Ewings P. How effective is teaching critical appraisal skills to health care professionals? A systematic review of the literature [poster #263]. 5th Annual Cochrane Colloquium; 1997 Oct; Amsterdam. 
  30. Wyer PC. The critically appraised topic: closing the evidence-transfer gap. Ann Emerg Med 1997; 30: 639 – 640.
    [Takes the CAT out of Hamilton & Oxford and hits the big city ED with it.]

  31. Norman GR, Shannon SI. Effectiveness of instruction in critical appraisal (evidence-based medicine) skills: a critical appraisal. Can Med Assoc J 1998; 158: 177 – 181. 
  32. Sackett DL, Parkes J. Teaching critical appraisal: no quick fixes. Can Med Assoc J 1998; 158: 203 – 204.
    [Concisely describes where we’ve been and where we’re heading.]

 Rational Clinical Examination Series:
[This series of articles gets ‘off the shelf’ in a hurry. They summarize what is known about the diagnostic power of items from the history and physical and conclude with a clinical bottom line. The topics grab the attention of experts and novices alike, and serve as a great bridge into EBM for beginning students who’ve just learned how to examine patients. These articles are worth collecting into a notebook for use wherever you teach.] 

  1. Sackett DL. A primer on the precision and accuracy of the clinical examination. JAMA 1992; 267: 2638 – 2644.
  2. Sackett DL, Rennie D. The science of the art of the clinical examination [Editorial]. JAMA 1992; 267: 2650 – 2652. 
  3. Williams JW, Simel DL. Does this patient have ascites? How to divine fluid in the abdomen. JAMA 1992; 267: 2645 – 2648. 
  4. Deyo RA, Rainville J, Kent DL. What can the history and physical examination tell us about low back pain? JAMA 1992; 268: 760 – 765. 
  5. Stephenson BJ, Rowe BH, Haynes RB, Macharia WM, Leon G. Is this patient taking the treatment as prescribed? JAMA 1993; 269: 2779 – 2781. 
  6. Williams JW, Simel DL. Does this patient have sinusitis? Diagnosing acute sinusitis by history and physical examination. JAMA 1993; 270: 1242 – 1246. 
  7. Grover SA, Barkun AN, Sackett DL. Does this patient have splenomegaly? JAMA 1993; 270: 2218 – 2221. 
  8. Sauve JS, Laupacis A, Ostbye T, Feagan B, Sackett DL. Does this patient have a clinically important carotid bruit? JAMA 1993; 270: 2843 – 2845.
  9. Detsky AS, Smalley PS, Chang J. Is this patient malnourished? JAMA 1994; 271: 54–58. 
  10. Froehling DA, Silverstein MD, Mohr DN, Beatty CW. Does this dizzy patient have a serious form of vertigo? JAMA 1994; 271: 385 – 388. 
  11. Goldstein LB, Matchar DB. Clinical assessment of stroke. JAMA 1994; 271: 1114 – 1120. 
  12. Naylor CD. Physical examination of the liver. JAMA 1994; 271: 1859 – 1865. 
  13. Kitchens JM. Does this patient have an alcohol problem? JAMA 1994; 272: 1782 – 1787. 
  14. Holleman DR, Simel DL. Does the clinical examination predict airflow limitation? JAMA 1995; 273: 313 – 319. 
  15. Siminoski K. Does this patient have a goiter? JAMA 1995; 273: 813 – 817. 
  16. Reeves RA. Does this patient have hypertension? How to measure blood pressure. JAMA 1995; 273: 1211 – 1218. 
  17. Turnbull JM. Is listening for abdominal bruits useful in the evaluation of hypertension? JAMA 1995; 274: 1299 – 1301. 
  18. Cook DJ, Simel DL. Does this patient have abnormal central venous pressure? JAMA 1996; 275: 630 – 634. 
  19. Wagner JM, McKinney P, Carpenter JL. Does this patient have appendicitis? JAMA 1996; 276: 1589 - 1594. 
  20. Etchells E, Bell C, Robb K. Does this patient have an abnormal systolic murmur? JAMA 1997; 277: 564 – 571.
  21. Simel DL, Rennie D. The clinical examination: an agenda to make it more rational [Editorial]. JAMA 1997; 277: 572 – 574. 
  22. Badgett RG, Lucey CR, Mulrow CD. Can the clinical examination diagnose left-sided heart failure in adults? JAMA 1997; 277: 1712 – 1719.
  23. Bastian LA, Piscitelli JT. Is this patient pregnant? Can you reliably rule in or rule out early pregnancy by clinical examination? JAMA 1997; 278: 586 – 591. 
  24. Metlay JP, Kapoor WN, Fine MJ. Does this patient have community-acquired pneumonia? Diagnosing pneumonia by history and physical examination. JAMA 1997; 278: 1440 – 1445. 
  25. Margolis P, Gadomski A. Does this infant have pneumonia? JAMA 1998; 279: 308 – 313.

[More installments to follow]

While we’re on the topic of the clinical examination, here are 3 more resources that may help you get off the shelf using evidence in practicing and teaching it ... 

  1. Sapira JD. The Art & Science Of Bedside Diagnosis. Baltimore, Urban & Schwarzenberg, 1990.
  2. Schneiderman H, Peixoto AJ. Bedside Diagnosis: An Annotated Bibliography of Literature On Physical Examination And Interviewing. 3/e, Philadelphia, American College of Physicians, 1997. 
  3. Hatala R, Smieja M, Kane SL, Cook DJ, Meade MO, Nishikawa J. An evidence-based approach to the clinical examination. J Gen Intern Med 1997; 12: 182 – 187.

Still More Resources You May Find Useful For Practising And Teaching EBM

 

  1. Gray JAM. Evidence-Based Healthcare: How to Make Health Policy and Management Decisions, New York, Churchill-Livingstone, 1997.
    [This book focuses on using evidence in decisions at the level of policy and management, which may help you get the attention of your Chair, Dean or President. And Muir’s delightful Scottish brogue comes through in his writing.]

  2. Panzer RJ, Black ER, Griner PF, eds. Diagnostic Strategies For Common Medical Problems, Philadelphia, American College of Physicians, 1991.
  3. Sox HC, Jr, ed. Common Diagnostic Tests. Philadelphia, American College of Physicians, 1991. 
  4. Eddy DM, ed. Commong Screening Tests. Philadelphia, American College of Physicians, 1991.
    [Although somewhat dated, these 3 books still provide good starting places for using evidence about tests in clinical practice. Rumor says Panzer book will be out in 3rd edition this year]
  5. Enkin M, Keirse MJNC, Renfrew M, Neilson J. A Guide to Effective Care in Pregnancy & Childbirth, 2/e, Oxford, Oxford University Press, 1995.
    [An example of an overtly evidence-based text, this book is useful for internists when consulting on obstetrical patients and in answering questions about ‘old wives tales’.]

  6. U.S. Preventive Services Task Force. Guide to Clinical Preventive Services, 2/e, Baltimore, Williams & Wilkins, 1996.
    [A compendium of recommendations with supporting evidence.]

  7. Fourth American College of Chest Physicians Consensus Conference on Antithrombotic Therapy. Chest 1995 (Oct); 108: 225S – 522S.
    [Another overtly evidence-based monograph.]

  8. Sauve S, Lee HN, Meade MO, Lang JD, Farkouh M, Cook DJ, Sackett DL. The critically appraised topic: a practical approach to learning critical appraisal. Ann Roy Coll Phys Surg Canada 1995; 28: 396 – 398.
    [Introduces the CAT (critically appraised topic), a handy way to record concisely the results of your EBM work (question, search, appraisal, use)

  9. Lee BW, Hsu SI, Stasior DS, eds. Quick Consult Manual of Evidence-Based Medicine, Philadelphia, Lippincott-Raven, 1997. 
    [Includes some summaries of evidence that support the recommendations concerning 34 topics, mostly in hospital-based medicine. Does not include explicit criteria for selection of evidence, and occasionally the tables have p values without NNTs.]

  10. Dale DC, Federman D, eds. Scientific American Medicine and SAM-CD. Available from Scientific American Medicine, 415 Madison Avenue, New York, NY, 10017, USA. (800-545-0554)
    [An example of a textbook of medicine that works hard to keep up to date and cites specific evidence for new recommendations. Seldom include study results and effects sizes.]

  11. Greenhalgh T. How To Read A Paper: The Basics Of Evidence-Based Medicine. London, BMJ Publishing Group, 1997.
    [This introductory text grew out of a series of articles published in the BMJ on how to read different sorts of papers. Includes a chapter on reading qualitative research.]

  12. Burns R, Pahor M, Shorr RI. Evidence-based medicine holds the key to the future for geriatric medicine. J Am Geriatr Soc 1997 Oct; 45: 1268 – 1272. 
  13. Geyman JP. Evidence-based medicine in primary care: an overview. J Am Board Fam Pract 1998 Jan; 11: 46 – 56. 

Periodicals & Electronic Tools for Learning and Teaching EBM: 

  1. ACP J Club: published by ACP. It is the first, evidence-based ‘secondary’ publication; it includes structured abstracts of high quality studies that are relevant to the practice of Internal Medicine.
  2. Evidence-Based Medicine: from the ACP and BMJ Publishers; uses the same format and methods as ACP J Club, but also including studies relevant to Pediatrics, Obstetrics, Surgery, Psychiatry, General and Family Practice.
  3. Best Evidence: a CD-ROM that includes the complete contents of ACPJC & EBM, from 1991 onward, updated quarterly, in a very easy to search database.
  4. ACP Library On Disk: contains Best Evidence plus many other ACP evidence-based information products, including Diagnostic Strategies for Common Medical Problems; Common Diagnostic Tests; Common Screening Tests; ACP Guidelines; Medical Knowledge Self-Assessment Program.
  5. Cochrane Library: a CD-ROM that contains the entire Cochrane Database of Systematic Reviews, along with some other systematic reviews and methodologic literature about evidence and how to use it soundly. Updated quarterly.
  6. Contact for Subscriptions to all the above:

    American College of Physicians
    Independence Mall West
    Sixth Street at Race
    Philadelphia, PA 19106-1572
    T: (800) 523-1546
    Or: (215) 351-2400
    F: (215) 351-2644

  7. Evidence-Based Cardiovascular Medicine: similar in approach to ACPJC and EBM, with the focus on cardiovascular disease. Turn your cardiologists onto this, if they aren’t already.
    Contact: Churchill Livingstone
    PO Box 3188
    Secaucus, NJ 07096-3188
    T: (800) 553-5426
    Or (201) 319-9800
    F: (201) 319-9659
    Email: orders@edinburgh.rsh.pearson-pro.com
  8. Evidence-Based Nursing: similar in approach to ACPJC and EBM. First issue is November, 1997. Published jointly by Royal College of Nurses Publishing Company and BMJ Publishing Group, with editorial offices at McMaster. Turn your nursing colleagues onto this, and consider starting some multi-disciplinary evidence-based rounds.
    Contact: BMJ Publishing Group
    BMA House, Tavistock Square
    London, WC1H 9JR, UK
    +44-171-383-6662
     
  9. Evidence-Based Mental Health: similar in approach to ACPJC and EBM. First issue is 1997 or 1998. Turn your mental health colleagues onto this, and invite them to your multi-disciplinary evidence-based rounds, too.
    Contact: BMJ Publishing Group
    BMA House, Tavistock Square
    London, WC1H 9JR, UK
    +44-171-383-6662
     
  10. Evidence-Based Health Policy & Management: again, similar in approach to ACPJC and EBM, although focused more at the policy level rather than at the patient level. First issue is 1997. Turn your administrator colleagues onto this, and invite them to pay for the food at your multi-disciplinary evidence-based rounds.
    Contact: Churchill Livingstone
    Order Department
    PO Box 3217
    Secaucus, NJ 07096-9930
    T: (800) 553-5426
    Or (201) 319-9800
    F: (201) 319-9659
    Email: orders@edinburgh.rsh.pearson-pro.com
  11. UpToDate: a quarterly CD-ROM text that provides a mix of basic and applied evidence supporting care in nephrology, cardiovascular medicine, endocrinology and diabetes.
    Contact: UpToDate Inc.
    PO Box 8?
    Wellesley MA 12098
    T: (617) 237-4788
    F: (617) 239-0391
    Email: uptodate@bdrinc.com
  12. Bandolier: this monthly newsletter out of Oxford, UK, contains bullet points (hence bandolier) of EBM. Available to folks outside the British NHS for a subscription fee; also available for free on internet (see below).
    Contact: Bandolier
    Hayward Medical Communications
    Rosemary House, Lanwades Park
    Kentford, Near Newmarket
    Suffolk CB8 7PW UK
    T: +44-1638-751515
    F: +44-1638-751517
    Email: hayward.newmarket@dial.pipex.com

Where In The World Can I Go To Learn More?

 

  1. Internet/World Wide Web Sites: 
  2. a. Evidence-Based Care Home Page
  3. b. Centre for Evidence Based Medicine (UK)
  4. c. New York Academy of Medicine
  5. c. SCHARR site (UK)
  6. d. Bandolier site (UK)
  7. e. Cochrane Collaboration  
  8. & if you read/speak Italian, there’s: http://www.gimbe/org/
    [Each site has a lot available, such as the Users’ Guides at McMaster, the CAT Bank and the EBM ToolBox at Oxford, and the massive bibliography at the SCHARR. Also, these sites include links to many other related sites, too numerous to list here. Don’t forget to send the site administrators your suggestions for improvements.] 
  9. Evidence-Based-Health Electronic Mail List
  10. to access: send a message to: mailbase@mailbase.ac.uk
    subject line: leave blank
    in the message type: join evidence-based-health your name here stop
    [Be prepared to kiss a lot of frogs in order to find some princes.]
  11. Listserv to discuss how to teach EBM, especially medical residents
  12. to access: send email to: listserv@health.nyam.org
    subject line: leave blank
    in the message type: subscribe EBMNY firstname lastname
    [A brand new listserve managed by the EBM initiative of the New York Academy of Medicine and the NY – ACP that is aspiring to provide a new forum for collaborative cross-fertilization between EBM educators. Inside tip: it’s a list to watch & contribute to]
  13. Workshops on ‘How To Teach Evidence-Based Medicine’:
  14. a. McMaster University, Canada: Please contact workshop coordinator:
    Deborah Maddock, Workshop Coordinator
    Dept. of Clinical Epidemiology & Biostatistics
    McMaster University HSC, Rm 2C9
    1200 Main Street West
    Hamilton, Ontario L8N 3Z5
    Phone: (905) 525-9140, X22900
    Fax: (905) 577-0017
    Email: maddock@fhs.mcmaster.ca
  15. b. Oxford University, UK: Please contact program manager:
    Douglas S. Badenoch, Programme Manager
    NHS R&D Centre for Evidence-Based Medicine
    Nuffield Department of Clinical Medicine
    University of Oxford
    Level 5, The John Radcliffe Hospital
    Headley Way, Headington
    Oxford OX3 9DU
    Phone: +44 1865 221321
    Fax: +44 1865 222901
    Email: badenoch@cebm.jr2.ox.ac.uk
  16. c. London, UK: Please contact course director:
    Dr. Trish Greenhalgh
    Senior Lecturer in Primary Health Care
    Unit for Evidence-Based Practice and Policy
    Joint Department of Primary Care & Population Sciences
    UCLMS/RFHSM
    Whittington Hospital
    London N19 5NF UK
    T: +44-171-288-5731
    F: +44-171-281-8004
    Email: p.greenhalgh@ucl.ac.uk
  17. d. Web: http://www.ucl.ac.uk/primcare-popsci/uebpp.htm
    [These 3 are week-long courses of enormous value to EBM teachers. All provide an intensive immersion in the field, and all recommend that you prepare ahead of time with the packets they send (and we strongly agree!). Slight differences exist (British course materials are on A4 paper, not letter size!), but we can recommend all of them highly. Similar courses are starting up in several places: Cardiff, Oslo, Madrid, Buenos Aires and Milan to name a few. Maybe someday there’ll be a course in the USA as well.]

The New York Academy of Medicine

 Library
1216 Fifth Avenue
New York, NY 10029-5293