Dr. Walter Himmel (the ‘walking encyclopedia of EM’) gave a fantastic talk from North York General’s Emergency Medicine Update Conference in Toronto, which has been edited with key commentary and summaries into a digestible podcast. Dr. Himmel eloquently shows us, through absolutely stunning personal cases, how evidence based medicine can be appropriately or inappropriately applied in real practice, resulting in major outcome differences for your patients. He elucidates the importance of clinical experience, patient values and ED resources in helping apply the medical literature to your practice. He reviews the essence of critical appraisal, the hierarchy of evidence and how to keep up with the emergency medicine literature. The famous NINDS thrombolysis for stroke trial is distilled down to a few key considerations and the NEJM transfusion for upper GI bleed trial from last year is dissected, analyzed and applied to Dr. Himmel’s personal cases, to help us understand exactly how to avoid BARF – the Brainless Application of Research Findings.
Blog post and written summary prepared by Keerat Grewal, edited by Anton Helman July 2014
Evidence Based Medicine (EBM)
Three Spheres of EBM1
David Sackett: An integration of factors are required to help improve patient outcomes when using EBM
1) Best research evidence 2) Clinical expertise 3) Patient values
Fig 1: Spheres of EBM
Hierarchy of Evidence (see Fig 2)
Strength of evidence increases from:
Expert opinion, case reports, cohort studies, RCTs (which attempt to minimize systematic error) and systematic reviews / meta-analysis (a combination of trials and the highest level of evidence)
Another model of levels of evidence is the 6S model2 (see Fig 3). The quality of evidence increases from single studies; synopses of single studies; syntheses (e.g. Cochrane reviews, can include systematic reviews); synopses of syntheses (e.g. summary of a systematic review); summaries (e.g. evidence-based textbooks, clinical practice guidelines); systems (fully developed clinical decision systems)
Fig 3: Levels of Evidence – 6S Model
EBM asks specific, useful, efficient questions (PICO):
P: who is the patient, population, or problem of interest
I: what is the intervention
C: what is the comparison
O: what are the outcomes
Minimizing Error in EBM
consider sources of error, or deviation from the truth, of evidence:
Systematic Error: is BIAS, or non-random error (e.g. failure to blind, unconcealed allocation, patient drop out, etc.
Random Error: due to chance, unavoidable; revealed by consideration of statistical concepts such as p-values and confidence intervals
Critical Appraisal of Evidence
When assessing evidence/literature, ask yourself questions about the study and the results. Avoid BARF (Brainless Application of Research Findings). Critical appraisal of a study can be based on the JAMA User’s Guide to Medical Literature3:
- Are the results valid? Is it free from bias?
Were the patients randomized? Were patients in the study groups similar? Was the study blinded? Was follow up complete?
- Are the outcomes important? Is there significant clinical importance to the patient?
What were the results? How large was the treatment effect? How precise were the results? Are all patient-oriented outcomes considered (harms and benefits)?
- Are the results relevant to my practice?
Were the study patients similar to your population (inclusion criteria)? Is the setting similar? Are the treatment benefits worth the potential harm and costs? What are your patient’s values? Is it feasible in your institution?
To improve your critical appraisal skills of landmark EM articles go to the Global EM Journal Club on the Academic Life in EM blog. Note that coming soon will be the JOURNAL JAM podcast, an EM Cases – ALiEM – Annals of EM collaboration, that summarizes the Global EM Journal Club and interviews the lead authors of the landmark EM articles.
For excellent critical appraisal tools- Ken Milne’s Skeptics Guide to EM BEEM tool kit.
Examples of Application of EBM
Tissue plasminogen activator for acute ischemic stroke (NINDS stroke study group, NEJM, 1995)4
Two part study, part 1: assessed whether TPA had clinical activity (resolution of symptoms, improvement in NIHSS); part 2: clinical outcomes assessed at 3 months
Randomized trial (TPA vs. placebo), blinded
Results: 26% of placebo arm had good function at 3 months (based on modified Rankin score); vs. 39% of the TPA arm. However, 6% of the TPA arm had intracerebral hemorrhage (with about half of those patients dying)
When critically appraising this article, it is important to note that patients in the placebo arm were sicker than the TPA arm (NIHHS 15 vs. 14, respectively). This suggests that randomization failed. Moreover, patients were not consecutively enrolled. Given the findings of this study, incorporation of patient values in administration of TPA is very important (i.e. ICH/mortality risk vs. functional recovery).
Go here for an in depth discussion with Walter Himmel and Daniel Selchen on Stroke Controversies including a review of the major thrombolysis trials, and here for our Journal Jam podcast dissecting the world’s literature on thrombolysis for stroke.
Transfusion strategies for acute upper gastrointestinal bleeding (Villaneuva et al., NEJM, 2013)5
N = 900 with UGIB (melena or hematemesis)
No active heart disease, no LGIB, no massive bleeding, no stroke history
Patients had access to gastroscopy within 6hrs
Patients were randomized into one of two groups:
1) restricted transfusion group: blood transfusion for Hb <70; or
2) liberal transfusion group: blood transfusion for Hb <90
If patient had SOB, was symptomatic, looked unwell, patient received blood transfusion regardless of Hb
Results: Mortality in restricted transfusion group was 5%, in liberal transfusion group, mortality was almost doubled. Adverse events in liberally transfused group was significantly higher than those in the restricted group.
When critically appraising this article, note that patients who were unwell, unstable or symptomatic were transfused as per clinical judgment, not according to the group they were randomized to. Moreover, patients in this study had access to gastroscopy within 6 hours, which may be difficult to achieve depending on your local resources. This is a good example of the need to carefully assess the details of patient enrolment (does your patient fit the precise enrolment criteria?) and whether the circumstances of a study are applicable to your Emergency Department (can you get a gastroscopy at your hospital within 6 hours?).
Go here for an in depth discussion with Walter Himmel, Katerina Pavenski & Jeannie Callum on Transfusions, Anticoagulants & Bleeding, where they discuss the upper GI bleed trial as well as transfusions in patients with coronary artery disease, transfusion reactions and more.
Dr. Helman and Dr. Himmel have no conflicts of interest to declare.
- Sackett, D. et al. Evidence based medicine: What it is and what it isn’t. 1996. BMJ, 312: 71. Free Full Text available at: http://www.bmj.com/content/312/7023/71
- DiCenso, a. et al. Accessing preappraised evidence: Fine-tuning the 5S into a 6S model. 2009. ACP Journal Club, 151(3): JC2-3. Abstract available at: http://www.ncbi.nlm.nih.gov/pubmed/19755349
- Guyatt, G. et al. JAMA User’s Guide to Medical Literature, 2nd edition. 2008. JAMA. Free Full Text: http://jamaevidence.com/resource/520
- NINDS Stroke Study Group. Tissue plasminogen activator for acute ischemic stroke.1995. NEJM, 333(24): 1581-1587. Free Full PDF available at: http://www.nejm.org/doi/full/10.1056/NEJM199512143332401
- Villaneuva, C. et al. Transfusion strategies for acute upper gastrointestinal bleeding. 2013. NEJM, 368(1): 11-21. Free Full PDF available at: http://www.nejm.org/doi/full/10.1056/NEJMoa1211801