This is Part 1 of EM Cases’ series on Diagnostic Decision Making with Walter Himmel, Chris Hicks and David Dushenski discussing the intersection of evidence-based medicine, cognitive bias and systems issues to effect our diagnostic decision making in Emergency Medicine. In this episode we first discuss 5 strategies to help you master evidence-based diagnostic decision making to minimize diagnostic error, avoid over-testing and improve patient care including:
1. The incorporation of patients’ values and clinical expertise into evidence-based decisions
2. Critically appraising diagnostic studies
3. Understanding that diagnostic tests are not perfect
4. Using the concept of test threshold to guide work-ups
5. Understanding that the predictive value of a test depends on the prevalence of disease
We then go on to review some of the factors that play into the clinician’s and patient’s risk tolerance in a given clinical encounter, how this plays into shared decision making and the need to adjust our risk tolerance in critical situations. Finally, we present some strategies to prevent over-testing while improving patient care, patient flow and ethical practice.
Written Summary and blog-post Prepared by Dr. Anton Helman, April 2015
Are doctors effective diagnosticians?
Diagnostic errors accounted for 17% of preventable errors in medical practice in the Harvard Medical Practice Study and a systematic review of autopsy studies conducted over four decades found that nearly 1 in 10 patients suffered a major ante-mortem diagnostic error; a figure that has fallen by only approximately 5% despite all of todays’ advanced imaging technology and increased testing utilization.
The factors that contribute to diagnostic error (adapted from the Ottawa M&M model) include patient factors such as a language barrier, clinician factors such as knowledge base, fatigue or emotional distress; cognitive biases (reviewed in Episode 11 with Chris Hicks and Doug Sinclair), teamwork failure, systems or process failure, and wider community issues such as access to care. A greater awareness of these factors that contribute to diagnostic error may improve the clinician’s diagnostic accuracy and improve patient outcomes.
Update 2015: NEJM article summarizing why diagnostic error can cause more harm now than ever before.
Five Strategies to master evidence-based diagnostic decision making
- Evidence Based Medicine incorporates patient values and clinical expertise – not only the evidence
For an explanation of the interaction of the 3 spheres of EBM see Episode 49: Walter Himmel on Evidence Based Medicine from the NYGH EMU Conference 2014
The intent of EBM is to…
- Make the ethical care of the patient its top priority
- Demand individualised evidence in a format that clinicians and patients can understand
- Use expert judgement rather than mechanical rule following
- Share decisions with patients through meaningful conversations
- Communicate risk whilst incorporating the patient’s values
- Apply these principles at the community level for evidence based public health
- Critically Appraise Diagnostic Studies
Critical Appraisal Checklist (adapted from Best Evidence in Emergency Medicine)
- Is the clinical problem well defined?
- Does the study population represent the target population (is there any spectrum bias)?
- Does the study population focus on ED patients or are they ICU patients, or admitted patients?
- Did the study recruit patients consecutively (was there a selection bias)?
- Was the diagnostic evaluation sufficiently comprehensive and applied equally to all patients (was there no verification bias)?
- Were all diagnostic criteria explicit, valid and reproducible?
- Was the reference standard appropriate?
- Was there good follow up?
- Was a likelihood ratio presented in the paper?
- Understand that Diagnostic Tests are NOT perfect
Himmel: “There is no absolute truth in medicine. Our job as diagnosticians is to estimate the truth.”
No test has 100% accuracy, so clinicians need to have a good understanding of the limitations of a particular test and how to apply the results to the patient.
The question we should ask is not ‘is the disease present or not’, but rather, ‘how does the test alter the probability of the diagnosis being present’
THE 4 PRINCIPLES OF DIAGNOSTIC DECISION ANALYSIS
(adapted from the landmark paper ‘Pathways through uncertainty’)
- In the diagnostic context, patients do not have disease, only a probability of disease.
- Diagnostic tests are merely revisions of probabilities.
- Test interpretation should precede test ordering.
- If the revisions in probabilities caused by a diagnostic test do not entail a change in subsequent management, use of the test should be reconsidered.
For how to apply likelihood ratios to physical exam maneuvers and and a deeper understanding of how physical exam testing effects diagnostic decision making Dr. Himmel recommends the book Evidence based Physical Diagnosis Expert Consult
- Use the concept of Test Threshold to guide work-ups
The Test Threshold is the probability of disease below which or above which there would be no further testing that would be considered necessary for that circumstance.
Tests do not give you a binary answer; rather, they change probabilities to a small or large degree until you reach a test threshold to trigger the clinician to act or not act.
- Understand that the Predictive Value of a test depends on the prevalence of the condition
Tests need to be considered in context. As the prevalence of a disease increases, so does the positive predictive value. The prevalence of some diseases varies greatly in different populations, so in order to come up with a predictive value of a test, you first need to know the prevalence of disease in your population.
For a deep dive podcast on smart testing go to David Newman’s SMART EM
For a practical review of Likelihood Ratios check out Boring EM
Risk Tolerance and Shared Decision Making in Diagnostic Decision Making
Some of the factors that play into the clinician’s and patient’s risk tolerance in a given clinical encounter include:
- The culture of the particular ED – does your department value speed or top-notch patient care?
- Years of practice – risk tolerance of the clinician typically increases after the first few years of practice and then decrease again after a few bad outcomes
- Medico-legal environment
- Financial incentives and hospital administration incentives to order tests
- Society’s tolerance for risk in general (which appears to be decreasing each year) and your local population’s tolerance for risk
- The particular disease –society has a very low risk tolerance for missing an MI compared to strep throat for example
- Whether it’s the beginning or end of your shift
- A recent bad outcome
Shared Decision Making requires taking into consideration the patient’s risk tolerance. It is important to realize that, in general, patients underestimate the risk of investigations and interventions and overestimate the benefit of action.
Adjusting Risk Tolerance in Critical Situations
Pitfall: With society’s and clinicians’ risk tolerance generally decreasing over the last decade, it becomes more difficult for the clinician to adjust from tolerating very little risk in working up a patient for an MI for example, to having to increase their risk tolerance and perform a life-saving action which may appear risky, in critically ill patients. One needs to incorporate situational awareness as well as resource management, and be willing to act in critical situations.
Go to Dr. Hick’s ‘Best Case Ever’ on ‘Taking Action in EM’ for a deeper understanding of the need for adjusting risk tolerance in critical situations.
For an analysis by George Kovacs of the book ‘Antifragile’ and it’s application to adjusting risk tolerance in critical care in EM go to this guest post on EMCrit.
Strategies to prevent over-testing
Hicks: “It is a false premise that more investigations necessarily means more diagnostic certainty.”
Choosing Wisely Canada is a campaign to help physicians and patients engage in conversations about unnecessary tests, treatments and procedures.
In a survey of physicians they found that nearly three quarters of them believe that the frequency of unnecessary tests and procedures in the health care system is a “very” or “somewhat” serious problem. The top reasons physicians say they order unnecessary tests and procedures are concern about malpractice issues, “just to be safe” and wanting more information for reassurance.
Other influences reported were patient’s insistence on getting the test, wanting to keep patients happy, not having enough time with patients, and having access to new technology in their practice. Simply understanding how these factors influence the way we make diagnostic decisions may help to curb over-testing.
Potential Solutions to Over-investigating
- Malpractice reform
- Evidence based recommendations
- Spend more time talking to your patients
- Changing the system of financial rewards for ordering tests
In part 2 of this series on Diagnostic Decision Making we will delve into the spheres of Cognitive Bias that can lead to diagnostic error, present Cognitive De-biasing Strategies, tips on avoiding diagnostic error, the concept of preferred error and much more.
For an introduction to Cognitive Decision Making and medical error based on the work by Pat Croskerry see Episode 11 with Chris Hicks & Doug Sinclair
Quote of the Month
Life is short
And the art long
The occasion instant
1. Troyen A. et al. Incidence of adverse events and negligence in hospitalized patients. NEJM. 1991;325;6;370-376. Full PDF
2. Calder, L. et al. Enhancing the quality of morbidity and mortality rounds: the Ottawa M&M model. Acad Emerg Med. 2014 Mar;21(3):314-21. Abstract
3. Diagnostic testing revisited. Pathways through uncertainty. Schechter, M. And Sheps, S. CMAJ, Vol. 132, Apr 1, 1985. Full PDF
4. Best Evidence in Emergency Medicine Critical Appraisal Tool for Diagnostic Tests. McMaster University. Full PDF