Key concepts in assessing drug efficacy trials
Risks of bias and industry sponsorship in drug trials
Industry sponsored clinical trials are more likely to produce positive results and positive conclusions compared to trials with other forms of sponsorship . Authors with financial ties to companies that make the drug vs. authors without ties portends an odds ratio of almost 3 for a positive study . It is important to assess whether there are any conflicts of interest of authors for any drug trial that you read as there are subtle ways in which conflicts of interest can bias a trial.
Consider using a checklist appraisal tool when reading drug trials so that subtle bias is more easily recognized.
Harms are often under-represented in trials
A systematic review compared the number of adverse events in matched published and unpublished studies. Relying only on published studies would have missed between 43% and 100% of adverse events . A study of 6 general medical journals showed that of all the published RCTs in those journals there was no information on severe adverse events in 27% of trials and no information on withdrawal of patients due to an adverse event in 47% . Some of the reasons why harms are often under-represented in trials include the huge cost of a large enough trial to detect harms, that many harms occur years later – long after the trial is completed, some trials include a ‘run-in’ period where all patients receive the drug and patients with major side effects are excluded from the subsequent randomized trial, and some serious harms are rare enough that the trial will not be powered to detect the harm. An important concept is that while benefit is relative depending on the population studied, harms are static across populations. It is thus important to consider whether or not the population in the trial is similar to the population of patients that you are treating.
When reading trials, remember that harms are often under-represented.
Lack of replication of study results
Perhaps because of the medical communities desire to find drugs that work, we often become overly optimistic about the positive results of a trial and don’t see a need to replicate the study results. Conversely, when multiple trials show negative results and subsequently one trial shows positive results, we tend to believe the positive results at the exclusion of the other trials.
Misunderstanding of p values
The p value of a trial does not tell us whether the results are true or false . Rather, the p value is a measure to identify data worthy of a second look. It is a statistical test to assess how unlikely the results of a trial are, if you assume a null hypothesis. Small differences between a drug arm and placebo arm may show statistical significance, but may not show a clinically significant difference.
As outlined in the landmark paper “Why Most Published Research Findings Are False”, the following are factors that make drug trials less likely to be true .
- The smaller the studies conducted
- The smaller the effect sizes
- The greater the number and the lesser the selection of tested relationships
- The greater the flexibility in designs, definitions, outcomes, and analytical modes
- The greater the financial and other interests and prejudices
- The hotter a topic (with more scientific teams involved)
Gabapentinoids for low back and radicular pain
In a 2018 CMAJ meta-analysis of 9 trials that compared gabapentinoids (topiramate, gabapentin or pregabalin) to placebo, these drugs were not effective at reducing pain or disability in low back pain or lumbar radicular pain at 1-12 weeks, or for lumbar radicular pain in the immediate term . Importantly (and not surprisingly) there was an increased risk of adverse events from use of gabapentinoids, based on high level evidence.
For neuropathic pain, gabapentin reduces pain scores by < 1 point on a 0-10 point scale and benefits about 15% of carefully selected patients with post-herpetic neuralgia and diabetic neuropathy (NNT=6-8). A similar proportion of people suffer harm (NNH=8) . Gabapentinoids have also been reported to be drugs of abuse . It is important to recognize that any treatment effect is similar in low doses and high doses. If using gabapentinoids use only the lowest dose and instruct patients to stop the medication if there is no effect in 3 days.
Take Home: Gabapentinoids are not recommended for routine use in ED patients with low back pain. Reserve them for patients with post-herpetic neuralgia and diabetic neuropathy using the lowest dose with cessation at 3 days if no effect.
Update 2019: A population based cohort study of 191973 patients showed gabapentinoids were associated with increased risk of suicidal behaviour, unintentional overdose, head/body injuries and road traffic incidents/offences. These observations were more often seen in pregabalin than in gabapentin. Abstract
NSAIDs and acetaminophen for low back pain
A Cochrane review in 2008 of 65 trials (total number of patients = 11,237) showed that NSAIDs and acetaminophen are similarly more effective than placebo at reducing short-term pain scores in acute and chronic mechanical low back pain patients, but acetaminophen had fewer side effects . All NSAIDs had similar effect. The selective COX-2 inhibitors showed fewer side effects compared to traditional NSAIDs, but other reviews have shown that COX-2 inhibitors are associated with increased cardiovascular risks in specific patient populations.
The PRECISION trial in 2016 compared ibuprorofen, naproxen and celocoxib taken daily for an average of > 20 months in patients with osteoarthritis and rheumatoid arthritis and found no difference in the rate of major GI bleeds (0.7% in all groups) . The rate of iron deficiency anemia thought to be from a GI source was 0.4% in the celocoxib group compared to 0.9% in the Naproxen group, however the rate of MI was higher in the celocoxib group.
Observational data suggests that naproxen has the lowest cardiovascular side effects of the NSAIDs, and ibuprofen, the lowest GI side effects. Remember too that all NSAIDs have a dose ceiling beyond which there is no improved efficacy, but there are increased side effects (e.g. ibuprofen 400mg po, ketorolac 10mg IV or IM, naproxen 375mg po).
Take Home: Acetaminophen and NSAIDs are equally as effective for mechanical low back pain with NSAIDs having more side effects. All NSAIDs are equally effective, with ibuprofen having the best GI side effect profile and naproxen, the best cardiovascular side effect profile.
Topical NSAIDs for acute strains and sprains
A Cochrane review showed that in patients with acute strains and sprains, topical NSAID gel such as diclofenac Emulgel, ketoprofen gel, piroxicam gel, have an NNT between 1.8 and 4.4 for a primary outcome of at least 50% pain relief at 7 days . Adverse event rates with topical NSAIDs (4.3%) were no greater than with topical placebo (4.6%) based on high quality evidence.
Take Home: Topical NSAIDs gels used for one week improve pain for patients with acute strains and sprains
Cyclobenzaprine for neck and back muscle strain/spasm
In a review of 46 RCTs of cyclobenzaprine for muscle spasm/pain of the neck or back prescribed for 1-2 weeks, there was a moderate improvement of pain and function with a NNT = 4-7 . However, this is likely an overestimate as a result of bias due to possible loss of blinding. The majority of trials used subjective, unvalidated, physician-rated assessments of pain and function. All trials were funded by the manufacturers of cyclobenzaprine. More importantly, this possible treatment effect needs to be balanced with a number needed to harm (NNH) = 4-5, mostly drowsiness and dizziness, which precludes patients from keeping active and getting back to work (two goals of treatment in patients with muscle strain/spasm). This likely underestimates real world harms because individuals at higher risk of experiencing adverse effects were excluded from RCTs. Anti- muscarinic effects, such as impaired visual accommodation, increased dental caries or gum disease, impaired bladder emptying, or constipation are less likely to have been captured in these trials. Our experts do not recommend the use of cyclobenzaprine for ED patients. If using the drug, start with the lowest dose of 5mg qhs and titrate to no more than 15mg for no longer than 7 days.
Take Home: The trial-based analgesic benefits of cyclobenzaprine are about equal to the harms, with real world harms likely outweighing the benefits. Our experts do not recommend the use of cyclobenzaprine for ED patients. If using the drug, start with the lowest dose of 5mg qhs and titrate to no more than 15mg for no longer than 7 days.
Caffeine as adjuvant analgesic
A Cochrane review of studies using acetaminophen or ibuprofen combined with 100 mg to 130 mg caffeine (equivalent to approximately one large brewed coffee) for dental pain, postpartum pain or headache, found a small but statistically significant benefit with caffeine which was not dependent on the pain condition or type of analgesic . About 5% to 10% additional patients achieved at least 50% reduction in pain over four to six hours with the addition of caffeine, with a NNT = 14 based on high quality evidence. The practicality and likely side effects (insomnia, jitteriness, anxiety) of drinking a large coffee every 6 hours along with acetaminophen or ibuprofen precludes the routine use of suggesting caffeine as an adjuvant analgesic.
Take Home: Caffeine (1 large coffee) is an effective adjunct analgesic with acetaminophen or NSAIDs, however the side effects may preclude their real world use.
Is tramadol better tolerated than morphine? Does it have less addiction potential? Is tramadol more effective than acetaminophen or NSAIDs?
Tramadol has been shown to be no more effective than NSAIDs or acetaminophen for post-op [16,17] and chronic pain .
Tramadol is an opioid that may have comparable, or even more addiction potential compared to other opioids. In one study, 14% of the patients prescribed tramadol continued to use opioids at one year vs 5-9% if the initial prescription was for another short acting opioid. . In another study of 445,000 post-op patients found that receiving a prescription for tramadol, compared with hydrocodone or oxycodone, was associated with significantly higher risk for additional opioid prescriptions at 6 months . Similar to codeine, the potency of tramadol is strongly influenced by one of the cytochrome P450 enzymes, which varies widely from person to person. Respiratory depression can occur in ultra-rapid metabolizers. This may explain reports of overdosing and underdosing after standard dosing of both codeine and tramadol. Tramadol additionally has SNRI-like effects which may result in serotonin syndrome, hypoglycemia, hyponatremia and seizures. Compared to morphine, the efficacy of tramadol varies more between patients. Our experts recommend against the use of tramadol.
Take Home: Tramadol is no more effective than acetaminophen or NSAIDs, has highly unpredictable analgesic effect, and has been shown to carry a higher addiction potential compared to other short acting opioids. It is not recommended by our experts.
Steroids reduce pain in patients with pharyngitis
Based on a Cochrane review  one dose of dexamethasone up to 10mg in adults with pharyngitis significantly reduces pain and improves symptom resolution. In addition to any effect of antibiotics and analgesia, corticosteroids increased the likelihood of complete resolution of pain at 24 hours by more than three times and at 48 hours by 1.7 times. This meta-analysis found no significant harms, however all the RCTs were small, so by design, harms were likely underestimated. Our experts recommend reserving dexamethasone in pharyngitis for patients who have failed acetaminophen/NSAIDs and have severe pain.
Take Home: A single dose of dexamethasone (up to 10mg) reduces pain and improves symptom resolution for patients with pharyngitis. Because of potential adverse events not represented in the trials, consider it only in those who have failed acetaminophen and/or NSAIDs and who are experiencing severe pain, rather than routinely for pharyngitis.
Calcium channel blocker ointment and flavanoids for hemorrhoids and anal fissures
There is no evidence supporting he use of steroid-based or lidocaine-based ointments such as Anusol or Preparation H in patients with painful hemorrhoids or anal fissures. According to a 2017 practice guideline for management of anal fissures  and a Cochrane review , nitrate and calcium channel blocker ointment have similar efficacy in healing rate and pain, with calcium channel blockers resulting in less side effects. Similar efficacy has been shown with painful hemorrhoids when comparing nifedipine 0.3% + lidocaine 1.5% vs placebo + lidocaine 1.5%, with complete relief in 86% vs 50% at 7 days, and complete resolution of hemorrhoids 92% vs 46% at 2 weeks .
Our experts recommend diltiazem 2% or nifedipine 0.3% ointment for both painful hemorrhoids and anal fissures as a bridge to surgical therapy.
Fiber supplementation with ispaghula husk, psyllium, sterculia or unprocessed bran has been shown to decrease bleeding and hemorrhoid recurrence with a relative risk of 0.47 but has no significant effect on prolapse, pain and itch .
A meta-analysis of 14 RCTs comparing flavonoids (diosmin, micronized purified flavonoid fraction and rutosides) with placebo or no therapy in patients with symptomatic hemorrhoids showed a beneficial effect on bleeding, itch and recurrence (RR = 0.53) .
Take Home: Diltiazem 2% or nifedipine 0.3% ointment are effective analgesics and improve healing for both painful hemorrhoids and anal fissures as a bridge to surgical therapy. Over the counter preparations such as Anusol and Preparation H are not efficacious.
Buscopan for abdominal pain or renal colic
When compared to NSAIDs or acetaminophen, Buscopan has no added benefit for adult patients with abdominal pain or renal colic [27,28].
A 2019 RCT compared Buscopan 10 mg plus placebo to oral acetaminophen 15 mg/kg plus placebo in 225 children 8-17 years presenting to a single center ED with colicky abdominal pain presumed to be functional . At 80 minutes, the mean pain scores in the acetaminophen and Buscopan groups were no different and there were no significant differences in adverse effects, although this trial may have been underpowered to detect adverse events.
Take Home: Buscopan is no better than acetaminophen in adults and children with non-surgical abdominal pain.
Morphine or hydromorphone for acute pain?
When equal analgesic doses of opioids are used, efficacy for acute pain control are similar. When comparing morphine to hydromorphone, the same principle applies, however morphine is more often under-dosed in the ED. Hydromorphone 0.015 mg/kg IV is equivalent to morphine 0.1 mg/kg IV . While 1mg of hydromorphone is commonly used in the adult ED patient, less than 7mg of morphine is often used. In patients with renal failure, morphine’s opioid metabolites are excreted less readily essentially converting morphine into a longer acting opioid, whereas with hydromorphone, the neuro-excitatory metabolites are excreted less readily, increasing the risk of delirium, myoclonus and possibly hallucinations. According to the European Palliative Care guidelines , morphine and hydromorphone is still recommended in patients with renal failure, however it is recommended to lower the dose and increase the dosing intervals. Hydromorphone is more expensive than morphine in Ontario.
Take Home: Morphine and hydromorphone have similar analgesic efficacy at opioid equivalent doses, but hydromorphone is more expensive. Morphine tends to be under-dosed in the ED. The correct dose for acute severe pain is 0.1mg/kg. Beware of the different side effects of morphine vs hydromophone in patients with renal failure and modify dosing accordingly.
- Lundh A, Lexchin J, Mintzes B, Schroll JB, Bero L. Industry sponsorship and research outcome. Cochrane Database of Systematic Reviews 2017, Issue 2. Art. No.: MR000033.
- Rosa Ahn, Alexandra Woodbridge, Ann Abraham, Susan Saba, Deborah Korenstein, Erin Madden, W John Boscardin, Salomeh Keyhani. Financial ties of principal investigators and randomized controlled trial outcomes: cross sectional study. BMJ 2017;356:i6770.
- Golder S, Loke YK, Wright K, Norman G (2016) Reporting of Adverse Events in Published and Unpublished Studies of Health Care Interventions: A Systematic Review. PLoS Med 13(9): e1002127.
- Isabelle Pitrou, Isabelle Boutron, Nizar Ahmad, Philippe Ravaud. Reporting of Safety Results in Published Reports of Randomized Controlled Trials. Arch Intern Med 2009;169:1756-1761.
- Amrhein V, Greenland S, McShane B. Retire statistical significance. Nature 2019;567:305-307.
- John Ioannidis. Why most published research findings are false. PLoS Medicine 2005;2(8):e124.
- Enke O, New HA, New CH, et al. Anticonvulsants in the treatment of low back pain and lumbar radicular pain: a systematic review and meta-analysis. CMAJ. 2018;190(26):E786-E793.
- Wiffen PJ, McQuay HJ, Rees J, Moore RA. Gabapentin for acute and chronic pain. Cochrane Database of Systematic Reviews 2005, Issue 3. Art. No.: CD005452.
- Goodman CW, Brett AS. A clinical overview of off-label use of gabapentinoid drugs. JAMA Internal Medicine. Published on-line March 25, 2019.
- Wiffen PJ, McQuay HJ, Rees J, Moore RA. Gabapentin for acute and chronic pain. Cochrane Database of Systematic Reviews 2005, Issue 3. Art. No.: CD005452.Lundh A, Sismondo S, Lexchin J, et al. Industry sponsorship and research outcome. Cochrane Database of Systematic Reviews 2012, Issue 12. Art. No.: MR000033.
- Roelofs PD, Deyo RA, Koes BW, Scholten RJ, Van tulder MW. Non-steroidal anti-inflammatory drugs for low back pain. Cochrane Database Syst Rev. 2008;(1):CD000396.
- Nissen SE, et al. Cardiovascular safety of celecoxib, naproxen, or ibuprofen for arthritis. NEJM, 2016. 375(26):2519-2529.
- Derry S, Wiffen PJ, Kalso EA, et al. Topical analgesics for acute and chronic pain in adults – an overview of Cochrane Reviews. Cochrane Database Syst Rev. 2017;5:CD008609.
- Is cyclobenzaprine useful for pain? Therapeutics Letter No. 105, July 24, 2017.
- Derry CJ, Derry S, Moore RA. Caffeine as an analgesic adjuvant for acute pain in adults. Cochrane Database Syst Rev. 2014;(12):CD009281.
- Banerjee M, Bhaumik DJ, Ghosh AK. A comparative study of oral tramadol and ibuprofen in postoperative pain in operations of lower abdomen. J Indian Med Assoc. 2011;109(9):619-22, 626.
- Edwards JE, McQuay HJ, Moore RA. Combination analgesic efficacy: individual patient data meta-analysis of single-dose oral tramadol plus acetaminophen in acute postoperative pain. Journal of pain and symptom management. 2002; 23(2):121-30.
- Toupin April K, Bisaillon J, Welch V, Maxwell LJ, Jüni P, Rutjes AWS, Husni M, Vincent J, El Hindi T, Wells GA, Tugwell P. Tramadol for osteoarthritis. Cochrane Database of Systematic Reviews 2019, Issue 5. Art. No.: CD005522.
- Shah A, Hayes CJ, Martin BC. Characteristics of Initial Prescription Episodes and Likelihood of Long-Term Opioid Use – United States, 2006-2015. MMWR. Morbidity and mortality weekly report. 2017; 66(10):265-269.
- Thiels CA, Habermann EB, Hooten WM, Jeffery MM. Chronic use of tramadol after acute pain episode: cohort study. BMJ. 2019;365:l1849./fusion_toggle]
- Hayward G, Thompson MJ, Perera R, Glasziou PP, Del mar CB, Heneghan CJ. Corticosteroids as standalone or add-on treatment for sore throat. Cochrane Database Syst Rev. 2012;10:CD008268.
- Stewart DB, Gaertner W, Glasgow S, Migaly J, Feingold D, Steele SR. Clinical Practice Guideline for the Management of Anal Fissures. Dis Colon Rectum. 2017;60(1):7-14.
- Nelson RL, Thomas K, Morgan J, Jones A. Non surgical therapy for anal fissure. Cochrane Database Syst Rev. 2012;(2):CD003431.
- Perrotti P, Antropoli C, Molino D, De Stefano G, Antropoli M. Conservative treatment of acute thrombosed external hemorrhoids with topical nifedipine. Dis Colon Rectum. 2001 Mar. 44(3):405-9.
- Alonso-Coello P, Mills E,Heels-Ansdell D, et al. Fiber for the Treatment of Hemorrhoids Complications: a Systematic Review and Meta-Analysis Am J Gastroenterol.2006;101:181-8
- Alonso-Coello P, Zhou Q,Martinez-Zapata MJ, et al. Meta-analysis of flavonoids for the treatment of haemorrhoids. Br J Surg 2006;93:909-20.
- Mackway-Jones K. Buscopan (hyoscine butylbromide) in abdominal colic. Best Bets 15th May 2003.
- Mackway-Jones K. No evidence for the use of buscopan (hyoscine butylbromide) in renal colic. Best Bets July 31, 2003.
- Poonai, N.Hyoscine butylbromide (Buscopan) for abdominal pain in children: a randomized controlled trial. CJEM. Volume 21, Supplement S1, May 2019, p. S20.
- Felden L, Walter C, Harder S, et al. Comparative clinical effects of hydromorphone and morphine: a meta-analysis. Br J Anaesth. 2011;107(3):319-28.
- Caraceni A, Hanks G, Kaasa S, et al. Use of opioid analgesics in the treatment of cancer pain: evidence-based recommendations from the EAPC. Lancet Oncol. 2012;13(2):e58-68.
Drs. Helman, Lexchin and Morgenstern have no conflicts of interest to declare