Evidence Based Practice

Results of the Mingrone Study

Experimental Event Rate (EER) = 15 / 20 = 75%
outcome present / total in experimental group

Control Event Rate (CER) = 2 / 20 = 10%
outcome present / total in control group

Absolute Benefit Increase (ABI) = 75% - 10% = 65%
arithmetic difference between the rates of events in the experimental and control group. An Absolute Benefit Increase (ABI) refers to the increase of a good event as a result of the intervention. An Absolute Risk Reduction (ARR) refers to the decrease of a bed event as the result of the intervention. [ARR = EER-CER]

Relative Risk  (RR) = .75 / .10 = 7.5
ratio of the risk in the experimental group compared to the risk in the control group.proportional reduction in risk between the rates of events in the control group and the experimental group. [RR = EER/CER]

Relative Benefit Increase (RBI) = 65% / 10% = 650%
proportional increase in benefit between the rates of events in the control group and the experimental group. [RBI = EER - CER / CER]

Numbers Needed to Treat (NNT) = 1 / .65 = 2
number of patients who need to be treated to prevent one bad outcome or produce one good outcome. In other words, it is the number of patients that a clinician would have to treat with the experimental treatment compared to the control treatment to achieve one additional patient with a favorable outcome. [NNT = 1/ARR]

Clinical vs. Statistical Significance 

Statistical significance indicates a result is unlikely to be due to chance, while clinical significance determines if that result has a meaningful impact on patient care. A finding can be statistically significant but not clinically significant if the effect is too small to matter in practice

"To be clinically important requires a substantial change in an outcome that matters. Statistically significant changes, however, can be observed with trivial outcomes. And because statistical significance is powerfully influenced by the number of observations, statistically significant changes can be observed with trivial (small) changes in important outcomes. Large studies can be significant without being clinically important and small studies may be important without being significant." (Effective Clinical Practice, July/August 2001, ACP)

Clinical significance has little to do with statistics and is a matter of judgment. Clinical significance often depends on the magnitude of the effect being studied. It answers the question "Is the difference between groups large enough to be worth achieving?" Studies can be statistically significant yet clinically insignificant.

For example, a large study might find that a new antihypertensive drug lowered BP, on average, 1 mm Hg more than conventional treatments. The results were statistically significant with a P Value of less than .05 because the study was large enough to detect a very small difference. However, most clinicians would not find the 1 mm Hg difference in blood pressure large enough to justify changing to a new drug. This would be a case where the results were statistically significant (p value less than .05) but clinically insignificant.

Source: Guyatt, G. Rennie, D. Meade, MO, Cook, DJ. Users' Guide to Medical Literature: A Manual for Evidence-Based Clinical Practice, 2nd Edition, 2008.

Knowledge Check
Read the information on this page then answer the following questions.

Question 1) If no patients in the medical-therapy group achieved remission, but remission rates were 75% and 95% for the two surgical groups respectively, why might the researchers choose to assume remission in two medical-therapy patients who dropped out when calculating risk ratios?

A) To create a more conservative estimate of the surgical benefit.
B) To increase the observed difference between groups.
C) To make the statistical test more favorable toward surgery.

Question 2) A new diabetes intervention shows an Experimental Event Rate (EER) of 75% and a Control Event Rate (CER) of 10%. Which of the following best interprets the Absolute Benefit Increase (ABI) of 65% in clinical terms?

A) For every 100 patients treated, 65 more will achieve remission compared to the control group.
B) The treatment increases the likelihood of remission by 65% compared to control.
C) Only 10% of patients in the control group benefit from treatment.

Question 3) If the Numbers Needed to Treat (NNT) for the experimental therapy is 2, what does this imply about the treatment’s effectiveness?

A) The treatment prevents remission in 50% of patients compared with control.
B) Two patients must receive the treatment for one additional patient to benefit compared with control.
C) Only one in every two treated patients will experience remission overall.