In a recent review from the New England Journal of Medicine, Stuart Pocock and Gregg Stone take a close look at the evaluation of “positive” clinical trials, providing readers with a strategy to effectively interpret the results from such trials.

The authors recognise that in-depth examination of trial data is necessary to determine whether the findings provide sufficient evidence to change clinical practice. They propose that answering a set of key questions may assist in this process:

1. Does a P value of <0.05 provide strong enough evidence?
A significance level of 5% for the primary efficacy outcome is the minimum requirement for a trial to be declared “positive”. However, achieving this level of significance may not always represent sufficiently strong enough evidence of efficacy and should prompt deeper inspection into the secondary outcomes and how the study was conducted.

2. What is the magnitude of treatment benefit?
The treatment difference should be large when both the relative and absolute treatment effects, as well as the extent of uncertainty as indicated by the 95% confidence interval, are examined.

3. Is the primary outcome clinically important (and internally consistent)?
For some diseases, a surrogate primary outcome measure is used to indicate a clinical outcome. Even if this outcome is positive, an analogous effect on important clinical measures, such as mortality, may not be evident. Further to this, positive composite primary outcomes require careful inspection to determine which elements are driving the result.

4. Are secondary outcomes supportive?
Positive, pre-specified secondary outcomes can improve confidence in the overall “positivity” of a trial, while negative secondary outcomes can cast doubt on the primary outcome result.

5. Are the principal findings consistent across important subgroups?
Relative and absolute treatment effects may vary according to patient characteristics. Importantly, there may be subgroups of patients in a “positive” trial that do no benefit from the new treatment and will need protecting from an ineffective (or harmful) treatment.

6. Do concerns about safety counterbalance positive efficacy?
Whether there are any safety concerns that may offset efficacy benefits should be assessed; absolute benefits and risks should be presented in terms of differences in percentages and numbers needed to treat analyses may help to assess clinical benefit.

7. Is the efficacy–safety balance patient-specific?
Statistical and modelling techniques may be required to assess the trade-off between efficacy and safety for different patient populations.

8. Is the trial large enough to be convincing?
Small trials lack statistical power, warranting cautious interpretation.

9. Was the trial stopped early?
An interim estimate of treatment effect may be high as a result of random variation throughout the trial, relative to the true effect. Stopping a trial early may therefore exaggerate treatment efficacy and also reduce data collection for important secondary and safety outcomes.

10. Are there flaws in trial design and conduct?
Biases in the design and conduct of the trial, for example a lack of blinding, may negate any benefit substantiated by a highly significant positive primary outcome.

11. Do the findings apply to my patients?
The eligibility criteria of a trial should be scrutinised to check whether the findings can be generalised to other patients. Results from studies conducted at single centres should be interpreted with caution as centre-specific effects and geographical location may also affect generalisability. Moreover, concurrent advances in care may reduce the relevance of the findings to contemporary clinical practice.

Pocock and Stone provide examples from published clinical trials in cardiovascular disease to illustrate each of the above points, which can be easily applied to other therapy areas.

——————————————————–

Summary by Louise Niven, DPhil from Aspire Scientific.