Risk of Cardiovascular Serious Adverse Events With Varenicline
Risk of Cardiovascular Serious Adverse Events With Varenicline
This meta-analysis included all published randomised, placebo controlled trials of varenicline for tobacco cessation, examined events occurring during drug exposure or within 30 days of discontinuation, and analysed findings via four summary measures. None of these measures identified a significantly elevated risk of treatment emergent, cardiovascular serious adverse events with varenicline use, and the 95% confidence interval of the risk difference excluded an increase larger than 0.63%. In response to the FDA’s call for analysis of cardiovascular serious adverse events attributed to varenicline use, this meta-analysis of 22 independent trials and more than 9000 individuals had high power to detect a significant treatment effect and found negligible variation in the evidence across the trials.
Study participants tended to be chronic, heavy smokers, averaging more than one pack of cigarettes a day for more than two decades, placing them at elevated risk of serious adverse events related to cardiovascular disease. Most trials included individuals with current (two trials) or past (11 trials) cardiovascular disease. More than a third of studies did not observe a cardiovascular serious adverse event; among these trials, five of eight included participants with past cardiovascular disease.
In their meta-analysis of the safety of varenicline, Singh and colleagues reported a Peto odds ratio of 1.72 (95% confidence interval 1.09 to 2.71) and concluded that “The use of varenicline among tobacco users was associated with a 72% increased risk of serious adverse cardiovascular events.” They questioned the safety of this medication, and in subsequent press interviews called for withdrawal of varenicline from the market. The researchers’ analysis of cardiovascular serious adverse events at any time during the trial duration was, on average, twice the duration of study drug exposure and did not account for longer follow-up in the varenicline group than in the placebo group.
The discrepancy between the conclusions of our meta-analysis and those of Singh and colleagues’ study is explained not only by differing periods over which events were collected but also by the choice of statistics used to summarise the results, which affected the trials included in the meta-analyses. For direct comparison in our study, we analysed data from all 22 trials using the full study follow-up, because the observation period of interest and the calculated risk difference was 0.47% (95% confidence interval 0.04 to 0.91), far less inflammatory than the previous meta-analysis’s reported risk based on the Peto odds ratio, yet still biased in favour of finding an effect due to the differential inclusion of events related to disease and not just treatment. By contrast, our calculated risk difference of treatment emergent, cardiovascular serious events was 0.27% (-0.10 to 0.63).
When study participants are selected by outcome status (for example, case or control), an odds ratio must be used to summarise an association with exposure status. However, when participants are selected by exposure status (for example, active or placebo treatment), a risk difference or relative risk can be used. These statistics are more natural choices for randomised trials because they explicitly estimate and contrast effects of interest—namely, event rates in the active and placebo arms. Treatment effects based on relative risks always are as or less extreme than those based on odds ratios. The (absolute) risk difference has a further advantage because it can be calculated for trials in which zero events occur, whereas relative statistics cannot be calculated for these trials and therefore can bias summaries against the null hypothesis of “no effect.”
Furthermore, relative statistics are unitless, which hides the fact that a low response rate remains very low even when scaled up by a seemingly large effect; by contrast, the risk difference retains the units of the measurement scale, showing that a difference between low response rates is itself very small. Vandermeer and colleagues’ comprehensive reanalysis of findings from 1613 meta-analyses of safety data indicated that the Peto odds ratio statistic was particularly biased. We further demonstrate that, regardless of sample size allocation, when all events are in one study arm, Mantel-Haenszel odds ratios match relative risks well whereas Peto odds ratios are far more extreme. For clinical considerations and in the setting of rare events, the risk difference most clearly conveys the relevant effect.
Meta-analysis is an important analytical technique for synthesising treatment effects across trials for maximum power and is particularly useful for analysis of serious adverse events, which can occur with low frequency. Bias in methods, however, is a real concern. Our comparison of four summary statistics identified conditions under which the Peto odds ratio produced extreme estimates that did not reflect the underlying event rates, and identified cases in which it produced smaller estimates than it theoretically should have done. Our results accord with other reports that the Peto statistic can lead to incorrect conclusions. The consequence of inflated risk estimates, such as those from Singh and colleagues’ meta-analysis concerning the effect of varenicline on serious adverse events related to cardiovascular disease, can be unnecessary public alarm and real harm, since patients may discontinue their drug treatment out of fear of adverse effects and clinicians may recommend cessation treatments of reduced efficacy or discourage use of the drug treatment altogether.
Smoking is the leading preventable cause of death worldwide. Half of long term smokers die from their tobacco use, and smokers die from cardiovascular disease more than from any other cause. Varenicline is a first line treatment for quitting smoking, and quitting smoking is central to the prevention of cardiovascular disease. Our meta-analysis of treatment emergent, cardiovascular serious adverse events, with attention to bias and critical design issues, indicates that the risk of these events associated with varenicline use is small, and statistically and clinically insignificant.
Discussion
This meta-analysis included all published randomised, placebo controlled trials of varenicline for tobacco cessation, examined events occurring during drug exposure or within 30 days of discontinuation, and analysed findings via four summary measures. None of these measures identified a significantly elevated risk of treatment emergent, cardiovascular serious adverse events with varenicline use, and the 95% confidence interval of the risk difference excluded an increase larger than 0.63%. In response to the FDA’s call for analysis of cardiovascular serious adverse events attributed to varenicline use, this meta-analysis of 22 independent trials and more than 9000 individuals had high power to detect a significant treatment effect and found negligible variation in the evidence across the trials.
Study participants tended to be chronic, heavy smokers, averaging more than one pack of cigarettes a day for more than two decades, placing them at elevated risk of serious adverse events related to cardiovascular disease. Most trials included individuals with current (two trials) or past (11 trials) cardiovascular disease. More than a third of studies did not observe a cardiovascular serious adverse event; among these trials, five of eight included participants with past cardiovascular disease.
Comparison With Other Studies
In their meta-analysis of the safety of varenicline, Singh and colleagues reported a Peto odds ratio of 1.72 (95% confidence interval 1.09 to 2.71) and concluded that “The use of varenicline among tobacco users was associated with a 72% increased risk of serious adverse cardiovascular events.” They questioned the safety of this medication, and in subsequent press interviews called for withdrawal of varenicline from the market. The researchers’ analysis of cardiovascular serious adverse events at any time during the trial duration was, on average, twice the duration of study drug exposure and did not account for longer follow-up in the varenicline group than in the placebo group.
The discrepancy between the conclusions of our meta-analysis and those of Singh and colleagues’ study is explained not only by differing periods over which events were collected but also by the choice of statistics used to summarise the results, which affected the trials included in the meta-analyses. For direct comparison in our study, we analysed data from all 22 trials using the full study follow-up, because the observation period of interest and the calculated risk difference was 0.47% (95% confidence interval 0.04 to 0.91), far less inflammatory than the previous meta-analysis’s reported risk based on the Peto odds ratio, yet still biased in favour of finding an effect due to the differential inclusion of events related to disease and not just treatment. By contrast, our calculated risk difference of treatment emergent, cardiovascular serious events was 0.27% (-0.10 to 0.63).
Choice of Summary Statistics
When study participants are selected by outcome status (for example, case or control), an odds ratio must be used to summarise an association with exposure status. However, when participants are selected by exposure status (for example, active or placebo treatment), a risk difference or relative risk can be used. These statistics are more natural choices for randomised trials because they explicitly estimate and contrast effects of interest—namely, event rates in the active and placebo arms. Treatment effects based on relative risks always are as or less extreme than those based on odds ratios. The (absolute) risk difference has a further advantage because it can be calculated for trials in which zero events occur, whereas relative statistics cannot be calculated for these trials and therefore can bias summaries against the null hypothesis of “no effect.”
Furthermore, relative statistics are unitless, which hides the fact that a low response rate remains very low even when scaled up by a seemingly large effect; by contrast, the risk difference retains the units of the measurement scale, showing that a difference between low response rates is itself very small. Vandermeer and colleagues’ comprehensive reanalysis of findings from 1613 meta-analyses of safety data indicated that the Peto odds ratio statistic was particularly biased. We further demonstrate that, regardless of sample size allocation, when all events are in one study arm, Mantel-Haenszel odds ratios match relative risks well whereas Peto odds ratios are far more extreme. For clinical considerations and in the setting of rare events, the risk difference most clearly conveys the relevant effect.
Conclusions and Clinical Implications
Meta-analysis is an important analytical technique for synthesising treatment effects across trials for maximum power and is particularly useful for analysis of serious adverse events, which can occur with low frequency. Bias in methods, however, is a real concern. Our comparison of four summary statistics identified conditions under which the Peto odds ratio produced extreme estimates that did not reflect the underlying event rates, and identified cases in which it produced smaller estimates than it theoretically should have done. Our results accord with other reports that the Peto statistic can lead to incorrect conclusions. The consequence of inflated risk estimates, such as those from Singh and colleagues’ meta-analysis concerning the effect of varenicline on serious adverse events related to cardiovascular disease, can be unnecessary public alarm and real harm, since patients may discontinue their drug treatment out of fear of adverse effects and clinicians may recommend cessation treatments of reduced efficacy or discourage use of the drug treatment altogether.
Smoking is the leading preventable cause of death worldwide. Half of long term smokers die from their tobacco use, and smokers die from cardiovascular disease more than from any other cause. Varenicline is a first line treatment for quitting smoking, and quitting smoking is central to the prevention of cardiovascular disease. Our meta-analysis of treatment emergent, cardiovascular serious adverse events, with attention to bias and critical design issues, indicates that the risk of these events associated with varenicline use is small, and statistically and clinically insignificant.
Source...