Impact of Information Lost to Follow-Up in RCTs
Impact of Information Lost to Follow-Up in RCTs
Objective To assess the reporting, extent, and handling of loss to follow-up and its potential impact on the estimates of the effect of treatment in randomised controlled trials.
Design Systematic review. We calculated the percentage of trials for which the relative risk would no longer be significant under a number of assumptions about the outcomes of participants lost to follow-up.
Data sources Medline search of five top general medical journals, 2005-07.
Eligibility criteria Randomised controlled trials that reported a significant binary primary patient important outcome.
Results Of the 235 eligible reports identified, 31 (13%) did not report whether or not loss to follow-up occurred. In reports that did give the relevant information, the median percentage of participants lost to follow-up was 6% (interquartile range 2-14%). The method by which loss to follow-up was handled was unclear in 37 studies (19%); the most commonly used method was survival analysis (66, 35%). When we varied assumptions about loss to follow-up, results of 19% of trials were no longer significant if we assumed no participants lost to follow-up had the event of interest, 17% if we assumed that all participants lost to follow-up had the event, and 58% if we assumed a worst case scenario (all participants lost to follow-up in the treatment group and none of those in the control group had the event). Under more plausible assumptions, in which the incidence of events in those lost to follow-up relative to those followed-up is higher in the intervention than control group, results of 0% to 33% trials were no longer significant.
Conclusion Plausible assumptions regarding outcomes of patients lost to follow-up could change the interpretation of results of randomised controlled trials published in top medical journals.
Loss to follow-up in randomised controlled trials could bias results if the unavailability of data is associated with the likelihood of outcome events. For example, patients might fail to return for assessment because of deterioration in their medical condition, resulting in a higher frequency of adverse outcomes of interest associated with that condition. If the distribution of such patients differs between study arms, the prognostic balance created by randomisation will be disturbed. Although analysis of patients for whom outcome data are available in the groups to which they are randomised will avoid bias as a result of factors such as non-adherence, it will not protect against potential bias associated with loss to follow-up.
Although investigators strive to reduce the amount of missing data, in most instances they will fail to achieve complete follow-up. Indeed, 60-89% of randomised controlled trials have some missing outcome data. Interpretation of results is compromised when, as is often the case, investigators do not report strategies for handling such data. The most commonly reported strategy among trials that do report their approach is to restrict analyses to participants with full outcome information (complete case analysis).
Most previous analyses have not found significant associations between the extent of loss to follow-up and the magnitude of treatment effect. These findings suggest that substantial loss to follow-up sometimes leads to overestimates of treatment effects and sometimes to underestimates. Another approach to examining the possibility of bias associated with loss to follow-up in randomised controlled trials is to evaluate the potential impact on effect estimates of various assumptions regarding the outcomes of participants lost to follow-up. We examined this question in a sample of randomised controlled trials from five general medical journals with the highest impact factors. We also described the reporting, extent, and handling of loss to follow-up.
Abstract and Introduction
Abstract
Objective To assess the reporting, extent, and handling of loss to follow-up and its potential impact on the estimates of the effect of treatment in randomised controlled trials.
Design Systematic review. We calculated the percentage of trials for which the relative risk would no longer be significant under a number of assumptions about the outcomes of participants lost to follow-up.
Data sources Medline search of five top general medical journals, 2005-07.
Eligibility criteria Randomised controlled trials that reported a significant binary primary patient important outcome.
Results Of the 235 eligible reports identified, 31 (13%) did not report whether or not loss to follow-up occurred. In reports that did give the relevant information, the median percentage of participants lost to follow-up was 6% (interquartile range 2-14%). The method by which loss to follow-up was handled was unclear in 37 studies (19%); the most commonly used method was survival analysis (66, 35%). When we varied assumptions about loss to follow-up, results of 19% of trials were no longer significant if we assumed no participants lost to follow-up had the event of interest, 17% if we assumed that all participants lost to follow-up had the event, and 58% if we assumed a worst case scenario (all participants lost to follow-up in the treatment group and none of those in the control group had the event). Under more plausible assumptions, in which the incidence of events in those lost to follow-up relative to those followed-up is higher in the intervention than control group, results of 0% to 33% trials were no longer significant.
Conclusion Plausible assumptions regarding outcomes of patients lost to follow-up could change the interpretation of results of randomised controlled trials published in top medical journals.
Introduction
Loss to follow-up in randomised controlled trials could bias results if the unavailability of data is associated with the likelihood of outcome events. For example, patients might fail to return for assessment because of deterioration in their medical condition, resulting in a higher frequency of adverse outcomes of interest associated with that condition. If the distribution of such patients differs between study arms, the prognostic balance created by randomisation will be disturbed. Although analysis of patients for whom outcome data are available in the groups to which they are randomised will avoid bias as a result of factors such as non-adherence, it will not protect against potential bias associated with loss to follow-up.
Although investigators strive to reduce the amount of missing data, in most instances they will fail to achieve complete follow-up. Indeed, 60-89% of randomised controlled trials have some missing outcome data. Interpretation of results is compromised when, as is often the case, investigators do not report strategies for handling such data. The most commonly reported strategy among trials that do report their approach is to restrict analyses to participants with full outcome information (complete case analysis).
Most previous analyses have not found significant associations between the extent of loss to follow-up and the magnitude of treatment effect. These findings suggest that substantial loss to follow-up sometimes leads to overestimates of treatment effects and sometimes to underestimates. Another approach to examining the possibility of bias associated with loss to follow-up in randomised controlled trials is to evaluate the potential impact on effect estimates of various assumptions regarding the outcomes of participants lost to follow-up. We examined this question in a sample of randomised controlled trials from five general medical journals with the highest impact factors. We also described the reporting, extent, and handling of loss to follow-up.
Source...