Selective Participation in Clinical Trials Biases Estimates of Side Effects

Clinical trials use randomized controlled experiments to evaluate the safety and efficacy of new drugs. However, trial participants are not randomly selected: they are recruited from selected trial sites, with physicians exercising discretion regarding enrollment offers and patients choosing whether to participate. As a result, clinical trial participants may not be representative of the intended population of patients. 

In Trials Avoid High Risk Patients and Underestimate Drug Harms (NBER Working Paper 34534), researchers Jason Abaluck, Leila Agha, and Sachin Shah demonstrate that patients who are at higher risk of serious side effects are less represented in cancer drug trials and that this selective participation can bias estimates of side effects. 

The estimated risk of a serious drug-induced adverse event after treatment initiation is 15 percent higher for the full population of eligible patients than for trial participants.

Using data from the Surveillance, Epidemiology, and End Results (SEER) Program linked to Medicare claims, the researchers identify a cohort of Medicare beneficiaries who initiate treatment with cancer drugs within 6 months of a diagnosis. They use variation across patients in the exact timing of treatment initiation to show that cancer medications, on average, increase the risk of hospitalization due to infection or organ injury by 2.4 percentage points per month (250 percent) in the three months following treatment initiation.

This hospitalization risk varies predictably with patient characteristics. The researchers use each patient’s comorbidities, frailty, age, and gender to construct a prediction of their likelihood of hospitalization for these drug-induced adverse events. The two most important predictors of risk are chronic kidney disease and an index of frailty. Patients at the 90th percentile of this predicted risk distribution face a 3.6 percentage point per month increase in the risk of serious adverse events following treatment initiation, compared to a 1.4 percentage point increase for patients at the 10th percentile. 

Comparing patients who participated in 112 distinct clinical trials to nonparticipants who would have been eligible for the treatments under consideration, the researchers find a negative correlation between a patient’s risk of serious side effects and their clinical trial participation. Patients at the 90th percentile of the risk distribution have only a 0.2 percent rate of clinical trial participation, compared to a 0.8 percent rate for patients at the 10th percentile. On average, trial participants are three years younger than nonparticipants and are less likely to have each of 20 measured comorbidities.

This disparity in risk between clinical trial participants and nonparticipants implies that the rate of side effects observed in clinical trials may not generalize to the full population of patients. The estimated risk of a serious drug-induced adverse event after treatment initiation is 15 percent higher for the full population of eligible patients than for trial participants. This difference corresponds to one additional hospitalization for every 25 patients per year of treatment. This estimate is likely to understate the full gap in adverse events between trial enrollees and treatment-eligible patients because the analysis only covers patients aged 65 and over. The findings suggest that selective participation in clinical trials could contribute to observed gaps between efficacy in clinical trials and in clinical practice.

- Robin McKnight

The researchers acknowledge support from National Institute on Aging grant #K76AG074919.