Concerned that reports of clinical trials can exaggerate a treatment’s benefits and downplay its risks, two research groups will sift through data from tests of drugs involving thousands of people with chronic pain or depression. The question for those reanalyzing the data is whether dozens of papers on the trials’ outcomes painted a complete picture, and what details may have gone unmentioned about the drugs’ effects.
“Bias and spin are incredibly common in the publication of clinical trials,” says Peter Doshi, a health services researcher at the University of Maryland School of Pharmacy in Baltimore and an editor at The BMJ. He and others have long been concerned about how trial results wend their way to doctors and patients, and whether both groups are fully informed when considering a specific medication. So in 2013 Doshi formed RIAT, which stands for Restoring Invisible and Abandoned Trials. It sought to encourage researchers to obtain unpublished clinical trial information, which would include deidentified patient-level data. With those data in hand, the researchers could reanalyze a trial’s results and publish what they found, which may or may not jibe with the original reports.
Who’s Funding the Research, and Why?
In 2017, RIAT received $1.4 million from the Laura and John Arnold Foundation (now called Arnold Ventures), and last year awarded its first grant for clinical trial reanalysis.
Why? It provided $150,000 to look at a U.S. government–funded trial of antidepressants in more than 300 teenagers. That trial’s reporting had been criticized in part because some arms of the study were unblinded and because it described outcomes that it didn’t originally set out to analyze—called post hoc analysis. Trial investigators reported that taking Prozac alone or combining it with cognitive-behavioral therapy eased depression, and the combination strategy reduced suicidality. “This trial is much more influential than any other antidepressant trial in kids,” says child psychiatrist Jon Jureidini at Women’s and Children’s Hospital in Adelaide, Australia, who is leading the reanalysis. Jureidini acknowledges that even if his reanalysis turns up something new, changing practice patterns may be tough. “But that has to be our target,” he says.
This week, RIAT is announcing its second and third grants, each for another $150,000; next year, it will award three more.
Recognizing Patterns of Bias
“It’s an enormous amount of work,” Doshi says. “We’re talking about doing something that hasn’t traditionally been done.” A paper published in 2014 in JAMA, led by epidemiologist John Ioannidis at Stanford University in Palo Alto, California, underscored how unusual reanalyses are: The authors could find only 37 examples of patient-level data reanalyzed across the medical literature. Only five were, like RIAT’s projects, performed by independent authors. Just over one-third of the 37 reanalyses led to new interpretations of the data.
“Reanalysis can do a lot of good,” says Arnold Monto, an infectious disease epidemiologist at the University of Michigan in Ann Arbor. But, he adds, those conducting reanalyses must be mindful of their own biases. “The danger there is you’re looking for something, you’re looking for something that is publishable,” he says.
Call to Action: Encouraging transparency across clinical research is a mission TrialSite News will absolutely stand behind. Sign up for our Daily Digest to receive more news as we work towards completely transparent coverage of the clinical trial industry.