Real-World Evidence can only Replicate 15% of Randomized Controlled Trials: So What?

Real world-evidence (RWE) based research has been the buzz word in medical research circles. Some may assume that RWEs should or could ultimately replace randomized controlled trials (RCT), the gold standard of research, as RCTs are lengthy, expensive, and often challenging to secure sufficient numbers of qualified investigators and patients. The infusion of RWE into clinical research mostly represents an opportunistic move to better understand the performance and behavior of drugs in broader patient groups, not to mention as a tool to accelerate RCTs, making RWE and RCT more complementary than competitive. One recent study revealed that RWE could only replicate 15% of RCTs. The research question was premised on the faulty premise that society wants one over the other.

A recent study published in the JAMA Network Open revealed that real-world data (RWD) sources in all reality could only be used to replicate 15% of clinical trials feasibly. Dr. Andrew Norden recently covered the study for MedCity News

What is RWE?

RWE requires real-world data (RWD), which is obtained from observational data outside of the context of RCT and generated during routine clinical practice. In the U.S., the 21 Century Cures Act helped accelerate an ongoing provider transformation from paper to digital records. With pervasive cloud technologies, big data and AI algorithms, the primary ingredient—observational data—was in place. This data could be accessed from a multiplicity of the source from a health systems’ electronic health record to a government payer population health system, from state payer claims databases to accumulating amounts of patient-generated data from smartphones and smart health devices. Now observational studies, whether retrospective or prospective to observational registries, could be tapped into and leveraged for in-depth, vibrant, and extensive analyses. For example, researchers could now study the parameters of an RCT trial and seek to replicate the results from RWE. Moreover, RCTs are limited, and RWE can be used when clinical trials cannot indeed account for the entire patient population of a particular disease.

The Study Findings

The study’s authors, originating from a number of prominent academic medical centers, determined that based on a cross-sectional survey of 220 clinical trials published in “high-impact journals” in 2017 that only 15% of these studies could be replicated in any feasible way using available real-world data (RWD) sources.

The implication for these findings is clear if this is, in fact, true: RWE-based studies don’t obviate the need for traditional (and costly) RCT.

A Small Number but so What

As Dr. Norden explains in his MedCity News piece, the findings could be deemed a disappointment given the tremendous excitement and promise for RWE, but he reminds all—at the end of the day–the end- goal of leveraging RWE isn’t to replace clinical trials but rather boost our overall understanding of health in a number of critical areas. For example, we can use RWE to improve our knowledge of drug performance in broader patient groups; we can leverage RWE to identify expanded uses for existing approved drugs or consider RWE as an augmentation to accelerate clinical trials by utilizing RWD-based external control groups.

Academic Research vs. Real World

Norden acknowledges that based on the specific research question, the answer of 15% made sense. But the research question is based on a faulty premise: that is, RWE exists to replace RCT—it doesn’t. Instead, RWE offers society a more significant opportunity to bolster and improve our knowledge by not only complementing RCT but also extending, supporting, clarifying, or accelerating RCTs. After all, RCTs have limitations, and in many cases, RCTs may not even be feasible. Enter RWE as a complementary force for medical research.

Lead Research/Contact

Dr. Andrew Norden