Randomized Controlled Trials (RCTs)
The sidebar
January 8, 2020
Photo via Canva
A randomized controlled trial (RCT) is a widely used method to measure the effectiveness of clinical and pharmaceutical treatments but less commonly used to evaluate health policies. It is the most robust and reliable way to determine if there is a causal link between an intervention, such as a drug or policy, and a desired effect, such as weight loss.
In the inaugural edition of our series, The Sidebar, co-hosts Dan Gorensten and Sayeh Nikpay dive a bit deeper into the RCT as a policy evaluation tool. How do these trials work and what are some of their tradeoffs?
Listen to the conversation below or scroll down for more information.
How Does It Work?
At its most basic level, an RCT is trying to answer a question: Does an intervention (a drug, a treatment, a policy, etc.) make a significant difference in achieving a particular outcome?
To do that, researchers randomly assign a group of people to two different groups: one that receives the intervention and one that doesn’t. The one that doesn’t receive the intervention is known as the “control” group, and it provides a baseline to which the effect of the intervention can be compared.
Want to See an RCT in Action?
Our episode “‘Hot Spotters’ on Trial” tells the story of how an RCT was used to evaluate one of the 21st century’s most highly touted health care initiatives.
Tradeoffs
Strengths
- Establishes causation, rather than just correlation, between an intervention and an observed effect
- Clearly demonstrates, via the control group, what would have happened in the absence of the intervention
- Rigorously randomizes participants, ensuring the impact observed cannot be attributed to other factors
- Well regarded by policymakers, funders, regulators, academic journals, and other influential decision makers
Limitations
- Can be very expensive and take a long time, especially if a large sample size is required to detect expected effects
- Offers little explanation as to why an intervention succeeded or failed, leaving the path forward to implement or iterate unclear
- Findings may not be easily generalized to other patient populations, care settings, etc.
- Not all interventions can be easily or ethically randomized, such as the effect of clean drinking water on people’s health
- Many ways for the trial to become biased, such as participants dropping out during the study
Health Policy RCTs You Should Know...
When: 1971-1986
What: Compared how different levels of cost sharing impacted health care spending, quality of care and health outcomes.
Why it matters: It demonstrated people use less care when they pay out of pocket and is often cited as evidence that high-deductible health plans and high coinsurance may hurt patients.
When: 2008-present
What: Studied impact of Medicaid expansion by comparing those selected via lottery to receive expansion and those not selected.
Why it matters: One of, if not the largest health care policy studies in U.S. history. Demonstrated numerous positive effects of Medicaid.
Additional Resources
Understanding and Misunderstanding Randomized Controlled Trials (Angus Deaton and Nancy Cartwright, National Bureau of Economic Research, 2017)
Evidence for Health Decision Making — Beyond Randomized, Controlled Trials (Thomas Frieden, New England Journal of Medicine, 2017)
In Defense of the Randomized Controlled Trial for Health Promotion Research (Laura Rosen, Orly Manor, Dan Engelhard and David Zucker; American Journal of Public Health, 2006)
Using Randomized Evaluations to Improve the Efficiency of US Healthcare Delivery (Amy Finkelstein and Sarah Taubman, J-PAL North America, 2015)
Mostly Harmless Econometrics, Chapter 2 (Joshua Angrist and Jorn-Steffen Pischke, 2008)