Regression discontinuity design studies: a guide for health researchers
BMJ 2024; 384 doi: https://doi.org/10.1136/bmj-2022-072254 (Published 27 February 2024) Cite this as: BMJ 2024;384:e072254
- Sebastian Calonico, professor1,
- Neal Jawadekar, doctoral candidate2,
- Katrina Kezios, postdoctoral researcher2,
- Adina Zeki Al Hazzouri, professor2
- 1Department of Health Policy and Management, Mailman School of Public Health, Columbia University, New York, NY 10032, USA
- 2Department of Epidemiology, Mailman School of Public Health, Columbia University, New York, NY, USA
- Correspondence to: S Calonico sebastian.calonico{at}columbia.edu (or @scalonico on Twitter)
- Accepted 17 November 2023
Everyday medical practice is largely shaped by evidence from clinical research activities, such as understanding treatment effects of prescribed drugs, changes in clinical guidelines, or program evaluation of medical services. While such evidence would ideally come from randomized controlled trials, these trials are not always feasible and are often conducted under strict criteria that might differ from real world settings; therefore, clinicians and other medical researchers often must resort to observational studies for answering key research questions. However, associational observational analysis is prone to multiple sources of bias, which leaves many medical researchers with limited options for conducting defensible causal inference research.
Quasi-experimental methods, when used appropriately, can help to improve the validity of causal evidence generated from observational settings while also providing estimates potentially more applicable in real world settings than those from randomized controlled trials. One such method, regression discontinuity design (RDD), compares the outcomes of individuals who are just above and below a specific threshold for recommending treatment as a quasi-experiment to investigate causal relations using observational data. Regarded for its capacity to approximate experimental conditions in observational studies, RDD has gained popularity in recent years, particularly in the social sciences and, more recently, in the medical sciences.12345678910111213141516171819202122 While relying on some of the advantages of observational data such as long follow-up periods and larger and more diverse samples than are typically seen in randomized controlled trials, the widespread interest in this method can be explained by its simple structure, intuitive assumptions, and conceptual similarities to randomized experiments.
Figure 1 illustrates how quasi-experimental study designs such as RDD combine the advantages of both observational studies and randomized experiments. Despite those advantages, clinical training often emphasizes traditional study designs, especially randomized controlled trials, and clinical researchers probably are less familiar with and have less training in quasi-experimental designs. Furthermore, to our knowledge, no guided examples exist for how to conduct an RDD analysis and how to report RDD studies.
Premise of quasi-experimental study methods. These methods, such as randomized discontinuity designs, provide a bridge between observational and experimental study designs, combining the advantages of both types
This article builds on the RDD theory discussed in previous publications in The BMJ1123 and complements this work by focusing on implementation. We provide a detailed checklist and glossary on how to conduct an RDD analysis, walk through an RDD application, and briefly discuss the results of the application. We also include sample statistical codes, a list of software packages, and a dataset modified for illustrative purposes that can be used by clinicians to apply RDD in the context of a well defined research question. The purpose of this article is to help clinicians read, conduct, and interpret this method. To facilitate reading and understanding our discussions of RDD, we define common terms used in the glossary below (box 1). More comprehensive background with detailed descriptions of these terms can be found elsewhere.1123
Glossary of common terms used in studies with regression discontinuity designs
Bandwidth
Range determining the size of the neighborhood around the cut-off value where the empirical analysis is conducted.24
Bias
Systematic tendency that causes differences between the population parameter of interest (eg, average treatment effect) and the mean value of an estimator under repeated sampling.25
Continuity
Exchangeability is conceptualized as continuity of average (or some other feature of) potential outcomes at the cut-off value.2627
Cut-off value
Threshold value, such that everyone with a score value above it (or below it) is eligible for the treatment (control) group.26
Discontinuity (or jump)
Change in the probability of receiving treatment by crossing the cut-off value.24
Exchangeability
Assumption that the risk of the outcome within treatment group 1 would be the same as the risk of the outcome within treatment group 2, had those in group 1 received the same treatment assigned to those in group 2.2829
Local average treatment effect
Measure used to compare treatments or interventions. This effect measures the local difference (near the cut-off value) in mean outcomes between units assigned to the treatment and the same units assigned to the control.3031
Quasi-experiment
A real world study that mimics a randomized trial, whereby the probability of intervention (or some other relevant factor) is tied to a policy unassociated with individual characteristics that might influence risk of the outcome.323334
Score or running variable
Continuous variable used to determine treatment assignment, either completely or partially.26 Deviations from continuous running variables (ie, discrete) have been studied, but such scenarios require additional assumptions.35
Summary points
Regression discontinuity design (RDD) is a quasi-experimental method that can help to improve the validity of causal evidence generated from observational studies
RDD studies are less vulnerable to confounding than conventional observational study designs
Appropriately conducting RDD depends on a set of assumptions, with formal approaches for falsification, estimation, and inference
Although RDD can leverage clinical decision rules from everyday medical practice, clinical researchers are likely to be less familiar with and have less training in quasi-experimental designs such as RDD; furthermore, no guided RDD examples exist
With a focus on implementation, this article provides a checklist, glossary, and guided example on how to conduct and interpret an RDD study
Randomized discontinuity design provides causal inference without needing actual randomization
Suppose, for example, we wanted to investigate the treatment effect of antihypertensive drugs on cardiovascular disease and other related outcomes (eg, dementia). Randomized controlled trials for blood pressure treatment and cardiovascular disease3637 as well as cognitive impairment or dementia383940414243 have relatively short follow-up periods and are often conducted in highly selected patient populations. Therefore, one source of evidence to complement randomized controlled trial work would be non-experimental evidence from observational studies. Owing to difficulties around confounding, however, associational evidence can often be biased. To resolve this problem, investigators often attempt to either adjust for confounders using traditional regression methods or incorporate them in propensity scores,44454647 which can then be used in reweighting or matching techniques.484950 Even then, those methods might not provide perfect confounding balance across treated and untreated patients, because measuring all potential confounders or measuring them correctly is not always possible, leaving studies vulnerable to residual or unmeasured confounding.
With its conceptual underpinnings rooted in causal frameworks, RDD is a quasi-experimental, design based method to estimate causal effects by comparing the outcomes of individuals who have values of a continuous variable (known as the running variable) above and below a specific threshold meant to represent a treatment decision rule.2651 As such, RDD models can be described by three main components: a score or running variable, a cut-off value, and a recommended treatment. We illustrate these concepts in figure 2 using the example of the causal effect of blood pressure treatment on the incidence of cardiovascular disease. The estimation approach we propose in this article, which is standard in RDD, relies on local polynomials, a non-parametric estimation method that approximates the regression function at a particular point using nearby observations. Thus, there are no assumptions imposed regarding the characteristics of the outcome variable, and the method works for continuous, discrete, or any other type of variable.
Example of sharp regression discontinuity (RD) used to analyze the effect of antihypertensive treatment on cardiovascular disease incidence, with a systolic blood pressure cut-off value of 140 mm Hg. Sharp RD assumes perfect treatment uptake, and thus is equivalent to an intention-to-treat analysis. Only untreated individuals below the cut-off value (purple dots) and treated individuals above it (yellow dots) are observed. Dashed purple line represents the counterfactual for the intervention group had they not been treated (despite being above the cut-off). Regression discontinuity design is implemented by fitting two polynomial regressions of the outcome on the running variable above and below the cut-off value, using only observations within the bandwidth (heavy purple and yellow dots). The final local average treatment effect estimate is given by the vertical distance between the two estimated curves at the cut-off value (solid vertical arrow)
In certain contexts, a model that fits specific outcomes, such as binary outcomes or counts, might be preferable. These extensions are limited in the regression discontinuity literature; they are difficult to generalize owing to the non-parametric nature of local polynomials, and therefore we do not focus on them here. Meanwhile, the score, or running variable, is the systolic blood pressure reading or any other clinical instrument (eg, risk score of cardiovascular disease at 10 years) or biomarker that is used to indicate blood pressure treatment. The cut-off value is the threshold based rule for which treatment assignment is made by the provider; this value is often of clinical relevance (eg, systolic blood pressure value ≥140 mm Hg based on the blood pressure guidelines by the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure).52 Finally, the recommended treatment rule (eg, blood pressure treatment v no treatment) is determined by falling above or below the chosen cut-off value, which is associated with an increase or decrease in the likelihood of being prescribed antihypertensive drugs.
In brief, RDD leverages the discontinuity (or jump) in the probability of receiving treatment at the cut-off value to estimate causal effects (fig 2 shows the jump in data points at the 140 mm Hg cut-off value). This jump at the cut-off value is regarded as an exogenous variation in treatment status, which then enables the estimation of a causal effect. The estimated average treatment effect, here the effect of antihypertensive drugs on incidence of cardiovascular disease, is simply the vertical distance between the blue (treated group) and red lines (untreated group) at the cut-off value. This distance represents the risk difference of cardiovascular disease at the cut-off value, obtained by comparing individuals in a neighborhood above and below the cut-off value. Under perfect treatment compliance, this estimand is interpreted as an average treatment effect of antihypertensive drugs on the risk of cardiovascular disease at the cut-off value (also known as the local average treatment effect).
This RDD model is also known as sharp regression discontinuity, which assumes perfect treatment uptake, and thus is equivalent to an intention-to-treat analysis. When there is imperfect compliance with treatment assignment, a fuzzy regression discontinuity would be used, which is equivalent to a per protocol analysis53—and where the estimated effect is weighted on the basis of the compliance rate (ie, the percentage of participants who took their assigned treatment). Fuzzy regression discontinuity uses recommended treatment as an instrumental variable to estimate the causal effect of treatment received among patients willing to follow the recommendation. In other words, the estimand is interpreted as a local average treatment effect, but only for the group of compliers. In our application, we will illustrate both sharp and fuzzy regression discontinuity (fig 3 compares these concepts for an RDD study with a randomized controlled trial design).
Comparison of regression discontinuity design with randomized controlled trial design
Assumptions relied on by regression discontinuity
The ability to estimate a valid causal effect with RDD depends on a key assumption about exchangeability (comparing units below and above the cut-off value), which is based on a continuity condition. This assumption enables us to emulate trials in situations where randomization is not possible.
Exchangeability and continuity assumptions
Valid estimation of a causal effect with RDD depends on comparability (exchangeability) of units below and above the cut-off value (fig 2). Intuitively, we expect that individuals above and below but close to the cut-off value should have a similar distribution of measured and unmeasured prognostic characteristics or confounders (ie, they are exchangeable). Thus, if the analytical sample is restricted to individuals close to the cut-off value, the RD treatment recommendation can be viewed as a random treatment assignment.
In effect, we assume the fact that someone with a systolic blood pressure of 141 mm Hg and another individual with a corresponding value of 139 mm Hg landed on opposite sides of the cut-off value (thus receiving different treatment recommendations) due to chance, because very small fluctuations in their recorded blood pressure would have resulted in different treatment recommendations. By contrast, a pair of individuals further apart from the cut-off value (eg, with systolic blood pressure values of 160 mm Hg v 110 mm Hg) would probably differ for many reasons other than chance—in particular, many underlying health risks or other confounding factors predicting the outcome, both measured and unmeasured (eg, lifestyle, pre-existing conditions, genetics). In our application, we will illustrate how to provide support the regression discontinuity assumptions using a combination of falsification and validation methods.
In consideration of the exchangeability assumption, an integral aspect of regression discontinuity is the choice of a bandwidth (fig 2), which is the range of values around the cut-off value where the analysis will be conducted. Using our blood pressure example and the 140 mm Hg cut-off value, a bandwidth of 15 mm Hg would mean that blood pressure readings below and above the cut-off value would range from 125 to 140 mm Hg and from 140 to 155 mm Hg, respectively.
The question ultimately becomes how far researchers are willing to go above and below the cut-off value without compromising exchangeability, while also maintaining adequate precision (sample size). Bandwidth selection generally entails a trade-off between bias and variance; while a small bandwidth might result in lower bias, this setting can also create a larger variance because of the smaller number of observations used in the analysis. Conversely, a larger bandwidth will result in a greater number of observations used in the analysis, thereby lowering the variance while increasing the bias, making the exchangeability assumption less credible. Therefore, the most common choice for bandwidth selection in practice is based on minimizing the mean squared error of the regression discontinuity estimator, which balances bias and variance. Further technical discussion on this topic can be found elsewhere.5455
Conducting regression discontinuity design analyses
Briefly, RDD models are usually estimated by use of local polynomial models, a non-parametric estimation method that is both flexible (because it does not rely on linearity or other functional forms assumptions) and easy to implement (because it can be computed by weighted linear regression models).2651 After determining the running variable (ie, the instrument) and its cut-off value, the main analytical decision that must be made when implementing a regression discontinuity model is bandwidth selection that entails a trade-off between bias and variance. As discussed above, the optimal bandwidth choice balances these two factors and is referred to as the mean squared error-optimal bandwidth. Below, we provide a checklist of the statistical steps entailed in an RDD analysis (fig 4), followed by a detailed discussion of each step.
Checklist of analytical steps needed to conduct an analysis with a regression discontinuity design
Statistical steps for regression discontinuity analysis
We discuss each analytical step (fig 4) in more detail below. We also distinguish between estimation and inference, and provide additional suggestions regarding validation of the regression discontinuity model.
(1) Identify the main components of the regression discontinuity model: outcome variable, running variable, and cut-off value.
(2) A standard, recommended practice is to plot the regression discontinuity data by constructing a combination of binned scatterplots of the outcome as a function of the running variable, together with global polynomial functions fitted on each side of the cut-off value.56 The binned scatterplots (ie, dots generated in the figure) represent local sample means over non-overlapping partitions (bins) of the running variable. These binned means are included to capture the behavior of the cloud of points and to show any other discontinuities in the data away from the cut-off value. The second component of the figure consist of two smooth, global, polynomial, regression curve estimates for control and treatment units separately, which are meant to give a flexible global approximation of the regression functions at each side of the cut-off value.
(3) Set up the local polynomial model, which will encompass performing the three choices below. Setting up the local polynomial model can be implemented in the same step (web appendix).
(i) A common approach is to provide higher weight to observations closer to the cut-off value, with a weighting function (known as a kernel). Typically, a triangular kernel is chosen.
(ii) Choose a polynomial order to model the outcome as a function of the running variable on either side of the cut-off value; a linear fit is the preferred choice.
(iii) Compute the bandwidth to define the study sample. The mean squared error-optimal bandwidth can be computed relying entirely on the data, which makes it less susceptible to manipulation.57
(4) Regression discontinuity effect estimation. The bandwidth choice limits the original study sample to individuals within that range. The polynomial regression model consists of a local, weighted linear approximation of the outcome across levels of the running variable. This model estimates a local average treatment effect parameter, the causal effect of the treatment on the outcome, among participants near the cut-off value.
(5) Regression discontinuity inference. In addition to estimating the treatment effect, researchers might also be interested in testing hypotheses and constructing confidence intervals. Confidence intervals and P values should incorporate the local weighted approximation. Robust inference methods specifically tailored for this context were developed by Calonico et al5558 and can be implemented using the rdrobust software package.5960
(6) Robustness checks and validation. Several intuitive procedures can be used to provide empirical support for the regression discontinuity model.6162
(i) Density/manipulation test for continuity assumption: we would expect the regression discontinuity model to fail when individuals are able to manipulate their value of the running variable, because this could affect the continuity assumption required to recover the regression discontinuity effect. McCrary63 proposes to test for manipulation by looking at the continuity of the running variable density function at the cut-off value. The idea is to check whether the density of observations near the cut-off value is relatively constant; by doing so, researchers can rule out having too many individuals concentrated very close together at either side of the cut-off value. This test became one of the most common ways to provide an initial validation of the regression discontinuity model.
(ii) Regression discontinuity covariate balance: we can also exploit the availability of additional, pre-determined covariates to validate our model. If exchangeability holds in a neighborhood of the cut-off value, we expect individuals to be balanced on the value of their observed covariates not affected by the treatment. Thus, we can also compute regression discontinuity effects for these covariates and test for a null effect (ie, no jump in the conditional regression functions at the cut-off value) following the steps previously described.
(7) Imperfect compliance. Compliance with treatment assignment can be checked by comparing actual treatment uptake for those above and below the cut-off value. An easy way to assess this association is by making a plot as in step 2, and by computing the regression discontinuity effect using actual treatment uptake as the outcome. Perfect compliance means that treatment is withheld for everyone below the cut-off value, while it is received by everyone above it. If compliance is not perfect, then researchers can implement a fuzzy regression discontinuity model following steps 1-6 as described above. The model relies on using treatment assignment (being above or below the cut-off value) as the instrument, and this treatment effect of the fuzzy regression discontinuity model can be easily estimated using the same software packages (see below for additional details).
Application of the regression discontinuity design
Introducing the dataset
For the regression discontinuity design application, we use a teaching version of the Framingham Heart Study dataset, available on request at the Biologic Specimen and Data Repository Information Coordinating Center. This dataset contains redacted data and is intended for teaching purposes only, and as such any findings presented in the RDD application should not be used for interpreting actual results. The Framingham Heart Study is a longitudinal cohort study that began in 1948 and has since helped to uncover some of the determinants of cardiovascular disease.64 Therefore, in our application, we leverage this study’s extensive resource in cardiovascular health data and seek to estimate the causal effect of antihypertensive drugs on the incidence of cardiovascular disease among individuals in this cohort. We decided to investigate this question in our application because it is customary when demonstrating a method to benchmark results against a research question for which there is well established evidence about the strength and direction of the effect size of interest. In this example, the running variable is systolic blood pressure, the cut-off value for treatment is a systolic blood pressure of 140 mm Hg based on established clinical guidelines, the treatment is antihypertensive drugs, and the outcome is cardiovascular disease (yes/no). For simplicity, we assume age (year) to be the only covariate of interest.
How to conduct regression discontinuity analysis and present and interpret findings
To conduct the RDD analysis into the causal effect of antihypertensive drugs on incident cardiovascular disease, we will follow all seven steps listed figure 4. We briefly discuss below each analytical step, how to interpret the results, and provide the accompanying statistical code. The web appendix provides a more detailed walkthrough of these steps (as well as detailed statistical codes and outputs).
In step 1, we first define the components of our regression discontinuity model—the running variable (systolic blood pressure), cut-off value (systolic blood pressure of 140 mm Hg), treatment (antihypertensive drug use), and outcome (cardiovascular disease). In step 2, we proceed by plotting the overall association between systolic blood pressure (x axis) and cardiovascular disease (y axis), separately at each side of the cut-off value. As shown in figure 5, an overall positive association between systolic blood pressure and cardiovascular disease can be seen, which is expected and is the reason why we do not simply compare all individuals above and below the cut-off value, but instead focus on a neighborhood around the cut-off value. In addition, a discontinuity at the cut-off value can be seen, whereby individuals just above the cut-off value have a lower risk of the outcome than individuals just below the cut-off value.
Results selected for steps 1-5 for the analysis of an example model for sharp regression discontinuity. Sharp RD assumes perfect treatment uptake, and thus is equivalent to an intention-to-treat analysis. The model estimates the treatment effect of antihypertensive drugs on incident cardiovascular disease. Steps 1-5 are outlined in figure 4 and detailed in the main text. Top graph refers to step 2: plotting binned scatterplot showing association between the running variable and outcome. Bottom graph refers to steps 4-5: results from sharp regression discontinuity model, and plotting regression discontinuity effect within the selected bandwidth (9.571); the distance between pink lines at the cut-off value provides the average treatment effect, which corresponds to the parameter estimate provided in the middle table. *Y axis scale measures predicted probability of the outcome. CI=confidence interval
In steps 3-5, we set up the local polynomial model, select the bandwidth, and run the sharp regression discontinuity model (these results can also be seen in fig 5). In brief, the selected bandwidth for mean squared error was 9.571 mm Hg (ie, 9.571 mm Hg points above and below the 140 mm Hg cut-off value). Here, the coefficient value (−0.051) corresponds to the regression discontinuity design’s average treatment effect of interest—that is, antihypertensive treatment as indicated by the cut-off value of 140 mm Hg (assuming perfect treatment uptake) is associated with an absolute risk reduction of cardiovascular disease of 5.1 percentage points. The 95% confidence interval for this estimate ranges from a reduction of 14.7 percentage points to an increase of 1.4 percentage points. We then replot the same data, except this time by limiting the functions on either side of the cut-off value to the individuals selected within the selected bandwidth. Again, we see a discontinuity at the cut-off value. The distance between the lines at the cut-off value equates to the average treatment effect, which corresponds to the parameter estimate provided in the summary function (−0.051 or −5.1%). Box A2 of the web appendix shows the statistical codes needed to run the analysis and the output of steps 1 to 5.
After conducting the estimation and inference for the main regression discontinuity model, we next perform robustness checks (step 6). Figure 6 shows a density plot for testing the continuity assumption (of observations around the cut-off value). If this assumption holds, we should expect to see a relatively consistent frequency of observations at each side of the cut-off value. From both the plot and the results of the test we can conclude that there is no evidence of a jump in the density of the running variable at the cut-off value, which is consistent with a lack of manipulation. As an additional robustness check, we assess covariate balance by taking a given covariate (eg, age) and treating it as an outcome in the regression discontinuity model. As shown in figure 6, an overall positive association is seen between age and systolic blood pressure, which is expected and the reason why we do not simply compare all individuals above and below the cut-off value, but instead focus on a neighborhood around the cut-off value. In other words, although these polynomials are plotted across the entire range in the sample, the actual regression discontinuity effect for assessing covariate balance is computed at the cut-off value (systolic blood pressure of 140 mm Hg). If the exchangeability assumption holds, then we should expect the levels of this covariate age to be comparable below and above the cut-off value (ie, the regression discontinuity effect for that covariate should be close to 0). In other words, the null hypothesis refers to no regression discontinuity effects in the covariates—that is, the conditional expectations of these variables (eg, age) for the treatment and control groups should be the same at the cut-off value point. To test this assumption, we evaluate the estimated function at the cut-off value that, for the default choice of a linear approximation, includes a constant term plus a slope, which are allowed to be different across treatment and control groups (so that we get a linear prediction of each regression function evaluated at the cut-off value).
Results selected for step 6 for the analysis of an example model of sharp regression discontinuity. Sharp RD assumes perfect treatment uptake, and thus is equivalent to an intention-to-treat analysis. The model estimates the treatment effect of antihypertensive drugs on incident cardiovascular disease. Step 6 is outlined in figure 4 and detailed in the main text. Figure shows the visual inspection of two main assumptions for regression discontinuity. Under the continuity assumption (top graph), the plot shows no evidence of a jump in the density of the running variable at the cut-off value, which is consistent with a lack of manipulation; this lack of evidence suggests that the continuity assumption is probably not violated. Under the exchangeability assumption (bottom graph), if covariate age is used as an example, this plot shows no significant difference in age comparing individuals below and above the cut-off value, providing some evidence that the exchangeability assumption is probably satisfied
Lastly, in step 7, we discuss the implementation of a fuzzy regression discontinuity design, which distinguishes between treatment assignment (in this example, determined by being above the 140 cut-off value) to actual treatment uptake (or compliance). Imperfect compliance occurs when some patients below the cut-off value receive the treatment or some patients above the cut-off value fail to receive the treatment. This scenario is common in medical applications, where clinical guidelines are not binding and physicians can decide whether to prescribe a treatment—and where patients can be encouraged but not forced to take drug treatment. Fuzzy regression discontinuity is conceptually similar to an instrumental variables design in a local neighborhood around the cut-off value, where the score acts as an instrument for use of the treatment for patients whose score is sufficiently close.
To implement a fuzzy regression discontinuity model, we use two steps. First, we compute the first stage, defined as the difference in treatment uptake, by computing the treatment compliance at each side of the cut-off value. In other words, we calculate the difference in the per cent of patients above and below the cut-off value (locally) who were actually treated with the antihypertensive drugs. Note that this difference in treatment uptake will be equal to 1 under perfect compliance (sharp regression discontinuity), because treatment uptake is 100% above the cut-off value and 0% below it. Second, we compute our estimate for fuzzy regression discontinuity as the ratio of the sharp regression discontinuity effect, which in this context can be interpreted as an intention-to-treat parameter, to the difference in treatment uptake (the first stage) computed in the previous step.
In our application, we find a first stage of +0.024 (ie, 2.4 percentage points), meaning that the change in treatment uptake at the cut-off value is extremely low. In particular, within the bandwidth of 18.68, the treatment uptake for the control group (ie, those values left of the cut-off value) is 5.6%, while this value equals 8% for the treatment group; thus the obtained coefficient is +0.024. These treatment uptake percentages can be seen in figure 7, and can also be computed using the statistical code in box A2 of the appendix. Thus, in this example, the strength of the instrument is weak. As in the case of instrumental variable models, a weak instrument compromises the ability of the RDD to be informative about the effect of the treatment on the outcomes of interest. But we can still proceed with the analysis simply as an illustration. A formal discussion of weak fuzzy regression discontinuity can be found elsewhere.65
Compliance analysis and fuzzy regression discontinuity (RD) results for an example model estimating the treatment effect of antihypertensive drugs on incident cardiovascular disease. Fuzzy RD assumes imperfect compliance with treatment assignment, and is equivalent to a per protocol analysis. This process represents step 7 of the analysis, which is outlined in figure 4 and detailed in the main text. CI=confidence interval
In figure 7, we present our fuzzy regression discontinuity estimates, with a coefficient value (−0.23) corresponding to the per protocol treatment effect. In other words, the complier specific effect of antihypertensive treatment is associated with an absolute risk reduction of cardiovascular disease of 23 percentage points. The complier specific treatment effect is computed by dividing the sharp regression discontinuity effect (−0.6 percentage points) by the compliance rate (2.4 percentage points). The 95% confidence interval is −3.18 to 1.73 and is obtained by use of the same code in box A2 of the web appendix (step 7). Again, given the low compliance rate in our empirical example, we must be cautious about interpreting this fuzzy regression discontinuity analysis. Box A2 of the web appendix also shows the statistical codes needed to run the analysis and the output of steps 6 and 7.
Summary
As illustrated in this paper, some of the most acknowledged advantages of regression discontinuity are its intuitiveness at emulating a randomized controlled trial, the array of empirical methods it offers that can enhance validity, including a straightforward graphical representation, and the formal approaches that exist for falsification, estimation, and inference. Through implementation of all the technical steps and detailed statistical code ready to use by clinical investigators, this practical guide serves as a first step for demystifying and better understanding how to conduct an RDD analysis. More importantly, we hope this paper highlights the opportunities and breadth of research questions that RDD can help leverage, further strengthening collaborations across fields that often do not speak to one another, such as medicine, economics, and epidemiology.
Use of causal inference approaches based on quasi-experiments, including regression discontinuity methods, combines major advantages of both traditional experimental and observational research. RDD allows investigators to estimate causal effects in non-randomized settings, overcoming the many sources of bias that could compromise internal validity in observational studies, and thus improving on traditional approaches to satisfy exchangeability by exploiting a treatment rule observed in practice.
In this article, while we focused on RDD as an ex-post-design strategy (ie, based on previously established guidelines), another potentially relevant contribution of this approach is at the design stage, where RDD could be conducted instead of randomized controlled trials or other quasi-experimental methods. While these intentional RDDs remain relatively under-explored, they have already been already implemented to evaluate education and health interventions.6667 Furthermore, treatment guidelines and their associated cut-off values could change over time in the light of new evidence or other criteria. As such, another relevant contribution of RDDs could also be to explore what the best treatment regimen(s) should be.6869 However, RDD is not without limitations, including limited extrapolation beyond the cut-off value—that is, their local nature implies that it is not possible to learn about treatment effects away from the cut-off value without additional assumptions. In our context, a treatment effect observed at the 140 cut-off value might not extrapolate well to individuals with a higher or lower systolic blood pressure. Still, this potential limitation does not necessarily affect the external validity of the approach (for instance, to compare the effect across different groups or geographical areas) because in many cases, the cut-off value is based on clinically relevant values. Additionally, several formal approaches have been proposed in recent years to enable valid extrapolation of regression discontinuity treatment effects away from the cut-off value.7071
Finally, with no existing practical guides, especially those with replicable examples and syntax, clinical researchers might not be familiar with the method and could be unsure of how to implement it. As such, we hope that our practical guide can help overcome some of these limitations and encourage wider adoption of RDD in practice, by familiarizing health researchers with RDD, alerting them to its advantages and situations in which its use is warranted, and providing them with analytical tools and procedures to perform their own RDD analyses.
Everyday medical practice has traditionally been shaped by evidence from clinical research conducted within a randomized trial framework. In this article, we argue that, in many cases, RDD can produce equally rigorous and complementary evidence to support clinical research activities. We hope that RDD becomes a standard tool in health researchers’ toolkits, helping to inform our understanding of treatment effects of prescribed drugs, changes in clinical guidelines, or program evaluation of medical services.
Footnotes
Contributors: SC and AZ conceived the study and SC wrote the original draft. NJ and KK contributed to the development of the regression discontinuity design application as well as the supplementary didactic material. All authors were involved in the reviewing and editing of the manuscript. SC is the guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Funding: National Institute on Aging-National Institutes of Health (NIH/NIA) grant R56 AG061177and R01AG081973. The funders had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report, or decision to submit the article for publication.
Competing interests: All authors have completed the ICMJE uniform disclosure form at https://www.icmje.org/disclosure-of-interest/ and declare: support from the NIH/NIA for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Provenance and peer review: Not commissioned; externally peer reviewed.