Accounting for site-selection bias in before–after studies for continuous distributions: Characteristics and application using speed data

The before–after study is still the most popular method used by traffic engineers and transportation safety analysts for evaluating the effects of an intervention. Compared to the cross-sectional study, the before–after study has lower within-subject variability since it directly accounts for changes that have occurred at the study sites. However, although this kind of study may offer superior performance, it can still be plagued by important methodological limitations, which could significantly alter the study outcome. They include the regression-to-the-mean (RTM) and site-selection effects. The primary objective of this study consists of presenting a method that can reduce the selection effects when an entry criterion is used in before–after studies for continuous data (e.g. speed, reaction times, etc.), without relying on the use of a control group. The distribution of the data could follow a normal or lognormal distribution. The study objective was accomplished using simulated and observed speed data collected in Florida. The proposed method documented in this paper was compared to the Naïve, Control Group (CG) and the Analysis of Covariance (ANCOVA) methods. The simulation results show that the proposed method provides a more precise estimate than the Naïve method, as expected. In addition, the method performs better than the CG and the ANCOVA methods when similar control group data are not available. The results also show that higher entry criteria, lower between-subject variances, and higher within-subject variances cause higher selection biases. When traffic engineers and urban planners evaluate or compare different strategies, the proposed method can be applied to adjust naïve estimators of treatment effectiveness documented in previous studies without similar control group data.

► Before–after studies can be plagued by important methodological limitations. ► This study presents a method that can reduce selection effects. ► The study was accomplished using simulated and observed speed data. ► Entry criteria, between-subject and within-subject variances affect the bias. ► The proposed method provides a more precise estimate than the Naïve method.