| Article ID | Journal | Published Year | Pages | File Type |
|---|---|---|---|---|
| 5057921 | Economics Letters | 2017 | 9 Pages |
â¢We compare experimental and observational estimates of environmental program impact.â¢We expand the sample of comparison units to improve covariate balance.â¢Despite similarity of covariates and baseline trends, bias of the estimator worsens.â¢Fixed-effects panel estimators and indirect tests of their validity are no panacea.
We compare experimental and nonexperimental estimates from a social and informational messaging experiment. Our results show that applying a fixed effects estimator in conjunction with matching to pre-process nonexperimental comparison groups cannot replicate an experimental benchmark, despite parallel pre-intervention trends and good covariate balance. The results are a stark reminder about the role of untestable assumptions-in our case, conditional bias stability-in drawing causal inferences from observational data, and the dangers of relying on single studies to justify program scaling-up or canceling.
