Strengthening Causal Inferences

Abstract

Education researchers are frequently interested in examining the causal impact of academic services and interventions; however, it is often not feasible to randomly assign study elements to treatment conditions in the field of education (Adelson, 2013). When assignment to treatment conditions is non-random, the omission of any variables relevant to treatment selection creates a correlation between the treatment variable and the error in regression models. This is termed endogeneity (Ebbes, 2004). In the presence of endogeneity, treatment effect estimates from traditionally used regression approaches may be biased. The purpose of this study was to investigate the causal impact of an integrated student support model, namely City Connects, on student academic achievement. Given that students are not randomly assigned to the City Connects intervention, endogeneity bias may be present. To address this issue, two novel and underused statistical approaches were used with school admissions lottery data, namely Gaussian copula regression developed by Park and Gupta (2012), and Latent Instrumental Variable (LIV) regression developed by Peter Ebbes (2004). The use of real-world school admissions lottery data allowed the first-ever comparison of the two proposed methods with Instrumental Variable (IV) regression under a large-scale randomized control (RCT) trial. Additionally, the researcher used simulation data to investigate both the performance and boundaries of the two proposed methods compared with that of OLS and IV regression. Simulation study findings suggest that both Gaussian copula and LIV regression are useful approaches for addressing endogeneity bias across a range of research conditions. Furthermore, simulation findings suggest that the two proposed methods have important differences in their set of identifying assumptions, and that some assumptions are more crucial than others. Results from the application of the Gaussian copula and LIV regression in the City Connects school lottery admissions study demonstrated that receiving the City Connects model of integrated student support during elementary school has a positive impact on mathematics achievement. Such findings underscore the importance of addressing out-of-school barriers to learning.