Data vs. Methods: Quasi-Experimental Evaluation of Alternative Sample Selection Corrections for Missing College Entrance Exam Score Data

Thumbnail Image



Journal Title

Journal ISSN

Volume Title

Repository Usage Stats



In 2007, Michigan began requiring all high school students to take the ACT college entrance exam. This natural experiment allows us to evaluate the performance of several parametric and semiparametric sample selection correction models. We apply each model to the censored, prepolicy test score data and compare the predicted values to the uncensored, post-policy distribution. We vary the set of model predictors to imitate the varying levels of data detail to which a researcher may have access. We find that predictive performance is sensitive to predictor choice but not correction model choice. All models perform poorly using student demographics and school- and district-level characteristics as predictors. However, all models perform well when including students’ prior and contemporaneous scores on other tests. Similarly, correction models using group-level data perform better with more finely disaggregated groups, but produce similar predictions under different functional form assumptions. Our findings are not explained by an absence of selection, the assumptions of the parametric models holding, or the data lacking sufficient variation to permit useful semiparametric estimation. We conclude that “data beat methods” in this setting: gains from using less restrictive econometric methods are small relative to gains from seeking richer or more disaggregated data.








Robert J Garlick

Assistant Professor of Economics

Robert Garlick joined the economics department as an assistant professor in 2014. He previously worked as a postdoctoral researcher at the World Bank's Development Research Group and holds a PhD in economics and public policy from the University of Michigan. Garlick was born and raised in South Africa, where he studied economics, mathematics and philosophy at the University of Cape Town and managed a small education nonprofit.

Garlick studies education and labor economics in developing countries. He is currently working on peer and network effects in education, determinants of education investments by households, and transitions between education and the labor market. This work spans empirical and methodological topics, and includes primary data collection in and analysis of secondary data from Kenya, Nigeria, and South Africa.

Material is made available in this collection at the direction of authors according to their understanding of their rights in that material. You may download and use these materials in any manner not prohibited by copyright or other applicable law.