Click on the following links. Please note these will open in a new window.

Journal Article 1: Garcia, F. M., & Wantchekon, L. (2010). theory, external validity, and experimental inference: Some conjectures. The ANNALS of the American Academy of Political and Social Science, 628, 132–147.

Abstract: It is often argued that experiments are strong on causal identification (internal validity) but weak on generalizability (external validity). One widely accepted way to limit threats to external validity is to incorporate as much variation in the background conditions and in the covariates as possible through replication. Another strategy is to make the theoretical foundations of the experiment more explicit. The latter requires that we develop trajectories of experiments that are consistent with a theoretical argument. In other words, new experiments should not simply consist of changing the context of old ones, but do so in ways that explicitly test various aspects of a theory in a coherent way.

Journal Article 2: Camburn, E. M., Goldring, E., Sebastian, J., May, H., & Huff, J. (2015). An examination of the benefits, limitations, and challenges of conducting randomized experiments with principals. Educational Administration Quarterly, 52, 187–220.

Abstract: Purpose: The past decade has seen considerable debate about how to best evaluate the efficacy of educational improvement initiatives, and members of the educational leadership research community have entered the debate with great energy. Throughout this debate, the use of randomized experiments has been a particularly contentious subject. This study examines the potential benefits, limitations, and challenges involved in using experiments to evaluate professional development for principals. Approach: We present a case study of an experimental evaluation of a professional development program for principals. The case study is grounded in key themes in recent debates about the use of experiments in educational research, scholarship on challenges in conducting experiments, and experimental studies involving principals. Setting and Sample: The case study was conducted in an urban school district with 48 principals. Implications for Research: The experimental component of the study allowed us to form a trustworthy summary inference about whether or not a professional development program had an overall effect on principals. However, the experiment did not illuminate why or how the program failed to influence principal practice. Using descriptions of the intended curriculum for principals, professional development attendance records, and interview data, we developed an understanding of why the program failed to achieve its intended goals. Based on our experiences, we support continued advocacy of research designs that bring rich evidence to bear about causal mechanisms, implementation conditions, potential measures of delivery of and adherence to treatment protocols, and measures of participants’ exposure to treatment.