Click on the following links. Please note these will open in a new window.

(11.1). Experimental researchers sometimes ignore the importance of external validity because they are so concerned with internal validity. This article addresses this and makes the point that general theories may need to be contextualized to local places.

Journal Article 1: Leviton, L. C., & Trujillo, M. D. (2017). Interaction of theory and practice to assess external validity. Evaluation Review, 41, 436–471. doi:10.1177/0193841X15625289

Abstract:

Background: Variations in local context bedevil the assessment of external validity: the ability to generalize about effects of treatments. For evaluation, the challenges of assessing external validity are intimately tied to the translation and spread of evidence-based interventions. This makes external validity a question for decision makers, who need to determine whether to endorse, fund, or adopt interventions that were found to be effective and how to ensure high quality once they spread.

Objective: To present the rationale for using theory to assess external validity and the value of more systematic interaction of theory and practice.

Methods: We review advances in external validity, program theory, practitioner expertise, and local adaptation. Examples are provided for program theory, its adaptation to diverse contexts, and generalizing to contexts that have not yet been studied. The often critical role of practitioner experience is illustrated in these examples. Work is described that the Robert Wood Johnson Foundation is supporting to study treatment variation and context more systematically.

Results: Researchers and developers generally see a limited range of contexts in which the intervention is implemented. Individual practitioners see a different and often a wider range of contexts, albeit not a systematic sample. Organized and taken together, however, practitioner experiences can inform external validity by challenging the developers and researchers to consider a wider range of contexts. Researchers have developed a variety of ways to adapt interventions in light of such challenges.

Conclusions: In systematic programs of inquiry, as opposed to individual studies, the problems of context can be better addressed. Evaluators have advocated an interaction of theory and practice for many years, but the process can be made more systematic and useful. Systematic interaction can set priorities for assessment of external validity by examining the prevalence and importance of context features and treatment variations. Practitioner interaction with researchers and developers can assist in sharpening program theory, reducing uncertainty about treatment variations that are consistent or inconsistent with the theory, inductively ruling out the ones that are harmful or irrelevant, and helping set priorities for more rigorous study of context and treatment variation.

(11.2). This article provides an additional and useful look at validity in qualitative research.

Journal Article 2: Cho, J., & Trent, A. (2006). Validity in qualitative research revisited. Qualitative Research, 6, 319–340.

Abstract: Concerns with the issues of validity in qualitative research have dramatically increased. Traditionally, validity in qualitative research involved determining the degree to which researchers’ claims about knowledge corresponded to the reality (or research participants’ construction of reality) being studied. The authors note that recent trends have shown the emergence of two quite different approaches to the validity question within the literature on qualitative research. The authors categorize and label these ‘transactional’ validity and ‘transformational’ validity. While useful, the authors assert that neither approach is sufficient to meet the current needs of the field. The authors propose a recursive, process-oriented view of validity as an alternative framework.

(11.3). This article also provides an additional and useful look at validity in qualitative research. Basically, unlike in quantitative research, there is no single set of validity types in qualitative research. Therefore, it is useful to examine multiple articles on the topic.

Journal Article 3: Tracy, S. J. (2010). Qualitative quality: Eight “big-tent” criteria for excellent qualitative research. Qualitative Inquiry, 16, 837–851. doi:10.1177/1077800410383121

Abstract: This article presents a model for quality in qualitative research that is uniquely expansive, yet flexible, in that it makes distinctions among qualitative research’s means (methods and practices) and its ends. The article first provides a contextualization and rationale for the conceptualization. Then the author presents and explores eight key markers of quality in qualitative research including (a) worthy topic, (b) rich rigor, (c) sincerity, (d) credibility, (e) resonance, (f) significant contribution, (g) ethics, and (h) meaningful coherence. This eight-point conceptualization offers a useful pedagogical model and provides a common language of qualitative best practices that can be recognized as integral by a variety of audiences. While making a case for these markers of quality, the article leaves space for dialogue, imagination, growth, and improvisation.

(11.4). This important article shows that causation is important in qualitative and quantitative research. It shows that taking a mixed methods perspective can provide an excellent and more complete view of causation.

Journal Article 4: Johnson, R. B., Russo, F., & Schoonenboom, J. (2017). Causation in mixed methods research: The meeting of philosophy, science, and practice. Journal of Mixed Methods Research, 11, 156–173 doi:10.1177/1558689817719610

Abstract: This article provides a systematic and pluralistic theory of causation that fits the kind of reasoning commonly found in mixed methods research. It encompasses a variety of causal concepts, notions, approaches, and methods. Each instantiation of the theory is like a mosaic, where the image appears when the tiles are appropriately displayed. This means that researchers should carefully construct a causal mosaic for each research study, articulating what is causally relevant given their particular research questions, purposes, contexts, methods, methodologies, paradigms, and resources. Our theory includes 11 propositions that can guide researchers addressing causation.