Complex Systems Approaches to Educational Research: Introduction to the Special Issue

Abstract

To explore the movement toward increased complex systems (CS) research in education, this special issue of the Journal of Experimental Education was developed to identify some under-examined places where CS approaches have advanced research. The articles provide empirical examples of research leveraging methods and analyses from complexity science. In this introduction article to the special issue, we discuss the authors’ contributions to defining and explicating concepts and methods that fall under the umbrella of CS perspectives and how these methods were used to investigate complex processes in their topics of study. The articles capture the goal of the special issue to demonstrate how the research questions framed by CS assumptions can change our expectations about the very nature of the processes under study in education. We then take a more global perspective and acknowledge the commonalities amongst the papers, including (a) the reliance on intensive data to answer research questions, and (b) the use of dynamic approaches that yield findings about the stability and change of these systems and phenomena.     

Meta-Analysis in Higher Education: An Illustrative Example Using Hierarchical Linear Modeling

The purpose of this article is to provide higher education researchers with an illustrative example of meta-analysis utilizing hierarchical linear modeling (HLM). This article demonstrates the step-by-step process of meta-analysis using a recently-published study examining the effects of curricular and co-curricular diversity activities on racial bias in college students as an example (Denson, Rev Educ Res 79:805–838, 2009). The authors present an overview of the meta-analytic approach and describe a meta-analysis from beginning to end. The example includes: problem specification; research questions; study retrieval and selection; coding procedure; calculating effect sizes; visual displays and summary statistics; conducting HLM analyses; and sensitivity analyses. The authors also offer guidelines and recommendations for improving the conduct and reporting of research which in turn can provide the information necessary for future and more comprehensive meta-analytic reviews.     

Multivariate Meta-Analysis as Structural Equation Models

Multivariate meta-analysis has become increasingly popular in the educational, social, and medical sciences. It is because the outcome measures in a meta-analysis can involve more than one effect size. This article proposes 2 mathematically equivalent models to implement multivariate meta-analysis in structural equation modeling (SEM). Specifically, this article shows how multivariate fixed-, random- and mixed-effects meta-analyses can be formulated as structural equation models. metaSEM (a free R package based on OpenMx) and Mplus are used to implement the proposed procedures. A real data set is used to illustrate the procedures. Formulating multivariate meta-analysis as structural equation models provides many new research opportunities for methodological development in both meta-analysis and SEM. Issues related to and extensions on the SEM-based meta-analysis are discussed.     

Our Invited Essay Series: “Critical Engagements with the Politics of ‘Extremism’ and Schooling”

This issue of Diaspora, Indigenous, and Minority Education features two essays responding to Adeela Arshad-Ayaz and M. Ayaz Naseem'’s lead article for this series, “Creating “Invited” Spaces for Counter-Radicalization and Counter-Extremism Education.” Drawing upon metaphor and empirical research, these essays impart important theoretical tools for scholars who work on questions of migration, social justice, belonging, and security in education.     

A consolidation and appraisal of science meta-analyses

This article is directed to consolidating information on selected matters reported in two or more of the separate meta-analysis reported in this issue of the Journal and examining the relationship between the results of these meta-analyses and other work of this nature reported in other places. A high degree of consistency was found among the results of these various meta-analyses.     

Should we be encouraging pupils to ask more questions?

This article investigates the issue of secondary school pupils asking questions. This is an important topic on which very little has been published to date. The article reviews the current literature, which almost exclusively reports the lack of student initiated, content related questioning in classrooms. A small study is described that challenges this view, finding a significantly greater level of student participation, a high percentage of inquiry driven questions and little reluctance on the part of pupils to participate actively in lessons. However, when these questioning patterns, which appear to indicate exemplary lessons on paper, are examined in terms of pupil learning, it is found that large numbers of student questions can lead to severe disruption unless adequately controlled. For effective learning to take place, it is a vital that the learner has the opportunity to ask questions of the teacher. The vast bulk of the literature explores this topic through quantitative analysis of questioning patterns and suggested strategies for increasing the quality and frequency of pupil’s questions. This article reviews this research and explores the issue further through classroom observations, pupil questionnaires and interviews with teachers. The findings are compared with the current paradigm and the issue is discussed in terms of impact on pupil learning.     

What Future for Phenomenographic Research? On Continuity and Development in the Phenomenography and Variation Theory Research Tradition

Volume 60(3) of the Scandanavian Journal of Educational Research is a special issue on current trends in the phenomenography and variation theory research tradition. In the introduction to the special issue, Rovio-Johansson and Ingerman look towards the potential theoretical and methodological future of phenomenography and pose two questions: “Is phenomenography complete and finalised with no further development necessary?” and “Is phenomenography relatively insignificant in the future development of the tradition as it has been transcended by variation theory?” The aim of this paper is to address these two questions, and in the process elaborate the relationship between phenomenography and variation theory in educational research.     

A Complexity Approach Toward Mind–Brain–Education (MBE); Challenges and Opportunities in Educational Intervention and Research

In the context of an educational or clinical intervention, we often ask questions such as “How does this intervention influence the task behavior of autistic children?” or “How does working memory influence inhibition of immediate responses?” What do we mean by the word influence here? In this article, we introduce the framework of complex dynamic systems (CDS) to disentangle the meaning of words such as influence, and to discuss the issue of education and intervention as something that takes place in the form of complex, real‐time, situated processes. What are the applied implications of such a CDS framework? Can we use it to improve education? Five general principles—process laws—are introduced, which can be used to guide the way we formulate research questions and methods, and the way we use the results of such research. In addition, we briefly discuss a project in progress, in which we ourselves attempt to apply the process laws that govern educational activities. Finally, we report about a discussion about the usability of the process laws, both in educational research and in the classroom, as was held during our workshop at the Mind, Brain, and Education Conference, November 2014.     

Writing culture; inscribing lives: a reflective treatise on the burden of representation in native research

This paper complicates the contested assumptions surrounding native research by exploring the burden of representation placed on native researchers because they are seen as insiders. This particular issue of representation is important for native researchers to consider, especially in instances where the research is on an understudied or politically charged group, because of the ways in which the socio-political climate influences both the telling and the reading of such research. Drawing on the author’s personal struggles as a Pakistani researcher conducting ethnographic research with Pakistani immigrants in the United States in the post-9/11 climate, this paper explores and critiques the role of the native researcher and the issues involved with representing understudied groups. Specifically, the paper focuses on authenticity, positionality, audience, and accountability. Thus, the paper is a call for researchers to be more reflective and to think more deeply about their positionality and its impact on the various constituents as they research and write. To encourage such reflexivity, the article provides a set of questions for researchers to consider at different stages of the research and writing process.     

Adding results to a meta-analysis: Theory and example

Meta-analysis has been used as a research method to describe bodies of research data. It promotes hypothesis formation and the development of science education laws. A function overlooked, however, is the role it plays in updating research. Methods to integrate new research with meta-analysis results need explication. A procedure is presented using Bayesian analysis. Research in science education attitude correlation with achievement has been published after a recent meta-analysis of the topic. The results show how new findings complement the previous meta-analysis and extend its conclusions. Additional methodological questions adddressed are how studies are to be weighted, which variables are to be examined, and how often meta-analysis are to be updated.     

Detecting potential hucksterism in meta-analysis using a follow-up fail-safe test

Meta-analysis is an analysis of analyses. It is a technique widely used by researchers and practitioners to aggregate and summarize statistically reported empirical educational research. In 10 years, meta-analysis appeared more than 600 times in research journals and dissertation abstracts. Although most meta-analyses were reported as significant, few of the findings determined how many unpublished “no-effect” studies, if sampled, would have invalidated significance. If significant meta-analysis results are over represented through selective sampling, hucksterism in the form of sampling bias exists. An explanation for using a follow-up test called the fail-safe N is provided with tables constructed to assist researchers and practitioners to estimate, without calculation, the relative stability of meta-analysis results. The implication is that failsafe N should routinely be used and reported in meta-analysis research.     

Design Research as a Mechanism for Consultants to Facilitate and Evaluate Educational Innovations

Schools across the nation are implementing innovative practices; however, questions remain regarding how to facilitate quality implementation. Research designs that emphasize high degrees of control over independent variables result in findings with internal validity, but that may not generalize to complex, dynamic educational systems. The purpose of this article is to propose a design research framework as a mechanism for consultants to facilitate and evaluate innovation implementation. Information on design research principles and processes is provided, and issues to consider when applying the framework are discussed. An illustration of how a design research framework was applied in a large-scale initiative to implement and evaluate Response to Intervention (RtI) implementation is also provided. Finally, issues and questions to consider relative to consultants' use of design research principles are explored.     

Levels of Analysis in Higher Education Research

This article examines the levels of analysis adopted in higher education research, using a database of 567 articles published in 15 leading higher education journals internationally during 2010. Level of analysis is a relatively overlooked issue in this field of research, when compared with methodology, theory and/or topic—to which it is, of course, closely related—but is worthy of more investigation. The article seeks to address the question of whether certain levels of analysis are more or less common for different kinds of research. Eight levels of analysis are recognized, varying from the individual, through the course, department, institution, region, nation and system up to the international. The relationships between level of analysis, journal, theme, methodology and authors' characteristics are considered. It is argued that, given the constraints of available data and intended audiences, while valuable research may be conducted at all levels of analysis, the key issue is choosing an appropriate level for the questions being explored, and the methods and theories being applied.     

Teacher knowledge,curriculum materials,and quality of instruction: Unpacking a complex relationship

The set of papers presented in this issue comprise a multiple-case study which attends to instructional resources—teacher knowledge and curriculum materials—to understand how they individually and jointly contribute to instructional quality. We approach this inquiry by comparing lessons taught by teachers with differing mathematical knowledge for teaching who were using either the same or different editions of a US Standards-based curriculum. This introductory paper situates the work reported in the next four case-study papers by outlining the analytic framework guiding the exploration and detailing the methods for addressing the research questions.     

Decision-oriented educational research in school districts: The role of dissemination processes

This article addresses the broad issue of how research information can be integrated into the policy and decision processes of educational systems. Our specific foci here is on one type of educational system -- school districts; on research designed to answer specific questions for that district; and, on one major element of the research process -- dissemination. This paper draws heavily upon our experiences in a large, urban American school district, where we work on the problem of how to improve the system's evaluation research capability. This work began in 1978 and continues today. This article has four major section. Background is provided on the nature of the collaboration with the school district. Then, basic strategies for conducting decision-oriented educational research that have emerged from the work are described briefly. This is followed by a discussion of the broad issue of research use. The final section is a detailed review of what we have learned about the relationship between dissemination and utilization.     

International capacity building: Improving sales skills in a for‐profit business in South Africa‐part 1

How many times do clients ask: What is a CPT? What do CPTs do? Perhaps the best way to answer these questions is to look at some of the case studies submitted by Certified Performance Technologists applicants as part of the CPT application process. The applicants are asked to identify three projects they have done in the past 10 years that demonstrate proficiency in ISPI Standards 1 to 4 and at least three other Standards. In this article, a successful CPT applicant describes one of his projects and aligned his role as a performance improvement consultant with all 10 ISPI Standards. The story begins in this issue as Wessel Van Reede Van Oudtshoorn describes how he focused on results, took a systemic viewpoint, added value, used partnerships, and systematically assessed the need and opportunity. The story will continue in the February 2012 issue of PIJ. Joan Conway Dessinger     

Examining the Double-Deficit Hypothesis in an Orthographically Consistent Language

As many of you know, this is my final issue as editor of Scientific Studies of Reading. The new editor, Charles Hulme, has the work well in hand and has actually completed the editing of two articles which appeared in Issue 3 of this volume. The journal will be in good hands. I want to thank the many board members, reviewers, and authors for their energy, talents, and enthusiasm in the cause of advancing the journal. It's been a great privilege to be able to work with you and to guide the journal for 5 years. The current issue is a special issue, which had its inception in an idea of Mike Pressley's to have authors assess the present and future of key areas of literacy research. The first three articles in this issue are part of the special issue. The fourth article is a rewritten version of Charles Perfetti's Distinguished Scientific Contribution presentation at the July 2006 SSSR meeting. I append comments about Mike Pressley, written by Joanna Williams, who appropriately was the first editor of Scientific Studies of Reading and who had a hand in this special issue.     

Higher education and the achievement (and/or prevention) of equity and social justice

The article examines the theoretical and empirical literature on higher education’s role in relation to social equity and related notions of citizenship, social justice, social cohesion and meritocracy. It considers both the education and the research functions of higher education and how these impact upon different sections of society, on who benefits and who loses from them. Questions for future research on the wider impact of higher education are posed as well as some research questions on the narrower issue of widening participation.     

Detecting bias in meta-analyses of distance education research: big pictures we can rely on

This article has two interrelated purposes. The first is to explain how various forms of bias, if introduced during any stage of a meta-analysis, can provide the consumer with a misimpression of the state of a research literature. Five of the most important bias-producing aspects of a meta-analysis are presented and discussed. Second, armed with this information, we examine 15 meta-analyses of the literatures of distance education (DE), online learning (OL), and blended learning (BL), conducted from 2000 to 2014, with the intention of assessing potential sources of bias in each. All of these meta-analyses address the question: “How do students taking courses through DE, OL, and BL compare to students engaged in pure classroom instruction in terms of learning achievement outcomes?” We argue that questions asked by primary researchers must change to reflect issues that will drive improvements in designing and implementing DE, OL, and BL courses.