Misconceptions about data-based decision making in education: An exploration of the literature

Research on data-based decision making has proliferated around the world, fueled by policy recommendations and the diverse data that are now available to educators to inform their practice. Yet, many misconceptions and concerns have been raised by researchers and practitioners. To better understand the issues, a session was convened at AERA’s annual convention in 2018, followed by an analysis of the literature based on misconceptions that emerged. This commentary is an outgrowth of that exploration by providing research, theoretical, and practical evidence to dispel some of the misconceptions. Our objective is to survey and synthesize the landscape of the data-based decision making literature to address the identified misconceptions and then to serve as a stimulus to changes in policy and practice as well as a roadmap for a research agenda.     

Factors promoting and hindering data-based decision making in schools

Although data-based decision making can lead to improved student achievement, data are often not used effectively in schools. This paper therefore focuses on conditions for effective data use. We studied the extent to which school organizational characteristics, data characteristics, user characteristics, and collaboration influenced data use for (1) accountability, (2) school development, and (3) instruction. The results of our hierarchical linear modeling (HLM) analysis from this large-scale quantitative study (N = 1073) show that, on average, teachers appear to score relatively high on data use for accountability and school development. Regarding instruction, however, several data sources are used only on a yearly basis. Among the factors investigated, school organizational characteristics and collaboration have the greatest influence on teachers’ data use in schools.     

Effects of a data-based decision making intervention on student achievement

Data-based decision making (DBDM) is becoming important for teachers due to increasing amounts of digital feedback on student performance. In the quasi-experimental study reported here, teachers, principals, and academic coaches from 42 schools were trained for two years in using the results of half-year interim assessments for providing students with tailor-made instruction. Our results did not show any main effects of this DBDM training trajectory on student achievement but did indicate interaction effects with students’ low prior achievement levels and socioeconomic status. Teachers experience difficulties in translating student progress data into adaptive instruction in the classroom. Implications of our findings for teacher professionalization are discussed.     

On the value of data-based decision making in education: The evidence from six intervention studies

Although some studies have investigated the impact of data-based decision making (DBDM) on student achievement, the overall findings are not straightforward, because of the studies’ methodological flaws and their mixed results. This article first presents a breakdown of the DBDM concept as applied in the Dutch context. Next, it explains the theoretical foundations of DBDM in feedback and goal-setting theory and then discusses various factors influencing DBDM effectiveness. The results of six Dutch DBDM interventions with an explicit focus on student achievement effects of DBDM and with strong research designs are then presented. Significant positive effects on student performance as measured by means of standardized tests are reported for four interventions. The interpretation of student progress data from student monitoring systems does not seem to be problematic for teachers, if they are deliberately trained for it, but teachers do find it difficult to translate student progress data into tailor-made instruction.     

School characteristics influencing the implementation of a data-based decision making intervention

There is an increasing global emphasis on using data for decision making, with a growing body of research on interventions aimed at implementing and sustaining data-based decision making (DBDM) in schools. Yet, little is known about the school features that facilitate or hinder the implementation of DBDM. Based on a literature review, the authors identified 12 potentially critical factors for the successful implementation of DBDM. Following an intensive DBDM intervention, 16 schools’ characteristics were studied and related to the success of DBDM implementation after participating in an intensive DBDM intervention for 2 subsequent academic years. Strong instructional leadership, maximum exposure to the intervention, standardization of work processes, as well as staff continuity and a strong academic coach appear to be strongly related to implementation success. The remaining school context features and the culture and structure of the schools were not associated with the success of DBDM implementation.     

The effects of professional development on the attitudes,knowledge and skills for data-driven decision making

The Dutch government and School Inspectorate encourage schools to use the student performance data they can obtain from their student monitoring systems to maximize student performance in a systematic and goal-oriented way. Research by the same Inspectorate (Inspectie van het Onderwijs, 2010) shows that students in schools which do so outperform students in schools where data-driven decision making (DDDM) is as yet less developed. The University of Twente developed a training course in which school teams learn to utilize data from computerized student monitoring systems in order to improve instructional quality and student performance. Parallel to the training activities, training effects are studied. The research findings show that the training activities had a positive effect on school staff's DDDM knowledge and DDDM skills. Staff attitudes towards DDDM were already high on the pre-tests and remained high on the post-tests.     

Teachers learning how to use data: A synthesis of the issues and what is known

This article provides a synthesis of the articles found in the special issue on data use. The synthetic piece contextualizes how the articles contribute to the knowledge base of how teachers use data. It synthesizes the findings by identifying key common themes. It then describes gaps in the current knowledge base and identifies the requisite steps needed to address those gaps.     

What does it mean for teachers to be data literate: Laying out the skills,knowledge, and dispositions

This article reports on the evolution of a conceptual framework for a construct called data literacy for teachers. Data use has become an emphasis in education but few educators have received sufficient training or preparation pertaining to data literacy skills. This article lays out the framework, identifying the specific knowledge, skills, and dispositions teachers need to use data effectively and responsibly. It concludes with a call to schools of education and teacher preparation programs to begin to integrate data literacy into curricula and practical experiences.     

Exploring data use practices around Europe: Identifying enablers and barriers

In this article we explore what data-based decision making use looks like in schools in five different countries (United Kingdom, Germany, Poland, Lithuania and the Netherlands). We explore for what purposes data are used in these countries and what the enablers and barriers to data use are. The case study results show that schools in all five countries use data for school development, accountability, and instructional improvement. Also, the schools in the five countries struggle with the same type of problems: e.g. lack of access to high quality data, lack of professional development in using data, and a lack of collaboration around the use of data. Finally, we discuss how some enablers can turn into barriers for effective data use.     

The development of data use,data skills,and positive attitude towards data use in a data team intervention for teacher educators

This study focuses on understanding how a participation in a data team develops data skills and data use in individual teacher educators. Five teacher educators collaborated in a data team that used data to solve the problem of student teachers dropping out during their course of study. This study aimed at understanding how teacher educators learn from their participation in the data team. We collected data through interviews, surveys, and a knowledge test, and gained insight into the development of data skills, attitude towards data use, and the teacher educators' data use in daily practice. The results show that the data team members’ data skills and attitudes towards data use changed in different ways during the data team intervention depending on their initial situation, and that overall, their data use for school development increased.     

Implementation of a collaborative data use model in a United States context

Collaborative data use requires educator capacity in moving data to action to address root causes of student underperformance. Implementation of the model used in the present study has proved promising in European countries for building educator capacity and addressing problems-of-practice, but this model has not been studied in the United States (US), where structural issues and accountability factors present different challenges. In the present study, we explored enabling and hindering factors of the implementation in an elementary school in the US to better understand how differences in policy and practice contexts influence collaborative data use. Organizational structures and some policies in the US hindered implementation. Drawing on our findings, we suggest shifting thinking around data use to accommodate for both short cycles of data use (for straightforward problems) and intentionally slow cycles for stickier problems; furthermore, nesting collaborative data use within high-priority initiatives may help mitigate barriers to future implementations.     

Data-based decision making for teacher and student learning: a psychological perspective on the role of the teacher

Data-based decision-making has the potential to increase student achievement results. Data-based decision-making can be defined as teachers’ systematic analysis of data sources in order to study and adapt their educational practices for the purpose of maximizing learning results. Teachers must apply the findings from their data use to their personal teaching activities. Therefore, data-based decision-making may be influenced by individual teachers' psychological characteristics. The present study aimed to explore which psychological factors contribute to teachers’ data use in a Dutch primary school context. A questionnaire-based quantitative methodology was employed. We included the following psychological constructs: affective and instrumental attitudes, perceived control, social norms, self-efficacy, collective efficacy, and intentions regarding data use. Results of the path analysis showed that perceived control, instrumental attitude, and intention regarding data use all significantly influenced data use. Additionally, intention was found to be a mediator of the relation between affective attitude and data use. Interventions aimed at data-based decision-making should take these psychological factors into account to increase teachers’ implementation of data-based decision-making for instruction and, consequently, educational quality.     

Data-informed curriculum reform: Which data,what purposes,and promoting and hindering factors

Schools face a lot of data on the functioning of their school which they can use to make improvements in teaching, learning and the organization. For data use to lead to improvement, it is important to further research the concept data-driven decision making. The results of this explorative study in the Netherlands show that teachers mainly use classroom level data for making instructional decisions at classroom level, and school leaders mainly use school level data for policy development decisions. This article ends with suggestions with regard to enhancing the effectiveness of data-driven decision making, for example by stressing the importance of developing teachers' competence in the use of data.     

职业决策研究的新进展——PIC模型

职业决策研究是国内外职业心理学研究的难点。PIC模型是由以色列职业心理学家Gaff提出的一种系统的职业决策方法，其构建兼顾理论验证与实践运用。详析这一模型提出的理论基础及操作阶段，对团体和个人职业决策有重要帮助。     

Professional Development in the Use of Data: From Data to Knowledge in Data Teams

Schools need support in the use of data. To provide this support, a data team intervention was developed. A prior study conducted in the Netherlands showed that several factors can enable or hinder the work of data teams. The current replication study focuses on the factors influencing data use in data teams and the perceived effects of the data teams’ work, but looking at data teams in Sweden. The results of this qualitative study show that the data teams’ work is influenced by the same factors as in the Netherlands: Data characteristics (e.g., relevance of the data), team characteristics (e.g., heterogeneity of the team), and school organizational characteristics (e.g., school leader support).     

Computer-based tools for instructional design: An introduction to the special issue

Modern instructional theories are characterized by their focus on rich, multidisciplinary and often collaborative learning tasks that are somehow representative for authentic, real life tasks. This new view on learning heavily increases the complexity of the design process and the resulting instructional systems. It is argued that computer-based instructional design (ID) tools may help to deal with this growing complexity. A framework to distinguish different kinds of ID tools is presented. This framework is then used to introduce the contributions to this special issue.     

An introduction to the special issue on wisdom and moral education

ABSTRACT

This essay introduces the present special issue on wisdom and moral education, which draws on a conference held in Oxford in 2017. Some of the seven contributions (by Sanderse; Ferkany; and Hatchimonji et al.) make use of the Aristotelian concept of phronesis, or practical wisdom, while others focus more on the wisdom concept as it has developed in contemporary psychology (Huynh and Grossman; Ardelt; and Brocato, Hix and Jayawickreme). One (by Swartwood) straddles the distinction between the two. All the contributions, however, address in different ways practical questions about how wisdom can be evaluated and how it relates to issues of moral development and education.     

Complexity and then some: Theories of action and theories of learning in data-informed decision making

This article presents a synthesis of the complexity of using data in education and highlights some of the promise and challenges of adequately understanding data use across the continuum of accountability to instructional influence in the classroom. Several common themes around the complexity of data use were identified: (1) The nature of the data used, (2) the importance and difficulty of educator data capacity building and sense-making, (3) the importance of characteristics of collaboration, (4) the influence of contextual factors of schools, districts and international settings, (5) the potential negative influence of data use in equity considerations, and (6) the difficulty of determining impact of data use and discerning explanations of why data use initiative work some of the times were examined across the articles. This synthesis concludes with a call for an approach to research in data use in education that concurrently engages multiple aspects of the educational system with a focus on multiple levels of theories of learning that address students, teachers, and organizations.     

The language of written language: an introduction to the special issue

This special issue includes five articles dealing with selected aspects of writing disorders, crafted by key contributors to the literature on written language. Articles addressing definitional issues and diagnostic questions, assessment strategies, selected neurodevelopmental components of written expression, the manifestation of writing problems in children and young adults, direct intervention, training for elementary teachers in the early grades in the area of literacy, and brain-behavior linkages are presented. This introduction provides an overview of these articles tied to specific issues confronting the field of written language. Some thoughts and directions for the future are articulated.     

Listening and reflecting: an introduction to the special issue

The topic of listening has been broadly neglected in educational studies. This special issue presents studies on the connections between listening and reflecting in educational contexts.