Evaluation: judgemental or developmental?

An increasing number of studies are concerned with the international comparison of the results that different education systems achieve. For example, TIMSS, PIRLS and PISA publish the results from different countries in mathematics, science and literacy. The European Commission has also published indicators and benchmarks relating to the quality of school education. This type of external evaluation is often perceived by teachers as judgemental and controlling. In contrast, internal school‐based evaluation aims to be seen as a developmental process contributing to improved teacher and student learning, where ownership of the evaluation processes remains with the school stakeholders. This paper examines the possible tensions and conflicts that may arise between these two types of evaluation and asks the question ‘How can we overcome the negative attitudes that may have resulted from external evaluation in order to encourage schools to undertake internal school‐based evaluation?’ The paper explores, using Scottish case studies, ways in which schools through their internal self‐evaluation processes may use the data in different ways to meet some, if not all, of the external demands placed on them. This strategy, it is argued, ensures that evaluation is largely done for school purposes and as an aid to the learning process, rather than as an external assessment designed to hold schools accountable for the learning achieved to date.

Il y a un nombre grandissant d'études qui s'occupent de la comparaison internationale des résultats atteints par de différents systèmes éducatifs. Par exemple, TIMSS, PERLS et PISA publient les résultats de différents pays en maths, en sciences et en alphabétisation. La Communauté européenne publie aussi des indicateurs et des standards qui se rapportent à la qualité d'éducation dans les écoles. Ce genre d'évaluation externe est souvent perçu par les enseignants comme un moyen de jugement et de contrôle. Par contraste, les évaluations internes, basées et créés dans les écoles essaient d'établir un processus de développement qui contribue à améliorer l'apprentissage des enseignants et des étudiants et qui es la copropriété des preneurs de l'école. Cet article examine les tensions et les conflits qui peuvent se présenter entre les deux genres d'évaluation et pose la question: “Comment peut‐on combattre les attitudes negatives qui résulteraient après des évaluations externes et comment peut‐on par la suite encourager les écoles à entreprendre leur propre évaluation externe?”Plus particulièrement, en utilisant des études de cas de quelques écoles en Ecosse, l'article explore aussi les moyens par lesquels les écoles peuvent utiliser les données obtenues par les processus d'auto‐évaluations internes afin d'adresser au moins quelques‐unes des exigences externes qui s'imposent à l'école. Cette strategie, l'on soutient, garantit que l'évaluation bénéficit en grande partie aux méthodes d'enseignement à l'école au contraire des évaluations externes qui généralement poussent les écoles à rendre des comptes.

Eine zunehmende Anzahl von Untersuchungen befaßt sich mit dem internationalen Vergleich der Leistung, die in verschiedenen Ausbildungssystemen erzielt wird. Zum Beispiel veröffentlichen TIMSS, PIRLR und PISA die Ergebnisse von verschiedenen Ländern in den Bereichen Mathematik, Naturwissenschaften und Lesen/Schreiben. In Bezug auf die Qualität schulischer Ausbildung hat die Europäische Kommission auch Indikatoren und Anforderungsmaßstäbe veröffentlicht. Diese Art von externer Evaluation wird von Lehrern oft als bewertend und als ein Element von Steuerungsmaßnahmen angesehen. Im Gegensatz dazu sollte die interne schulbezogene Evaluation, deren Kontrolle bei den Betroffenen in der Schule verbleibt, als Teil eines Entwicklungsprozesses aufgefaßt werden, der auf die Verbesserung des Lernens von Lehrern und Schülern abzielt. Dieser Beitrag untersucht die möglichen Spannungen und Konflikte, die sich aus den beiden verschieden Evaluationsansätzen ergeben. Es wird die Frage aufgeworfen: “Wie können wir die negativen Einstellungen, die mit externer Evaluationen leicht verbunden sind, überwinden, um damit die Schulen bereit zu machen, interne schulbezogene Evaluationen durchzuführen?” Auf der Grundlage von schottischen Fallstudien zeigt der Beitrag Möglichkeiten auf, wie Schulen im Rahmen von internen Selbstevaluationsprozessen Daten verschiedenster Art nutzen können, um zumindest einigen Anforderungen, die an sie von außen gestellt werden, genügen zu können. Die Verfasser gehen davon aus, daß eine solche Strategie sicherstellt, daß die Evaluation im wesentlichen für schulische Zwecke durchgeführt wird: Daß sie vor allem den weiteren Lernprozeß unterstützt und weniger einer externen Beurteilung dient, die darauf ausgerichtet ist, Schulen für ihre bisherigen Lernleistungen zur Verantwortung zu ziehen.

Ultimamente ha habido un incremento importante de estudios sobre los resultados que han obtenido diversos sistemas educativos en comparaciones internacionales. Por ejemplo, TIMSS, PERLS y PISA publican los resultados de diversos países en matemáticas, ciencias y alfabetización. La Comisión Europea también ha publicado indicadores y bases de datos, referidos a la calidad de la educación de las escuelas. Este tipo de evaluación externa a menudo es percibido por los profesores como un control y una crítica a su labor. En contraste, la evaluación interna realizada desde la escuela, apunta a ser considerada como un proceso de desarrollo profesional que contribuye a la mejora del profesorado y al aprendizaje de los alumnos, y donde el propietario del proceso de evaluación es el mismo centro educativo. Este documento examina las posibles tensiones y los conflictos que pueden darse entre estos dos tipos de evaluación y se pregunta, “¿cómo podemos superar las actitudes negativas que pudieron originar la evaluación externa de manera que se pueda animar a las escuelas a emprender una evaluación interna desde la escuela? El documento explora, usando estudio de casos escoceses, las maneras en las cuales las escuelas a través de sus procesos de autoevaluación interna pueden utilizar los datos de diversas maneras y dar respuesta a algunas, sino a todas, las demandas que reciben del exterior. Esta estrategia, se dice, que asegura que la evaluación esté hecha en gran parte para el propósito de la escuela y como ayuda al proceso de aprendizaje más que un examen externo diseñado para que las escuelas rindan cuentas sobre los resultados que han obtenido.     

The development of flexibility in equation solving

This paper explores the development of students’ knowledge of mathematical procedures. Students’ tendency to develop rote knowledge of procedures has been widely commented on. An alternative, more flexible endpoint for the development of procedural knowledge is explored here, where students choose to deviate from established solving patterns on particular problems for greater efficiency. Students with no prior knowledge of formal linear equation solving techniques were taught the basic transformations of this domain. After instruction, students engaged in problem-solving sessions in two conditions. Treatment students completed the “alternative ordering task,” where they were asked to re-solve a previously completed problem but using a different ordering of transformations. Those completing alternative ordering tasks demonstrated greater flexibility.     

Conscientious internationalisation in higher education: contextual complexities and comparative tensions

Asia Pacific Education Review - In this paper, authors focus on how internationalisation is defined, interpreted, and responded to by Universities in Australia and Canada, two decades after de...     

Spoken language development in oral preschool children with permanent childhood deafness

This article documented spoken language outcomes for preschool children with hearing loss and examined the relationships between language abilities and characteristics of children such as degree of hearing loss, cognitive abilities, age at entry to early intervention, and parent involvement in children's intervention programs. Participants were evaluated using a combination of the Child Development Inventory, the Peabody Picture Vocabulary Test, and the Preschool Clinical Evaluation of Language Fundamentals depending on their age at the time of assessment. Maternal education, cognitive ability, and family involvement were also measured. Over half of the children who participated in this study had poor language outcomes overall. No significant differences were found in language outcomes on any of the measures for children who were diagnosed early and those diagnosed later. Multiple regression analyses showed that family participation, degree of hearing loss, and cognitive ability significantly predicted language outcomes and together accounted for almost 60% of the variance in scores. This article highlights the importance of family participation in intervention programs to enable children to achieve optimal language outcomes. Further work may clarify the effects of early diagnosis on language outcomes for preschool children.     

An evaluation of the special needs referral process for low-income preschool children with emotional and behavioral problems

The present study is an evaluation of the special needs referral system in a large urban Head Start program. To assess the validity of this system, two assessments were conducted. The first examined a representative sample of 105 children who were formally referred for special needs services. The Early Intervention Screening Profile (EISP) was used to indicate the distribution of problem behaviors of these special needs children. Two independent raters evaluated all cases. Reliable evaluations revealed that all of the referred children displayed speech and language problems, with 70% showing only speech problems and 30% showing speech plus emotional and behavioral problems. The second assessment examined reports of teachers’ concerns about children program-wide by obtaining a representative sample of 203 children nominated by teachers as most likely to be identified as having special needs. Teachers completed a modified EISP. Results indicated that over 70% of the children evidenced emotional and behavioral problems with no accompanying speech difficulties and 30% displayed emotional and behavioral problems plus speech problems. Implications of referral biases were discussed.     

Maltreatment following reunification: Predictors of subsequent Child Protective Services contact after children return home

ObjectivesThis study examined risk of maltreatment among children exiting foster care using a statewide sample of children reunified between 2001 and 2004 in Rhode Island. The objectives were: (1) to compare rates of maltreatment following parental reunification for youth in care as a result of maltreatment with those in care for other reasons; and (2) to assess the effects of child, family, and case characteristics on rates of re-maltreatment among children placed in foster care due to maltreatment.MethodA longitudinal dataset of all reunified cases was matched with state records of substantiated Child Protective Service (CPS) investigations. Two Cox proportional hazards models were tested. The first model compared rates of subsequent maltreatment for two groups: children in foster care as a result of maltreatment, and those in care for other reasons. The second model investigated the effects of child, family, and case characteristics on re-maltreatment rates for those in care as a result of maltreatment.ResultsChildren in foster care due to maltreatment were significantly more likely to be maltreated following reunification. Among children in foster care due to maltreatment, factors that raised risk for re-maltreatment included a previous foster care placement, exiting care from a non-relative foster home, and removal due to neglect. Older adolescents had lower rates of re-maltreatment than infants. Child neglect was the primary type of recurrent maltreatment that occurred following reunification.ConclusionsSupports are needed for families about to be reunified, particularly when the removal was prompted by incidents of abuse or neglect. Incidents of neglect are particularly likely and appropriate services should specifically target factors contributing to neglect. Cases involving youth with a history of repeated foster care placement or in which non-relative placements are utilized may need additional supports.Practice implicationsThis study suggests that services should be developed to minimize the risk for recurrent maltreatment following reunification. Services would be most useful for high-risk cases prior to reunification and during the first year following reunification. Understanding the risks associated with maltreatment will help guide development of appropriate interventions.     

Exploring cultural differences in critical thinking: Is it about my thinking style or the language I speak?

Critical thinking is deemed as an ideal in academic settings, but cultural differences in critical thinking performance between Asian and Western students have been reported in the international education literature. We examined explanations for the observed differences in critical thinking between Asian and New Zealand (NZ) European students, and tested hypotheses derived from research in international education and cultural psychology. The results showed that NZ European students performed better on two objective measures of critical thinking skills than Asian students. English proficiency, but not dialectical thinking style, could at least partially if not fully explain these differences. This finding holds with both self-report (Study 1) and objectively measured (Study 2a) English proficiency. The results also indicated that Asian students tended to rely more on dialectical thinking to solve critical thinking problems than their Western counterparts. In a follow-up data analysis, students' critical thinking was found to predict their academic performance after controlling for the effects of English proficiency and general intellectual ability, but the relationship does not vary as a function of students' cultural backgrounds or cultural adoption (Study 2b). Altogether, these findings contribute to our understanding of the influence of culture on critical thinking in international education.     

Inside Out: Action Research from the Teacher–Researcher Perspective

Teachers enrolled in the master of natural science program for high school science teachers at a large research university must complete a year-long action research study. This account, by the program’s action research coordinator, describes both process and outcomes of this research experience from the perspectives of the research coordinator and the teacher–researchers, shedding light on the organizational learning that takes place, and the ways in which the research experience affected individual teacher–researchers. Teachers reported that their action research experience changed them in fundamental ways, providing them with a framework for deepening their understanding of student thinking, challenging their folk wisdom about teaching and learning, building confidence in their abilities and renewing their commitment to teaching as a vocation.     

Recent commissioned reports on tertiary distance education in Australia: context and critique

Within one year there appeared nine major reports or evaluation studies on Australian distance education — all commissioned by the Australian federal government's former statutory authority advising on tertiary education. The political context in which these studies were initiated is considered first. Then each of the commissioned evaluation studies is examined in turn and finally there is an overall assessment of the likely impact of the reports.