Evaluation: judgemental or developmental?

An increasing number of studies are concerned with the international comparison of the results that different education systems achieve. For example, TIMSS, PIRLS and PISA publish the results from different countries in mathematics, science and literacy. The European Commission has also published indicators and benchmarks relating to the quality of school education. This type of external evaluation is often perceived by teachers as judgemental and controlling. In contrast, internal school‐based evaluation aims to be seen as a developmental process contributing to improved teacher and student learning, where ownership of the evaluation processes remains with the school stakeholders. This paper examines the possible tensions and conflicts that may arise between these two types of evaluation and asks the question ‘How can we overcome the negative attitudes that may have resulted from external evaluation in order to encourage schools to undertake internal school‐based evaluation?’ The paper explores, using Scottish case studies, ways in which schools through their internal self‐evaluation processes may use the data in different ways to meet some, if not all, of the external demands placed on them. This strategy, it is argued, ensures that evaluation is largely done for school purposes and as an aid to the learning process, rather than as an external assessment designed to hold schools accountable for the learning achieved to date.

Il y a un nombre grandissant d'études qui s'occupent de la comparaison internationale des résultats atteints par de différents systèmes éducatifs. Par exemple, TIMSS, PERLS et PISA publient les résultats de différents pays en maths, en sciences et en alphabétisation. La Communauté européenne publie aussi des indicateurs et des standards qui se rapportent à la qualité d'éducation dans les écoles. Ce genre d'évaluation externe est souvent perçu par les enseignants comme un moyen de jugement et de contrôle. Par contraste, les évaluations internes, basées et créés dans les écoles essaient d'établir un processus de développement qui contribue à améliorer l'apprentissage des enseignants et des étudiants et qui es la copropriété des preneurs de l'école. Cet article examine les tensions et les conflits qui peuvent se présenter entre les deux genres d'évaluation et pose la question: “Comment peut‐on combattre les attitudes negatives qui résulteraient après des évaluations externes et comment peut‐on par la suite encourager les écoles à entreprendre leur propre évaluation externe?”Plus particulièrement, en utilisant des études de cas de quelques écoles en Ecosse, l'article explore aussi les moyens par lesquels les écoles peuvent utiliser les données obtenues par les processus d'auto‐évaluations internes afin d'adresser au moins quelques‐unes des exigences externes qui s'imposent à l'école. Cette strategie, l'on soutient, garantit que l'évaluation bénéficit en grande partie aux méthodes d'enseignement à l'école au contraire des évaluations externes qui généralement poussent les écoles à rendre des comptes.

Eine zunehmende Anzahl von Untersuchungen befaßt sich mit dem internationalen Vergleich der Leistung, die in verschiedenen Ausbildungssystemen erzielt wird. Zum Beispiel veröffentlichen TIMSS, PIRLR und PISA die Ergebnisse von verschiedenen Ländern in den Bereichen Mathematik, Naturwissenschaften und Lesen/Schreiben. In Bezug auf die Qualität schulischer Ausbildung hat die Europäische Kommission auch Indikatoren und Anforderungsmaßstäbe veröffentlicht. Diese Art von externer Evaluation wird von Lehrern oft als bewertend und als ein Element von Steuerungsmaßnahmen angesehen. Im Gegensatz dazu sollte die interne schulbezogene Evaluation, deren Kontrolle bei den Betroffenen in der Schule verbleibt, als Teil eines Entwicklungsprozesses aufgefaßt werden, der auf die Verbesserung des Lernens von Lehrern und Schülern abzielt. Dieser Beitrag untersucht die möglichen Spannungen und Konflikte, die sich aus den beiden verschieden Evaluationsansätzen ergeben. Es wird die Frage aufgeworfen: “Wie können wir die negativen Einstellungen, die mit externer Evaluationen leicht verbunden sind, überwinden, um damit die Schulen bereit zu machen, interne schulbezogene Evaluationen durchzuführen?” Auf der Grundlage von schottischen Fallstudien zeigt der Beitrag Möglichkeiten auf, wie Schulen im Rahmen von internen Selbstevaluationsprozessen Daten verschiedenster Art nutzen können, um zumindest einigen Anforderungen, die an sie von außen gestellt werden, genügen zu können. Die Verfasser gehen davon aus, daß eine solche Strategie sicherstellt, daß die Evaluation im wesentlichen für schulische Zwecke durchgeführt wird: Daß sie vor allem den weiteren Lernprozeß unterstützt und weniger einer externen Beurteilung dient, die darauf ausgerichtet ist, Schulen für ihre bisherigen Lernleistungen zur Verantwortung zu ziehen.

Ultimamente ha habido un incremento importante de estudios sobre los resultados que han obtenido diversos sistemas educativos en comparaciones internacionales. Por ejemplo, TIMSS, PERLS y PISA publican los resultados de diversos países en matemáticas, ciencias y alfabetización. La Comisión Europea también ha publicado indicadores y bases de datos, referidos a la calidad de la educación de las escuelas. Este tipo de evaluación externa a menudo es percibido por los profesores como un control y una crítica a su labor. En contraste, la evaluación interna realizada desde la escuela, apunta a ser considerada como un proceso de desarrollo profesional que contribuye a la mejora del profesorado y al aprendizaje de los alumnos, y donde el propietario del proceso de evaluación es el mismo centro educativo. Este documento examina las posibles tensiones y los conflictos que pueden darse entre estos dos tipos de evaluación y se pregunta, “¿cómo podemos superar las actitudes negativas que pudieron originar la evaluación externa de manera que se pueda animar a las escuelas a emprender una evaluación interna desde la escuela? El documento explora, usando estudio de casos escoceses, las maneras en las cuales las escuelas a través de sus procesos de autoevaluación interna pueden utilizar los datos de diversas maneras y dar respuesta a algunas, sino a todas, las demandas que reciben del exterior. Esta estrategia, se dice, que asegura que la evaluación esté hecha en gran parte para el propósito de la escuela y como ayuda al proceso de aprendizaje más que un examen externo diseñado para que las escuelas rindan cuentas sobre los resultados que han obtenido.     

Identifying sex differences in reading disability

The issue of sex differences in reading disability has been of recent interest in relation to sex ratios in families with reading disabled children and to possible sex biases in referred populations. Data from a study of 570 twins are used to develop alternative definitions of reading disability that vary in the manner to which sex effects are taken into account. These definitions include discrepancies between reading quotients and IQ, the use of the regression of reading onto IQ and chronological age/reading age differences. In each case the reading and spelling disability was defined either separately for the sexes or based upon the data for the sexes combined and with and without an IQ>90 exclusion criterion. The consequences of using the alternative definitions for prevalence, sex ratio and heritability are examined. The results demonstrate that the characteristics of reading disabled children vary with the way disability is defined. The excess of males seems to be a robust finding. Definitions that take into account differences in mean score for males and females reduce but do not eradicate the sex ratio. From the genetic analysis, there is no support for the suggestion that the genetic effect on reading is greater for females than males. It is concluded that the use of regression based procedures for identifying reading disability is desirable but that at present there is insufficient evidence to justify the adoption of separate regression procedures for the two sexes.     

Do parents make a difference to children’s academic achievement? Differences between parents of higher and lower achieving students

Differences in family factors in determining academic achievement were investigated by testing 432 parents in nine independent, coeducational Melbourne schools. Schools were ranked and categorized into three groups (high, medium and low), based on student achievement (ENTER) scores in their final year of secondary school and school improvement indexes. Parents completed a questionnaire investigating their attitudes towards the school environment, their aspirations, expectations, encouragement and interest in their child’s education (adapted from scales constructed by Marjoribanks). They also responded to six open‐ended questions on their attitudes to achievement and to their (child’s) school. Multiple regression analyses revealed that parental expectations of their children’s educational level made the strongest unique prediction of high achievement followed by the length of time they had maintained their expectations. Limitations discussed include the disparity in meaning associated with the definition of school success and whether these results can be generalized to all students considering the biased sample (socio‐economic status).     

Impact of sustained professional development in STEM on outcome measures in a diverse urban district

Sustained professional development can support STEM (Science, Technology, Engineering, and Mathematics) reform. The authors describe a 3-year study of sustained professional development for 3 diverse urban schools across the salient factors of fidelity of implementation of project-based learning, development of professional learning communities, and student achievement. Qualitative and quantitative data were collected. The students who experienced the greatest fidelity of implementation exhibited the greatest gains (d = 1.41–2.03) on standardized test scores, while those with the lowest fidelity of implementation exhibited negative gains (d = –0.16 to –0.08). Qualitative data indicated teachers perceived there were multiple benefits from the implementation of project-based learning.     

Spectrum Approach to Mentoring: an evidence-based approach to mentoring for academics working in higher education

Anecdotal and empirical evidence indicates that mentoring can be a successful strategy for supporting professional learning, yet limited literature exists on approaches to mentoring designed specifically for academics working in higher education. The aim of this study was to create an approach to mentoring tailored to the needs of academics and underpinned by evidence from the literature. Participatory action research was chosen as the methodology for the project as this enabled a process of inquiry to be embedded within academic practice. The outcome was the development of the Spectrum Approach to Mentoring (SAM). This three-step approach is goal orientated, and encompasses mentoring relationships that may be ongoing and sustained over time through to those that are short-term and aligned with a particular task or focus. SAM provides a suite of resources that can be used by academics to promote valuable opportunities for professional learning through the initiation of mentoring relationships.     

Genetic etiology of spelling deficits in the Colorado and London twin studies of reading disability

The basic multiple regression model for the analysis of selected twin data (DeFries and Fulker 1985, 1988) was fitted to spelling data from 100 pairs of MZ twins and 71 pairs of same-sex DZ twins tested in the Colorado Reading Project (DeFries, Olson, Pennington and Smith 1991), and to data from 12 pairs of MZ twins and 15 pairs of same-sex DZ twins tested in the London twin study of reading disability (Stevenson, Graham, Fredman and McLoughlin 1984, 1987). Estimates of h _g ² obtained from analyses of these data suggest that about 60% of the deficit of probands is due to heritable influences in both samples. When a regression model was fitted separately to data from males and females in the combined Colorado and London samples, resulting estimates of h _g ² were 0.66±0.18 and 0.56±0.19, respectively, a nonsignificant difference. Collaborative analyses of data from additional twin studies of reading disability would facilitate more rigorous tests of hypotheses of differential genetic etiology as a function of group membership.     

Assessing Multiliteracies and the New Basics

This paper addresses the skills and characteristics required of successful learners, workers and citizens in the knowledge economy. The authors trace the shifting commercial, technological and cultural conditions characteristic of this economy, and highlight the key qualities now required for individual success. Effective learners will increasingly need to be autonomous and self-directed, flexible, collaborative, of open sensibility, broadly knowledgeable, and able to work productively with linguistic and cultural diversity. While still prevalent, it is held that standardised testing and a 'back to basics' approach to curriculum are unable to promote and measure effectively these skills and sensibilities. Instead, a broader and more creative approach to curriculum and assessment is recommended. A 'new basics' is argued for at the level of curriculum, with correlative assessment techniques such as analysis of portfolios, performance, projects and group work.     

Multicultural Complexity: an Intersectional Lens for Clinical Supervision

The author discusses the need for counselor educators and supervisors to promote the intersectional nature of identities in counselor trainees and supervisees, especially as supervision serves as the vanguard for professional development and provides a more individually tailored space. This article introduces the concept of multicultural complexity, along with its current dearth in the counseling literature. The utility of multicultural complexity is explored, and recommendations made in the form of sample action plans (Appendix) regarding how counselor educators and supervisors can further incorporate the concept into the practice of clinical supervision.