International comparisons of student attainment: some issues arising from the PISA study

This paper raises some methodological concerns about the conduct, analysis and interpretation of results from the Programme for International Student Assessment (PISA) study. While in many respects PISA represents an advance on previous attempts at international comparative assessment studies, it retains certain problematic aspects. The article comments on the restricted nature of the data modelling and analysis, and the resulting interpretations. It points to certain features of the results that raise questions about the adequacy of the data and it stresses the failure to introduce a longitudinal component. The paper makes suggestions for ways in which such studies can be improved.     

How Do Young Children Misbehave in the Grocery Store and in the School? The Preschoolers' Perspective

This research assessed young children's perceptions about what misconduct behaviors peers are likely to commit across two contexts, the school and the grocery store. In addition, participants heard one of two versions in which the protagonist was either a boy or a girl. The participants were 70 preschool children (40 males and 30 females) and ranged in age from 36 to 77 months (M = 57 months). The results showed that a total of 242 non-repetitive behaviors were generated. Most of the behaviors generated either concerned acts having negative consequences to others (i.e., moral transgressions) or violations of social norms (i.e., conventional transgressions). The results also showed that children generated more moral than conventional misbehaviors. Moral acts were expected to occur more often in the school context than in grocery context, whereas social conventional misbehaviors were expected to occur in both contexts. Children described three specific types of moral misbehaviors: physical harm, property violations, and interpersonal violations. Furthermore, children's expectations of peers' misbehaviors were a function of the gender of the character committing the misdeed as well as the story context.     

Francis Galton,measurement, psychometrics and social progress

This paper looks at Galton’s work from the perspective of its influence on subsequent developments in assessment and especially psychometric latent variable models. It describes how Galton’s views on scientific validity and assumptions about data analysis came to be incorporated into later perspectives.     

Methods in School Effectiveness Research∗

This paper discusses the methodological requirements for valid inferences from school effectiveness research studies. The requirements include long term longitudinal data and proper statistical modelling of hierarchical data structures. The paper outlines the appropriate multilevel statistical models and shows how these can model the complexities of school, class and student level data.

     

The Influence of Secondary and Junior Schools on Sixteen Year Examination Performance: A Cross‐classified Multilevel Analysis∗

This paper presents a model for the explanation of examination (GCSE) performance at the age of 16 years in terms of both secondary and junior school attended together with prior achievement measures and certain background factors. Using a cohort of 758 students in 48 junior schools and 116 secondary schools it compares the variation in performance due to secondary schools with that due to junior schools in a multilevel cross‐classified analysis. It shows that the variation among junior schools is substantially larger than that among secondary schools. It also demonstrates that those junior schools with high average achievement scores for the students when they leave junior school also tend to have high average scores for their students at the age of 16. The implications of these findings, if replicated, are profound. They imply that current attempts to measure the ‘effectiveness’ of secondary schools using achievement measured at the start of secondary schooling may be fruitless and they point to the need for school effectiveness research to become involved in very long term studies of schooling, rather than being restricted to a single phase.

     

A Multi‐Level Analysis of School Improvement: Changes in Schools' Performance over Time

The improvement of schools takes place over extended periods of time. Consequently longitudinal studies which track successive cohorts of pupils through their schooling are required if estimates of the extent of improvement are to be established. To date, hardly any studies have collected the necessary data. Those studies which have had appropriate data have tended to emphasise the extent of stability of schools' effectiveness over time rather than the extent of any changes. A shift in conceptual framework is called for if improvements in schools' effectiveness are to be the central focus of concern.

The study is based on three successive cohorts of pupils passing through some 30 English secondary schools. It uses examination results as the outcome measure and includes a prior attainment measure amongst the variables used to control for differences between schools' intakes. A multi‐level strategy for conceptualising and modelling data on schools' changes in performance over time is offered.

In common with earlier studies the research shows that there is a good deal of stability in schools' effectiveness from year‐to‐year; only a small proportion of the schools in the study (between a fifth and a quarter) were improving or deteriorating in terms of their effectiveness. A particularly striking finding of the research was that whilst several schools improved in effectiveness only one initially ‘ineffective’ school did so consistently. The implications of the study for future research on school improvement are discussed.     

The broadcaster's responsibility for advertising

Far too often, one comes upon people who are unwilling or unable to tear down the artificial walls between “academic research” and “practical broadcasting.” Naturally, many of the most fruitful researchers recognize the advantages of tying their work to the “real world,” and many of the most efficient broadcasters rely strongly upon rigorous research rather than “seat of the pants” intuition. In the following article yet another link between research and broadcasting is demonstrated: the use of research, conducted by or on behalf of a station, to provide a wealth of newsworthy stories for the broadcaster. Whereas most journal articles emphasize the substantive results of a research, here they serve primarily as examples, with emphasis on reporting and the application of research techniques in the newsroom.

Dr. McCombs is an associate professor of journalism in the University of North Carolina at Chapel Hill. Mr. Waters is assistant news manager of station WBTV in Charlotte, North Carolina.