The interpretation of norm-based scores from individualized testing using the Iowa tests of basic skills

School teachers and administrators are often faced with the dilemma of deciding what level of an achievement test to assign a child whose developmental rate is atypical of his peers. It is not attractive to mismatch the developmental and achievement test level; however, alternative procedures often call for extra testing time. The ITBS has an out-of-level option which allows for a developmental/achievement test level match that does not require additional testing. Since the procedures used by ITBS to assign grade equivalent scores do not take grade level into account, questions have been raised about the interpretation of grade equivalent scores achieved from out-of-level testing. This research addresses the question of the comparability of equal scores on the same test from children in different grades. The results indicate that the scores are comparable and support the assignment of ITBS levels that match the child's developmental level.     

Operationalization of Bachman’s model via a multimodal reading comprehension test: Screening test method facets and testees’ characteristics

Test performance is a function both of the test takers’ personal attributes and of the test method facets. However, much of the previous research has addressed the covariates of assessment preferences of pupils rather than those of their actual performances. Following the microsystem perspective and as part of a larger project, this study was set out to detect the learner factors and linguistic parameters which mediate performance on different test formats. A number of language learners responded to the group embedded figures test, willingness to communicate scale, Michigan proficiency test, and a reading comprehension test battery. Based on the previous empirical research, a hypothetical model was designed and tested using structural equation modeling. The findings were as follows: (a) Performance on controlled and constructed response tests is substantially mediated by testees’ characteristics (cognitive source); (b) Target ability (linguistic source) is the most significant determinant of performance on free-response tasks.     

On the meaning of cross-cultural differences in simple cognitive measures

A set of 5 reaction time tests of increasing cognitive complexity were administered to 35 secondary school pupils in Zimbabwe and The Netherlands at 4 consecutive school days in order to explore the existence and nature of cross-cultural differences on reaction time tests measuring basic cognitive operations. No cross-cultural differences were observed on the simple tests, while the Dutch pupils were faster than Zimbabwean pupils on the more complex tests. Zimbabwean pupils showed larger performance increments at retesting. Two Wechsler Intelligence Scale for Children (WISC) subtests (Vocabulary and Digit Span Forward and Backward), a tapping test, and parental socioeconomic status were used to predict the cross-cultural performance differences. Socioeconomic status and digit span could explain all these differences. It is argued that the cross-cultural performance differences on elementary cognitive tests cannot be interpreted at face value because of their susceptibility to non-target factors, such as test understanding and previous test exposure.     

Is an intervention using computer software effective in literacy learning? A randomised controlled trial

Background: computer software is widely used to support literacy learning. There are few randomised trials to support its effectiveness. Therefore, there is an urgent need to rigorously evaluate computer software that supports literacy learning.

Methods: we undertook a pragmatic randomised controlled trial among pupils aged 11–12 within a single state comprehensive school in the North of England. The pupils were randomised to receive 10 hours of literacy learning delivered via laptop computers or to act as controls. Both groups received normal literacy learning. A pre‐test and two post‐tests were given in spelling and literacy. The main pre‐defined outcome was improvements in spelling scores.

Results: 155 pupils were randomly allocated, 77 to the ICT group and 78 to control. Four pupils left the school before post‐testing and 25 pupils did not have both pre‐ and post‐test data. Therefore, 63 and 67 pupils were included in the main analysis for the ICT and control groups respectively. After adjusting for pre‐test scores there was a slight increase in spelling scores, associated with the ICT intervention, but this was not statistically significant (0.954, 95% confidence interval (CI) – 1.83 to 3.74, p = 0.50). For reading scores there was a statistically significant decrease associated with the ICT intervention (?2.33, 95% CI ?0.96 to ?3.71, p = 0.001).

Conclusions: we found no evidence of a statistically significant benefit on spelling outcomes using a computer program for literacy learning. For reading there seemed to be a reduction in reading scores associated with the use of the program. All new literacy software needs to be tested in a rigorous trial before it is used routinely in schools.     

Some Bases Used by Students in Choosing Partners in the Chemistry Laboratory

Abstract

The effects of mobility upon performance on standardized group tests of mental ability of disadvantaged children in 18 elementary schools in New York City were assessed.

Among nontransient pupils there was no significant difference between the mean Otis Alpha test score at the third grade and the mean Otis Beta score which was taken three years later at the sixth grade. However, the Otis Alpha scores of these nontransients were significantly higher than that of their third-grade transient classmates. Also, the Otis Beta scores of these same non transients were significantly better than that of their sixth-grade classmates.

This study emphasized differences in the test performance of transient and nontransient pupils in a school population of disadvantaged children and pointed up the need for separate longitudinal analyses of these two groups.     

Assessing pupils’ intelligence through self,parental, and teacher estimates

This study examined the accuracy of self‐ and other‐estimated intelligence in relation to tested cognitive ability and gender. Three groups of raters were examined: 187 (102 male, 85 female; mean age 14.33 years, SD = .32) pupils of single‐sex comprehensive schools, 109 (55 mothers and 54 fathers) parents, and six teachers of the pupils. Pupils estimated their own overall IQ, while their parents and teachers estimated the pupils’ overall, mathematical, spatial, and verbal abilities. Self‐ and other‐estimates were compared to each other, and to the child’s psychometric test scores in verbal, quantitative, and figural/non‐verbal reasoning ability. Results suggested that participants were reasonably accurate at estimating pupils’ intelligence – teachers significantly more so than parents, and pupils significantly more so than fathers. Although both parents significantly overestimated their child’s IQ, this overestimation was more pronounced in fathers.     

Differences in Effectiveness Between Primary Schools and their Impact on Secondary School Recommendations

In school effectiveness research differences between schools are mainly expressed in percentages of the variance in cognitive test scores. This article focuses on the impact these differences in effectiveness have on the school career of pupils. The analyses were executed on the data of a school effectiveness study carried out in 1987 and 1988. Two groups of outlier primary schools were selected. The pupils were divided into four categories based on their IQ-score, and into four categories based on their SES-score. Four dependent variables were used: test scores on language, test scores on mathematics, expectations for the follow-up secondary school type based on these test scores, and the actual secondary school recommendation given by the teacher. For all groups of pupils the average scores on mathematics, language and the total standard test score differ significantly for both low- and high-effective schools. The actual recommendation for type of secondary education does not differ significantly. This leads to the conclusion that the recommendations given to the other pupils in the same grade seem to be important; schools tend to recommend a secondary school type according to a normal distribution. High-effective schools tend to recommend beneath, while low-effective schools tend to advise above the expectations.     

The relationship between attitudes toward science and science achievement

This study investigated the relationship between elementary pupils' attitudes toward science and their science achievement. Residualized gain scores were used to analyze the data. By using residualized gain scores, the effects of individual differences can be minimized. In addition to controlling for these differences, residualized gain scores do not possess the measurement errors that are normally associated with simple change scores. The subjects of this study were 583 intermediate elementary pupils. The average class size was 21. A pretest-posttest design was used. To insure consistency in the teaching of the lesson, each teacher was given an identical detailed science lesson that included all the instructions and materials needed for the activity. The pupils were pre- and posttested. The pupils' science achievement was assessed by a test, the “Hough Pupil Process Test.” It consisted of multiple choice and fill-in questions. The attitude instrument, the “Hough Attitude Inventory,” was given to the elementary pupils involved in this study. It was field tested and found to discern attitudes. The instrument consisted of six statements to which the subjects responded by circling either yes, I don't know, or no. The analysis revealed that there was a significant relationship between the pupils' residualized gain scores on the “Hough Pupil Process Test” and their residualized gain scores on the “Hough Attitude Inventory” (r = 0.45).     

English language acquisition and educational attainment at the end of primary school

This paper analyses the national key stage 2 test results for 2300 11‐year‐old pupils in an inner London LEA. A range of concurrent pupil background data was also collected, including whether pupils spoke English as an additional language (EAL), and if so, their stage of fluency in English. EAL pupils at the early stages (1–3) of developing fluency had significantly lower KS2 test scores in all subjects than their monolingual peers. However, EAL pupils who were fully fluent in English achieved significantly higher scores in all KS2 tests than their monolingual peers. The negative association with attainment for the early stages of fluency remained significant after controls for a range of other pupil characteristics, including age, gender, free school meal entitlement, stage of special educational need and ethnic group, although these factors effectively explained the higher attainment of the ‘fully fluent’ group. We conclude that EAL is not itself a good guide to levels of attainment, and a measure of stage of English fluency is necessary to interpret associations with test performance. Alternative measures which focus only on the very early stages of English proficiency, such as the QCA ‘language in common’ steps, are inadequate to assess the impact of bilingualism for all but the very earliest learners of English. Given the uneven distribution of EAL pupils across the country, those schools and local education authorities with high concentrations of pupils in the early stages of learning English are likely to be adversely affected in school achievement and attainment tables. The policy implications for national data collection and for the use of such data are considered.     

An intervention program for promoting deaf pupils' achievement in mathematics

Our past research identified two aspects of deaf children's functioning that places them at risk for underachievement in mathematics. The first is their reduced opportunities for incidental learning, and the second is their difficulty in making inferences involving time sequences. This article examines the effectiveness of an intervention program to promote deaf children's numeracy that was designed to deal with these two factors. The design involved a comparison of 23 deaf pupils participating in the project with a baseline group formed by 65 deaf pupils attending the same schools in the previous year. The project pupils were tested before and after the intervention on the NFER-Nelson Age Appropriate Mathematics Achievement Test. The intervention was delivered by the teachers during the time normally scheduled for mathematics lessons. The project pupils did not differ from the baseline group at pretest but performed significantly better at posttest. They also performed at posttest better than expected on the basis of their pretest scores, according to norms provided by the NFER-Nelson Age Appropriate Mathematics Test for assessing the progress of hearing pupils. We conclude that the program was effective in promoting deaf pupils' achievement in numeracy.     

The impact of test language on PISA scores. New evidence from Wales

In this paper we investigate the link between the language in which pupils take the Programme for International Student Assessment (PISA) test and the scores they achieve in this assessment, focusing upon the case of Wales. Using five rounds of PISA data and an instrumental variable approach, we show how pupils who took the test in Welsh score around 0.3 standard deviations (30 PISA test points) lower in reading, mathematics and science than their peers who took the test in English. This finding is robust to different model specifications and statistical approaches. We argue that this may indicate that the academic performance of teenagers in Wales may be underestimated in PISA – particularly amongst those who take this test in Welsh.     

Does living with a female head-of-household affect the arithmetic achievement of black fourth-grade pupils?

It was the purpose of this investigation to examine the effects of living with a male or female head-of-household upon the arithmetic performance of fourth graders. No significant differences were found in mean scores between male and female pupils. Significant differences were found favoring those pupils living with male heads-of-households, particularly for female pupils and for pupils with IQ scores greater than 100. These findings should not be considered as simple cause and effect relationships, but rather as indicative of potential difficulty for some pupils.     

When and why do initially high-achieving poor children fall behind?

In this paper, we examine the trajectories of initially higher- and lower-achieving children from lower and higher socio-economic status families from primary school through to university in England for the first time. We also explore what explains these trajectories. This enables us to provide new insights into when and why the performance of children with similar initial achievement diverges on the basis of their socio-economic background. Our results indicate that pupils from poor backgrounds who are higher achievers in primary school fall behind their better-off but lower-achieving peers during secondary school. This suggests that secondary school may be a critical period to intervene to prevent poor children from falling behind their richer peers. Our analysis suggests that there is less divergence in performance between pupils from different socio-economic backgrounds who attend the same schools. This result is particularly strong for children with low initial achievement. While we remain cautious about the implications of these findings, they provide suggestive evidence that schools (or the sorting of pupils into schools) play an important role in explaining why the test scores of richer and poorer children diverge over time.     

民族地区小学生数学学习态度调查研究

本研究以民族地区194名少数民族和汉族小学生为被试,使用问卷法对小学生数学学习态度的现状进行调查,结果表明：1．少数民族学生对“数学用处”和“得到教师的关注少”的认识比汉族学生低;2．四年级数学态度得分最高,五、六年级出现下降趋势;3．女生数学态度均分高于男生,但男女生的数学成绩没有差异;4．数学态度在“对教师的看法”上存在地域差异,市区学生得分优于郊区和山区学生;5．数学学习成绩好的,其数学态度得分较高。数学态度与数学学习成绩是双向影响的,这在民族地区的数学教学中应给予充分重视．     

A comparative judgement approach to the large-scale assessment of primary writing in England

ABSTRACT

Writing assessment is a key feature of most education systems, yet there are limitations with traditional methods of assessing writing involving rubrics. In contrast, comparative judgement appears to overcome the reliability issues that beset the assessment of performance assessment tasks. The approach presented here extends previous work on comparative judgement by directly involving teachers in a large number of schools in the judging of young pupils’ writing. To ensure quality control the process incorporated a process of ‘anchoring’ that ensured that teachers could not artificially inflate their own pupils’ scores. The approach was used to assess the writing of 55,599 primary pupils in England in 2017–2018. Overall, the results showed that a comparative judgement approach to writing incorporating anchoring shows promise in providing a fair and robust large-scale method to assess writing.     

What Is Happening with Portable Computers in Schools?

This study was an exploratory of the Landes initiative to supply all pupils and teachers with portable computers and Internet at school. These surveys involved 817 pupils and parents and produced a response rate of 98% for pupils from the first survey at the beginning of the year and 798 pupils at the end of the year. The approach, which consists of identifying any variances in individual perceptions of technology, was tested in an initial In-Tele research programme in which we were involved. The questionnaire-based survey reveals low usage of information technology in terms of teaching, with no difference between the portable and test groups. Moving on from the first level study, characterizing the sites and studying their viewing frequency produces the surprising outcome that the very high number of sites viewed actually conceals a much more pessimistic reality about pedagogical uses.     

The effect of standardized test information on teachers’ assessment of ability in their classes

Abstract

The responses of three groups of teachers to a rating task in which they were asked to indicate how many of their pupils were “bright”, “above average”, “below average” and “dull” were compared. In two of the groups, teachers had been provided with test information based on performance on nationally standardized ability and attainment tests. In the third group tests had been administered but no results were provided.

No differences between the groups were found in terms of the extent to which teacher ratings of ability levels correspond with mean ability test scores. In addition, teachers, irrespective of the group to which they belonged, were found to display a tendency to place more pupils in the above average categories than the below average categories. Finally, no support was found for an hypothesis which suggested that test information would differentially affect the ratings of teachers of classes with pupils who were typical and untypical with respect to age.

The fact that the correlation between mean ratings and mean test scores were found to be fairly high in all three groups (they ranged from .51 to .60) suggests a reason for the failure of test information to impact on teachers’ judgements. The degree of agreement between teachers and tests that the correlations reveal means that there is less scope for a convergence of ratings on tests to occur than might otherwise be the case.     

Self-efficacy for self-regulated learning, academic self-efficacy, and internet self-efficacy in web-based instruction

Effects of student motivation on performance in Web-based instruction (WBI) were examined. In particular, applicability of the self-efficacy theory to WBI contexts was tested. A total of 152 junior high school students in Seoul, Korea, participated in WBI during regular science classes. Participants completed motivational surveys before the onset of WBI and took the written and search tests at the end of WBI. Path analyses revealed that students' self-efficacy for self-regulated learning positively related to their academic self-efficacy, strategy use, and Internet self-efficacy. Academic self-efficacy predicted students' performance on the written test, which comprised problems on topics covered during the previous WBI sessions. Students' scores on the WBI search test were significantly and positively predicted by their self-efficacy in using the Internet. More interesting, students' academic self-efficacy beliefs were not able to predict their search test performance, whereas students' Internet self-efficacy beliefs were not able to predict their written test performance.This research is based on data collected as part of Ha-Jeen Choi's master's thesis completed at Ewha Womans University.The authors thank Steven Ross and three anonymous reviewers for their helpful comments on earlier versions of this article.     

Do migrant girls perform better than migrant boys? Deviant gender differences between the reading scores of 15-year-old children of migrants compared to native pupils

In this paper, we analyse the gender differences between the educational performance of 15-year-old children of migrants from specific regions of origin countries living in different destination countries with the Programme for International Student Assessment (PISA) 2009 wave. We study whether this gender difference of migrant pupils deviates from the gender difference between the native pupils in their destination country. We analyse the educational performance of 16,612 daughters and 16,804 sons of migrants in destination countries across Asia, Europe, Latin America, and Oceania. We distinguish 62 origin countries and 12 origin areas in 30 destination countries. Female migrant pupils have both higher reading and math scores than comparable male migrant pupils, and these gender differences among migrant pupils are larger than among comparable native pupils. Parental socioeconomic background has an equal effect on the educational performance of daughters of migrants and on that of sons of migrants. The variation in educational performance by region of origin is, however, not clearly related to the poverty or traditionalism of regions.     

ACHIEVEMENT TEST RELIABILITY AS A FUNCTION OF PUPIL-RESPONSE PROCEDURES

Muller, Calhoun, and Orling (1972) conclude that test reliability is dependent on the type of answer document used by elementary pupils. The present study was designed in part to assess the differential effect of two pupil response procedures (answering directly in the test booklet versus on a separate answer folder) on Metropolitan Achievement Tests scores of grades 3 and 4 pupils.
Over 4000 pupils from nine school systems took the Metropolitan , half responding in their booklets and half using answer folders. The two groups were matched by grade in general scholastic aptitude.
Although the separate answer folder group received lower scores than did the group responding in the test booklets, the score reliabilities did not differ significantly for any test. Additionally, these reliabilities did not differ significantly from comparable Metropolitan normative reliabilities. For survey achievement tests such as Metropolitan , test reliability would not appear to depend on pupil response mode.