Comparability of GCSE examinations in different subjects: an application of the Rasch model

The comparability of examinations in different subjects has been a controversial topic for many years and a number of criticisms have been made of statistical approaches to estimating the ‘difficulties’ of achieving particular grades in different subjects. This paper argues that if comparability is understood in terms of a linking construct then many of these problems are resolved. The Rasch model was applied to an analysis of data from over 600,000 candidates who took the General Certificate of Secondary Education (GCSE) examinations in England in 2004. Thirty‐four GCSE subjects were included in the final model, which estimated the relative difficulty of each grade in each subject. Other subjects failed to fit, as did the fail grade, U. Significant overall differences were found, with some subjects more than a grade harder than others, though the difficulty of a subject varied appreciably for different grades. The gaps between the highest grades were on average twice as big as those between the bottom grades. Differential item functioning (DIF) was found for male and female candidates in some subjects, though it was small in relation to variation across subjects. Implications of these findings for various uses of examination grades are discussed.     

Learning and expertise in support for parents of children at risk: a cultural-historical analysis of partnership practices

A key predictor of young people’s future outcomes is their level of academic achievement whilst at school. In England this is most commonly measured by achievement in GCSEs. However, not all pupils will have taken the same set of GCSE examinations as, for example, they may make different subject choices. For this reason, GCSE performance is often aggregated into a simple measure such as ‘mean GCSE grade’ before being used in statistical models. This paper investigates the merits of using an alternative method, based upon the relatively new technique of Generalised Boosting Models, which does not require for GCSE results in different subjects to be aggregated together. The importance of this research is that by evaluating the predictive performance of such a method we can ascertain how much useful information is lost in the process of GCSE aggregation. The results show that traditional predictions based upon simple aggregated measures of GCSE attainment are fairly similar to those based upon the more complex approach. This provides some confidence that, for the majority of outcomes, only a small amount of predictive information will be lost through the use of aggregated measures of GCSE performance.     

Assessing pupils at the age of 16 in England – approaches for effective examinations

In England, pupils aged 16 take the General Certificate of Secondary Education (GCSE) examinations for a range of subjects. The current assessment models for GCSE include a two-tier structure for some subjects and a non-tier model for the others. The tiered subjects have a higher tier designed for high achieving pupils and a lower tier for low achieving pupils. The higher tier paper is targeted at grades A*–D (with A* the highest grade available), while the lower tier paper at grades C–G (with G the lowest grade). The UK government has proposed a comprehensive reform of GCSEs. It suggested that, with tiered papers, pupils are forced to choose between higher and lower tier papers, which will place a cap on the ambition of those entering for the lower tier. The government therefore suggests avoiding tiering in the reformed GCSEs when possible. This paper discusses the technical and equity issues with the use of tiered examinations in current GCSEs and reviews potential alternative assessment approaches for effective differentiation between pupils for the reformed GCSEs.     

Establishing and applying performance standards for curriculum-based examinations

This paper describes how a state education system in Australia introduced standards-referenced assessments into its large-scale, high-stakes, curriculum-based examinations in a way that enables comparison of performance across time even though the examinations are different each year. It describes the multi-stage modified Angoff standard-setting procedure used to establish cut-off scores on subject examinations, and how the results from this exercise were then used to develop standards packages. These packages illustrate the performances of students at the borders between the various bands.

The paper also shows how originally it was intended to use a Rasch measurement model to create the statistical feedback used in the standard-setting procedure. It also describes the modifications to the feedback that were necessary to meet the real-time constraints of this large-scale examination programme. It argues that consideration should now be given to using the Rasch model to provide this feedback instead of the current approach.     

Date of birth and achievement in GCSE and GCE A‐level

General Certificate of Secondary Education (GCSE) examinations are taken by almost all pupils in England, Wales and Northern Ireland at age 16 years. General Certificate of Education Advanced level (GCE A‐level) examinations are normally taken by relatively able students at age 18. The effect of month of birth on attainment in these public examinations is investigated through a database which brings together the 1991 GCSE results and the 1993 GCE A‐level results of all candidates born between September 1974 and August 1975. Older pupils perform best at GCSE but not at GCE A‐level. Selection effects on entry to A‐level courses appear to explain why this is the case.     

Psychometric Aspects of Maintaining Standards of Examinations

Through pilot studies and regular examination procedures, the National Institute for Educational Measurement (CITO) in The Netherlands has gathered experience with different methods of maintaining the standards of examinations. The present paper presents an overview of the psychometric aspects of the various approaches that can be chosen for the maintenance of standards. Generally speaking, the approaches to the problem, can be divided into two classes. In the first approach the examinations are a fixed factor, i.e. the examination is already constructed and cannot be changed, and the link between the standards of both examinations is created by some test equating design. In the second approach the items of both examinations are selected from a pre‐tested pool of items, in such a way that two equivalent examinations are constructed. In both approaches the statistical problems of simultaneously modelling possible differences in the ability level of different groups of examinees and differences in the difficulty of the items are solved within the framework of item response theory. It is shown that applying the Rasch model for dichotomous and polytomous items results in a variety of possible test‐equating designs which adequately deal with the restrictions imposed by the practical conditions related to the fact that the equating involves examinations. Especially the requirement of secrecy of the content of new examinations must be taken into account. Finally it is shown that, given a pool of pre‐tested items, optimisation techniques can be used to construct equivalent examinations.     

Comparing Examination Standards: is a purely statistical approach adequate?

There has recently been a renewed interest in three types of comparability of standards in the United Kingdom [1] public examination system: between years, between subjects and between the six examination boards. Whilst comparisons of raw grade distributions are now generally acknowledged to be invalid indicators of relative standards, comparisons are regularly made for this purpose between adjusted grade distributions. Such adjustments are typically the result of statistically controlling for some of the relevant variables. The dangers of such an approach are that only easily quantifiable variables are used in the adjustment and that any residual differences between distributions will automatically be attributed to difference in standard. Using candidate‐level data from four 1994 Advanced level (A level) [2] mathematics examinations (designed for 18‐year‐old students), and paying particular attention to the Schools’ Mathematics Project (SMP) 16‐19 [3] syllabus, the paper reports on two such analyses. It then discusses some reasons why attributing differences in the adjusted grade distribution to differences in standard could be invalid. Whilst the study focuses on four A level mathematics syllabuses, the same principles apply irrespective of the context in which statistical comparisons of examination results are made. The methodologies, their shortcomings and the pleas for caution are not, therefore, specific to this study, this type of comparison or this examination system.     

Fears for tiers: are candidates being appropriately rewarded for their performance in tiered examinations?

Tiering is a multi‐stage test design whereby teachers allocate students to a particular difficulty level (tier) of a test. This approach to the challenge of delivering assessments to students with a heterogeneous ability distribution is normal practice in UK public examinations at the age of 16. This study uses Item Response Theory number‐correct score equating to examine the relative standards that are set between tiers on certain GCSE assessments. It finds evidence to suggest that candidates on the foundation tier are being over‐rewarded, while those on the higher tier are being under‐rewarded. It concludes that the use of IRT test equating could help improve standard setting on tiered tests and that the issue of restricted grade ranges on these tests may need to be reconsidered.     

University Admissions and the Prediction of Degree Performance: an Analysis in the Light of Changes to the English Schools’ Examination System

The United Kingdom's Department for Education has recently changed the nature of the AS‐level examinations normally taken by students aspiring to enter higher education degree courses one year into their post‐compulsory education. In the face of protests from universities and other institutions that this would both harm students’ progression towards the A‐level qualifications, on which entry to English universities is normally based, and make it difficult for universities determining which of their applicants were best placed to benefit from their degree courses, the Department conducted research which it claimed showed that degree outcome could be predicted as well from the results of GCSE examinations taken one year before AS‐levels as from AS‐levels themselves. This paper critiques those analyses and their conclusions showing, through a re‐analysis and extended interpretation of the Department's data, that AS‐levels provide a more reliable predictor of degree performance in 2011 than GCSEs and that many students who performed better at AS‐level than at GCSE gained a place, and performed well, at a university with high entrance standards.     

What are the effects of ability grouping on GCSE attainment?

The use of ability grouping is frequently justified on the grounds that it is an effective means of raising attainment. Little large‐scale quantitative research has been undertaken since the introduction of the National Curriculum in England and Wales. The aim of this article is to examine the effects of setting on students' achievement in English, mathematics and science General Certificate of Secondary Education (GCSE) examinations. Data were collected from a cohort of over 6000 Year 9 students in 45 mixed secondary comprehensive schools who were followed up in Year 11 when they sat GCSE examinations. Multilevel modelling was used to estimate the effect of setting on GCSE attainment, taking account of prior attainment, social disadvantage, gender and attendance. There were no significant effects of setting in English, mathematics or science. Effects on higher and lower attaining students were not consistent in the three subjects. Socially disadvantaged students achieved significantly lower grades and girls achieved higher grades than boys, especially in English. In all three subjects, students of similar ability achieved higher GCSE grades when they were placed in higher sets. Theoretical and practical implications are discussed.     

The SES equity gap and the reform from modular to linear GCSE mathematics

This article addresses whether the introduction of end-of-course, linear General Certificate of Secondary Education (GCSE) examinations changed the socio-economic equity gap in England. The GCSE is a national examination offered in a wide range of subjects and taken by almost the entire 16-year-old age cohort. Between the years 2007 and 2014, it underwent a number of reforms to both the underlying curriculum and the examination structure. At the beginning of the period, examinations were primarily modular in structure where the course was decomposed into discrete units tested in a staged manner. By 2014, all GCSE examinations were linear and the whole course content was tested simultaneously when study was complete. These structural changes and the curriculum reforms mean that the impact of modular and linear testing on the performance of students has been the focus of recent interest. Some educational commentators suggested that modular examinations are more suitable for lower-performing students, including those with lower socio-economic status (SES). This research has been conducted to monitor the socio-economic equity gap in the light of the structural changes. It focuses on GCSE mathematics and concludes that, although there is still a clear gap in attainment between disadvantaged students and their peers, this gap does not seem to be exacerbated by the examination structure. In other words, the linearisation of GCSE mathematics is unlikely to have increased inequity between students of high and low SES.     

Gender differences in approaches to studying for the GCSE among high‐achieving pupils

This study explores gender differences in approaches to studying for GCSE among high‐achieving pupils. The sample comprised 310 Year 10 and 11 pupils from two single‐sex schools. Pupils completed a self‐reported questionnaire designed to assess approaches to studying for GCSE, including statements relating to coursework, examinations, research, study strategies and homework. Boys gained a higher score overall in the questionnaire, indicating a more effective approach to studying for GCSE. Gender differences were found in approaches to examinations and study but not in approaches to coursework. The boys reported doing less homework than the girls. The findings suggest that overall high‐achieving boys have better studying strategies than high‐achieving girls. They achieve high standards while doing less homework. Approaches to studying among highachieving girls may be mediated by anxiety that manifests itself in surface approaches to studying for examinations     

It's not which school but which set you're in that matters: the influence of ability grouping practices on student progress in mathematics

The mathematics achievement of a cohort of 955 students in 42 classes in six schools in London was followed over a 4‐year period, until they took their General Certificate of Secondary Education examinations (GCSEs) in the summer of 2000. All six schools were regarded by the Office for Standards in Education (Ofsted) as providing a good standard of education, and all were involved in teacher training partnerships with universities. Matched data on Key Stage 3 test scores and GCSE grades were available for 709 students, and these data were analysed in terms of the progress from Key Stage 3 test scores to GCSE grades. Although there were wide differences between schools in terms of overall GCSE grades, the average progress made by students was similar in all six schools. However, within each school, the progress made during Key Stage 4 varied greatly from set to set. Comparing students with the same Key Stage 3 scores, students placed in top sets averaged nearly half a GCSE grade higher than those in the other upper sets, who in turn averaged a third of a grade higher than those in lower sets, who in turn averaged around a third of a grade higher than those students placed in bottom sets. In the four schools that used formal whole‐class teaching, the difference in GCSE grades between top and bottom sets, taking Key Stage 3 scores into account, ranged from just over one grade at GCSE to nearly three grades. At the schools using small‐group and individualized teaching, the differences in value‐added between sets were not significant. In two of the schools, a significant proportion of working‐class students were placed into lower sets than would be indicated by their Key Stage 3 test scores.     

Photography in Schools: Current Trends in Theory and Practice

This paper reviews the place of photography in formal education. It is argued that as a specific subject of study photography remains marginal within primary and secondary education, and is viewed predominantly as a leisure activity or as a vocational subject more appropriate to further education. Despite this however, it is argued that photography can make a unique contribution to general education, and that it is particularly well placed for teachers seeking to develop innovative and cross-curricular work. Two contexts for photography in education are considered in detail: GCSE and ‘A’ level photography; and the National Curriculum for Art, and GCSE art and design examinations. In conclusion I look at the potential photography offers for developing cross-curricular work in formal education.     

Cognitive Style and GCSE Performance in Mathematics,English Language and French

A preliminary study was undertaken of cognitive style and performance in General Certificate of Secondary Education (GCSE), the British public examination for pupils at 16 years. The positions of 182 pupils on two fundamental cognitive styles dimensions (Wholist‐Analytic and Verbal‐Imagery) were assessed by means of the Cognitive Styles Analysis (CSA). The pupils were from two comprehensive secondary schools and all took Mathematics, English Language and French in the GCSE administered by the Midland Examining Group (MEG) in 1991. Comparison of cognitive style and GCSE performance indicated that, for overall performance across the subjects, the pattern was a modified dome shape with the candidates who were intermediate on both dimensions of cognitive style doing best. There was a significant interaction between the styles of the candidates and the GCSE subjects in their effect on performance. The results were discussed in terms of the implications for GCSE and for further research.     

Does school‐based initial teacher training affect secondary school performance?

This article investigates the effects of trainee teachers on secondary school student outcomes. The additional resources which schools receive from being involved in teacher training offer them an opportunity to raise standards, but this has to be set against the possible losses due to school students being taught by inexperienced beginning teachers and the diversion of mentors' efforts away from the classroom. Inspection evidence is used to assess whether trainee teachers affect school students' test and examination results. The findings of this research are that the number of trainees has no significant effect on school results at A‐level or General Certificate of Secondary Education (GCSE), or on the overall value added between Key Stage 3 and GCSE level. However, at Key Stage 3 level at age 14, while there appears to be a very small depressing effect on achievement in schools with low numbers of trainees, there is a significant positive effect on achievement in schools with larger numbers of trainees.     

Can Teachers Enter Candidates Appropriately for Examinations Involving Differentiated Papers?

The ability of teachers to enter candidates for appropriate combinations of differentiated papers is considered. The results of experimental work suggest that teachers would be able to predict their pupils’ examination performance accurately enough to enter almost all pupils at appropriate levels of such examinations; and that they would be able to do this as early as the January preceding the examination. However, they will be able to enter candidates effectively only if the standards required for the overlapping grades are the same at all levels of an examination. There is some evidence to suggest that this condition may not always hold. In addition, results from some Joint 16+ examinations suggest that there may be a considerable number of inappropriate entries to GCSE examinations which use differentiated papers.     

Deciding to provide a 'reader' in examinations for the General Certificate of Secondary Education (GCSE): questions about validity and 'inclusion'

In England, Wales and Northern Ireland, the General Certificate of Secondary Education (GCSE) was designed as an 'inclusive' qualification, access to which has been supported by providing a 'reader' to some candidates during examinations. A candidate reading age criterion of ten years has been employed by the GCSE awarding bodies to determine eligibility for this provision. In this paper, Kevin Woods, a member of the Educational Support and Inclusion Research and Teaching Group in the Faculty of Education at the University of Manchester, examines the rationale and evidence for adopting this criterion and describes an investigation of the reading needs of a cross-sectional sample of 38 GCSE examination candidates in trial examinations. The investigation found a low level of candidate need for a reader, with candidate reading age and self-prediction being unreliable indicators of this need. Kevin Woods highlights the implications for the assessment process used to determine eligibility for a reader in GCSE examinations and pays particular attention to the feasibility and validity of including all students as eligible.     

Patterns of Subject Uptake and Examination Entry 1984–1997

In 1984, the APU science survey collected information on the courses followed by Year 11 pupils. In this paper, the APU survey will be compared with recent GCSE examination level data and will describe the impact of the National Curriculum on the sexes and on pupils of differing ability. In 1984, there were considerable differences in uptake by the sexes and by ability. In 1997, pupils were taking more examinations than were pupils in 1984. Also, girls were taking more GCSEs than boys in 1997. This could be the result of changes in the provision of subjects. The subjects favoured by males, such as science and technology, tend to have been merged with other subjects. The amount of physics studied by boys has decreased with the introduction of the GCSE (from a whole subject to half a subject). Subjects stereotypically preferred by girls such as modern languages, drama and English literature have increased. In this sense, it could be said that there has been a feminisation of the curriculum, but these changes would only account for some of the differential performance between the sexes.