Setting Reliable National Curriculum Standards: a guide to the Angoff procedure

The Northern Ireland Curriculum, like the English National Curriculum, records pupil achievement on a 10‐level scale. The level to which a pupil is ‘assigned’ at the end of a Key Stage is based upon two sources of assessment information: classroom‐based measures provided by the teacher and summative information from Common Assessment Instruments (CAIs), which are pen‐and‐paper tests taken at the end of the Key Stage. CAIs play a central role in confirming the accuracy with which teachers judge the level at which a pupil is working. While the teacher might judge a pupil to have mastered level 7 in Algebra, for example, based upon observation in class, test data and homeworks, the CAI will only confirm this level if the pupil scores above the level 7 cutscore on the CAI. If this cutscore does not accord with a reliable measure of what constitutes level 7 performance in Algebra in the classroom, there is likely to be misclassification of pupils with attendant difficulties for the efficient planning of teaching and learning. Misclassifications can be minimised when examiners and teachers interpret level 7 achievement in Algebra similarly. The Angoff standard‐setting procedure was used to establish level 5 cutscores in the Number and Handling Data tests of the mathematics CAI so that comparisons might be made between the published level 5 cutscores and those which result from a judgemental standard‐setting procedure. The 21 teachers involved in the procedure were offered the opportunity to recommend a level 5 ‘standard’ using the Angoff methodology, and to review their recommendations in the light of test data from the February 1993 CAI administration. A further opportunity was offered following a discussion during which individual teachers articulated their reasons for the standards they recommended. The results confirm that the reliability of recommended standards increases both as a consequence of receiving normative data and of discussion. All statistical measures reported in this article indicate that the procedure could command the confidence of examiners, teachers and the public. While the recommended cutscore for Number is in close accord with that published by the examiners, the extent of the mismatch in the Handling Data test is such as might give rise to some misclassification of pupils. It is important to stress that this mismatch had no real consequences since 1993 was a pilot year and no test outcomes were reported. The article concludes with an outline of the contribution which the Angoff methodology can make to the resolution of some of the difficulties faced by English national assessment, as identified in Sir Ron Dealing's interim report “The National Curriculum and its Assessment”.