Harnessing technology to improve formative assessment of student conceptions in STEM: forging a national network

Concept inventories, consisting of multiple-choice questions designed around common student misconceptions, are designed to reveal student thinking. However, students often have complex, heterogeneous ideas about scientific concepts. Constructed-response assessments, in which students must create their own answer, may better reveal students' thinking, but are time- and resource-intensive to evaluate. This report describes the initial meeting of a National Science Foundation-funded cross-institutional collaboration of interdisciplinary science, technology, engineering, and mathematics (STEM) education researchers interested in exploring the use of automated text analysis to evaluate constructed-response assessments. Participants at the meeting shared existing work on lexical analysis and concept inventories, participated in technology demonstrations and workshops, and discussed research goals. We are seeking interested collaborators to join our research community.     

The National Advisory Body Planning Exercise 1987–88: A Shot in the Foot

The National Advisory Body was established to advise the Secretary of State for Education and Science on the allocation of money to support advanced work in polytechnics and colleges. The committee system of NAB was representative of local authority and college interests and there was always a tension between NAB and the Department of Education and Science. The paper charts the decisions that were taken by NAB in progressing the 1987–88 planning exercise. The policy context of these decisions is analysed and the political consequences are discussed. The paper argues that the 1987–88 exercise was the wrong approach to the planning problems and that the resulting conflict with the Government destroyed the credibility of NAB.     

Development of the Nine-Ball Skills Test to discriminate elite and high-level amateur golfers

Currently, there is a lack of appropriate skill assessments available for use in golf. The aim of this study was to examine the discriminative validity and the test-retest reliability of the newly developed "Nine-Ball Skills Test". Participants of two ability levels (elite, n = 14; high-level amateur, n = 16) each hit nine golf shots of differing combinations of trajectory (straight, fade, draw) and height (normal, high, low) at an individually determined target area. Each shot was scored on its percent error index from the target and whether it achieved the maximum height as required. Participants completed the test twice using a 5-iron club. The elite group scored significantly higher (P < 0.05) than the amateur group for both the first and second rounds of the test as well as the combined scores. The between-round test-retest reliability was deemed to be not acceptable, thus we propose that the test's protocol should include use of the two rounds as standard. Due to the importance of ball striking and flight control to performance in golf, the Nine-Ball Skills Test is appropriate for providing a measure of this skill component in elite and high-level amateur golfers.     

Teacher Verbal Feedback Directed Towards Secondary Pupils with Challenging Behaviour and its Relationship to their Behaviour

There is a body of evidence that has linked teachers’ verbal feedback to pupils with pupil behaviour. In this study teacher verbal behaviour that was directed towards those pupils that the teachers had nominated as being especially difficult to teach was examined. A series of lessons was observed in a secondary school. The quality and quantity of teacher verbal feedback directed to the class as a whole and to the designated pupils was recorded, as was the on‐task behaviour of the pupils. It was found that the designated pupils were less on‐task than their peers and were more likely to ‘shout‐out’ in lessons. However, they were found to behave appropriately in well run lessons, where on‐task rates were high for all pupils. Teachers tended to give more attention to the designated pupils in the form of positive feedback directed towards their work, but also negative attention directed towards their behaviour. A positive relationship was found between teachers’ use of positive feedback and on‐task rates of the designated pupils. The results are discussed both in terms of the strategies used by teachers but also the effect that labelling of the pupils might have on the behaviour of the teachers and the pupils themselves.     

Self‐help in research and development relating to assessment: a case study

This article briefly chronicles nearly a decade of research and development activity undertaken in the area of assessment by a group of committed volunteers, ‘The Student Assessment and Classification Working Group’ (SACWG). However, greater attention is given to demonstrating what a self‐help approach can achieve in respect of research and development in higher education, and to identifying the factors that contribute to success in this respect. It is suggested that the approach has transfer value, provided that certain conditions are met.     

The empirical derivation of equations for predicting subjective textual information

Parameters pertaining to information processing by human beings have, in the past, been determined by learning and memory experiments with nonsense syllables, number sequences, etc. However, in the real world we are concerned with the processing of meaningful information, not senselles texts. Therefore, the determination of subjective information directly from meaningful material becomes extremely important in the instruction-learning process.This study derives an equation for predicting the subjective textual information contained in a text of material written in the English language. Specifically, this investigation describes, by a mathematical equation, the relationship between the subjective information content of written textual material and the relative number of errors committed by a learner when asked to predict, letter by letter, the content of given textual material. This relationship shows that the subjective information of a given text for a specific learner is directly proportional to the number of wrongly guessed signs made by that learner. This is expressed mathematically by% MathType!MTEF!2!1!+- % feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn % hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr % 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq-Jc9 % vqaqpepm0xbba9pwe9Q8fs0-yqaqpepae9pg0FirpepeKkFr0xfr-x % fr-xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaGaamysaiabg2 % da9iaaiodacaGGUaGaaGymaiaabccacaWGfbaaaa!3B57! \[ I = 3.1{\rm{ }}E \] 0935 0212 VWhere: I = Information in bits E = Number of wrongly guessed signsThe application of Shannon's guessing procedure (1951) in this study permits the measurement of the subjective information of a given text for a specific learner. Unlike senseless texts, the subjective information of a meaningful text varies from learner to learner. Therefore, the derived equation permits the measurement of information in terms of a value that is dependent not only upon the inherent qualities of the subject matter, but also upon the internal state of the learner.The derived equation for the English language is then compared to an equation derived by Weltner (1967) for the German language and found to be remarkably similar.This work was, in part, supported by United States Air Force Contract F41609-71-C-0027 entitled Flying Training Research on Airborne and Ground Training Systems, and a research grant from Arizona State University.