An a‐mazing way to deal with performance‐based assessment in early childhood education

Abstract

This article discusses the lessons learned by the implementation of Colorado Senate Bill 99–154, a performance‐based teacher education bill with implications for other Early Childhood Programs that want to promote a developmentally appropriate standard‐based program. A parable about change is used to make a point.     

(Mis)appropriations of criteria and standards‐referenced assessment in a performance‐based subject

This paper draws on semi‐structured interview data and participant observations of senior secondary Physical Education (PE) teachers and students at two school sites across 20 weeks of the school year. The data indicated that the teachers in this study made progressive judgements about students’ level of achievement across each unit of work without explicit or overt reference to the criteria and standards represented in the schools’ work programmes and in the Senior PE syllabus. The teachers’ justification for such an approach was that the criteria and standards had become for them sufficiently ‘internalised’. Determining students’ levels of achievement was for the teachers somewhat ‘intuitive’, being reliant on their memory of students’ performances, and influenced by the construct‐irrelevant affective characteristics of the students. It is argued in this paper that such construct‐irrelevance compromised the construct validity and possible inter‐rater reliability of the decisions made and advantaged some students and marginalised others on the basis of characteristics that were not specifically related to the learning expected from following the syllabus. The potential inequities of such an approach are discussed and suggestions are made for the consolidation of the validity and reliability of teachers’ judgements.     

Evaluating computer‐based assessment in a risk‐based model

There are three purposes for evaluation: evaluation for action to aid the decision making process, evaluation for understanding to further enhance enlightenment and evaluation for control to ensure compliance to standards. This article argues that the primary function of evaluation in the ‘Catherine Wheel’ computer‐based assessment (CBA) cyclic model is evaluation for action and secondary functions are evaluation for understanding and control. By studying segment and cyclic dependencies, conducting a risk analysis and identifying key stakeholders it is possible to identify where action evaluation has the most effect, what is required to be evaluated, who will perform the action and how it will be controlled. Each segment and cycle in the CBA model must therefore be monitored and the observations made must have meaning for the elimination or reduction of pedagogic, operational, technical non web‐based, web‐based and financial risks. It is important to consider stakeholders because they have differing views on what is most important in the evaluation process and inevitably will be involved in the evaluation process itself. CBA system stakeholders include management, academic, support and administrative staff together with external verifiers and students. The information gained from these stakeholders provides quality enhancement and assurance and forms the basis for change.     

Discipline‐based educational development

Abstract

This paper grows out of my academic journey from a long period teaching and researching geography to recently taking on the role of an educational developer. For most staff, their primary allegiance is to their discipline. In my experience, educational developers often fail to recognize and value staff's focus on their discipline ‐ or see it as an obstacle to improving the quality of their teaching. My central argument is that, to have a significant impact on the broad mass of staff, educational developers should work with these discipline‐based concerns. I detail how this can be achieved, drawing on much good practice from the UK, Australia and North America. Approaches include recognizing particular curricular and pedagogic concerns of the disciplines; helping staff to develop as scholars in the teaching of their discipline; for some staff to develop careers in the teaching of their discipline; and for educational developers to work with disciplinary organizations to promote discipline‐based teaching initiatives.     

Introducing Computer‐based Learning

Education and Training are adapting cautiously and clumsily to incorporate computer‐based learning. Potential benefits of CBL are enormous. It is possible, now, to produce a framework for successful development, requiring collaboration between computer manufacturers, professional trainers and teachers, institutions and national agencies. Diana Laurillard of the Open University Institute of Educational Technology (and currently Visiting Fellow at the University of Melbourne) offers such a framework.     

Design‐based science and real‐world problem‐solving

Design‐based science (DBS) is a science pedagogy in which new scientific knowledge and problem‐solving skills are constructed in the context of designing artifacts. This paper examines whether the enactment of a DBS unit supported students’ efforts to construct and transfer new science knowledge and ‘designerly’ problem‐solving skills to the solution of a new real‐world design problem in a real‐world setting. One hundred and forty‐nine students participated in the enactment of a DBS unit. Their understanding of the curricular content was assessed by identical pre‐instructional and post‐instructional written tests. They were then given a new design problem as a transfer task. There was a statistically significant increase on scores from pre‐test to post‐test with an effect size of 1.8. There was a stronger correlation between the scores of the transfer task and those of the post‐test than with those of the pre‐test; we use this finding to suggest that the knowledge that was constructed during the unit enactment supported the solution of the transfer task. This has implications for the development of science curricula that aim to lead to the construction of knowledge and skills that may be useful in extra‐classroom settings. Whether participation in consecutive enactments of different DBS units increases transfer remains to be investigated in more depth.     

Potential stereotype threat and face validity effects on cognitive‐based test performance in the classroom

Evaluation is an inherent part of education for an increasingly diverse student population. Confidence in one’s test‐taking skills, and the associated testing environment, needs to be examined from a perspective that combines the concept of Bandurian self‐efficacy with the concept of stereotype threat reactions in a diverse student sample. Factors underlying testing reactions and performance on a cognitive ability test in four different testing conditions (high or low stereotype threat and high or low test face validity) were examined in this exploratory study. The stereotype threat manipulation seemed to lower African‐American and Hispanic participants’ test scores. However, the hypothesis that there would be an interaction with face validity was only partially supported. Participants’ highest scores resulted from low stereotype threat and high face validity, as predicted. However, the lowest scores were not in the high stereotype threat/ low face validity condition as expected. Instead, most groups tended to score lower when the test was perceived to be more face valid. Stereotype threat manipulation affected Whites as well as non‐Whites, although differently. Specifically, high stereotype threat increased Whites’ cognitive ability test scores in the low face validity condition, but decreased them in the high face validity condition. Implications for testing and classroom environment design are discussed.     

Moving Towards Problem‐based Learning

Abstract

This paper is based upon the reflections of teachers and students as they experience problem‐based learning (PBL) for the first time. A PBL exercise was integrated into a curriculum‐based Biology field course to meet specific learning requirements of students. The exercise created a new learning environment for both students and teachers, and PBL was evaluated as an educative strategy. The exercise could have been improved through more effective student support. Our experiences of PBL have encouraged us to look for other areas of the curriculum where problem‐based approaches could be incorporated to enrich student learning.     

Explorations in using arts‐based self‐study methods

Research methods courses typically require students to conceptualize, describe, and present their research ideas in writing. In this article, the author describes her exploration in using arts‐based techniques for teaching research to support the development of students’ self‐study research projects. The pedagogical approach emerged from the author’s sociocultural theoretical perspective in using symbols and dialogue as mediating tools. Three arts‐based research projects were utilized to assist doctoral students in articulating research interests, framing research proposals, and reflecting on their development as researchers. Data included students’ implementation and assessment of projects; the instructor’s teaching logs; and students’ course evaluations. The projects served to illuminate the subtleties of research interests, self‐understanding and understanding of other’s research, and learning about self‐study by practicing it. Research methods instructors are prompted to consider what happens when students are asked to demonstrate their understanding of research using the arts and when they study the ‘so what’ of using them.     

The problem of education‐based discrimination

While the research, theory and policy literature on race, class and gender discrimination in education is extensive, the problem of education‐based discrimination itself has been widely overlooked. Indeed, the dominant ideologies of meritocracy and human capital (into which we are inculcated throughout our lives by schools, media and the state) proclaim that higher levels of education are and should be linked with greater reward. In a world where education is regularly invoked to legitimate inequality, it can appear nonsensical even to raise concern about education‐based discrimination as a matter of social injustice. We need, however, to challenge those who have taught us not to see what has essentially become an elephant in our living room. Otherwise, we will find ourselves unable ever to use our public systems of education for universal emancipation and empowerment.     

Case‐based Long‐term Professional Development of Science Teachers

Reform efforts are often unsuccessful because they failed to understand that teachers play a key role in making educational reforms successful. This paper describes a long‐term teacher professional development (PD) program aimed at educating and training teachers to teach interdisciplinary topics using case‐based method in science. The research objective was to identify, follow and document the processes that science teachers went through as they assimilated the interdisciplinary, case‐based science teaching approach. The research accompanied the PD program throughout its 3‐year period. About 50 teachers, who took part in the PD program, were exposed to an interdisciplinary case‐based teaching method. The research instruments included teacher portfolios, which contained projects and reflection questionnaires, classroom observations, teacher interviews, and student feedback questionnaires. The portfolios contained the projects that the teachers had carried out during the PD program, which included case studies and accompanying student activities. We found that the teachers gradually moved from exposure to new teaching methods and subject matter, through active learning and preparing case‐based team projects, to interdisciplinary, active classroom teaching using the case studies they developed.     

Factors underlying distance learner satisfaction with televised college‐level courses

Abstract

Two investigations were conducted to identify the major dimensions of distance learner satisfaction with live‐broadcast, interactive (one‐way video, two‐way audio) televised college‐level courses. In the first study, factor analyses were used to explore the responses of 201 currently enrolled students to the Telecourse Evaluation Questionnaire. The analyses identified seven distinct dimensions of course satisfaction. A comparable study was conducted one year later using a different sample of 177 distance learners who were also enrolled in the televised courses. A factor analysis of these data validated the original results in that, as predicted, seven comparable dimensions were identified. Overall results are discussed in terms of the practical benefits the research offers to both program personnel and evaluation researchers.     

Stopping to thinks school‐based and higher education‐based primary student teachers' reflections on their learning

This paper examines trainees' perceptions of the impact, on secondary mathematics trainees, of collaborative school‐based work involving small groups of trainees, a university tutor and a class teacher. The findings suggest that trainees' become better able to plan and deliver a coherent yet diverse sequence of lessons, that their awareness of children's learning styles and misconceptions was raised and that preconceived notions about teaching and learning were challenged and mediated. A few trainees found the experience unhelpful and threatening. The role of the tutor was found to influence trainees' responses to the program and, to an extent, moderate the effects of variations in teacher involvement. Some implications are discussed.