The effects of technology‐enhanced formative evaluation on student performance on state accountability math tests

Technology‐enhanced formative evaluation (TEFE) could enhance student achievement while diminishing the burden of formative evaluation for educators. The current study examined data from 360 randomly selected schools that either used a TEFE program for 1 year to 4 years 11 months, for 5 or more years, or not at all. Analyses of covariance found a significant and moderate to large (d = .51 and .78) effect on percentage of students who scored proficient on state accountability tests in math. Moreover, among schools that used TEFE, those with a non‐White student population of at least 50% scored in the proficient range on state math tests at a rate equal to that of those with at least 50% White students. A significant and moderate to large (d = .71) effect was found, however, between these two groups of schools among schools that did not use a TEFE program. © 2010 Wiley Periodicals, Inc.     

Dimensions of personalisation in technology‐enhanced learning: A framework and implications for design

Personalisation of learning is a recurring trend in our society, referred to in government speeches, popular media, conference and research papers and technological innovations. This latter aspect—of using personalisation in technology‐enhanced learning (TEL)—has promised much but has not always lived up to the claims made. Personalisation is often perceived to be a positive phenomenon, but it is often difficult to know how to implement it effectively within educational technology. In order to address this problem, we propose a framework for the analysis and creation of personalised TEL. This article outlines and explains this framework with examples from a series of case studies. The framework serves as a valuable resource in order to change or consolidate existing practice and suggests design guidelines for effective implementations of future personalised TEL.     

Transition and technology—Evaluation of blended learning delivered by university staff to 6th form students

This paper tackles two issues which are closely related: transition from pre‐entry learning to studying at university and the role blended learning strategies might have in supporting this transition. It draws on the pilot delivery of two credit‐bearing modules which were delivered in two pre‐entry institutions. The modules followed two different models in terms of their alignment to the pre‐entry curriculum and in their mode of delivery. They were evaluated through questionnaires and focus groups with students. Both university tutors and coordinators at the schools were also interviewed in order to identify barriers in the process from their perspectives. This paper contributes to research in this field by identifying, from the key stakeholders of students' and deliverers' vantage points, factors that are specific to the blended learning process and contribute to a more structured and positive transition experience for the students. It looks at how blended methodologies can enhance the transition process, as well as what some of the barriers might be. As a consequence, the paper will provide some suggestions as to how institutions may overcome some of those issues identified in order to replicate the practice successfully.     

The impact of mobile learning on students' learning behaviours and performance: Report from a large blended classroom

Chinese classrooms, whether on school grounds or online, have long suffered from a lack of interactivity. Many online classes simply provide recorded instructor lectures, which only reinforces the negative effects of passive nonparticipatory learning. At Shanghai Jiaotong University, researchers and developers actively seek technologic interventions that can greatly increase interactivity in large blended classes. They developed a cutting-edge mobile learning system that can deliver live broadcasts of real-time classroom teaching to students with mobile devices. Their system allows students to customise means of content-reception based on when and where they tune into the broadcast. The system also supports short text messaging and instant polls. Through these venues, students can ask questions and make suggestions in real time, and the instructor can address them immediately. This article describes this system in detail, and also reports results from a formal implementation of the system in a blended English classroom of 1000 students (with about 800 being online). As the data reveal, m-learning activities can much better engage students in the learning process. Students in this class changed from passive learners to truly engaged learners who are behaviourally, intellectually and emotionally involved in their learning tasks.     

A blended approach to learning: added value and lessons learnt from students' use of computer-based materials for neurological analysis

This paper examines BSc Physiotherapy students’ experiences of developing their neurological observational and analytical skills using a blend of traditional classroom activities and computer‐based materials at the University of Birmingham. New teaching and learning resources were developed and supported in the School of Health Sciences using Web Course Tools combined with a wide range of video clips of patients with neurological disorders on CD‐ROM. These resources provided students with the opportunity to observe “real patients” prior to clinical placements, thus bridging the gap between their theoretical understanding of these disorders and their practical experience of evaluating abnormal movement in the clinical setting. This paper considers how this blended approach to learning enhanced students’ experiences of developing their neurological skills and of preparing for their clinical placements. This paper also discusses the lessons that have been gained from students’ experiences to provide future or similar projects with the opportunity to learn from these experiences.     

Technology‐enhanced learning in higher education: How to enhance student engagement through blended learning

Blended learning has risen in popularity in the last two decades as it has been shown to be an effective approach for accommodating an increasingly diverse student population in higher education and enriching the learning environment by incorporating online teaching resources. Blending significant elements of the learning environment such as face‐to‐face, online and self‐paced learning leads to better student experiences and outcomes and more efficient teaching and course management practices if combined appropriately. Hence, an appropriate systematic and dynamic approach of blended learning design is crucial for a positive outcome, starting with planning for integrating blended elements into a course and creating blended activities and implementing them. Evaluating their effectiveness and knowing in which environments they work better and improving the blended activities designed from both the student’s and instructor’s perspective are critical for the next delivery of the course. This article aims to increase awareness of higher education educators about how traditional face‐to‐face learning can be transformed into blended courses so as to develop student engagement with both in‐class and online approaches, whilst being time effective for the instructor.     

Best practice in conducting RCTs: Lessons learnt from an independent evaluation of the Response-to-Intervention programme

This paper presents the findings of the first independent UK evaluation of a large-scale randomised controlled trial of Response-to-Intervention, used as a catch-up literacy intervention. A total of 385 pupils in their final year of primary school (Year 6) were involved in the study (181 treatment and 204 control). These were identified as those at-risk of not achieving the threshold Level 4 in English at Key Stage 2. The pupils came from 49 schools across England. Twenty-seven schools were randomised to receive treatment immediately and 22 schools, which formed the control, were randomised to receive the intervention later. RTI was delivered in the summer term in preparation for pupil transfer to secondary school. The overall impact based on the standardised New Group Reading Test (NGRT) showed an ‘effect’ size of +0.19, and of +0.48 when considering only free school meal eligible pupils. However, these results must be viewed with considerable caution given the high attrition (over 25%) especially from the control group, and unclear identification of pupils eligible for the intervention. The fact that the evaluators did not have direct contact with schools when trying to identify eligible pupils, and that the developers wanted to use the pre-test to identify eligible pupils, led to this being a weak trial. The significance of the work therefore lies at least as much in the lessons learnt as in the headline figures. We learnt that ideally no more than two parties should be involved in communicating with schools, so that relevant instructions are passed quickly and acted upon promptly. This helps minimise the risk of misunderstanding and dropout post-allocation. Prior training on the technicalities of trials and research in general is necessary for both developers and any staff delivering the intervention so that all parties involved understand their commitment and the need to provide accurate and complete data. In future trials of RTI, it would be better for individual eligible pupils to be randomised rather than schools. RTI should ideally be given a whole year to allow the full cycle to be implemented, and it should be delivered daily for at least 30 minutes.     

A pedagogy of larger concerns used as a lens to reflect on the design of student learning experiences

ABSTRACT

Although engineering educational endeavours are often described with regard to how well they prepare students to meet national or global engineering concerns, the purpose of this paper is to show how those efforts can also serve the larger educational goals of the educators themselves as they develop learning experiences for their students. This paper describes the reflections of four engineering educators who used a research-based ‘Pedagogy of Larger Concerns’ (PLC) as a conceptual lens for examining important aspects of their courses. A PLC is a set of teaching conceptions that was identified in a study exploring the thinking of outstanding professors in order to improve our understanding of the basis of their effectiveness.     

The conceptualisation of a modified formative assessment model for non‐verbal students with autism and severe learning difficulties

The popularity of formative assessment has increased since the publication of work by Black and Wiliam in 1998. Even though it is a useful teaching tool, in most cases it has only been possible to use it for students with high levels of cognitive and communicative ability. The aim of this article is to propose a modified, personalisable model of formative assessment for non‐verbal students with autism and severe learning difficulties. Five students with autism and severe learning difficulties participated in systematic video observations over a period of eight weeks, during which student behaviour and attainment were recorded. The behaviour checklist gave an accurate representation of the students' level of engagement and predicted attainment, but differences in passive learning objectives (those requiring passive co‐operation) and active ones (those requiring active contribution) were noted. The introduction of engaging resources improved engagement, but tangible rewards had a negative effect on attainment. Praise had a positive effect on engagement and attainment.     

Use of a technology‐enhanced version of the good behavior game in an elementary school setting

The purpose of this study was to investigate the effectiveness of a variation of the Good Behavior Game (GBG) in which teachers used ClassDojo to manage each team's progress. ClassDojo is a computer‐based program that enables teachers to award students with points for demonstrating target behaviors. Dependent variables included class‐wide disruptive and academically engaged behavior, teachers’ ratings on the Behavior Intervention Rating Scale (BIRS), and the rate of teacher praise statements delivered in each phase. Overall, results indicated that the GBG with ClassDojo was effective at reducing disruptive behavior, increasing academically engaged behavior, and was rated as socially valid. Additionally, when the intervention was in place, increases in the amount of behavior‐specific praise statements delivered were observed across all three classrooms.     

Supporting learning of variable control in a computer‐based biology environment: Effects of prompting college students to reflect on their own thinking

While instruction on control of variables has been shown to be effective, especially when it encourages students to focus explicitly on rules or procedures, little evidence of application to novel problems has been obtained. We hypothesized that prompting students to understand their own learning processes while doing experiments involving control of variables would allow them to activate their repertoire of knowledge and strategies and learn in a way that would enhance transfer of learning. Students were assigned to one of four versions of a computer‐based biology simulation learning environment, each employing a different type of prompt: reason justification, rule based, emotion focused, or none (control). Learning in this computer environment, college biology students designed and conducted experiments involving control of variables. Students' ability to solve both contextually similar (near transfer) and contextually dissimilar (far transfer) problems was assessed. The treatment groups performed equally well on contextually similar problems. However, on a contextually dissimilar problem, the reason justification group had significantly higher scores than the other groups. Qualitative data showed that the reason justification prompts directed students' attention to understanding when, why, and how to employ experiment design principles and strategies, and this in turn helped students to transfer their understanding to a novel problem. © 1999 John Wiley & Sons, Inc. J Res Sci Teach 36: 837–858, 1999     

Campus‐based student experiences of learning technologies in a first‐year science course

This study reports on an investigation into the campus‐based experience of university students studying mammalian physiology that was significantly supported with learning technologies. The design of the course enabled the students to interrogate the key ideas that they came across in their lectures and laboratories through online activities which prepared the students for practical classes. Close‐ended questionnaires were used to uncover qualitative variation in the population sample, particularly the differences in the concepts of learning technologies and approaches to learning technologies. Qualitative variation in concepts of, and approaches to, learning technologies was found to be significantly associated with variation in academic achievement. The outcomes have important implications for the approaches to teaching of campus‐based experiences of learning supported by learning technologies when we seek to support all students to realise their learning outcomes in technology‐mediated processes     

Development of a peer‐assisted learning strategy in computer‐supported collaborative learning environments for elementary school students

This study explores the effects of Electronic Peer‐Assisted Learning for Kids (EPK), on the quality and development of reading skills, peer interaction and self‐concept in elementary students. The EPK methodology uses a well‐developed, synchronous computer‐supported, collaborative learning system to facilitate students' learning in Chinese. We first review the theoretical framework and architecture of EPK. This study aimed to compare students learning Chinese language arts activities online using EPK with students learning face‐to‐face in a classroom setting. Students' online interactions were coded into four patterns of peer interactions: task coordination, tutor behaviour, tutee behaviour and off‐task. The online activity type significantly influenced the online peer interactions. Students in the online peer‐assisted learning group outperformed the face‐to‐face group on reading skills. Students learning online showed significant growth in self‐concept, and this was especially true for students who were passive participants. These findings demonstrate that EPK is an effective tool that can enhance elementary students' reading, as well as promote positive self‐concepts.     

Identification of gifted students with learning disabilities in a Response‐to‐Intervention era

The identification of children who are twice‐exceptional—those who are gifted and have concomitant learning disabilities (LDs)—has historically posed a number of challenges for school psychologists and other school personnel. With the reauthorization of the Individuals With Disabilities Education Act and the shift to the use of a Response‐to‐Intervention (RtI) model to identify students with LDs, the task of identifying those who are twice exceptional is even more daunting. This article proposes an integrated model for the identification of gifted children with LDs that blends standardized assessment methods with practices consistent with RtI. This balanced approach brings together the best of both worlds to more accurately identify twice‐exceptional students and better meet their educational needs. © 2010 Wiley Periodicals, Inc.     

Responsive evaluation as a guide to design and implementation: Case study of an e‐health learning system

Evaluation of the design and implementation of a web‐based e‐health application offers an opportunity to apply extensive research findings and evidence‐based practices from the learning and performance literature. In this study, we examined how interactions between stakeholders influenced the design, implementation, and outcomes of an e‐health education system. The online system was designed for adults with diabetes. A midsized suburban medical practice in the Midwest partnered with a private sector information technology provider and a university research team to develop the system for patients. Design of the support system was based on a model developed by Schaffer and colleagues, and the implementation strategy was informed by Ely's conditions‐of‐change model. Analyses of design and pilot study implementation processes were accomplished through the lens of responsive evaluation. Discussion of the successes, tensions, and challenges related to the design of learning and performance environments in the e‐health context will guide future system design and evaluation efforts.     

Structure and agency in learning: a critical realist theory of the development of capacity to reflect on academic practice

Theories of learning typically downplay the interplay between social structure and student agency. In this article, we adapt a causal hypothesis from realist social theory and draw on wider perspectives from critical realism to account for the development of capacity to engage in reflection on professional practice in academic roles. We thereby offer a theory of professional learning that explores how social and cultural structures and personal emergent powers combine to ensure variation in the emergence of such reflective capacity. The influence of these factors on professional learning is mediated through reflexive deliberation and social interaction, with the exercise of one's personal powers specifically identified as a stratum of social reality. We consider further the role of concerns, intention and attention in professional learning, drawing together issues that are rarely considered within the same theory. We thus offer a comprehensive account of professional learning, showing how a focus on structure and agency increases the explanatory power of learning theory.     

Use of interactive sessions and e‐learning in teaching anatomy to first‐year optometry students

Students enrolled in the Optometry program at the University of Manchester are required to take a functional anatomy course during the first year of their studies. Low mean scores in the written examination of this unit for the past two academic years energized staff to rethink the teaching format. Interactive sessions lasting 20 minutes each were introduced during the two hour lecture sessions. In these sessions students reinforced their anatomical knowledge learned in lectures, through playing games such as anatomy bingo and solving anatomical anagrams. In addition, five e‐learning modules were also introduced for students to complete in their own time. A pre‐ and postcourse questionnaire were distributed to obtain student views on their expectations of the course and interactive sessions. Comparisons were made between written examination results from 2008 to 2009 to written examination results from the previous five academic years to see if the interactive sessions and e‐learning modules had any impact on student knowledge. In addition, comparisons were made between student performances on the functional anatomy course with their performance in all of the other assessments taken by the students during their first year of study. Analysis of the questionnaires showed that student's expectations of the course were fulfilled and the interactive sessions were well received by the majority. There was a significant increase (P ≤ 0.01) in the mean examination score in 2008–2009 after introduction of the interactive sessions and e‐learning modules compared with scores in previous years. The introduction of interactive sessions has increased student enjoyment of the module and along with the e‐learning modules have had a positive impact on student examination results. Anat Sci Educ 3:39–45, 2010. © 2009 American Association of Anatomists.     

Transforming practice in Design and Technology: evidence from a classroom-based research study of students' responses to an intervention on inclusive design

This article offers evidence from a classroom-based research study in English secondary schools which developed an innovative approach to teaching in Design and Technology based on inclusive design (also known as universal design). The research evidence highlights how creative, problem-solving approaches in Design and Technology can be used to provide authentic, engaging learning experiences for students. This study, entitled ‘Designing our Tomorrows', introduced secondary school teachers and students to the principles and practices of inclusive design through an intervention using specially designed resources. In this article, findings from the study's qualitative pre- and post-intervention interviews are presented to illustrate how this type of approach can stimulate students' creativity and understanding of design.     

Listening to the ‘voices’ of students with severe learning difficulties through a task‐based approach to research and learning in physical education

In this article, Hayley Fitzgerald, Anne Jobling and David Kirk consider the physical education and sporting experiences of a group of students with severe learning difficulties. Their study is thought provoking, not only because of the important and somewhat neglected subject matter, but equally for the research approach adopted. The way in which the study engaged with the students and the insights gained from that engagement will be of particular interest to practitioner researchers.