How do instructional designers evaluate? A qualitative study of evaluation in practice

This study employed a qualitative research design to investigate how instructional designers use evaluation in everyday design practice. While past research has examined how designers spend their time, how they generally make decisions, and expert-novice differences, little attention has been paid to use of context, input, process, or product evaluation, from the perspective of practicing designers. Based on interviews of practitioners, our findings included ten themes regarding how designers use evaluation to improve their products. While these results substantiate to some degree the claim that practitioners believe clients will not pay for formal evaluations, they also suggest that practitioners use evaluation in important but less formal ways. Other conclusions regarding the role of evaluation in design are provided and future directions for training and research are discussed.     

Instructional design and project management: complementary or divergent?

This paper reports the results of a study to identify the extent to which organizations that develop educational/training products are committed to project management, as measured by their project management implementation maturity, as a methodology that is separate and distinct from the processes of instructional design. A Web survey was conducted among 103 public and private sector organizations worldwide that develop educational/training products. Results show no significant difference by project management maturity level in the roles of instructional designer and project manager, although there is some relationship between maturity level and how organizations perceive the skills/competencies of project managers versus those of instructional designers. Further, organizational decision-makers have very specific expectations about the formal education and training of educational/training product development project leaders. The findings should be of value to institutions of higher education in evaluating programs that prepare students for careers in instructional design.     

How young people respond to learning spaces outside school: A sociocultural perspective

This article focuses on educational enterprises outside the formal sector, such as museums, botanical gardens and interactive science centres. International research is drawn on to illuminate how design, culture, educational strategies and settings combine to affect the way in which young people respond to experiences on offer, leading to analysis of the impact of such settings in promoting learning, and the likely implications for those who staff such venues. Aikenhead’s concept of the educator as ‘culture broker’ is developed to suggest ways in which learning might be best supported. It envisages a shift from ‘delivery’ strategies targeted at large groups towards approaches which focus on what learners choose to know about using dialogue between children and ‘known and trusted people’. Analysis of observed responses in various settings is undertaken from a sociocultural perspective using the notion of communities of practice. Implications for the roles of education managers and their staff in further research are developed.     

Being an Artist Teacher: A Liberating Identity?

Education as a liberating and democratic process remains an aim and belief in discourses within the field and beyond. The arts also have a tradition in which ‘artistic freedom’ is valued even if what constitutes artistic freedom is contested. In this article an educational discourse in which dialogue is considered a means to personal and collective liberation through education is highlighted and related to ‘artistic freedom’ and the dual roles of artist and teacher, in which learners and teachers are encouraged to contribute to and change culture as well as study and absorb it. Conceptualisations of the artist teacher and professional development and practices associated with these are considered to open up creative possibilities for art teachers without undermining other positive aspects of identity as a teacher and practitioner.     

Technology enhanced feedback tools as a knowledge management mechanism for supporting professional growth and school reform

Attempts at school reform and improvement have experienced difficulty creating and implementing feedback systems that energize and sustain change efforts. If the call for reform at all levels of education is to be met, attention must be given to establishing effective feedback mechanisms in educational institutions as they embark on improvement efforts. The purpose of this article is to describe application of an educational design metaphor to create a Knowledge Management System that provides teachers with timely and constructive feedback about their professional practice. This system includes the way in which a corpus of professional knowledge about differentiated content and instruction, classroom grouping, evidence-based practice and classroom management can be embedded in a set of tools that provides multiple stakeholders (teachers, students, administrators) the opportunity to generate feedback about instructional practice. Use of the approach is discussed within the broader context of Knowledge Management as a vehicle to extend the role of feedback in school reform and improvement within a long term pilot application in one school.     

Analysis of self-regulated learning processing using statistical models for count data

Researchers often use measures of the frequency of self-regulated learning (SRL; Zimmerman, American Educational Research Journal, 45(1), 166–183, 2000) processing as a predictor of learning gains. These frequency data, which are really counts of SRL processing events, are often non-normally distributed, and the accurate analysis of these data requires the use of specialized statistical models. In this study, we demonstrate how to implement and interpret count statistical models in path and latent profile analyses to investigate the role of SRL as a mediator of the relation between pretest and posttest conceptual understanding. Our findings from a sample of 170 college students using a computer to learn about the circulatory system show that SRL does mediate the aforementioned relation, and that count models are a more accurate representation of SRL processing data than standard analysis models based on ordinary least squares estimation. The results of our path analyses revealed which specific SRL processes were related to learning, above and beyond the effect of prior knowledge. Our latent profile analysis revealed two groups of participants, indicative of Brophy’s (2004) model of schematic and aschematic learners. We conclude with implications and future directions for basic and applied SRL research.