How do instructional designers evaluate? A qualitative study of evaluation in practice

This study employed a qualitative research design to investigate how instructional designers use evaluation in everyday design practice. While past research has examined how designers spend their time, how they generally make decisions, and expert-novice differences, little attention has been paid to use of context, input, process, or product evaluation, from the perspective of practicing designers. Based on interviews of practitioners, our findings included ten themes regarding how designers use evaluation to improve their products. While these results substantiate to some degree the claim that practitioners believe clients will not pay for formal evaluations, they also suggest that practitioners use evaluation in important but less formal ways. Other conclusions regarding the role of evaluation in design are provided and future directions for training and research are discussed.     

Instructional design and project management: complementary or divergent?

This paper reports the results of a study to identify the extent to which organizations that develop educational/training products are committed to project management, as measured by their project management implementation maturity, as a methodology that is separate and distinct from the processes of instructional design. A Web survey was conducted among 103 public and private sector organizations worldwide that develop educational/training products. Results show no significant difference by project management maturity level in the roles of instructional designer and project manager, although there is some relationship between maturity level and how organizations perceive the skills/competencies of project managers versus those of instructional designers. Further, organizational decision-makers have very specific expectations about the formal education and training of educational/training product development project leaders. The findings should be of value to institutions of higher education in evaluating programs that prepare students for careers in instructional design.     

Neuroscience,Early Childhood Education and Play: We are Doing it Right!

This editorial examines neuroscience and its impact on the field of education. Starting with a narrative between two young children, the author intertwines research with basic principles of learning, using the interaction between two 4-year-olds to illustrate the precepts. The four principles are: (1) the brain is uniquely organized; (2) the brain is continually growing; (3) a “brain-compatible” classroom enables connection of learning to positive emotions; and (4) children’s brains need to be immersed in real-life, hands-on, and meaningful learning experiences. The editorial concludes with an illustration of how the brain works while two children are playing at the small animal center in their classroom.     

Measuring perceived social presence in distributed learning groups

Social presence—the degree to which ‘the other’ in a communication appears to be a ‘real’ person—has captured the attention of those dealing with learning in groups through computer-supported collaborative learning environments. The concept is important because it affects participation and social interaction, both necessary for effective collaboration and knowledge construction. This article reports on the construction and validation of a self-reporting (Dutch-language) Social Presence Scale to determine perceived social presence in distributed learning groups using computer-supported collaborative learning environments. The result is a one-dimensional scale consisting of five items with an internal consistency of .81. We used a nomological network of similar constructs for further validation. The findings suggest that the Social Presence Scale has potential to be useful as a measure for social presence.     

Between eclecticism and orthodoxy in instructional design

This paper presents the concept of critical flexibility as an alternative to eclecticism in instructional design. Eclecticism is often viewed as a persuasive alternative to theoretical orthodoxy (i.e., rigid use of a single perspective or process) due to the openness and flexibility it purports to offer. In contrast, the authors argue that eclecticism ignores or discourages critical reflection regarding background understanding (e.g., implicit assumptions and values) and perpetuates the lack of openness and flexibility commonly associated with orthodoxy. Critical flexibility, as an alternative to eclecticism, emphasizes an awareness of background understanding, but construes it as capable of being explicated, critically examined, adjusted in specific contexts, and refined or developed over time to facilitate increasingly flexible and effective design practices. The authors clarify the nature of critical flexibility as a general way of being involved in the design process, suggest how it helps overcome the traditional theory–practice split, and discuss several of its implications for scholarship and training.     

Teaching effectiveness: Comparisons between traditional and nontraditional college students

This article compares traditional college students' perceptions of effective teaching behaviors with nontraditional students' perceptions. A 15-item questionnaire was completed by undergraduates at a small Southeastern university. Nontraditional students viewed personality and interaction behaviors as more indicative of effective teaching, whereas traditional students focused on behaviors that potentially would enhance grades. The implications of the findings are discussed in light of techniques and approaches that facilitate the needs of both types of students.His major areas of interest include testing and counseling. Nina Mattie, M.A. is a recent graduate of the University of West Florida in the area of school psychology. Stephen J. Vodanovich, Ph.D. is an assistant professor in the psychology department at the University of West Florida. His major areas of interest are social psychology and industrial/organizational psychology. Chris Piotrowski, M.A. is a consultant in forensic psychology. His major areas of interest are personality assessment and online database searching.     

Effects of infant cries on alcohol consumption in college males at risk for child abuse

This analog study examines the effect of hearing a child's cries on alcohol consumption, comprising one link in a possible association between drinking and child abuse. Thirty-two male undergraduate volunteers were preselected by age, drinking practices, and abuse potential. Subjects were assigned to either a high or low abuse potential group and were then randomly assigned to hear either an infant cry or a smoke alarm. Blood pressure was measured during each of three stimulus presentations. Following each presentation, the subject was asked to rate the stimulus on a number of different dimensions (e.g., soothing-arousing). Additionally all subjects participated in a subsequent taste-rating task as an unobtrusive measure of alcohol consumption. The results indicate that, regardless of risk group, subjects who listened to the infant cry consumed significantly more alcohol (M = 206 ml, SD = 126.43) during the taste-rating task than did those who heard only the smoke alarm (Mr = 95.68 ml, SD = 62.94) which was also rated as aversive. There was no relationship between abuse potential and alcohol consumption. Although diastolic blood pressure did not differ between groups, analysis of the infant rating scales revealed that, regardless of risk group, subjects who heard the infant cry reported feeling more aversion, arousal, and distress, than did those who heard the smoke alarm. These results lend additional support to studies depicting the infant cry as a stressful and aversive event, capable of eliciting increased drinking. Implications for arousal reduction by alcohol and a model of alcohol-induced child abuse are discussed.     

Reading and listening progress in segregated primary schools: Does ethnic and socioeconomic classroom composition matter?

This study aimed to investigate the relationship of classroom composition factors with reading and listening comprehension achievement and progress in socially and ethnically segregated primary schools in Flanders (Belgium). Specifically, using a three‐level multivariate repeated measures analysis, it examined the association of reading and listening achievement and progress with ethnic diversity, the proportion of non‐native students and the average socioeconomic status of the class, taking into account student characteristics. At the beginning and end of the school year, reading tests, listening tests and questionnaires were administered to a sample of 7‐ and 8‐year‐old students (n = 683) in 42 second‐grade classes. Students' listening comprehension achievement at the beginning of the school year was negatively related to having a home language other than the language of instruction and to classes with a high proportion of non‐native students. However, progress in listening comprehension was not significantly associated with any student or classroom composition factors. Students whose mothers had a lower level of education performed lower on reading comprehension at the beginning of the school year, while at the end of the school year students whose mothers had a higher level of education were at a greater disadvantage. Limitations of this study and directions for future research are discussed.