How do instructional designers evaluate? A qualitative study of evaluation in practice

This study employed a qualitative research design to investigate how instructional designers use evaluation in everyday design practice. While past research has examined how designers spend their time, how they generally make decisions, and expert-novice differences, little attention has been paid to use of context, input, process, or product evaluation, from the perspective of practicing designers. Based on interviews of practitioners, our findings included ten themes regarding how designers use evaluation to improve their products. While these results substantiate to some degree the claim that practitioners believe clients will not pay for formal evaluations, they also suggest that practitioners use evaluation in important but less formal ways. Other conclusions regarding the role of evaluation in design are provided and future directions for training and research are discussed.     

Why Should I Use University Library Website Resources? Discipline Differences

Users across academic disciplines utilize different information sources based on the resource's usefulness and relevance. This study's findings show that users from arts and sciences disciplines are much more likely to utilize university library website resources and printed materials than business users who heavily rely on commercial websites.     

A Comparison of Tests for Equality of Two or More Independent Alpha Coefficients

This article extends the Bonett (2003a) approach to testing the equality of alpha coefficients from two independent samples to the case of m ≥ 2 independent samples. The extended Fisher-Bonett test and its competitor, the Hakstian-Whalen (1976) test, are illustrated with numerical examples of both hypothesis testing and power calculation. Computer simulations are used to compare the performance of the two tests and the Feldt (1969) test (for m = 2) in terms of power and Type I error control. It is shown that the Fisher-Bonett test is just as effective as its competitors in controlling Type I error, is comparable to them in power, and is equally robust against heterogeneity of error variance.     

Classification Consistency and Accuracy With Atypical Score Distributions

The current study aims to evaluate the performance of three non-IRT procedures (i.e., normal approximation, Livingston-Lewis, and compound multinomial) for estimating classification indices when the observed score distribution shows atypical patterns: (a) bimodality, (b) structural (i.e., systematic) bumpiness, or (c) structural zeros (i.e., no frequencies). Under a bimodal distribution, the normal approximation procedure produced substantially large bias. For a distribution with structural bumpiness, the compound multinomial procedure tended to introduce larger bias. Under a distribution with structural zeroes, the relative performance of selected estimation procedures depended on cut score location and the sample-size conditions. In general, the differences in estimation errors among the three procedures were not substantially large.     

Entering and exiting a teacher preparation program: What happens in between?

In determining whether graduates of the Master of Arts in Teaching (MAT) Program at Southern Oregon University were developing the necessary knowledge, skills, and dispositions to teach, faculty members found themselves examining their own practice. This is just what Goodlad (1988) suggested we do: clarify our own beliefs in order to better understand our teacher preparation program. We discovered that the development of the MAT program is primarily shaped by the people who teach within the program. Who we are as people, influences who we are as teachers and, who we are as teachers influences the nature of the teacher preparation program. This discovery underscored the importance of our efforts to steward the development of our students’ personhood as well as their knowledge and skills.     

The inspirited nature of mindful curricular enactment’s community (re)making

Intertwining case study with practitioner inquiry, a research team seeks language for the conduct of mindful curricular enactment in an alternative public middle school. The school is committed to valuing students’ narratives of experience, the resources all bring to the school, and the given particularities of contexts, subject matter, and situations, as the makings comprising learning within and through community. A collaborative approach to inquiry that involves all team partners in the research process and engages the unique strengths that each brings focuses on how this alternative school provides a case for accessing how and why mindful curricular enactment orients accordingly. It is the makings forming this curricular movement, and the room found within these makings to reorganize and reconstruct thinking, that primary attention is oriented towards. Deweyan ‘roominess’ negotiated through such curricular enactment is documented as mindful attention to curriculum-making that fosters the needed room—the conditions and supports sustaining an individual/collective movement of thinking. Such movement is revealed as necessarily receptive, inciting community (re)making. Its inspirited nature is revealed through participatory modes of being and associated habits embracing knowledge-making as generative, elemental to being human, in need of other(s), assuming temporal/spatial agency, and interdependent with imagination, instilling embodied understandings.