Relationships between student characteristics and self‐, peer and tutor evaluations of oral presentations

There are many influences on how assessors grade themselves and others. Oral presentations are useful for exploring such factors in peer, self‐ and tutor marked assessments, being rapidly completed and assessed, commonly used in HE and very difficult to anonymize. This opportunistic study examined the effects of gender and level of attainment on the triangulation of marks awarded to student presenters. Grades generated by peer assessment were associated more strongly with tutor‐awarded marks than those from self‐assessment. For self‐assessment there was a strong effect of gender (female students undervalued their performance compared with tutor grades). Peer assessment produced higher marks than from tutors, perhaps because of the close‐knit community developed during residential courses. For tutor marks, the greatest variability was at the lower end of the scale, whereas peer assessors were most variable when marking students who self‐evaluated or peer assessed highly. Students awarded a narrower range of marks to peers compared with tutors, but when self‐assessing used a larger range. Presentations by students who admitted to little sleep the night before received lower grades from both peers and tutors, but this was not reflected by self‐assessments, suggesting they were unaware of their poorer performances. Sessions with fewer talks (four rather than seven) reduced the ‘dip’ in marks previously observed in the middle of sessions. Findings are discussed in the context of bias in this mode of assessment.     

Science Teacher Efficacy and Extrinsic Factors Toward Professional Development Using Video Games in a Design-Based Research Model: The Next Generation of STEM Learning

Designed-based research principles guided the study of 51 secondary-science teachers in the second year of a 3-year professional development project. The project entailed the creation of student-centered, inquiry-based, science, video games. A professional development model appropriate for infusing innovative technologies into standards-based curricula was employed to determine how science teacher’s attitudes and efficacy where impacted while designing science-based video games. The study’s mixed-method design ascertained teacher efficacy on five factors (General computer use, Science Learning, Inquiry Teaching and Learning, Synchronous chat/text, and Playing Video Games) related to technology and gaming using a web-based survey). Qualitative data in the form of online blog posts was gathered during the project to assist in the triangulation and assessment of teacher efficacy. Data analyses consisted of an Analysis of Variance and serial coding of teacher reflective responses. Results indicated participants who used computers daily have higher efficacy while using inquiry-based teaching methods and science teaching and learning. Additional emergent findings revealed possible motivating factors for efficacy. This professional development project was focused on inquiry as a pedagogical strategy, standard-based science learning as means to develop content knowledge, and creating video games as technological knowledge. The project was consistent with the Technological Pedagogical Content Knowledge (TPCK) framework where overlapping circles of the three components indicates development of an integrated understanding of the suggested relationships. Findings provide suggestions for development of standards-based science education software, its integration into the curriculum and, strategies for implementing technology into teaching practices.     

Evidence for confounding in computer-based instruction studies: Analyzing the meta-analyses

A thirty percent sample of the computer-based instruction (CBI) studies meta-analyzed by Kulik et al was examined for evidence of confounding. The purpose of the analysis was to explore the validity of competing claims about the contribution of the computer to measured achievement gains found in CBI studies. Some of these claims propose that CBI effects are overestimated and others argue that CBI effects are underestimated. The result of the analysis strongly suggests that achievement gains found in these CBI studies are overestimated and are actually due to the uncontrolled but robust instructional methods embedded in CBI treatments. It is argued that these methods may be delivered by other media with achievement gains comparable to those reported for computers. Construct validity cautions are offered for those who wish to use meta-analytic results as evidence for implementing CBI in schools and for the design of future CBI research.     

Expectations,observations, and violations: Comparing their effects on course ratings

Knowledgeable observers generally agree that students can give reasonably unbiased assessments of their classes, as long as proper instruments and procedures are used. These same observers agree, however, that more research is needed to understand the subtle dynamics which affect evaluations. Students' expectations (obtained on the first day) and the violation of these expectations are two possible factors. Their effects on students' final interest and future intentions are compared with students' actual reports of the class given at the end of the semester (n=196). Regression analysis using a simultaneous entry of expectations, expectancy violations, and final reports shows that students' reports at the end of the course account for almost all of the explained variance in the course rating items.     

Gender differences in the academic career

This study examines sources of satisfaction, indicators of stress, academic performance, and reward structure in the academic careers of male and female faculty at a major midwestern state university. Faculty pairs were selected from five academic divisions. A total of 58 matched pairs were identified. Of these, 63 faculty responded, yielding data for 23 matched pairs. Subjects were mailed the Academic Career Development Survey, consisting of 144 items. Results indicated that self-reported physical and mental health, and professional and personal life satisfaction were high for both males and females. Both males and females placed high importance on their career and on marriage/other intimate relationships; however, both reported high levels of dissatisfaction with these factors. Gender differences were found in both satisfaction and type of stress reported; no gender differences were found in composite teaching and research performance indices. Salary data, obtained for 42 matched pairs, indicated similar salary levels for men and women. Results were discussed with respect to prior research on gender differences and the academic career.     

The becoming place: A study of educational change in a small college

One of the more astonishing characteristics of educational research is the relative lack of social organizational analyses of higher educational institutions. There is even a greater gap in the literature with regard to thechange process associated with a given university or college. The present study is an examination of one college's attempt to change from a traditional white female finishing college to a larger biracial and coeducational college committed to experimental and innovative educational programs. Two ideal types of educational social organizations are utilized as explicit aids for the specification of the educational change process observed for the Becoming Place during the time period 1969 to 1975. The major findings, which are based on several data sources, can be summarized as follows. (1) Even when a proposed educational change is characterized by ideals of maximum involvement in the decision making process for all members of a college community, resistance to the change can be very substantial. (2) Ironically, little or no involvement by the faculty, students, and alumni in the initial decision to innovate was a major factor for the resistance to educational social change. The research is concluded with a consideration of the findings in terms of possible implications for the future of small colleges.An earlier version of this paper was presented at the annual meeting of the Southern Sociological Society, Knoxville, TN, April 1980.     

The Effect of Functional Flow Diagrams on Apprentice Aircraft Mechanics' Technical System Understanding

Cognitive theory suggests that a key to expert performance lies in the internal organization of the expert's knowledge. The authors contend that the type of technical illustration used during instruction influences knowledge organization and greatly impacts students' understanding of the content. This paper describes an experimental study that tested the impact of one type of conceptual illustration on students' understanding of the structure, function, and behavior of complex technical systems. The results show that supplementing traditional technical instruction with functional flow diagrams can improve overall system understanding. The functional flow diagrams were also found to be an effective instructional aid for enhancing students' conceptual understanding of the causal behavior of systems. In addition, the use of the functional flow diagram was found to significantly improve the subjects' ability to construct conceptual models that were similar to those of an expert. The implications of using conceptual diagrams for technical instruction are discussed and recommendations for future research in this area are provided.