The impact of individual consultation on student ratings of teaching

Abstract

The present study addressed the impact of individual consultation on teaching improvement as measured by changes in student ratings. Subjects included 91 professors who presented naturally for individual consultation services over a seven‐year period at the teaching centre of a Canadian university. Interventions by the consultant fell into three categories: 1) Feedback‐Consultation, 2) Feedback‐Consultation‐Class Observation, and 3) Feedback‐Consultation‐Class Observation and Student Consultation. End of term student ratings for the course that was the subject of the consultation were compared with student ratings for the same course taught between one and three years prior to the consultation service, and for the same course taught between one to three years following consultation. The results showed that, overall, consultation was effective in improving the quality of the consultees’ teaching, as evidenced by an increase in mean student ratings of instruction. This effect persisted post consultation. Not all intervention groups, however, showed the same pattern of results. Change was evident immediately after the intervention except in the case of brief consultation, although follow‐up data showed improved teaching for the latter group. Control data provided evidence that the change in student ratings post consultation could reasonably be attributed to consultation effects.     

Teaching Family Communication Concepts through Family Stories: An Analysis of Stories and Rituals in David Bradley's “Harvest Home”

Objective: Students will be able to apply the concepts of stories and rituals to an analysis of the ritual in “Harvest Home,” gaining understanding of how stories and rituals affect and reflect family values, power structures and identities.

Course: Family Communication. Especially good for family communication taught online     

Maximizing Learning From Engaging Across Difference: The Role of Anxiety and Meaning Making

This study explores college students’ reactions to engaging in experiences with diverse others that students found uncomfortable, interactions that disrupted students’ current beliefs or values. Our examination of these experiences was grounded in theory and research on intergroup contact and anxiety and developmental models of self-authorship leading to intercultural maturity. We found three types of reactions to these experiences: Some students were “stuck,” unsure of how to respond; some were prone to continue exploring; and some reframed their perspectives to reflect capacities associated with advanced intercultural maturity. We offer recommendations for managing intergroup anxiety and scaffolding learning to support reframing one's perspectives to maximize learning from engaging in difference.     

An Investigation of Explanation Multiple-Choice Items in Science Assessment

Both multiple-choice and constructed-response items have known advantages and disadvantages in measuring scientific inquiry. In this article we explore the function of explanation multiple-choice (EMC) items and examine how EMC items differ from traditional multiple-choice and constructed-response items in measuring scientific reasoning. A group of 794 middle school students was randomly assigned to answer either constructed-response or EMC items following regular multiple-choice items. By applying a Rasch partial-credit analysis, we found that there is a consistent alignment between the EMC and multiple-choice items. Also, the EMC items are easier than the constructed-response items but are harder than most of the multiple-choice items. We discuss the potential value of the EMC items as a learning and diagnostic tool.     

Assessing Knowledge Integration in Science: Construct,Measures, and Evidence

In response to the demand for sound science assessments, this article presents the development of a latent construct called knowledge integration as an effective measure of science inquiry. Knowledge integration assessments ask students to link, distinguish, evaluate, and organize their ideas about complex scientific topics. The article focuses on assessment topics commonly taught in 6th- through 12th-grade classes. Items from both published standardized tests and previous knowledge integration research were examined in 6 subject-area tests. Results from Rasch partial credit analyses revealed that the tests exhibited satisfactory psychometric properties with respect to internal consistency, item fit, weighted likelihood estimates, discrimination, and differential item functioning. Compared with items coded using dichotomous scoring rubrics, those coded with the knowledge integration rubrics yielded significantly higher discrimination indexes. The knowledge integration assessment tasks, analyzed using knowledge integration scoring rubrics, demonstrate strong promise as effective measures of complex science reasoning in varied science domains.     

Conceptualising and measuring student engagement through the Australasian Survey of Student Engagement (AUSSE): a critique

Student engagement has rapidly developed a central place in the quality agenda of Australian universities since the introduction of the Australasian Survey of Student Engagement (AUSSE). The AUSSE is based on one developed in the USA. The main arguments given for adopting this survey in Australia are that it provides a valid instrument for measuring engagement and that it enables international comparisons. However, the survey instrument and scales have been adopted with little scrutiny of these arguments. This paper examines these arguments by considering different perspectives of engagement, examining the importance of contextual differences and evaluating the AUSSE engagement scales in the light of both. The paper concludes that the AUSSE results should be used by universities and policy‐makers with caution.     

Managerial coaching: A review of the literature

Managerial coaching has become increasingly popular despite limited empirical evidence of its impact on the individuals giving and receiving coaching, and its impact on the workplace overall. This article reviews the literature on the definition of practice of managerial coaching, and what managerial coaching looks like in terms of skills and behaviors, and examines the empirical evidence that currently exists. Models representing new current research and gaps within the literature are advanced, and future research implications are explored.     

Validating Measurement of Knowledge Integration in Science Using Multiple-Choice and Explanation Items

This study explores measurement of a construct called knowledge integration in science using multiple-choice and explanation items. We use construct and instructional validity evidence to examine the role multiple-choice and explanation items plays in measuring students' knowledge integration ability. For construct validity, we analyze item properties such as alignment, discrimination, and target range on the knowledge integration scale using a Rasch Partial Credit Model analysis. For instructional validity, we test the sensitivity of multiple-choice and explanation items to knowledge integration instruction using a cohort comparison design. Results show that (1) one third of correct multiple-choice responses are aligned with higher levels of knowledge integration while three quarters of incorrect multiple-choice responses are aligned with lower levels of knowledge integration, (2) explanation items discriminate between high and low knowledge integration ability students much more effectively than multiple-choice items, (3) explanation items measure a wider range of knowledge integration levels than multiple-choice items, and (4) explanation items are more sensitive to knowledge integration instruction than multiple-choice items.     

Diversity among Spanish-speaking English language learners: profiles of early literacy skills in kindergarten

This study explored heterogeneity in literacy development among 2,300 Hispanic children receiving English as a Second Language (ESL) services at the start of kindergarten. Two research questions guided this work: (1) Do Spanish-speaking English language learners receiving ESL services in the fall of kindergarten demonstrate homogeneous early literacy skills, or are there distinct patterns of achievement across measures of phonological awareness, alphabet knowledge, and orthography? and (2) if there are distinct profiles, to what extent do they predict literacy achievement at the end of kindergarten and the beginning of first grade? Using cluster analysis, the authors identified four distinct literacy profiles derived from fall kindergarten measures of phonological awareness, alphabet knowledge, and phonetic spelling. These profiles were found to be associated with literacy outcomes in spring of kindergarten and fall of first grade. The two profiles that were associated with greater success on later measures of concept of word in text, letter sound knowledge, word reading, and spelling were the two that included stronger performance on orthographic skills (i.e., alphabet knowledge and phonetic spelling). These findings demonstrated that there is heterogeneity among Hispanic ESL students at kindergarten entry and suggested that literacy instruction must be differentiated from the very beginning in order to meet students’ individual needs. The findings also suggested that orthographic skills should be assessed and taught early on. While phonological awareness may be a necessary precursor to reading, phonological awareness in the absence of orthographic skills may not be sufficient.     

A Technology‐Enhanced Unit of Modeling Static Electricity: Integrating scientific explanations and everyday observations

What trajectories do students follow as they connect their observations of electrostatic phenomena to atomic‐level visualizations? We designed an electrostatics unit, using the knowledge integration framework to help students link observations and scientific ideas. We analyze how learners integrate ideas about charges, charged particles, energy, and observable events. We compare learning enactments in a typical school and a magnet school in the USA. We use pre‐tests, post‐tests, embedded notes, and delayed post‐tests to capture the trajectories of students’ knowledge integration. We analyze how visualizations help students grapple with abstract electrostatics concepts such as induction. We find that overall students gain more sophisticated ideas. They can interpret dynamic, interactive visualizations, and connect charge‐ and particle‐based explanations to interpret observable events. Students continue to have difficulty in applying the energy‐based explanation.