Pedagogical Considerations for Examining Rater Variability in Rater‐Mediated Assessments: A Three‐Model Framework

Rater‐mediated assessments are a common methodology for measuring persons, investigating rater behavior, and/or defining latent constructs. The purpose of this article is to provide a pedagogical framework for examining rater variability in the context of rater‐mediated assessments using three distinct models. The first model is the observation model, which includes ecological/environmental considerations for the evaluation system. The second model is the measurement model, which includes the transformation of observed, rater response data to linear measures using a measurement model with specific requirements of rater‐invariant measurement in order to examine raters’ construct‐relevant variability stemming from the evaluative system. The third model is the interaction model, which includes an interaction parameter to allow for the investigation into raters’ systematic, construct‐irrelevant variability stemming from the evaluative system. Implications for measurement outcomes and validity are discussed.     

Making the Case for Early Identification and Intervention for Young Children at Risk for Learning Disabilities

The early identification of children with learning disabilities (LD) is difficult but can be accomplished. Observation of key behaviors which are indicators of LD by preschool and kindergarten teachers can assist in this process. This early identification facilitates the use of intervention strategies to provide a positive early experience for children at risk for academic difficulties.     

Working and Playing Together: Prediction of Preschool Social-Emotional Competence from Mother-Child Interaction

The impact of mother-child interaction on preschoolers' social-emotional competence was evaluated. Twenty-five girls and 23 boys (mean age = 44 mos) and their mothers were videotaped performing four play/teaching tasks. Preschool teachers completed the Baumrind Preschool Behavior Q-Sort and the Behar Problem Behavior Questionnaire. Mother-child interaction aggregates represented task orientation, positive emotion, and allowance of autonomy/reliance on mother. Results indicated that specific maternal interaction aggregates coherently predicted the children's Positive Social Behavior, Assertiveness, and Sadness in the preschool setting. Child aggregates were less effective predictors of teachers' ratings. Several gender differences in the prediction of social-emotional competence were found (e.g., maternal allowance of autonomy predicted girls' sadness, but boys' sadness was predicted by their own negative emotion). These findings' contributions to the parent-child literature are discussed.     

Gender effects in children's beliefs about school performance: a cross-cultural study

Do young boys and girls understand what leads to academic success (e.g., talent, effort, good teaching, luck) in the same way? Do young girls and boys have equivalent perceptions of their academic competence? Are these beliefs engendered in the same way across sociocultural contexts? In a cross-cultural study of over 3,000 children in grades 2 to 6, ages 7.2 to 13.6, we discovered that boys and girls around the world have very similar ideas about what generally leads to academic success. Moreover, in the few contexts where boys' and girls' academic performances were equal, their beliefs were also equal. However, when girls outperformed boys, their beliefs in their own talent were no greater than boys' beliefs, even though they did have stronger beliefs than boys in other facets of their achievement potential (e.g., putting forth effort, being lucky, getting their teacher's help). Our findings support the generally close correspondence between children's achievement and their competence-related beliefs, with the exception that young girls appear to specifically discount their talent. The effects held regardless of the children's achievement, intelligence, or age (approximately 8 to 13 years). Girls were more biased in some contexts than in others, however, suggesting that competence-related biases are rooted in culture-specific aspects of school settings.     

Microcounseling: Techniques in Assessing Clients' Attitudes toward Guidance Tests

Microcounseling techniques were used to conduct an experimental study. The objective was to train naive college students who were interested in counseling to assess clients' attitudes toward guidance tests. A correlation of .797 was obtained between clients' ratings of attitudes toward tests and the trainees' ratings of clients' attitudes toward tests. Findings were consistent with the results of other microcounseling studies. Individual differences among the trainees were noted and comparisons were made between the trainees' own ratings, judges' ratings, and clients' ratings of trainees. Lack of a control group imposes a serious limitation on interpretation of the results.     

Using Family Backpacks as a Tool to Involve Families in Teaching Young Children About Healthy Eating

Children’s obesity rates have increased substantially over the past several decades, due in part to unhealthy eating habits. About 75% of preschool-aged children consume fewer fruits and vegetables than recommended for health. Because children begin developing eating habits during early childhood, obesity prevention programs are increasingly targeting young children in early childhood education (ECE) settings and are involving their families in teaching about healthy eating. The purpose of this pilot study is to assess the effectiveness of a family involvement activity known as the family backpack in increasing fruit and vegetable consumption among 4- and 5-year-old children and their parents through hands-on activities that encourage them to explore and discuss fruits and vegetables at home. Forty-two families (22 experimental, 20 control) participated in the study. Families received a family backpack from their child’s ECE teacher and completed activities with their child at home during a specific week. Experimental-group backpacks contained a children’s book and three activities to encourage parent–child discussion about fruits and vegetables; control-group backpacks contained an unrelated book and activities. Children’s and parents’ fruit and vegetable consumption and frequency of serving fruits and vegetables at home were assessed using parent self-report surveys before and after completing backpack activities. Parents in the experimental group, but not the control group, reported increases in their own and their children’s fruit and vegetable consumption after completing family backpack activities. Family backpacks show promise as a tool for early childhood educators to help families reinforce and expand children’s learning and encourage healthy eating habits at home.     

Facilitating small-group learning: A comparison of student and staff tutors' behavior

This study focused on students' observations of student and staff tutors' behavior during two academic courses, using a thirtynine-item rating scale. The study took place within an integrated problem-based law curriculum. Six major factors in tutors' behavior were identified. Differences between student and staff tutors' performance were investigated. The results showed that student tutors were better at understanding the nature of the problems students face in attempting to master the subject-matter. Student tutors were also more interested in students' daily lives, study experiences and personalities. In addition, student tutors referred to end-of-course examinations more frequently than staff tutors to direct student learning. Alternatively, staff tutors used their subject-matter expertise more often and displayed more authoritarian behavior than student tutors. No differences were found with respect to tutors' focus on cooperation among group members. The results are interpreted in terms of the nature of the knowledge and experiences of students and staff with regard to problem-based learning and its requirements.     

Conceptual change through vicarious learning in an authentic physics setting

Recent research on principles of best practice for designing effective multimedia instruction has rarely taken into account students’ alternative conceptions, which are known to strongly influence learning. The goal of this study was to determine how well students of quantum mechanics could learn ‘vicariously’ by watching a student-tutor dialogue based on alternative conceptions. Two video treatments were created to summarize key aspects of quantum tunneling, a fundamental quantum mechanical phenomenon. One video depicted a student-tutor dialogue, incorporating many of the common alternative conceptions on the topic, and resolving inconsistencies in reasoning through discussion. The other presented the same correct physics material in an expository style without alternative conceptions. Second year physics students were randomly assigned to one of the two treatments and were tested before and after watching the video during a lecture. Results show a statistically significant (p < .01) advantage for the learners in the dialogue treatment (d = 0.71). Follow-up interviews of students yielded insight into the affective and cognitive benefits of the dialogue video.