Assessing the writing of deaf college students: reevaluating a direct assessment of writing

The NTID Writing Test was developed to assess the writing ability of postsecondary deaf students entering the National Technical Institute for the Deaf and to determine their appropriate placement into developmental writing courses. While previous research (Albertini et al., 1986; Albertini et al., 1996; Bochner, Albertini, Samar, & Metz, 1992) has shown the test to be reliable between multiple test raters and as a valid measure of writing ability for placement into these courses, changes in curriculum and the rater pool necessitated a new look at interrater reliability and concurrent validity. We evaluated the rating scores for 236 samples from students who entered the college during the fall 2001. Using a multiprong approach, we confirmed the interrater reliability and the validity of this direct measure of assessment. The implications of continued use of this and similar tests in light of definitions of validity, local control, and the nature of writing are discussed.     

Developing Teachers' Classroom Interactions: A Description of a Video Review Process for Early Childhood Education Students

This article describes a video review process for providing feedback to students and documents students' teaching practices using the CLASS in a practicum course and student teaching. Students videotaped themselves in their field-based settings and then met with the course instructors and classmates in small groups to review strengths and challenges of their teaching using the CLASS framework of teacher–child interactions. These videos were also coded by trained CLASS observers across 10 dimensions in the areas of emotional support, classroom organization, and instructional support. Results from preservice teachers' CLASS ratings indicate a pattern similar to national data sets using the CLASS, higher scores in the emotional support and classroom organization domains than in the domain of instructional support. Also, findings from the current study revealed that CLASS scores declined from students' practicum placement to the end of student teaching in the domain of emotional support and specifically in the dimensions of regard for student perspectives and behavior management. The process of using videos for providing feedback in field-based experiences is discussed as well as implications for teacher development in light of students' CLASS scores and changes in CLASS scores.     

Uncovering Multivariate Structure in Classroom Observations in the Presence of Rater Errors

We examine the factor structure of scores from the CLASS‐S protocol obtained from observations of middle school classroom teaching. Factor analysis has been used to support both interpretations of scores from classroom observation protocols, like CLASS‐S, and the theories about teaching that underlie them. However, classroom observations contain multiple sources of error, most predominantly rater errors. We demonstrate that errors in scores made by two raters on the same lesson have a factor structure that is distinct from the factor structure at the teacher level. Consequently, the “standard” approach of analyzing on teacher‐level average dimension scores can yield incorrect inferences about the factor structure at the teacher level and possibly misleading evidence about the validity of scores and theories of teaching. We consider alternative hierarchical estimation approaches designed to prevent the contamination of estimated teacher‐level factors. These alternative approaches find a teacher‐level factor structure for CLASS‐S that consists of strongly correlated support and classroom management factors. Our results have implications for future studies using factor analysis on classroom observation data to develop validity evidence and test theories of teaching and for practitioners who rely on the results of such studies to support their use and interpretation of the classroom observation scores.     

Rater calibration when observational assessment occurs at large scale: Degree of calibration and characteristics of raters associated with calibration

Observational assessment is used to study program and teacher effectiveness across large numbers of classrooms, but training a workforce of raters that can assign reliable scores when observations are used in large-scale contexts can be challenging and expensive. Limited data are available to speak to the feasibility of training large numbers of raters to calibrate to an observation tool, or the characteristics of raters associated with calibration. This study reports on the success of rater calibration across 2093 raters trained by the Office of Head Start (OHS) in 2008–2009 on the Classroom Assessment Scoring System (CLASS), and for a subsample of 704 raters, characteristics that predict their calibration. Findings indicate that it is possible to train large numbers of raters to calibrate to an observation tool, and rater beliefs about teachers and children predicted the degree of calibration. Implications for large-scale observational assessments are discussed.     

Predicting Kindergarteners' Achievement and Motivation From Observational Measures of Teaching Effectiveness

We investigated the premise that observation measures of instruction are indicators of effective teaching, using the definition of effectiveness articulated by departments of education: teaching that boosts student achievement. We argued that student motivation is equally as important as achievement in the evaluation of teaching effectiveness (TE); therefore, we examined students' (N = 145) achievement and motivation outcomes. We scored 40 lessons (from 10 kindergarten teachers) with two TE observation measures: the content-independent Classroom Assessment Scoring System (CLASS) and the content-specific Reformed Teaching Observation Protocol (RTOP). We found that the two measures' scores were related differently to student outcomes. Instructionally supportive practices (CLASS and RTOP total) predicted achievement and motivation. Emotional support (CLASS) was positively related to motivation but not to achievement. Classroom organization (CLASS) was negatively related to both motivation and achievement. The CLASS total score did not predict student outcomes; its use masked differences across domains of teaching practices.     

DAP in the ’hood: Perceptions of child care practices by African American child care directors caring for children of color

This paper presents a qualitative analysis concerning child care practices by six African American directors of subsidized child care centers located in a low-income, racial ethnic minority area of Los Angeles, California. These programs are traditionally African American programs that experienced an influx of Latino immigrant enrollment. Using the integrative framework for children of color proposed by Garcia Coll et al. [Garcia Coll, C., Crinic, K., Lamberty, G., Wasik, B., Jenkins, R., Garcia Vazquez, H., et al. (1996). An integrative model for the study of developmental competencies in minority children. Child Development, 67, 1891–1914], which places racial ethnicity in the center, we examined how these directors integrated professional standards of practice (developmentally appropriate practice) with community understandings of the role of preschool/child care in this community; the function of social stratification on their articulated practices; and their understanding as to how they include the Latino immigrant families. We discovered patterns that reflect a community-interpreted understanding of developmentally appropriate practice.     

Maternal employment and child cognitive outcomes in the first three years of life: the NICHD Study of Early Child Care. National Institute of Child Health and Human Development

With increased numbers of women employed in their children's first year of life and with increased attention being paid by parents and policy makers to the importance of early experiences for children, establishing the links that might exist between early maternal employment and child cognitive outcomes is more important than ever. Negative associations between maternal employment during the first year of life and children's cognitive outcomes at age 3 (and later ages) have been reported using data from the National Longitudinal Survey of Youth-Child Supplement. However, it was not known whether these findings would be replicated in another study, nor whether these results were due to features of child care (e.g., quality, type), home environment (e.g., provision of learning), and/or parenting (e.g., sensitivity). This study explored these issues using data on 900 European American children from the National Institute of Child Health and Human Development Study of Early Child Care, which provides information on child cognitive scores at 15, 24, and 36 months, as well as data about the home environment (as assessed by the Home Observation of the Measurement of the Environment Scale), parental sensitivity, and child-care quality and type over the first 3 years of life. Maternal employment by the ninth month was found to be linked to lower Bracken School Readiness scores at 36 months, with the effects more pronounced when mothers were working 30 hr or more per week and with effects more pronounced for certain subgroups (i.e., children whose mothers were not sensitive, boys, and children with married parents). Although quality of child care, home environment, and maternal sensitivity also mattered, the negative effects of working 30 hr or more per week in the first 9 months were still found, even when controlling for child-care quality, the quality of the home environment, and maternal sensitivity. Implications for policy are also discussed.     

Rater accuracy and training group effects in Expert- and Supervisor-based monitoring systems

This study evaluated rater accuracy with rater-monitoring data from high stakes examinations in England. Rater accuracy was estimated with cross-classified multilevel modelling. The data included face-to-face training and monitoring of 567 raters in 110 teams, across 22 examinations, giving a total of 5500 data points. Two rater-monitoring systems (Expert consensus scores and Supervisor judgement of correct scores) were utilised for all raters. Results showed significant group training (table leader) effects upon rater accuracy and these were greater in the expert consensus score monitoring system. When supervisor judgement methods of monitoring were used, differences between training teams (table leader effects) were underestimated. Supervisor-based judgements of raters’ accuracies were more widely dispersed than in the Expert consensus monitoring system. Supervisors not only influenced their teams’ scoring accuracies, they overestimated differences between raters’ accuracies, compared with the Expert consensus system. Systems using supervisor judgements of correct scores and face-to-face rater training are, therefore, likely to underestimate table leader effects and overestimate rater effects.     

Threshold for noise in daycare: Noise level and noise variability are associated with child wellbeing in home-based childcare

This study focuses on the association of noise levels and noise variability with children's wellbeing in home-based childcare. The sample contained 103 toddlers with their caregivers. Wellbeing was observed by independent researchers during three 8-minute episodes of regular childcare activity. Noise levels in decibels were measured during these observation episodes. Average noise level and noise variability (i.e., the average standard deviation of noise level across observation episodes) were related to child wellbeing, over and above child and childcare characteristics. Both relations were nonlinear: only when observed scores rose above a critical threshold, noise and noise variability were negatively related to wellbeing. We found no interaction effects for child gender, age, and temperament. Results indicate that noise variability as well as average noise level matters for child wellbeing in home-based childcare.     

Accounting for Rater Effects With the Hierarchical Rater Model Framework When Scoring Simple Structured Constructed Response Tests

Many large‐scale assessments are designed to yield two or more scores for an individual by administering multiple sections measuring different but related skills. Multidimensional tests, or more specifically, simple structured tests, such as these rely on multiple multiple‐choice and/or constructed responses sections of items to generate multiple scores. In the current article, we propose an extension of the hierarchical rater model (HRM) to be applied with simple structured tests with constructed response items. In addition to modeling the appropriate trait structure, the multidimensional HRM (M‐HRM) presented here also accounts for rater severity bias and rater variability or inconsistency. We introduce the model formulation, test parameter recovery with a focus on latent traits, and compare the M‐HRM to other scoring approaches (unidimensional HRMs and a traditional multidimensional item response theory model) using simulated and empirical data. Results show more precise scores under the M‐HRM, with a major improvement in scores when incorporating rater effects versus ignoring them in the traditional multidimensional item response theory model.     

Impact of model-based science curriculum and instruction on elementary students' explanations for the hydrosphere

Developing scientific literacy about water systems is critical for K-12 students. However, even with opportunities to build knowledge about the hydrosphere in elementary classrooms, early learners may struggle to understand the water cycle (Forbes et al., 2015 ; Gunckel et al., 2012 ; Zangori et al., 2015 ; Zangori et al., 2017 ). Scientific modeling affords opportunities for students to develop representations, make their ideas visible, and generate model-based explanations for complex natural systems like the water cycle. This study describes a comprehensive evaluation of a 5-year, design-based research project focused on the development, implementation, revision, and testing of an enhanced, model-centered version of the Full Option Science System (FOSS) Water (2005) unit in third grade classrooms. Here, we build upon our previous work (Forbes et al., 2015 a; b; Vo et al., 2015 ; Zangori et al., 2015 ; Zangori et al., 2017 ) by conducting a comparative analysis of student outcomes in two sets of classrooms: (1) one implementing the modeling-enhanced version of the FOSS Water unit developed by the research team (n = 6), and 2) another using the standard, unmodified version of the same curricular unit (n = 5). Results demonstrate that teachers in both conditions implemented the two versions of the curriculum with relative fidelity. On average, students exposed to the modeling-enhanced version of the curriculum showed greater gains in their model-based explanations for the hydrosphere. Engagement in scientific modeling allowed students to articulate hydrologic phenomena by (1) identifying various elements that constitute the hydrosphere, (2) describing how these elements influenced the movement of water in the hydrosphere, and (3) demonstrating underlying processes that govern the movement of water in the hydrosphere.     

Family Stress Processes and Children's Self-Regulation

Economic hardship can affect children's development through child–caregiver interactions, which may mediate cascading effects of other family stress processes. This study examined, simultaneously, the relations of financial strain, caregiver general stress, and child–caregiver conflict—each measured at two time points—with child self-regulatory outcomes in a high-poverty sample (age 5–7 years; n = 343). Increase in child–caregiver conflict mediated negative relations between other processes and development of executive function. In contrast, only increase in financial strain had direct, negative association with development of delay of gratification and did not significantly mediate relations between any other process and children's outcomes. Results have implications for understanding effects of family stress on self-regulatory outcomes and for interventions with low-income families.     

A typical morning in preschool: Observations of teacher–child interactions in German preschools

The study examined the applicability and generalizability of the Classroom Assessment Scoring System Pre-K (CLASS Pre-K; Pianta, La Paro, & Hamre, 2008) and the associated conceptual Teaching through Interaction framework to understand classroom processes in the German early education system. Three broad domains describe effective teacher–child interactions: Emotional Support, Classroom Organization, and Instructional Support. In the present study, we observed teacher–child interactions in 63 classrooms drawn from 26 different preschools using the CLASS Pre-K. Consistent with research from the United States, CLASS Pre-K scores demonstrated that the quality of teacher–child interactions varied widely. Data indicated that the levels of Emotional Support and Classroom Organization were moderate. In contrast, the level of Instructional Support was rather low and even decreased over the course of the morning. Furthermore, Emotional Support was found to decrease over the day in classrooms with a higher child–teacher ratio. Results have important implications for policy and practice with regard to the quality of care and education in German preschools.     

Child race and gender as moderators of the association between teacher–child relationships and school adjustment

This investigation examines teacher and child perceptions of teacher–child relationships and early school adjustment among children (N = 157) in low‐income urban schools. Consistent with prior research, findings indicated that teacher–child relationships were associated with early school adjustment; however, the strength of this association varied depending on perspective (i.e., teacher vs. student) and by student characteristic. Associations between predictor and criterion variables were stronger for within‐rater perspectives than across raters. Children's race moderated the association between teacher–child relationships and early school adjustment when examining data from the same source (i.e., teacher) and across raters (i.e., teacher and student). Gender did not moderate these associations. The implications of these findings for future research focused on understanding teacher–child relationships are discussed. © 2008 Wiley Periodicals, Inc.     

Children’s resilience and trauma-specific cognitive behavioral therapy: Comparing resilience as an outcome,a trait,and a process

Resilience, which is associated with relatively positive outcomes following negative life experiences, is an important research target in the field of child maltreatment (Luthar et al., 2000). The extant literature contains multiple conceptualizations of resilience, which hinders development in research and clinical utility. Three models emerge from the literature: resilience as an immediate outcome (i.e., behavioral or symptom response), resilience as a trait, and resilience as a dynamic process. The current study compared these models in youth undergoing trauma-specific cognitive behavioral therapy. Results provide the most support for resilience as a process, in which increase in resilience preceded associated decrease in posttraumatic stress and depressive symptoms. There was partial support for resilience conceptualized as an outcome, and minimal support for resilience as a trait. Results of the models are compared and discussed in the context of existing literature and in light of potential clinical implications for maltreated youth seeking treatment.     

Concurrent Validity of the Classroom Strategies Scale–Teacher Form: A Preliminary Investigation

The present study investigated the concurrent validity of the Classroom Strategies Scale–Teacher Form (CSS-T), a multidimensional teacher formative assessment of instructional and behavioral management practices. The CSS-T is compared with the Classroom Assessment Scoring System (CLASS), a well-known teacher assessment of overall classroom quality. A sample of 126 kindergarten through 5th-grade general education teachers self-reported on their usage of empirically supported instructional and behavioral management strategies as measured by the CSS-T while a certified independent observer completed the CLASS. Correlational analyses were used to compare CSS-T frequency and discrepancy scores and the CLASS scores. As hypothesized, results demonstrated significant positive (CSS-T frequency scale scores) and negative (CSS-T discrepancy scale scores) correlations between specific CLASS domains and dimensions, thus providing initial evidence for the concurrent and discriminant validity of the CSS-T. Implications of findings are discussed.     

Creating cooperation and empowerment in a child care center

Staff turnover in child development centers is epidemic, reaching as high as 41% (Whitebook, Howes, & Phillips, 1989a). Studies have shown that high rates of turnover are related to poor-quality child care, having detrimental effects on children in child care facilities (Clarke-Stewart, 1987; Galinsky, 1990; Whitebook et al., 1982). Variables associated with high staff turnover are low wages, poor working conditions, and minimal benefits.Michael Havercamp and Joanne Everts are with the Department of Human Development and Family Studies, University of Nevada Reno.     

Teacher–Child Interactions in Chile and Their Associations With Prekindergarten Outcomes

Quality of teacher–child interactions is central to prekindergarten children's learning. In the United States, the quality of teacher–child interactions is commonly assessed using the teaching through interactions conceptual framework and an associ/ated observational tool, the Classroom Assessment Scoring System (CLASS). This study examined: (a) whether there was evidence supporting three distinctive domains of teacher–child interactions in Chile (construct validity) and (b) whether these domains predicted end‐of‐prekindergarten language, academic, and executive function skills in Chile (predictive validity). The sample consisted of 91 Chilean prekindergarten classrooms (1,868 four‐year‐old children). The findings support both construct and predictive validity of the teaching through interactions conceptual framework as assessed by the CLASS in Chile. Implications for cross‐country comparison of quality of teacher–child interactions in prekindergarten classrooms are discussed.