Teachers’ epistemic cognition in situ: Evidence from classroom assessment

Assessment tasks require the coordination of multiple knowledge-related goals for various audiences, and therefore provide an authentic context to observe teachers’ epistemic cognition in practice. In this instrumental case study, we investigated seven, fifth grade English Language Arts teachers’ epistemic cognition as they evaluated students’ classroom assessments. Our analyses revealed that the components of epistemic cognition identified in the literature emerged in these teachers’ assessment processes. Moreover, we found evidence that teachers’ epistemic cognition was iterative and nuanced, and required shifts in aims and reliable processes. This resulted in teachers forming new kinds of “epistemic matters” and questions beyond those ideas noted in existing models of epistemic cognition. Significance and implications are discussed.     

Content Analysis of Google Alerts for Cultural Heritage Institutions

This article outlines evidence-based methods for cultural heritage institutions to look for news and mentions about their collections and services online. Google Alerts were created for repository names, websites, and finding aid websites for 66 institutions randomly selected from the Archive Grid database. Results were analyzed to determine Google Alerts' accuracy, what types of institutions would benefit most from using Google Alerts, what the content of the sources found by the alerts was, what types of publications returned the most results, and if Google Alerts could also identify image reuse. While Google Alerts' accuracy for returning usable results for analysis was not very high, the structure of alert e-mails makes discarding irrelevant results simple and quick. Mid- to large-sized museums and institutions staging frequent exhibits and events, as well as those with active collections, return the largest number of alerts. News publications and social media publish the most content related to cultural heritage institutions. Though the sample set was small, Google Alerts also was able to identify image reuse.     

Traffic Light Cards: a Cross and Modification Between the Minute Paper and Muddiest Point

Muddiest point and minute paper have been popular techniques to obtain feedback from students. However, focusing on the muddiest point too often can be discouraging for both students and instructors; in minute paper, students may often have off-topic comments. Here a cross and modified technique, the ‘traffic light’ card is introduced, to overcome the drawbacks in muddiest point and minute paper. About 2?minutes before the end of lecture, students “traffic light” the given topics on a notecard, in which their understanding is high (green), partial (yellow), or low (red). The results are used by the instructors to give appropriate review in the next lecture and help students gain better understanding in the content.     

A longitudinal quantitative investigation into the concurrent validity of self and peer assessment applied to English-Chinese bi-directional interpretation in an undergraduate interpreting course

One of the productive lines of research on self-assessment (SA) and peer assessment (PA) concerns their concurrent validity with respect to a criterion measure. However, similar research has rarely been conducted for spoken-language interpreting. This article therefore reports on a longitudinal study that investigated the validity of self and peer ratings on three performance dimensions of English-Chinese consecutive interpretation (i.e., information completeness, fluency of delivery, and target language quality), taking teachers’ ratings as a yardstick. Major findings include: although the students as a group were unable to replicate teachers’ ratings, they were able to rank-order their performances in a fairly accurate manner and improved their SA and PA accuracy over time. Interpreting directionality seems to moderate the correlational strength of self/teacher ratings and peer/teacher ratings. These results are discussed in relation to previous literature, and pedagogical suggestions are provided to improve SA and PA for bi-directional interpretation.     

New curve sprint test for soccer players: Reliability and relationship with linear sprint

ABSTRACT

The speed performance is involved not only in linear sprints, but also in a wide spectrum of multi-directional movements, such as curve sprinting. Curved sprint can be defined as sprint with gradual and continuous change of direction (COD). Although ~85% of the actions performed at maximum velocity in a professional soccer league are curvilinear sprints, there is not any specific test to assess this ability. This study aimed to analyse the reliability of a new curve sprint test, and compare its results with those obtained by soccer players in linear sprint. Forty experienced soccer players performed 3 attempts of curve sprint (using the penalty arc) to right and left side (17 m), and 3 linear sprints (17 m) in two different days. The ICCs (inter-session reliability) were 0.93 for sprint curve right side (CSRS) and 0.89 for sprint curve left side (CSLS), considered “acceptable”. The CVs (intra-session reliability) were 0.87% in CSRS and 1.15% in CSLS. The coefficient of determination (R²) between linear and curve sprinting was ~35%. The association between curve sides was “very large” (r = 0.878; p < 0.01). In summary, we showed that “curve sprint test” is highly reliable, and that curvilinear and linear sprints are different and independent actions.     

The zone of proximal creativity: What dynamic assessment of divergent thinking reveals about students’ latent class membership

Divergent Thinking is a domain-general mental attribute closely associated with creativity that can be quantified through the use of text-mining algorithms. Past research has shown that students’ Divergent Thinking is malleable in response to relatively simple contextual prompts. In addition, there is substantial variance in the degree to which individual students’ Divergent Thinking is malleable, suggesting the presence of a student-specific zone-of-proximal-development in relation to creativity. Here, we adopted a dynamic assessment paradigm that included multiple conditions under which student Divergent Thinking was measured and fit a latent profile analysis model to that dynamic assessment data. We found that, although on average the Originality of student responses can be augmented through a prompt to generate surprising or unusual ideas, three latent classes emerged that differed significantly on their patterns of augmentation. These three latent classes were termed: (a) Conventional Thinkers (7.80% of the sample), whose response to the Divergent Thinking task were highly constrained and unoriginal across all conditions (b) Prompted Shifters (66.56%), whose Originality significantly increased across conditions, and (c) Idea Generators (25.64%), whose responses were highly original across all conditions. These latent profiles were validated in regard to personality characteristics and domain-specific creative activities, with Idea Generators reporting significantly more Openness and Intellect, less Industriousness, and more creative activities across the domains of Literature, Music, Sports, Visual Art, Science, and Cooking than did the other latent classes.     

Automated Scoring of Constructed‐Response Science Items: Prospects and Obstacles

Content‐based automated scoring has been applied in a variety of science domains. However, many prior applications involved simplified scoring rubrics without considering rubrics representing multiple levels of understanding. This study tested a concept‐based scoring tool for content‐based scoring, c‐rater?, for four science items with rubrics aiming to differentiate among multiple levels of understanding. The items showed moderate to good agreement with human scores. The findings suggest that automated scoring has the potential to score constructed‐response items with complex scoring rubrics, but in its current design cannot replace human raters. This article discusses sources of disagreement and factors that could potentially improve the accuracy of concept‐based automated scoring.     

The potential formative benefits of portfolio assessment in second and foreign language writing contexts: A review of the literature

This article identifies the potential formative benefits of using portfolio assessment (PA) in second/foreign language classrooms by reviewing the up-to-date research. Three methods were used to search for relevant research: database search, manual search, and citation search. According to the literature, PA has potential benefits due to its process-oriented, authentic, integrated, interactive, and learner-centered characteristics. PA leads to increased motivation, learner autonomy, and improved writing performance. However, not many studies include observation as a method of collecting data. Through observation a researcher can investigate how the benefits of PA are acted upon and negotiated. There is a need for more empirical research in the juxtaposition of PA and formative assessment, particularly in primary and secondary education.     

Diagnosing Teachers’ Understandings of Rational Numbers: Building a Multidimensional Test Within the Diagnostic Classification Framework

We report a multidimensional test that examines middle grades teachers’ understanding of fraction arithmetic, especially multiplication and division. The test is based on four attributes identified through an analysis of the extensive mathematics education research literature on teachers’ and students’ reasoning in this content area. We administered the test to a national sample of 990 in‐service middle grades teachers and analyzed the item responses using the log‐linear cognitive diagnosis model. We report the diagnostic quality of the test at the item level, mastery classifications for teachers, and attribute relationships. Our results demonstrate that, when a test is grounded in research on cognition and is designed to be multidimensional from the onset, it is possible to use diagnostic classification models to detect distinct patterns of attribute mastery.