The effect of rating augmentation on inter-rater reliability: An empirical study of a holistic rubric

A two-stage process by which a holistic rubric is applied to the assessment of open-ended items, such as writing samples, is defined. The first stage involves scoring a performance by the assignment of an integer rating that is congruent with the proficiency level that is exhibited in the performance. The second stage is the subsequent assignment by the rater of an augmentation that indicates whether or not the writing competency reflected in the paper is a bit higher or lower than the competency level reflected in the benchmark paper for the given proficiency level. If the rater feels that the paper represents benchmark proficiency for the given level, no augmentation is assigned to the rating. The results of this study indicate that the use of rating augmentation can improve the inter-rater reliability of holistic assessments, as indicated by generalizability phi coefficients, correlation coefficients, and percent agreement indices. Implications and suggestions for follow-up research are discussed.     

Word Search Puzzle:Places

Instructions:Read the clues (information that helps you to find the answers) and find the words they describe.The words can go any way,but the letters are always next to each other.The first clue is done for you.     

Bare bones,or a rich feast? Taking care with context in a data rich world

Providing a rich context has become a sine qua non of principled teaching of applied statistical thinking. With increasing opportunities to access secondary data, there should be increasing opportunity to work with rich context. We review the contextual information provided in 41 data sets suitable for introductory tertiary statistics teaching, available in the R “datasets” package, and investigate the source information for four data sets. We find failure to describe and retain important contextual information, including aspects that raise questions about the credibility of the data for statistical inference. The sanitization of data reduces the opportunities for learning meaningful lessons in statistical thinking and the real-world application of statistics. We advocate for teachers and users of such data to be curious about the provenance and context, and for the curators and distributors to examine, where possible, the primary sources, to accurately preserve the context and optimize pedagogical opportunities.