The Critical Thinking Analytic Rubric (CTAR): Investigating intra-rater and inter-rater reliability of a scoring mechanism for critical thinking performance assessments |
| |
| Authors: | Emily Saxton Secret Belanger William Becker |
| |
| Institution: | 1. Center for Science Education, Portland State University, 1136 SW Montgomery Street, Portland, OR 97201, United States;2. Century High School, Hillsboro School District, 2000 SE Century Blvd., Hillsboro, OR 97123, United States;1. Center for Research and Development in Dual Language and Literacy Acquisition, Department of Educational Psychology, 407 Harrington, TAMU 4225, Texas A&M University, College Station, TX, 77843, United States;2. Education Leadership Research Center, Educational Administration and Human Resource Development, Texas A&M University, College Station, United States;3. Center for Research and Development in Dual Language and Literacy Acquisition, Department of Educational Psychology, Texas A&M University, College Station, TX, 77843, United States;1. Foreign Languages College, Jiangxi Normal University, 99 Ziyang Road, Nanchang, Jiangxi, 330022 China;2. Department of English Language Education, The Education University of Hong Kong, 10 Lo Ping Road, Tai Po, New Territories, Hong Kong, China;1. Scar Free Foundation Centre for Burns Research, University Hospital Birmingham NHS Foundation Trust, Queen Elizabeth Hospital Birmingham, Mindelsohn Way, Edgbaston, Birmingham, B15 2WB, UK;2. University Hospital Birmingham NHS Foundation Trust, Queen Elizabeth Hospital Birmingham, Mindelsohn Way, Edgbaston, Birmingham, B15 2WB, UK;3. Birmingham Clinical Trials Unit (BCTU), Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham, B15 2TT, UK;4. Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham, B15 2TT, UK;5. School of Chemical Engineering, University of Birmingham, Edgbaston, Birmingham, B15 2TT, UK;6. Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Edgbaston, Birmingham, B15 2TT, UK |
| |
| Abstract: | The purpose of this study was to investigate the intra-rater and inter-rater reliability of the Critical Thinking Analytic Rubric (CTAR). The CTAR is composed of 6 rubric categories: interpretation, analysis, evaluation, inference, explanation, and disposition. To investigate inter-rater reliability, two trained raters scored four sets of performance-based student work samples derived from a pilot study and subsequent larger study. The two raters also blindly scored a subset of student work samples a second time to investigate intra-rater reliability. Participants in this study were high school seniors enrolled in a college preparation course. Both raters showed acceptable levels of intra-rater reliability (α ≥ 0.70) in five of the six rubric categories. One rater showed poor consistency (α = 0.56) for the analysis category of the rubric, while the other rater showed excellent consistency (α = 0.91) for the same category suggesting the need for further training of the former rater. The results of the inter-rater reliability investigation demonstrate acceptable levels of consistency (α ≥ 0.70) in all rubric categories. This investigation demonstrated that the CTAR can be used by raters to score student work samples in a consistent manner. |
| |
| Keywords: | |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
