Critical Thinking in Preservice Teachers: A Rubric for Evaluating Argumentation and Statistical Reasoning

The authors implemented a cognitive apprenticeship learning community (Collins, Brown, & Newman, 1989) in a class of preservice teachers at the University of Missouri - Columbia to enhance their argumentation and critical thinking skills about complex, educational problems. A detailed rubric based on the literature in argumentation and scientific reasoning was developed to evaluate the students' performance before and after the intervention. The rubric was designed to measure students' (a) conceptions and use of evidence, (b) notions about research and its applicability in evaluating complex social problems, and (c) ability to consider alternative perspectives. This article describes the intervention, the theoretical underpinnings of the rubric, and the use of the rubric in measuring the development of critical thinking in a group of preservice teachers. Strengths and weaknesses of the rubric are outlined and avenues for future research are discussed.     

The effect of rating augmentation on inter-rater reliability: An empirical study of a holistic rubric

A two-stage process by which a holistic rubric is applied to the assessment of open-ended items, such as writing samples, is defined. The first stage involves scoring a performance by the assignment of an integer rating that is congruent with the proficiency level that is exhibited in the performance. The second stage is the subsequent assignment by the rater of an augmentation that indicates whether or not the writing competency reflected in the paper is a bit higher or lower than the competency level reflected in the benchmark paper for the given proficiency level. If the rater feels that the paper represents benchmark proficiency for the given level, no augmentation is assigned to the rating. The results of this study indicate that the use of rating augmentation can improve the inter-rater reliability of holistic assessments, as indicated by generalizability phi coefficients, correlation coefficients, and percent agreement indices. Implications and suggestions for follow-up research are discussed.     

Thinking Styles: Their relationships with modes of thinking and academic performance

This study aimed at investigating the nature of thinking styles as described in the theory of mental self-government. Two-hundred-and-twelve US university students responded to the Thinking Styles Inventory and the Styles of Learning and Thinking. Results from convergent statistical analysis procedures indicated that thinking styles and modes of thinking share certain common variance in the data. It was evident that the more creativity-generating and more complex thinking styles are significantly related to a holistic mode of thinking, and that the more norm-conforming and more simplistic thinking styles are significantly related to an analytic mode of thinking. Furthermore, multiple-regression analyses showed that both thinking styles and modes of thinking statistically contributed to students' self-reported grade point averages beyond what was explained by their self-rated ability scores. These findings are discussed in terms of practical implications for educators.     

The validity of performance assessments for teacher licensure: Connecticut''s ongoing research

This article was based on a paper presented at the annual meeting of the American Educational Research Association, San Francisco, March 27 to March 31, 1989.     

Action learning and critical thinking: a synthesis of two models

Recent scholarship and the news media have identified a lack of critical thinking and ethical behavior in the business world. These deficiencies have led to faulty decision-making, ineffective planning, and frequent organizational dysfunction. This situation has focused attention on both practitioners in the field of business and on the university programs that educate them. A number of upper-tier universities have begun to address these important issues by exploring ways of revising their Master of Business Administration programs to place greater emphasis on their graduates' ability to think critically and consider ethical implications before and after taking action. The authors propose a potential means of addressing these issues in both the business and academic environments through a synthesis of two well-established models based on the constructs of critical thinking and action learning. This synthesis has the potential to produce symbiotic and synergistic effects that may provide educators and practitioners with a new tool for encouraging critical thinking and ethical behavior. The implications for future research and practice are also discussed.     

Canadian and Japanese teachers' conceptions of critical thinking: a comparative study

Canadian and Japanese secondary teachers' conceptions of critical thinking were compared and contrasted. Significant cross‐cultural differences were found. While Canadian teachers tended to relate critical thinking to the cognitive domain, Japanese teachers emphasized the affective domain. The quantitative data, effectively reduced through factor analysis, showed a clear distinction and significant difference between Canadian and Japanese teachers on four of the five factors—Canadian teachers viewed critical thinking through Cognitive Strategizing and Relevance, while Japanese teachers favoured Conscientious Judgements and Intellectual Engagement. However, through further qualitative analysis, critical thinking was found to be a tacit and implicit teaching practice supported by most teachers surveyed, irrespective of gender, age, teaching experience, subject area and, most importantly, the cultures from which they came.     

Critical Thinking as a Source of Respect for Persons: A critique

Critical thinking has come to be defined as and aligned with ‘good’ thinking. It connects to the value placed on rationality and agency and is woven into conceptions of what it means to become a person and hence deserve respect. Challenges to the supremacy of critical thinking have helped to provoke richer and fuller interpretations and critical thought is prevalent in talk of what it is to become a person and more fundamentally to educate. The capacity for critical thought may indeed be one significant aspect of developed personhood; however an emphasis on critical thought as the main source of respect for persons raises a number of issues about what might therefore be excluded or neglected. A number of alternative views that try to retrieve a more ‘humanised’ view of how we exist in the world are examined and are found to suggest that human consciousness as a mark of personhood should be seen as rooted in bodily senses and a more aesthetic orientation towards the world that moves us away from critical thought and rationality as the single indicators of ‘good’ thinking.     

Investigating the quality of the school technology needs assessment (STNA) 3.0: A validity and reliability study

Schools and districts should use a well-designed needs assessment to inform important decisions about a range of technology program areas. Presently, there is a lack of valid and reliable instruments available and accessible to schools to effectively assess their educational needs to better design and evaluate their projects and initiatives. The School Technology Needs Assessment (STNA) is a free, user-friendly online survey tool that meets this need for planning and formative evaluation of technology projects in educational settings. This study used existing data from a robust sample (n = 1918) of educators from across North Carolina to examine the reliability and validity of STNA. A collective review of study results including the literature review, exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and internal consistently reliability analysis indicated that STNA was a high-quality instrument.     

Oral versus written assessments: a test of student performance and attitudes

Student performance in and attitudes towards oral and written assessments were compared using quantitative and qualitative methods. Two separate cohorts of students were examined. The first larger cohort of students (n = 99) was randomly divided into ‘oral’ and ‘written’ groups, and the marks that they achieved in the same biology questions were compared. Students in the second smaller cohort (n = 29) were all examined using both written and oral questions concerning both ‘scientific’ and ‘personal development’ topics. Both cohorts showed highly significant differences in the mean marks achieved, with better performance in the oral assessment. There was no evidence of particular groups of students being disadvantaged in the oral tests. These students and also an additional cohort were asked about their attitudes to the two different assessment approaches. Although they tended to be more nervous in the face of oral assessments, many students thought oral assessments were more useful than written assessments. An important theme involved the perceived authenticity or ‘professionalism’ of an oral examination. This study suggests that oral assessments may be more inclusive than written ones and that they can act as powerful tools in helping students establish a ‘professional identity’.     

Psychological knowledge for teaching critical thinking: the agency of epistemic activity,metacognitive regulative behaviour and (student-centred) learning

This study considers the case of a tutor whose students repeatedly evidenced significantly superior critical thinking in summative assessment. For the purpose of surfacing appropriate pedagogical action to promote critical thinking (Bassey, Case Study Research in Educational Settings, 1999), the singularity of one tutor’s reported pedagogical practice was explored through focus-group discussion. Qualitative analysis of the data, theoretically informed by phenomenography, suggested that the tutor’s reported practice, when compared with that of two peers, revealed clear pedagogical intentions to be necessary for teaching critical thinking; and that these intentions can be explained through the literatures on epistemic activity, metacognitive regulative behaviour and student-centred learning. It is argued that a synthesised understanding of the literature that explores the nature and purpose of critical thinking —as outlined in the first part of this paper—is a prerequisite for constructing domain-specific pedagogical intentions for developing learners’ critical thinking, and that it is this extensive psychologically informed knowledge base which attenuates the risk of educationally important aspects of learning being overlooked. (De Corte, Learning and Instruction 10:249–266, 2000).     

Validity inquiry process: Practical guidance for examining performance assessments and building a validity argument

Given the increased use of performance assessments (PAs) in higher education to evaluate achievement of learning outcomes, it is important to address the barriers related to ensuring quality for this type of assessment. This article presents a design-based research (DBR) study that resulted in the development of a Validity Inquiry Process (VIP). The study’s aim was to support faculty in examining the validity and reliability of the interpretation and use of results from locally developed PAs. DBR was determined to be an appropriate method because it is used to study interventions such as an instructional innovation, type of assessment, technology integration, or administrative activity (Anderson & Shattuck, 2012). The VIP provides a collection of instruments and utilizes a reflective practice approach integrating concepts of quality criteria and development of a validity argument as outlined in the literature (M.T. Kane, 2013; Linn, Baker, & Dunbar, 1991; Messick, 1994).     

Beyond translation: adapting a performance-task-based assessment of critical thinking ability for use in Rwanda

Critical thinking is frequently proposed as one of the most important learning outcomes of a university education. However, to date, it has been difficult to ascertain whether university students in low-income contexts are improving in their critical thinking skills, because the limited studies in this domain have relied on instruments developed in Western contexts, despite the clear dangers of such an approach. Cultural bias in assessment can best be overcome by explicitly developing tests for use in specific contexts. However, resource constraints often prevent this possibility. An alternative strategy is to adapt an existing instrument for use in a particular context. Although adaptation is the norm for high-stakes cross-cultural assessments, it is often not attempted for single country research studies. This may be due to an assumption that adaptation is excessively technical or will add significantly to a study timeline. In this article, which relies on data from a recent study in Rwanda, we present a methodology for adapting a performance-task-based assessment of critical thinking. Our experience with this methodology suggests that small teams can adapt instruments in a relatively short time frame, and that the benefits of doing so far outweigh any cost.     

某类解析函数的Fekete-Szeg(o)不等式

引入了两个新的函数类Nα,b(φ)和Nλα,b(φ),讨论了这两个函数类的Fekete-Szeg(o)不等式,得到了准确的结果,推广了一些相关结果.     

Interrater reliability of the original and a revised scoring system for the developmental test of visual-motor integration

This study investigated the interrater reliability of teachers' and school psychology ex-terns' scoring of protocols for the Developmental Test of Visual-Motor Integration (VMI). Previous studies suggest that the scoring criteria of the VMI are ambiguous, which when coupled with raters' lack of scoring experience, as well as limited knowledge of testing issues, contributes to low rater reliability. The original manual scoring system was used by four trained teachers with no VMI experience and by four experienced raters. A VMI scoring system, revised to eliminate ambiguous scoring criteria, was used by an additional four teachers inexperienced with the VMI and by four experienced raters. High reliability coefficients (>.90) were found for all raters, regardless of the scoring system employed. The influence on interrater reliability of factors such as training, nature of the training setting, characteristics of the raters, and ambiguity of scoring criteria is discussed.     

Leniency and halo bias in industry-based assessments of student competencies: a critical,sector-based analysis

Industry placements are popular means to provide students with an opportunity to apply their skills, knowledge and experience in a ‘real world’ setting. Within this context, supervisor feedback allows educators to measure students’ performance beyond academic objectives, by benchmarking it against industry expectations. However, industry assessments appear to be frequently clouded and overwhelmingly positive by nature, which questions the reliability and validity of supervisors’ judgement of competencies. Supervisor bias has been paid much attention within the context of clinical placements, as well as within the domain of social work. However, the concept has been largely ignored within business education, despite the increasing emphasis on – and deep integration of – work-integrated learning in the business curriculum. This paper sets out to address this gap by examining variances in mark distribution and apparent leniency in the context of a final-year, compulsory placement unit, based on observations and data collected over nine semesters (n?=?546). The focus of this study is on gaining an understanding of the reasons behind assessment bias and the pressures placed on industry assessors. The data indicate that different types of placement locations apply dissimilar standards when assessing student performance. The author identifies three statistically different placement types (small business, not-for-profit and professional), which influence the strength and risk of grader bias, hence ultimately the assessment outcome.     

Establishing the validity and reliability of a modified tool for assessing innovative thinking of engineering students

Abstract

The goal of this study was to establish the validity and reliability of a self-report tool for assessing individual differences in innovative thinking, based on a modification of the ‘innovative behavior scale’. A five-stage study was conducted among engineering students worldwide, who enrolled in a massive open online course in Nanotechnology and Nanosensors. Content validity was established by a panel of experts in engineering education. Construct validity was established through exploratory and confirmatory factor analysis, indicating a four-factor solution with 13 items loaded above critical level. Known-groups validity revealed differences among learners with distinct expertise. The scale’s stability across populations and over time was confirmed, providing there has been no intentional intervention. Finally, the scale’s concurrent validity was reinforced through data triangulation with actual practice - the fabrication of innovative nanosensors.