Implementing multicultural and global studies: Selected resources about materials and their uses by teacher educators,inservice providers,and K‐12 educators

We conducted a content analysis of 27 assessment textbooks to determine how assessment planning was framed in texts for preservice teachers. We identified eight assessment planning themes: alignment, assessment purpose and types, reliability and validity, writing goals and objectives, planning specific assessments, unpacking, overall assessment plan, and other. Themes were used to code the all texts and evaluated the depth of coverage each theme received: mentioning, elaboration, and how to. Findings indicate that classroom assessment textbooks (a) lack a clear focus on assessment planning, (b) demonstrate wide variation in the depth of coverage with little focus on “how to” related to assessment planning, and (c) lack theoretical connections between assessment and instructional practices.     

Baseline assessment of writing: the need for reconsideration

This article questions the usefulness to teaching of items to be used in baseline assessment of writing in the UK. Current requirements for national baseline assessment are intended to serve two very different purposes: assessment for management and assessment for teaching. It is argued that this duality of purpose can threaten teaching and overemphasise assessment for management. This article describes a small study investigating items on the baseline assessment scale developed by the Schools Curriculum and Assessment Authority (SCAA, 1997a), which suggests that the SCAA criteria for baseline assessment of writing need to be reconsidered. This reconsideration can be sought in the light of research and evidence of children’s achievements. Finally the paper concludes that there needs to be greater clarity of purpose and a redefinition of the misleading term ‘baseline assessment’.     

Identifying Canadian teacher candidates' needs for training in the use of inclusive classroom assessment

To identify teacher candidates' needs for training in inclusive classroom assessment, the present study investigated teacher candidates' beliefs about inclusive classroom assessments for all students educated in regular classrooms, including those with special needs and English language learners. An innovative theoretical assessment model, consisting of four concepts of assessment for, as, of learning as well as accommodations and modifications, was built to compare with questionnaire responses from a group of Canadian teacher candidates (n?=?295). Our data yielded five factors: (1) beliefs about accommodations and modifications for students with diverse needs (ACC), (2) assessment of learning, (3) assessment as learning, (4) misconceptions of inclusive assessment, and (5) negative beliefs about assessment for learning (NAFL). Contrary to expectations, assessment concepts are not fully interrelated with each other in teacher candidates' belief systems. In addition, participating teacher candidates may have some misconceptions about inclusive assessments given that they possessed positive beliefs towards them. The implications for teacher education programmes and professional development were also discussed in this study.     

From substitution to redefinition: A framework of machine learning-based science assessment

This study develops a framework to conceptualize the use and evolution of machine learning (ML) in science assessment. We systematically reviewed 47 studies that applied ML in science assessment and classified them into five categories: (a) constructed response, (b) essay, (c) simulation, (d) educational game, and (e) inter-discipline. We compared the ML-based and conventional science assessments and extracted 12 critical characteristics to map three variables in a three-dimensional framework: construct, functionality, and automaticity. The 12 characteristics used to construct a profile for ML-based science assessments for each article were further analyzed by a two-step cluster analysis. The clusters identified for each variable were summarized into four levels to illustrate the evolution of each. We further conducted cluster analysis to identify four classes of assessment across the three variables. Based on the analysis, we conclude that ML has transformed—but not yet redefined—conventional science assessment practice in terms of fundamental purpose, the nature of the science assessment, and the relevant assessment challenges. Along with the three-dimensional framework, we propose five anticipated trends for incorporating ML in science assessment practice for future studies: addressing developmental cognition, changing the process of educational decision making, personalized science learning, borrowing 'good' to advance 'good', and integrating knowledge from other disciplines into science assessment.     

Principles for Fair Student Assessment Practices for Education in Canada

Developed in response to examples of unacceptable assessment practices, the Principles for Fair Student Assessment Practices for Education in Canada contains a set of principles and guidelines generally accepted by professional organisations as indicative of fair assessment practice within the Canadian educational context. Assessments depend on professional judgment: the ’Principles’ identify the issues to consider in exercising this professional judgment and in striving for the fair and equitable assessment of all students.     

Bases of competence: an instrument for self and institutional assessment

The Bases of Competence model provides a general framework for learner‐centred skill development and programme‐focused outcomes assessment. Based on previous research, the Bases of Competence model describes 17 skills and four base competencies important to graduates to achieve high performance in the workplace. Taking this work from research to relevant educational application as a tool for student self‐assessment and institutional outcomes assessment is the focus of this paper. Results from a multi‐year, multi‐course assessment initiative indicate that students rate themselves stronger in the foundation base competencies of Communicating and Managing Self, and weaker in more complex competencies of Managing People and Tasks and Mobilising Innovation and Change. Comparisons of skill confidence within each base competence as well as between year, student level, gender and beginning versus end of semester are presented as well. These results are discussed and suggestions made for programme design.     

Using teacher inquiry to support technology-enhanced formative assessment: a review of the literature to inform a new method

In this paper we review the literature on teacher inquiry (TI) to explore the possibility that this process can equip teachers to investigate students’ learning as a step towards the process of formative assessment. We draw a distinction between formative assessment and summative forms of assessment [CRELL. (2009). The transition to computer-based assessment: New approaches to skills assessment and implications for large-scale testing. In F. Scheuermann & J. Björnsson (Eds.), JRC Scientific and technical reports. Ispra: Author; Webb, M. (2010). Beginning teacher education and collaborative formative e-assessment. Assessment & Evaluation in Higher Education, 35, 597–618; EACEA. (2009). National testing of pupils in Europe: Objectives, organisation and use of results. Brussels: Eurydice; OECD. (2010b). Assessing the effects of ICT in education (F. Scheuermann & E. Pedró, Eds.). Paris: JRC, OECD]. Our review of TI is combined with a review of the research concerning the way that practices with technology can support the assessment process. We conclude with a comparison of TI and teacher design research from which we extract the characteristics for a method of TI that can be used to develop technology-enhanced formative assessment: teacher inquiry into student learning. In this review, our primary focus is upon enabling teachers to use technology effectively to inquire about their students’ learning progress.     

Promoting assessment efficacy through an integrated system for online clinical assessment of practical skills

This paper presents evaluation outcomes from an externally funded research project involving the online clinical assessment of practical skills (eCAPS) using web-based video technologies within a university medical programme. eCAPS was implemented to trial this web-based approach for promoting the efficacy of practical skills assessment in knee joint examinations for a pre-clinical cohort of second year medical students. eCAPS involves a progressive organisation of online video experiences and task expectations for formative and summative assessments of selected competencies. Data are presented from semi-structured interviews with the medical students (N?=?40). eCAPS successfully supported students’ skill development in knee joint examinations on fellow students and allowed remote assessment of candidates’ performances by clinicians experienced in musculoskeletal examinations. An ‘indicative standard’ paradigm, involving formative assessment of one randomly selected performance from the submission of all students’ responses within a small group, offered an efficient and efficacious avenue for providing consolidated feedback to students and promoted desirable learning behaviours. Overall, there was evidence of reciprocal learning benefits to the ‘blending’ of an online learning and assessment approach with an existing face-to-face environment. The medical students were able to successfully engage online with high quality and consistent practical skill-based materials in a flexible, independent and individual manner.     

Kindergarten entrance assessment: Is it worth the effort?

School psychologists frequently are involved in assessment programs for children entering kindergarten. A comparison is made between screening and assessment related to norm-based and criterion-referenced assessment. Normbased testing can be either categorical or deficit-centered, while the criterion approaches tend to be academic or cognitive-developmental. In order to make appropriate decisions, we need to know the problems and outcomes related to each approach. It is also felt that assessment should be related to the underlying issues in developmental psychology and the type of kindergarten experiences provided for the child.     

Learning Stories – crossing borders: introducing qualitative early childhood observation techniques to early childhood practitioners in Saudi Arabia

Early childhood education has become a focus of government policy across the world. Part of the present increased interest in early childhood education has been a focus on curriculum frameworks and socio/cultural methods of assessment. Currently, New Zealand has emerged as a world leader in early childhood education, and observation and assessment techniques, developed in New Zealand, have become an international focus of research and pedagogic practice. One exemplar practice to have emerged from research in New Zealand is the assessment of children's learning. An assessment project, conducted at the instigation of the New Zealand Ministry of Education, was designed to recognise key outcomes from the New Zealand curriculum, Te Whāriki, and to provide practitioners with a tool that would assist in the development of assessment ideas and procedures. The result was Learning Stories. This present research explored the introduction of Learning Stories into Australia and investigated the potential of Learning Stories as an assessment tool for early childhood practitioners in the context of Saudi Arabia.     

Educational Assessment Knowledge and Skills for Teachers

The 1990 Standards for Teacher Competence in Educational Assessment of Students ( AFT, NCME, & NEA, 1990 ) made a documentable contribution to the field. However, the Standards have become a bit dated, most notably in two ways: (1) the Standards do not consider current conceptions of formative assessment knowledge and skills, and (2) the Standards do not consider teacher knowledge and skills required to successfully work in the current accountability and “standards‐based reform” context. This article briefly reviews the 1990 Standards and their influence, describes some other lists of assessment knowledge and skills that might be considered in updating them, and then proposes educational assessment knowledge and skills for teachers that reflect current teacher assessment needs. This set of competencies should help focus the work of teachers, teacher supervisors, professional developers, teacher educators, and others responsible for teachers’ assessment knowledge and skills.     

Multilevel Assessment for Discourse,Understanding, and Achievement

Evaluating the impact of instructional innovations and coordinating instruction, assessment, and testing present complex tensions. Many evaluation and coordination efforts aim to address these tensions by using the coherence provided by modern cognitive science perspectives on domain-specific learning. This paper introduces an alternative framework that uses emerging situative assessment perspectives to align learning across increasingly formal levels of educational practice. This framework emerged from 2 design studies of a 20-hr high school genetics curriculum that used the GenScope computer-based modeling software. The 1st study aligned learning across (a) the contextualized enactment of inquiry-oriented activities in GenScope, (b) “feedback conversations” around informal embedded assessments, and (c) a formal performance assessment; the 2nd study extended this alignment to a conventional achievement test. Design-based refinements ultimately delivered gains of nearly 2 SD on the performance assessment and more than 1 SD in achievement. These compared to gains of 0.25 and 0.50 SD, respectively, in well-matched comparison classrooms. General and specific assessment design principles for aligning instruction, assessment, and testing and for evaluating instructional innovations are presented.     

Assessment leaders’ perspectives of institutional cultures of assessment: a Delphi study

Institutional cultures of assessment are praised as beneficial to student learning. Yet, extant studies have not explored the theoretical foundations and pragmatic approaches to shaping cultures of assessment. The researchers used the Delphi method to explore 10 higher education assessment leaders’ attitudes and theoretical perspectives regarding cultures of assessment. These expert assessment leaders were iteratively surveyed until a reasonable threshold of consensus was reached. Study participants viewed buy-in as a necessary component of a positive campus culture of assessment, and advice on reshaping negative cultures was offered. Assessment leaders’ guiding theoretical frameworks were implied and loosely defined with metaphors. Finally, advice is offered for improving cultures of assessment by symbolically connecting assessment to student learning through dialogue.     

How to encourage a lifelong learner? The complex relation between learning strategies and assessment in a medical curriculum

Abstract

To foster lifelong learning skills, we need new didactic approaches with aligned assessment methods. Therefore, we investigated whether the outcomes of a project assignment show a different relation to learning strategies than a longitudinal knowledge-based assessment. We studied learning strategies of first year students of medicine and biomedical sciences (n?=?248) and performed hierarchical regression analyses for the learning strategies and grades of the longitudinal knowledge-based test and project assignment. Scores of students, measured with the Motivated Strategies for Learning Questionnaire (Likert scale 1–7), were relatively low for critical thinking (3.53), compared to rehearsal (4.40), elaboration (4.82), organisation (4.69) and metacognitive self-regulation (4.33). Knowledge based tests showed a significant relation to elaboration (p?<?0.01). For the project-based assessment, we did not find a significant relation to any learning strategy (p?=?0.074). Explained variance of the grades was low for all learning strategies (R² < 0.043). Different types of assessment did not discriminate between students with high or low scores on learning strategies associated with lifelong learning. An explanation is that the curriculum is not aligned with assessment, or students do not benefit in terms of grades. We conclude that, if assessment is to drive lifelong learning skills, this is not self-evident.     

Student attitudes that predict participation in peer assessment

Peer assessment has been widely applied to actively engage students in learning to write. However, sometimes students resist peer assessment. This study explores reviewers’ attitudes and other underlying factors that influence students’ participation in online peer assessment. Participants were 234 Chinese undergraduates from two different academic backgrounds: engineering majors (n = 168) and English majors (n = 66). Gender, academic background and prior experience with peer assessment were all related to participation levels. Moreover, factor analyses revealed three attitudinal factors: (1) positive attitude (a general endorsement of the benefits of peer assessment), (2) interpersonal negative (concerns about the negative effects on interpersonal relationships), and (3) procedural negative (doubts about the procedural rationality of peer assessment). Among the attitudinal factors, procedural negative was negatively associated with participation, as expected. Interestingly, interpersonal negative was associated with greater participation, and positive attitude was associated with lower participation, in part because students worked hard on each review rather than doing many reviews superficially. Implications for instruction are discussed.     

An instrument to audit teachers' use of assessment for learning

This paper traces the design, development and trialling of an assessment for learning audit instrument (AfLAi) in use currently in the Republic of Ireland to gauge teachers' baseline understanding of assessment for learning (AfL) practices and the extent to which AfL is embedded in their teaching. As described in the paper, the AfLAi consists of 58 items distributed across four scales based on the following key AfL strategies: sharing learning intentions and success criteria, questioning and classroom discussion, feedback and peer- and self-assessment. Preliminary data from the study provide a window into current formative assessment practices in Irish primary schools and teachers' professional needs in AfL.     

Mathematical Modeling in the Elementary Grades: Developing and Testing an Assessment

Mathematical modeling is a high-leverage topic, critical for college and career readiness, participation in STEM education, and civic engagement. Mathematical modeling involves connecting real-world situations, phenomenon, and/or data with mathematical models, and in this way applies across various STEM disciplines, including mathematics, engineering, and science. Although research has begun to explore mathematical modeling instruction in the elementary grades, questions remain about how to assess student learning at the elementary level. We addressed this need by designing an assessment of mathematical modeling competencies for students in grades 3 through 5. Informed by international research, our assessment includes a hybrid structure to assess mathematical modeling competencies holistically (as students engage in the complete modeling process) and atomistically (as students engage in different components of the modeling process, including making sense of phenomena and real-world situations, setting up and operating on mathematical models, and interpreting results in relation to the real-world context). We conducted student interviews, followed by two rounds of pilot testing to inform item development and ensure acceptable psychometric properties. The final assessment included 13 items (9 multiple choice, 3 open-response, and 1 complete modeling task). We describe our assessment development process, and provide sample assessment items and detailed coding rubrics. We summarize quantitative analyses which established high reliability and low standard error for our assessment, supporting its use for grades 3 to 5. Implications of our framework and assessment for mathematical modeling instruction and future research on STEM learning are discussed.

     

Assessing Professional Know‐How

This article considers how professional knowledge should be assessed. It is maintained that the assessment of professional know‐how raises distinctive issues from the assessment of know‐how more generally. Intellectualist arguments which suggest that someone's giving an account of how to F should suffice for attributing to them knowledge of how to F are set out. The arguments fail to show that there is no necessary distinction between two kinds of know‐how, namely the ability to F and knowing that w is a way to F, such that the latter is more fundamental. The consequences of this failure for our understanding of professional assessment are then considered. The issue of the assessment of tacit knowledge is then addressed. It is concluded that there is no context‐dependent codifiable or articulable propositional knowledge of how to F which could be substituted for being able to F and that therefore tacit knowledge can only be assessed in performance. The parallel with Gettier cases is reviewed and it is concluded that the provenance of accounts of and justifications for the attribution of know‐how are not matters of indifference to its assessment. Finally, the question of evaluability or what Ryle would have called the applicability of intelligence epithets is discussed in relation to its relevance to our procedures for assessing practical knowledge. Once again, it is concluded that excellent performance is necessary to attribute excellence in know‐how. However, the ability to give an account of how and why an agent would do something in hypothetical circumstances is also very important for the assessment of professional knowledge.