Assessment in the science classroom

This paper reports on research into two teachers' views and practices about assessment at the classroom level. Emphasis was given to practical work and its assessment. Findings suggest it is unhelpful to define practical work as distinct from other activities in the science classroom. Various methods used for assessing activity within the participant teachers' classrooms are described. The participant teachers were found to be primarily concerned about issues of ‘fairness’: task validity, reliability of assessment based on co-operative work and assessment of the affective domain. The place of teacher intuition in assessment is raised and briefly discussed. Directions for the ongoing research are foreshadowed. Specializations: science education, technology education, assessment and curriculum development.     

Classroom-based elementary school affective education programs: A critical review

Twenty-three studies on the effectiveness of classroom-based, elementary school affective education programs were selected by predetermined criteria from published and unpublished research dating from 1970. The resulting research evidence was evaluated by methodological rigor, program type, grade level, program length, and leader's profession. The lack of positive significant findings in the more carefully designed studies provides little support for the effectiveness of affective education programs in promoting positive changes on either behavioral or affective measures, especially for programs with an internal focus. Studies of social-cognitive problem-solving programs produced promising results, but require further evidence of effectiveness. The results provide important implications for future research and professional practice.     

COMMUNITY‐BASED OUTREACH MULTIDIMENSIONAL GERIATRIC ASSESSMENT: A CLINICAL AND EDUCATIONAL MODEL

The shifting emphasis from institutionalto community‐based care for the elderly in Canada necessitates a new approach to geriatric assessment and education. Geriatric assessment outreach teams using a multidimensional clinical and educational model are described in the context of a regionally based geriatric program in Ottawa. This community‐based multidimensional model is compared to the traditional institutional‐based multidisciplinary team model, particularly in terms of its advantages and relevance for the approximately 33% of Canada's elderly living in small towns and rural areas. The educational model and training program for multidimensional geriatric outreach assessors are described. The clinical operations and advantages of the multidimensional model are fully described, particularly their critical role in community‐based assessment and triage to further specialized geriatric services.     

Effects of affective education interventions in the elementary school

Affective education programs have had an uneven impact upon children, teachers, and school systems, as have also affective interventions. By integrating environmental press research with Aptitude × Treatment Interaction (ATI) inquiry, affective education appears to be able to merge individual differences with assessment, intervention, and evaluation. This paper describes an experimental intervention and two active control programs and uses an ATI analysis for reporting their impact in the affective domain.     

A National Survey of Assessment Instruments Taught by Counselor Educators

This study reports results of a questionnaire of 210 counselor educators regarding assessment instruments covered in their assessment courses. Instruments are ranked on how extensively they are covered, and comparisons of coverage are made by assessment category (e.g., educational, clinical) and by instructor and program characteristic (e.g., Council for Accreditation of Counseling and Related Educational Programs [CACREP] vs. non‐CACREP). Suggestions regarding which assessment techniques to cover in a basic counselor education testing and assessment course are made.     

Curricular innovation in an undergraduate medical program: What is “appropriate” assessment?

In post-secondary education, there is a widely-held belief in a “gold standard” for evaluative studies of curricular innovations. In this context, “appropriate” assessment is understood to refer to experimental designs and statistically significant differences in group outcomes. Yet in our evaluative study of a medical undergraduate program, we did not find these concepts to be particularly applicable. Based on our experience, we now feel that it is appropriate to assemble an eclectic mix of scientific findings, show how they have been used for program improvement, and articulate the program’s theoretical rationale and social significance. In the absence of statistically significant differences, this comprehensive argument can be used to justify the deployment of curricular innovations. The same may be true of other educational programs that target hard-to-measure changes in affective domains.     

The Long,Wide View of Biology

Significant life experience research suggests that outdoor experiences foster proenvironmental outcomes. Time spent outdoors is more frequently identified as the source of proenvironmental behavior than is education, suggesting that cognition may be less important than affect. Yet, environmental education field programs are often evaluated on cognitive outcomes alone. The authors piloted a mixed-methods evaluation, measuring both cognitive and affective responses to a field education program. Quantitative responses suggest that field-based participants demonstrated greater cognitive understanding than classroom-based participants. Qualitative responses suggest that field programs foster different affective reactions than do classroom programs. These results have critical implications for field-based programming, classroom instruction, and evaluation.     

e is for exploration: Assessing hard-to-measure learning outcomes

The focus of this paper is on the use of e‐technologies to enable higher education to better assess aspects of learning that have proved difficult to assess using more conventional means. Higher education describes the knowledge and abilities it intends its graduates to have acquired before graduation, and it has a wide range of approaches to assess these. Higher education also seeks affective outcomes in the form of values, attitudes, behaviours and related attributes or dispositions, and these have consistently proved more difficult to assess by examination or assignment. After graduation, however, graduates are often assessed within the professions via portfolios, interview and peer or expert review. Assessment may focus on teamwork and networking skills, productivity, creativity and values fit to the profession. How can e‐technologies help with these forms of assessment? This paper reviews some of the e‐based approaches and explorations that have supported or could support assessment of affective attributes. At each stage, the paper seeks to establish the common elements of assessment in the different regimes and how e‐assessment contributes, or may contribute in the future. The paper concludes that many of the strengths of e‐assessment lend themselves to an evaluation paradigm rather than to conventional assessment for intended learning outcomes.     

Development of a distance education assessment instrument

The study described in this article takes the view that distance education programs are dynamic systems with multiple constituencies. Literature on distance education programs as complex systems and on the critical constituency approach to program evaluation was used to guide the development of a questionnaire designed to assess the components of a distance education program. Instrument development was conducted in four phases using both quantitative and qualitative methods. The data from this study suggest that there are four major components of effectiveness in distance education programs: instruction, management, telecommuting, and support. Within each of these components there are from two to five subcomponents, 14 components in all, which can be used to assess distance education programs. The assessment scale which was developed measures these four components and their subcomponents.     

People and Environment

Abstract

Recently, authentic assessments have been used extensively in the classroom setting; however, their use in informal settings, such as outdoor education centers, has been very limited. We review several classroom techniques and present examples that illustrate the use of authentic assessment at an ecology education center. We conclude with a discussion of general issues such as the correlation of learning objectives and program content, development and scoring, and other considerations associated with implementation of authentic assessment.     

Using Qualitative Evaluation Methods to Identify Exemplary Practices in Early Childhood Education

The proliferation of early childhood programs for children with and without disabilities has occurred without clear indicators or standards of program quality. Evaluations of these programs often consist of compliance monitoring or outcome assessment. Little attention is given to program context or the quality of individual components of the program. In this paper, an alternative evaluation model, a connoisseurship or professional review model using on-site observations, interviews, and document review, was used to identify exemplary practices in early childhood education in one large midwestern state. The results indicated that with proper training and support, early childhood professionals without experience in qualitative methods can become proficient in interviewing techniques, observation, and document review. The resulting case study reports were useful for identifying and describing exemplary practices across the state, demonstrating the potential for this alternative approach to evaluation in early childhood education.     

Using Qualitative Evaluation Methods to Identify Exemplary Practices in Early Childhood Education

The proliferation of early childhood programs for children with and without disabilities has occurred without clear indicators or standards of program quality. Evaluations of these programs often consist of compliance monitoring or outcome assessment. Little attention is given to program context or the quality of individual components of the program. In this paper, an alternative evaluation model, a connoisseurship or professional review model using on-site observations, interviews, and document review, was used to identify exemplary practices in early childhood education in one large midwestern state. The results indicated that with proper training and support, early childhood professionals without experience in qualitative methods can become proficient in interviewing techniques, observation, and document review. The resulting case study reports were useful for identifying and describing exemplary practices across the state, demonstrating the potential for this alternative approach to evaluation in early childhood education.     

情感计算在网络远程教育系统中的应用：功能、研究现状及关键问题

情感能力是人类智能的重要标志,情感的缺失会影响网络远程教育的教学质量和学习者的学习效果。情感计算是和谐人机交互与人工智能领域中新的研究方向。在网络远程教育系统中应用情感计算理论与技术,可以进一步优化网络远程教育的功能,帮助教师监测远程学习者的情感变化,调整教学策略和方法,实时给予学习者情感反馈,使教学质量达到最佳。目前国内外情感计算在远程教育方面的应用研究还处于起步与探索阶段,所构建的远程教学系统对远程教学中师生情感生理特征与心理特征、多模情感信息融合机制及网络虚拟人机情感交互特性等考虑不足,系统原型在情感合成与表达、智能人机情感交互实现上存在着很大的难度。因此,构建与人类情感系统相吻合的、自然和谐的、人性化和智能化的网络远程教育系统,需要有效解决师生面部表情、语音情感特征信号的准确识别与提取,师生多模情感信息融合机制与和谐人机情感交互技术等关键问题。     

Assessing Multicultural Initiatives in Higher Education Institutions

As educational institutions engage in promoting multiculturalism on campuses, it becomes essential to assess the quality and success of those initiatives. In this paper, the plan being implemented at Northern Illinois University to assess multicultural initiatives is described. The plan is to ensure that faculty, staff, and students participate and benefit from multicultural curricular and related program, courses and curricula continue to be multiculturally transformed, multicultural curricular transformation and support program needs are being met, students obtain and demonstrate the necessary multicultural competencies, and the campus is supportive of multicultural initiatives at all levels of the university. The paper describes the range of multicultural initiatives pursued in higher education institutions and the considerations necessary in assessing such initiatives. The plan described in this paper makes use of existing assessment mechanisms and as well as a few new ones designed and implemented in several phases. The results and findings from the assessment along with recommendations for improving the initiatives are discussed in this paper. The paper concludes with a discussion on the real challenges of the plan and issues to consider when assessing multicultural initiatives in a higher education institution.     

The teacher work sample: A tool for scaffolding and assessing preservice teachers' early field experiences

Abstract

This article describes adapting teacher work sample methodology from the student teaching level to a 4‐week, 25‐hour field experience occurring relatively early in a teacher education program. The goals of the project were twofold: (a) to examine whether an adapted version of the teacher work sample would promote planning and reflection in preservice teachers, and (b) to explore the potential of an adapted teacher work sample as an assessment tool. Adaptations to the original, such as emphasizing formative over summative assessment, are briefly described. Survey and focus group data from a pilot combined with survey data from a semester of full implementation with 270 teacher candidates indicate the teacher work sample is useful for promoting reflection and an understanding of student assessment. The advantages of the teacher work sample are maximized when teacher candidates are engaged in whole class instruction.     

Procedures for Evaluating the Impact of Complex Educational Interventions

The application of information and computer technology (ICT) to education, particularly when ICT is a component of a major school reform, raises difficult issues for program evaluation. In many cases even defining the program goal is difficult because so many different changes are often instituted at the same time: installation of a local area network, development of technical support, staff development for integrating ICT into the curriculum and for new teaching methods, and so on. To assist program developers and evaluators in gaining control of such complex interventions in education, two basic procedures are described: the history of the future and program logic mapping. The former is a goals and methods clarification scheme, built around a press release that describes at the projected end of the project what was achieved and how. The latter procedure draws on the goals defined in the history of the future, along with the project strategy, to make explicit all of the major links between project inputs and project outputs. This map then serves as a guide for planning an evaluation that would verify causal linkages as well as outcomes. For each type of node in such a diagram, appropriate evaluation techniques are defined.     

Program portfolios: documenting teachers’ growth in reflection‐based inquiry

Meeting the challenge of program accountability is a goal for teacher education programs across the USA. In this context, achieving effective assessment practices that provide concrete evidence of program participants’ knowledge and skills has become both an increasingly significant issue and a challenge to teacher education programs seeking to document the attainment of their program learning outcomes. This qualitative study examined the portfolio reflections of 51 teachers enrolled in an advanced master’s degree program whose learning outcomes are aligned with the core propositions of the National Board for Professional Teaching Standards. Using the four levels of reflection and a fifth level that emerged from data analysis, we examined the levels of development and change in teachers’ reflections across the program. This analysis helped the researchers to determine to what degree teachers developed a reflection‐based inquiry stance in their classrooms during the program. The study contributes new findings to the body of literature on the role and function of portfolios in teacher education programs seeking to document teachers’ understanding and application of specific program goals and professional standards as a result of professional development coursework. As an authentic assessment tool, portfolio data (i.e., evidence such as the reflections contained therein) can provide an important lens for capturing teachers’ approach to teaching and learning and provides insight into the complexity of professional development for practicing teachers.     

美国教师培养项目增值评价探析 ——以路易斯安那州为例

世纪之交以来,美国教师培养面临内部质量提升和外部质量问责的双重压力.在此背景下,伴随数据系统和分析方法的发展以及教师增值评价的推广,教师培养项目的增值评价应运而生.教师培养项目增值评价基于教师培养项目→教师教学质量→学生学业成绩的因果逻辑,通过统计计量模型来评估教师培养项目对学生成绩的贡献度.其中项目效能的证据选择、效能测量以及效能增值结果的意义阐释构成教师培养项目增值评价的关键内容.教师培养项目增值评价虽然在一定程度上响应了公众对于强化教师教育绩效责任的呼声,但依然存在着信效度和适用范围等问题.     

Automatic assessment of cognitive and emotional states in virtual reality-based flexibility training for four adolescents with autism

Tracking students’ learning states to provide tailored learner support is a critical element of an adaptive learning system. This study explores how an automatic assessment is capable of tracking learners’ cognitive and emotional states during virtual reality (VR)-based representational-flexibility training. This VR-based training program aims to promote the flexibility of adolescents with autism spectrum disorder (ASD) in interpreting, selecting and creating multimodal representations during STEM-related design problem solving. For the automatic assessment, we used both natural language processing (NLP) and machine-learning techniques to develop a multi-label classification model. We then trained the model with the data from a total of audio- and video-recorded 66 training sessions of four adolescents with ASD. To validate the model, we implemented both k-fold cross-validations and the manual evaluations by expert reviewers. The study finding suggests the feasibility of implementing the NLP and machine-learning driven automatic assessment to track and assess the cognitive and emotional states of individuals with ASD during VR-based flexibility training. The study finding also denotes the importance and viability of providing adaptive supports to maintain learners’ cognitive and affective engagement in a highly interactive digital learning environment.