From substitution to redefinition: A framework of machine learning-based science assessment

This study develops a framework to conceptualize the use and evolution of machine learning (ML) in science assessment. We systematically reviewed 47 studies that applied ML in science assessment and classified them into five categories: (a) constructed response, (b) essay, (c) simulation, (d) educational game, and (e) inter-discipline. We compared the ML-based and conventional science assessments and extracted 12 critical characteristics to map three variables in a three-dimensional framework: construct, functionality, and automaticity. The 12 characteristics used to construct a profile for ML-based science assessments for each article were further analyzed by a two-step cluster analysis. The clusters identified for each variable were summarized into four levels to illustrate the evolution of each. We further conducted cluster analysis to identify four classes of assessment across the three variables. Based on the analysis, we conclude that ML has transformed—but not yet redefined—conventional science assessment practice in terms of fundamental purpose, the nature of the science assessment, and the relevant assessment challenges. Along with the three-dimensional framework, we propose five anticipated trends for incorporating ML in science assessment practice for future studies: addressing developmental cognition, changing the process of educational decision making, personalized science learning, borrowing 'good' to advance 'good', and integrating knowledge from other disciplines into science assessment.     

Facilitating Learning from Formative Feedback in Level 3 Assessment

This paper presents the development, through action research, of formative elements in assessment in a level 3 compulsory module of the BSc Health Studies and BSc Nursing programmes at the University of Sunderland. The paper reviews three cycles of planning, implementing and evaluating change in assessment strategy and is written in the first person to emphasise the connections between the writer and the material. From a consideration of the format and characteristics of the assessment within the module, the action research is reported through the implementation of actions taken to facilitate more effective use of formative feedback. The evaluation of these actions through my own reflections, student performance, dialogue with team colleagues and student feedback through the production of short narrative accounts of their learning experience is outlined. The paper demonstrates that through explicitly using the learning potential within assessment, learning can be facilitated through challenging students to move from 'doing' assignments, to reflexive thinking about their writing.     

Reconfiguring and realigning the assessment feedback processes for an undergraduate criminology degree

A reconfigured and realigned system of assessment feedback was implemented with undergraduates taking criminology modules at Swansea University. The reformulated system integrated explicit engagement with assessment criteria in feedback given on an electronic template form with the use of a statement bank and the offer of follow‐up, feedback consolidation meetings with the tutor. Student module evaluation feedback questionnaires demonstrated a positive qualitative impact upon students' ratings of their awareness and understanding of module learning outcomes, assessment criteria and required levels of learning; moving the focus of assessment feedback from teaching to (higher‐order) learning. Substantive qualitative impact has also been evidenced by departmental adoption of the reconfigured assessment process as standard. Quantitative impact was evidenced in the form of improvements in assessment performance within‐modules and within students from 2004–2005 to 2005–2006. The educational implications of the study findings are discussed.     

教育游戏“农场狂想曲2”中的学习迁移

教育仿真游戏是否对学生的迁移具有作用?本文采用个案法研究农场狂想曲2中学生的学习迁移,结果发现,有提示问题的个案发生了远迁移,迁移的知识层次达到了分析、综合和评价的高阶思维水平,而没有提示问题的个案即发生了近迁移或者没有迁移。     

Multimedia tools for use in materials handling classes

Materials handling is a vital function in a manufacturing or distribution system. Efficient handling of material allows such systems to operate at high levels of productivity. US companies invest over US$60 billion annually in materials handling technology. It is therefore extremely important for universities to prepare materials handling engineers who understand the principles of materials handling, as well as the design, implementation, operation and control of materials handling systems so that they can ensure the cost-effectiveness of this investment. This paper pertains to development of an interactive, multimedia-based system for use in materials handling classes. Modules in the system introduce students to: the 10 principles of materials handling; and two major equipment categories—conveyors and automated storage and retrieval systems. They illustrate industrial applications of material handling equipment and teach the problem-solving process through a case study and an extensive series of models, algorithms, problems and solutions. The modules have been formally evaluated and results are summarized. They are being distributed at a nominal cost to US and international universities. The module described in this paper on the ‘10 principles of materials handling’ was selected as a finalist for the 2002 PREMIER Award for Excellence in Engineering Education Courseware.     

海峡两岸高等教育评估实施过程比较分析

文章从高等教育评估入手,对大陆和台湾同一时期开展的高等教育评估工作做了简要介绍,并对两岸评估实施过程中的组织架构、具体流程及其评估结果的处理等进行对比分析和简要评价。文章认为,台湾大学评鉴工作经过多年的发展逐步形成自身特色,尤其在评估法制化、国际化和高校建立自我约束机制等方面,对完善大陆高等教育质量保障体系有参考价值。     

Parents' views on nursery education: Perceptions in context

The analysis describes the broad consensus which is currently emerging about the nature of the educational changes which are necessary for the creation of a 'learning society' and contrasts this vision with the contemporary reality of an education system which is every day more constrained by formal assessment. The implications of these tightening bonds for the development of universities in the third millennium are explored in terms of research evidence which documents the impact of conventional forms of assessment on student learning. The argument is made that this emphasis on 'categoric' assessment if fundamentally incompatible with aspirations towards the creation of a 'learning society'. This is partly because institutions must necessarily give their attention to obtaining high scores and cannot risk the substantial changes in the reorganisation of teaching and learning that an 'empowering' educational environment would arguably require and partly because of the power of the prevailing assessment discourse to define priorities. The article uses Lyotard's concept of 'performativity' to examine these contemporary tensions in higher education, their origins and potential significance for the creation of a 'learning society'.     

电子书包中基于教育大数据的个性化学习评价模型与系统设计

根据大数据技术的模块构成和电子书包所包含的系统和功能,对电子书包中教育大数据的模块内容进行分析。之后,在评价内容上从课程内容学习、参与互动交流、考试与作品和课外资源学习四个方面进行了细分和聚类．构建了基于电子书包的个性化学习评价模型。在评价结果上,依据柯氏四级评估模式和布鲁姆教学目标分类理论设计了基于教育大数据的个性化评价层次塔,该层次塔包括学习成效、概念转变、学习迁移和学习力四个层级。最后,结合教育大数据、教育云服务、个性化评价模型和评价层次塔,设计了个性化学习评价系统模型,包括信息采集模块、数据分析与处理模块、个性化评价模块和可视化反馈模块,并通过云管理层实现对教育云服务平台、云存储池和云集群计算平台的调控和管理,以期为后面开展个性化评价系统的设计与开发提供有益的指导。     

Comparing multimethod assessment of approaches to learning among preschool children: Direct measure,teacher report,and parent report

A multi‐informant or multimethod approach has been suggested for use in educational evaluation and children's development assessment. However, in the study field of approaches to learning, most previous studies used one method to measure approaches to learning. In addition, compared with kindergarten and elementary children, younger children have received little attention. This study was dedicated to determining whether a multimethod approach (direct measure, teacher report, and parent report) was needed to assess preschool children's approaches to learning. A total of 713 preschool children were enrolled in this study. Correlations and multiple regressions were conducted to analyze the correlation among the three methods as well as their criterion validity based upon comparisons with an assessment of children's early childhood development. The results revealed significant but weak correlations among the three assessment methods. The direct measure of approaches to learning was more relevant to children's early childhood development than the teacher report and the parent report. The criterion validity of using the direct measure to assess preschool children's approaches to learning was also better than that of the teacher report and the parent report. Therefore, the direct measure was recommended for use in assessing preschool children's approaches to learning, and teacher report can be used as a supplement.     

Practical training and internships in engineering education: Educational goals and assessment

In many engineering curricula a period of practical training in industry is either compulsory or advised. Often the educational goals of such a period are defined as 'learning to put knowledge and skills into practice'. The assessment is usually based on a technical report of the activities during the period. In this paper, how the definition of educational objectives led to the introduction of practical training is discussed. These objectives are acquiring insight in the engineering profession, learning to 'survive' in a different culture and learning to apply as well as broadening technical knowledge and skills. Furthermore, an assessment procedure is introduced which allows for testing whether the students have met the objectives.     

Semantic-aware components and services of ActiveMath

ActiveMath is a complex web‐based adaptive learning environment with a number of components and interactive learning tools. The basis for handling semantics of learning content is provided by its semantic (mathematics) content markup, which is additionally annotated with educational metadata. Several components, tools and external services can make use of that content markup, eg, a course generator, a semantic search engine and user input evaluation services. The components and services have to communicate, pass content and state changes, actions, etc including mathematical semantics and educational markup. The novel event infrastructure supports this communication. This paper focuses on the usage of the content's semantics by selected novel components and sketches the communication.     

Using educational games and simulation software in a computer science course: learning achievements and student flow experiences

This study investigates how educational games impact on students' academic performance and multimedia flow experiences in a computer science course. A curriculum consists of five basic learning units, that is, the stack, queue, sort, tree traversal, and binary search tree, was conducted for 110 university students during one semester. Two groups of students participated in learning activities: the experimental group was instructed using the gaming method; and the control group was instructed using the non-gaming method. During the study, tests, a survey, and interviews were conducted with students. The evaluation results for academic performance and multimedia flow experiences show that compared to the non-gaming method, incorporating the gaming method into the learning process can enhance students' academic performance and multimedia flow experiences. The results also indicated that there is a non-significant and positive relationship between students' academic performance and multimedia flow experiences.     

Litigation and Learning: Tensions in improving university lecturers' assessment practice

Current debate about assessment in higher education raises educational and political issues. Lecturers who wish to make their assessment more reliable and rigorous, as well as more effective in improving students' learning, need more than technical help to do so. This paper reports findings from an action research project which focused on assessment practice at the University of Sunderland, UK. It highlights tensions between genuine educational concerns to improve practice and more instrumental pressures, for example, to defend one's assessment practice from challenges by students, colleagues and external bodies. It is argued that improvement, rather than mere change, will require the commitment of people who possess intimate day-to-day contextual knowledge of assessment, and who recognise its educational and political complexities. The findings highlight two areas for further research: ways of inducting and involving students in an 'assessment community', and institutional staff development designed to improve assessment practice.     

"The Cow Loves to Learn": The Hao-Xue-Xin Learning Model as a Reflection of the Cultural Relevance of Zhima Jie,China's Sesame Street

This study was part of the formative research conducted for the children's educational TV program, Zhima Jie, a Chinese adaptation of Sesame Street. Our particular goal was to examine the cultural basis and relevance o f Zhima Jie as a learning tool for Chinese children. Four hundred children aged 3-6 drawn from representative backgrounds participated in the study. Data were collected on relevant aspects of the Zhime Jie curriculum: preschoolers' desires for books and learning materials, and their reasoning and affect about learning. Results showed that the vast majority of children desired books and learning materials with a number of them also articulating reasons for that desire. While differences in gender and mother's education were small and inconsistent, disadvantaged children were more likely to desire books and learning materials than their peers from more advantaged backgrounds. Children also showed a greater tendency for such desires and expressions as their age increased. These findings reflected Confucian beliefs about learning, and they provided support for the cultural relevance of the educational efforts of Zhima Jie in China.     

What Counts as 'Success'? Hierarchical discourses in a girls' comprehensive school

In the UK, as in many parts of the world, educational policy is dominated by the 'standards agenda': the top-down drive to improve students' performance in examinations. Simultaneously, there is policy emphasis on (differing versions of) 'inclusive education', and mainstream schools are exhorted to remove barriers to learning and participation for students who would until recently have been educated separately in special schools. This paper examines one of the many tensions between these two policy imperatives. Using findings from an ethnographic study in one comprehensive girls' school in an English city, I identify three distinct versions of educational 'success'--'dominant', 'consolation' and 'really disabled'. This paper explores how students identified as having 'special educational needs' position themselves and are positioned by the three discourses, and suggests that the hierarchisation of what can count as 'success' is an important dimension in the enduring reproduction of educational and societal inequalities.     

Design for assessment of dissection in anatomy laboratory based on group identification of structures and peer evaluation

Dissection is a unique multisensory educational experience and is essential to learning the anatomical construction of the human and animal bodies. This study aims to introduce a specialized design for the assessment of dissection and to discuss the assessment's attributes. The design was a product of the “assessment drives learning” concept and was developed to motivate students to dissect. Students were awarded “dissection points” based on prior group dissection and identification of structures. Students' perception of the design was examined, and content analysis was performed. The assessment consisted of two parts: the first assigning each student group structures to “pin” on their previously dissected cadavers; the second was a group peer evaluation. The most critical factor for the assessment's success was careful selection of structures assigned to students to pin. The assessment was fit for the purpose, valid, reliable, and had a significant educational impact. Eighty-three percent of students (n = 116) recommended maintaining the assessment design, as they felt it promoted a deep approach to learning as well as teamwork while reducing stress to a minimum. A strong correlation (Spearman's rho = 0.46, P < 0.0001) was present between the high rating of the design and the number of structures learned, as well as, high rating and lower stress level (Spearman's rho = 0.40, P < 0.0001). There was no apparent influence of grades on student perception of the effectiveness of the assessment. This specific design of evaluation could be used as part of anatomy education in veterinary and medical schools.     

‘Just enough to make you take it seriously’: exploring students’ attitudes towards peer assessment

The use of peer learning and peer assessment has gained considerable interest in higher education driven by both its educational value and by its ability to provide students with the opportunity to develop important transferrable skills. This paper reports on the use of peer learning and peer assessment with a cohort of four-year undergraduate physiotherapy students and an 18 month taught post-graduate teacher education programme. The study observed the students’ engagement in the process, surveyed their opinions on the activity at the end of the experience and conducted one focus group discussion with a subset of students from each cohort. The study found that the vast majority of respondents felt that the experience was valuable and enjoyable. However, when asked to indicate whether it was a fairer method of assessment there were more varied responses. Similarly when asked whether their peers should have a greater say in their overall grade the majority disagreed. Views on the educational value of the experience appeared to differ between the two cohorts of students. The study highlights the influence of a prevailing assessment cultures on students’ engagement in peer learning which requires consideration when including such pedagogical approaches.     

Learning and Assessment in the Science Classroom in Thailand

The 1990 national curriculum of Thai education and educational reform states that, at primary and secondary educational levels, schools must prepare the child to become a smart, good and a happy individual in the Thai learning society. Teaching and learning must emphasise learning to think, to do and to solve problems. A study into integrating assessment and learning as a single activity was conducted by the Department of Curriculum and Instruction Development to demonstrate that good assessment is an integral part of good instruction. Three major elements of science education are learning science, learning about science and doing science. In the assessment of science, the teacher assesses knowledge and recall of scientific facts, the application of scientific knowledge, processes of science and scientific skills and, of course, attitudes and habits of mind acquired through science education. The assessment procedures are in the form of open-ended paper-and-pencil tests, practical tests, students' work and reports, teacher observation of practical work during the semester, and parents' and peers' comment and criticism. The assessment is a continuous activity in parallel with learning activity.     

Learning to be a doctor while learning anatomy!

This article presents results from a longitudinal study following a cohort of medical students. Semistructured interviews are conducted every year, in which the students tell about their experiences learning medicine, their daily life, and their social activities in relation to university. The aim of the study is to analyze how medical students develop their professional competencies, values, and attitudes. This article focuses on the medical student's professional development in relation to the process of studying and learning anatomy. We analyze interviews conducted while the students are in their third and fifth semester. Anatomy plays a significant role in the medical student's educational process, on both a cognitive and emotional level. It seems that students in learning the subject matter adapt to fundamental values in the medical profession and are thus transformed into real medical students, sharing a unique experience. The implications for curricular development and professionalism are discussed, and this article argues that the issue of professionalism must be addressed in the educational process in the preclinical years and in relation to the basic sciences. Anat Sci Ed 2008. © 2008 American Association of Anatomists.     

Investigating the effectiveness of three-dimensionally printed anatomical models compared with plastinated human specimens in learning cardiac and neck anatomy: A randomized crossover study

Three-dimensional printing (3DP) technology has been increasingly applied in health profession education. Yet, 3DP anatomical models compared with the plastinated specimens as learning scaffolds are unclear. A randomized-controlled crossover study was used to evaluate the objective outcomes of 3DP models compared with the plastinated specimens through an introductory lecture and team study for learning relatively simple (cardiac) and complex (neck) anatomies. Given the novel multimaterial and multicolored 3DP models are replicas of the plastinated specimens, it is hypothesized that 3DP models have the same educational benefits to plastinated specimens. This study was conducted in two phases in which participants were randomly assigned to 3DP (n = 31) and plastinated cardiac groups (n = 32) in the first phase, whereas same groups (3DP, n = 15; plastinated, n = 18) used switched materials in the second phase for learning neck anatomy. The pretest, educational activities and posttest were conducted for each phase. Miller's framework was used to assess the cognitive outcomes. There was a significant improvement in students' baseline knowledge by 29.7% and 31.3% for Phase 1; 31.7% and 31.3% for Phase 2 plastinated and 3DP models. Posttest scores for cardiac (plastinated, 3DP mean ± SD: 57.0 ± 13.3 and 60.8 ± 13.6, P = 0.27) and neck (70.3 ± 15.6 and 68.3 ± 9.9, P = 0.68) phases showed no significant difference. In addition, no difference observed when cognitive domains compared for both cases. These results reflect that introductory lecture plus either the plastinated or 3DP modes were effective for learning cardiac and neck anatomy.