The Construction and Evaluation of Simulations to Assess Professional Interviewing Skills

Courses preparing students for interviews commonly held in organisations often form part of the curriculum of senior secondary, higher, university, and in‐service education. In these courses, students are prepared for their future work practice. Assessment of student performance after attending such a course requires a different assessment method from the traditional written examination. In this article we describe the construction and evaluation of simulations. The results of an investigation into their quality show that they are reliable in terms of measures of internal consistency and inter‐rater reliability. However, it turned out that a student's score is highly dependent on the content of the interview. We found support for the simulations’ construct and content validity. Although the simulation is not an efficient instrument, its benefits are high: students are stimulated to do their best in practising for the interviews, and weaknesses in students’ performances will be detected so that remedial teaching can be offered.     

Between authenticity and cognitive demand: Finding a balance in designing a video-based simulation in the context of mathematics teacher education

A key challenge for teachers is the on-the-fly assessment of student learning. Video-based simulations may provide a tool for measuring assessment skills and a basis for learning environments in teacher education. Based on the framework for teaching practice by Grossman et al. (2009), considerations for designing video-based simulations that balance authenticity and cognitive demand are derived. Results show that participants perceived the developed simulation as authentic, were mostly able to rank students according to their overall mathematical argumentation skills and showed potential for learning in their detailed assessment of students. Thus, results indicate the internal validity of the video-based simulation.     

The use of self,peer and teacher assessment as a feedback system in a learning environment aimed at fostering skills of cooperation in an entrepreneurial context

Education is criticized for producing inert knowledge and for paying too little attention to skills such as cooperating and problem solving. Powerful learning environments have the potential to overcome these educational shortcomings. The goal of this research was to find out ways in which a ‘learning enterprise’ can best be supported (coached) in order to constitute a powerful learning environment aimed at teaching certain cooperative skills in a business context. This ‘learning enterprise’ constitutes an entrepreneurial context in which students in secondary or higher education are working together to conceptualize and eventually commercialize a product. In this research, the impact of different ways of supporting a learning enterprise will be compared. These ways are based on existing guiding principles for the design of powerful learning environments and on a further elaboration of these principles in what is conceptualized as an ‘equilibrium model’. In this model, the balances that are needed between motivating students, activating them towards self‐regulated learning, coaching, structuring and steering the learning processes have been elaborated. Based on this model a differentiation between a ‘student‐controlled’, a ‘teacher‐controlled and a ‘coached approach’, as an equilibrated way between the various approaches to coaching a learning enterprise, has been worked out. We hypothesize that the coached approach will give the best learning results in relation to cooperative skills. A combination of self, peer and teacher assessment of these skills, and an adequate feedback‐strategy based on these assessments, should be an important part of approaches used. These approaches were put into practice in a design experiment, and the impact was compared by means of a pre‐test/post‐test design. Results confirmed the postulated hypotheses that there will need to be a balance between, on the one hand giving students enough freedom for self‐discovery and self‐regulation, and on the other hand steering the students in such a way that certain problems can be avoided and that every student can get optimal learning chances. An adequate assessment‐strategy is needed to search for this balance. Further, a systematic action research of the design experiences resulted in more information on how best to coach a learning enterprise. This information has been summarized in the form of general guidelines.     

师范生教学技能考核实践探索

为了加强综合性大学的"师范性",提高师范生的教学技能水平,一些综合性大学进行了一些探索和改革——尝试建立了由一般技能、学科特色技能和综合技能构成的师范生教学技能结构体系,研制出了中学、小学和学前各类师范生的教学技能考核标准,并建立了规范化的教学技能考核制度。通过研究证明,教学技能考核已初步显示出良好的效果,具有一定的实践价值和推广意义。     

指向创造性思维:PISA 2021的启示与借鉴

创造性思维是人类发展所需的必要能力,可以帮助人们适应不断变化的世界和应对充满挑战的未来。经济合作与发展组织确定在PISA 2021中增加对创造性思维能力的评估,其发布的《PISA 2021创造性思维评估框架草案(第三版)》明确阐述了创造性思维的内涵、表现形式和促成因素,以系统的通用框架、科学简易的"三维度四领域"能力模型向公众提供了一个操作性强的评估系统。通过此次评估,各参与国家和地区可获得学生创造性思维能力的可比数据,为未来教育政策的制定和教育实践的改进提供支持。基于PISA的经验,为了更好地评估和培养学生的创造性思维,我国可借鉴创造性思维能力模型,细化学科核心素养的考查;构建创造性课堂,加强学校创新氛围的建设;在课堂教学中以真实情境和实际问题为载体,培育和评价学生的创造性思维。     

Giving feedback after a lesson observation

Student teachers spend a large proportion of their ITT course teaching classes on their own. It has always been an expectation, and now more formally an entitlement, that they will be observed on some of their lessons and given feedback during a de‐briefing session. In order to explore the nature and quality of the feedback given to students a number of tutors and mentors have been videoed giving feedback to students after observing them teach. The focus of the research is on the style which the tutors and mentors adopt and the interactions between themselves and students. Encouraging reflective practice requires facilitative interactions for which counselling skills are invaluable.     

Training self-assessment and task-selection skills: A cognitive approach to improving self-regulated learning

For self-regulated learning to be effective, students need to be able to accurately assess their own performance on a learning task and use this assessment for the selection of a new learning task. Evidence suggests, however, that students have difficulties with accurate self-assessment and task selection, which may explain the poor learning outcomes often found with self-regulated learning. In experiment 1, the hypothesis was investigated and confirmed that observing a human model engaging in self-assessment, task selection, or both could be effective for secondary education students’ (N = 80) acquisition of self-assessment and task-selection skills. Experiment 2 investigated and confirmed the hypothesis that secondary education students’ (N = 90) acquisition of self-assessment and task-selection skills, either through examples or through practice, would enhance the effectiveness of self-regulated learning. It can be concluded that self-assessment and task-selection skills indeed play an important role in self-regulated learning and that training these skills can significantly increase the amount of knowledge students can gain from self-regulated learning in which they choose their own learning tasks.     

Prioritising progression over proficiency: limitations of teacher-based assessment within technician-level vocational education

This article examines the evolution of assessment policy and practice in technician-level vocational education. Using the example of an advanced-level BTEC National programme in Engineering in one college in the UK, the article highlights how the origins of current assessment practice lie in genuine concerns since the late 1950s about the appropriateness of examination-based assessment for assessing technician engineering skills, resulting in a shift to teacher-based, criterion-referenced assessment and an emphasis on formative feedback. Data are presented from a case study of assessment practice in an engineering department of one college of further and higher education in the UK during the academic years 2006–2008, which investigated salient influences and considerations underpinning lecturers’ constructs of assessment. The study found a departmental ethos of facilitating students to achieve a pass, and the use of assessment methods that would ensure that students got through. However, lecturers were dubious whether their assessment practices developed the proficiency required for students’ progression into HE or employment, and their practices did not appear to have a strong educational or occupational rationale for their approach. We argue that the limitations of teacher-based, criterion-referenced assessment and associated formative assessment practices, of which our study provides an example, contribute to a temptation to simply return to what is deemed more rigorous assessment through examination, which however may well bring back the problems identified in the past, of inappropriate approaches for assessing technician-level skills associated with low success rates.     

工科院校开展学生职业技能培训的几点思考

职业技能教育与培训是工科院校教学计划的重要组成部分。学校建设好能适应学生实习要求的职业技能培训中心和校外实习基地是提高学生职业技能培训质量的重要前提和根本保证。对学生进行职业技能培训要因材施教,不同的专业安排不同的教学计划与内容。学生是主体,要充分调动学生的主动性和积极性,把学生职业技能培训与国家职业技能资格证书考试相结合。教师是搞好学生职业技能培训的关键。要建设一支热心学生职业技能培训,不断进取有较强实践动手能力的教师队伍。     

注重职业能力教育的培养模式转型研究——基于四川电大学生职业能力的调查分析

职业能力是人顺利完成职业活动所必需的能力,主要包括专业能力、方法能力和社会能力三个方面。当前电大学生低龄化趋势明显,参加工作时间较短,更换职业的需求较强。大部分学生认为缺乏专业操作技能,欠缺工作创新能力、与人沟通能力和协调合作能力。这进一步可以追溯至电大课程体系适应性差、考核方式单一、实验实习手段薄弱和忽视文化建设等方面的原因。在今后的发展中,电大应加速以能力培养为主导的培养模式的转型,找准培养目标定位,加快课程体系和考核方式改革,合理规划实验实习基地和校园文化建设,真正做到培养学生的关键职业能力。     

开放式实践教学基地信息化建设探索——以浙江机电职业技术学院为例

利用网络通信、数据库和仿真模拟等技术,将实践教学基地中各类资源有效重组,可以实现基地资源信息化管理.浙江机电职业技术学院在实践教学基地信息化建设过程中,开发信息管理平台,应用虚拟仿真技术,建设数字化制造实训车间,为更好地培养学生技能、提高教学资源使用效益创造了条件.     

Self‐assessment and the disadvantaged student: potential for encouraging self‐regulated learning?

The Science Foundation Programme (SFP) at the University of KwaZulu Natal, Pietermaritzburg attempts to address past educational inequalities by providing disadvantaged matriculants with the skills, resources and self‐confidence needed to embark on their tertiary studies. Students entering the Programme typically adopt a surface approach to learning with emphasis being placed on high score achievement which results in a mark‐driven attitude towards assessment. Students also lack the metacognitive skills associated with a deep approach to learning. Within this mark‐driven culture, it is important to attempt to move students away from such a superficial approach to learning and assessment. Worldwide, self‐assessment practice has been gaining recognition, and it has been linked to the adoption of a deep approach to learning; self‐regulated learning and the development of metacognitive skills. In the biology module of the Programme, students are given two essay assignments, tasks that are routinely performed very poorly. In attempt to improve SFP students’ essay writing abilities, a self‐assessment scheme to accompany the essay tasks was instituted, hoping that this would get the students to engage with the assessment criteria. It was intended that students would not only fulfil the requirements of the task better, but also achieve the valuable skill of self‐assessment. The results provide overwhelming evidence that SFP students cannot accurately self‐assess. Their naivety and inexperience in fulfilling assessment criteria was also revealed. Overall, marks awarded by staff were significantly lower than those awarded by students, and there was no evidence to suggest that any one category of students based on academic ability were better able to self‐assess than another. Marker inconsistencies were revealed, but this was shown to have no effect on findings. In spite of these results, a case for pursuing self‐assessment is made and proposals for improving student self‐assessment practice in the SFP are suggested.     

Enhancing learning in clinical placements: reflective practice,self-assessment,rubrics and scaffolding

Professional preparatory health programmes generally involve clinical placements with a focus on integration of theory into real life practice. Reflective writing is often included in the assessment requirements for clinical placement courses. However enabling students to engage in deeper levels of reflective writing in action, on action and for action requires careful scaffolding and assessment task design within university clinical placement handbooks. The purpose of this case study was to explore iterative refinement of assessment exercises within clinical placement handbook activities. Specifically, the study examined students’ responses to counselling experiences in authentic pharmacy practice, with comparative analysis undertaken for the various versions of the assessment task. Results showed that provision of iteratively developed guidelines, provided to three different groups of students, to support students in the assessment task, increased the marks achieved for individual components of the assessment. The results provide insight into how the design of these types of assessment exercises can be improved.     

Technology-Enhanced Formative Assessment of Plant Identification

Developing plant identification skills is an important part of the curriculum of any botany course in higher education. Frequent practice with dried and fresh plants is necessary to recognize the diversity of forms, states, and details that a species can present. We have developed a web-based assessment system for mobile devices that is able to pose appropriate questions according to the location of the student. A student’s location can be obtained using the device position or by scanning a QR code attached to a dried plant sheet in a herbarium or to a fresh plant in an arboretum. The assessment questions are complemented with elaborated feedback that, according to the students’ responses, provides indications of possible mistakes and correct answers. Three experiments were designed to measure the effectiveness of the formative assessment using dried and fresh plants. Three questionnaires were used to evaluate the system performance from the students’ perspective. The results clearly indicate that formative assessment is objectively effective compared to traditional methods and that the students’ attitudes towards the system were very positive.     

The impact of content area focus on the effectiveness of a web‐based simulation

This paper describes an assessment of a web‐based interview simulation designed to teach empathetic helping skills. The system includes an animated character acting as a client and responses designed to recreate a simulated role‐play, a common assessment method used for teaching these skills. The purpose of this study was to determine whether learning gains found in previous evaluations would transfer to students needing similar skill sets but in different areas of study. Findings indicate a positive trend in learning gains across both areas of study (human services and counselling) with no significant differences in skill acquisition. Developers of the system conclude that the system is effective in the general training of empathetic helping skills regardless of the students' area of study.     

Engineering education - finding the centre or 'back to the future'

In this paper I attempt to list the body of knowledge and skills which future young engineers will need and how such knowledge and skills can best be transferred to students during the engineering education process. The educational environment should stimulate students and encourage them to develop the capacity for lifelong learning as a preparation for tackling the unknown problems which will occur in future decades. The formation of an engineer should comprise both education and practice. Some of the problems facing higher education today are discussed and some ideas for cross-border harmonization of engineering education are given. Finally, there is a brief comparison of today's current ideas on engineering study programmes and the course that I followed more than 50 years ago.     

Anonymity as an instructional scaffold in peer assessment: its effects on peer feedback quality and evolution in students’ perceptions about peer assessment skills

Although previous research has indicated that providing anonymity is an effective way to create a safe peer assessment setting, continuously ensuring anonymity prevents students from experiencing genuine two-way interactive feedback dialogues. The present study investigated how installing a transitional approach from an anonymous to a non-anonymous peer assessment setting can overcome this problem. A total of 46 bachelor’s degree students in Educational Studies participated in multiple peer assessment cycles in which groups of students assessed each other’s work. Both students’ evolution in peer feedback quality as well as their perceptions were measured. The content analysis of the peer feedback messages revealed that the quality of peer feedback increased in the anonymous phase, and that over time, the feedback in the consecutive non-anonymous sessions was of similar quality. The results also indicate that the transitional approach does not hinder the perceived growth in peer feedback skills, nor does it have a negative impact on their general conceptions towards peer assessment. Furthermore, students clearly differentiated between their attributed importance of anonymity and their view on the usefulness of a transitional approach. The findings suggest that anonymity can be a valuable scaffold to ease students’ importance level towards anonymity and their associated need for practice.     

A dutch contribution to counsellor education an in-service model. An in-service model

In the Netherlands counselling is seen as an integrated part of school life for the benefit of all students. Counsellor education at the University of Utrecht was provided as a part-time, in-service programme for teacher-counsellors. This structure enabled students to integrate the skills and theory learned with their practical application in schools; current school practice enriched learning on the programme. Examples of integrative activities which encouraged these benefits are given.     

Building reading resilience: re-thinking reading for the literary studies classroom

This paper introduces the concept of ‘reading resilience’: students’ ability to read and interpret complex and demanding literary texts by drawing on advanced, engaged, critical reading skills. Reading resilience is a means for rethinking the place and pedagogies of close reading in the contemporary literary studies classroom. Our research was across four Australian universities and the first study of its kind in the Australian context. We trialled three working strategies to support students to become consistent and skilled readers, and to equip teachers with methods for coaching reading: ‘setting the scene’ for reading, surveying students on their reading experiences and habits, and rewarding reading within assessment. We argue that the nature and pedagogy of close reading has not been interrogated as much as it should be and that the building of reading resilience is less about modelling or outlining best practice for close reading (as has traditionally been thought) and more about deploying contextual, student-centred teaching and learning strategies around reading. The goal is to encourage students to develop a broad suite of skills and knowledge around reading that will equip them long term (for the university and beyond). We measured the effectiveness of our strategies through seeking formal and informal student feedback, and through students’ demonstration of skills and knowledge within assessment.     

Exploring computer science students’ continuance intentions to use Kattis

Teaching programming concepts to enhance students’ problem solving and computational thinking skills is a challenging task, especially when students enter college with little to no preparation, or they lack the interest or capacity for programming. Online platforms that serve as automated practice and assessment systems have been offered as potential tools for supporting programming skills development, providing feedback, and motivating students. The present article discusses the use of an online automated practice and assessment system called Kattis for homework assignments and final project in three computer science courses. The goal of the present study was to ascertain students’ continuance intentions to use Kattis. We attempt to address this by using partial least squares on data from a survey of 50 students. The findings of the present study suggest that continuance intentions to use Kattis is driven by students’ level of satisfaction with the system, the degree of students’ confirmation of expectations, and the perceived usefulness of the system.