Focus: Design and Evaluation of a Software Tool for Collecting Reader Feedback

Reader feedback is generally considered to be valuable input for writers who want to optimize their documents, but a reader-focused evaluation is often time-consuming. For this reason, we have developed Focus, a software tool for collecting reader comments more efficiently. The design and rationale of the software are described in this article. In a small-scale evaluation study, the results we obtained using Focus were compared to the reader feedback collected under the plus-minus method. It appeared that the number of problems detected per participant did not differ, but there were differences in the types of problems found. Focus participants appeared to comment more from a reviewer's and less from a user's perspective. Although the two methods are not interchangeable, Focus can be said to be a promising evaluation tool, deserving further research.     

Feedback during active learning: elementary school teachers’ beliefs and perceived problems

Giving feedback during active learning is an important, though difficult, task for teachers. In the present study, the problems elementary school teachers perceive and the beliefs they hold regarding this task were investigated. It appeared that teachers believe conditional teacher skills, especially time management, hinder them most from giving good feedback. The most widely held belief was that “feedback should be positive”. Teachers also believed that it is important to adopt a facilitative way of giving feedback, but they found this difficult to implement. Only some teachers believed goal-directedness and a focus on student meta-cognition were important during active learning and teachers did not perceive problems regarding these aspects. It was discussed whether teachers’ feedback behaviour was in line with these perceived problems and beliefs. The results give directions for the professional development of teachers to improve their feedback during active learning.     

基于动态学习数据流的“伴随式评价”框架设计

在“为了学习而评价”的影响下,教学评价日益表现出以数据为导向、追踪教学全过程、面向学生全面发展的“伴随”取向。但囿于时间不足、技术条件限制等原因,数据类型单一、反馈不及时、脱离教学情境等仍然是教学评价所面临的重要难题。随着智慧课堂的介入,更多、更细、更高频率的数据可以得到及时的采集,处理、分析这些数据,将有助于发挥评价的导向、激励、调整功能。因此,研究以智慧课堂中动态学习数据的生成、流动与转换为契机,引入了“伴随式评价”的理念,结合智慧课堂中学情诊断—备课、互动参与—教学、成果检验—迁移三个节点,从评价的时间、目的、内容、主体和工具五个层面构建了“伴随式评价”的设计框架,并面向初中英语课程设计了案例。     

Using mid-semester course evaluation as a feedback tool for improving learning and teaching in higher education

The way in which mid-semester course evaluations are structured, administered and reported is important for generating rich and high-quality student feedback for the enhancement of learning and teaching. Mid-semester evaluations usually contain open-ended questions that trigger more elaborative feedback about what is going on in a class than that from end-of-semester evaluations with Likert scale-type questions. The anonymity of the process for students and the confidentiality of the process for instructors make the process more reflective for students and less stressful for instructors. This study describes how the mid-semester course evaluation process can be used as a feedback tool for improving the quality of teaching and learning at an institutional level. Through a longitudinal analysis of 341 mid-semester course evaluation reports, positive areas and areas of concern with respect to learning and teaching were identified, and changes in student evaluations over the years were examined meticulously to make an overall evaluation of the quality of learning and teaching at a non-profit Turkish university. This research showed that the value of mid-semester course evaluations can go beyond course-level if we use open-ended questions and are able to gather the reports together to make some comprehensive analysis at university level.     

Pupils’ Information Processing and Its Implications for Learning and Assessment: A Think-Aloud Study

A think-aloud study investigated 7th graders’ (N?=?30) information processing and seeking of feedback while reading four multiple partly contradictory texts. An extreme-case analysis shows that a highly-skilled reader uses deeper strategies, such as elaboration, evaluation, and paraphrasing of less contradictory information, than a less-skilled reader. Elaboration strategies are frequently used by the less-skilled reader, but often consist of irrelevant elaboration strategies, which seem to preclude comprehension and deeper learning. The highly-skilled reader seeks internal feedback on task-, process-, and self levels, while the less-skilled reader seeks feedback related to task- and self levels. The results provide useful knowledge regarding pupils’ trajectories into deeper learning and self-regulation processes in information-processing tasks.     

对分课堂教学模式在传感器课程中的应用研究

对分课堂教学模式是为解决当前高校课堂教学问题与困境而提出的中国原创教学模式[1]。文章以传感器课程为例进行实践,将对分课堂教学模式用于传感器课程课堂教学。介绍课堂教学设计及对分易教学平台的应用,关注学生学习的阶段性成果,对达成情况进行实时评价和反馈。     

Closing the feedback loop? Iterative feedback between tutor and student in coursework assessments

We evaluate the case for using feedback iteratively, to improve student engagement and learning. In this model, students were invited to respond to tutor feedback with students’ own responses. Among the three courses/modules (three tutors) studied, differences in feedback styles were evident from: (a) thematic analysis of tutor comments and, for one course, and (b) comments in a student focus group. Students were inconsistent in their evaluation of tutor feedback but, in a more detailed study of one of the courses, there was clear evidence for the benefits of iterative feedback. Lessons from the main study were then applied to a course that had not previously incorporated iterative feedback. Using this experience, we provide suggestions for applying iterative feedback in assessments.     

Usability testing of a multimedia e‐learning resource for electrolyte and acid‐base disorders

The usability of computer interfaces may have a major influence on learning. Design approaches that optimize usability are commonplace in the software development industry but are seldom used in the development of e‐learning resources, especially in medical education. We conducted a usability evaluation of a multimedia resource for teaching electrolyte and acid‐base disorders by studying the interaction of 15 medical doctors with the application. Most of the usability problems occurred in an interactive treatment simulation, which was completed successfully by only 20% of participants. A total of 27 distinct usability problems were detected, with 15 categorized as serious. No differences were observed with respect to usability problems detected by junior doctors as compared with more experienced colleagues. Problems were related to user information and feedback, the visual layout, match with the real world, error prevention and management, and consistency and standards. The resource was therefore unusable for many participants; this is in contrast to good scores previously reported for subjective user satisfaction. The findings suggest that the development of e‐learning materials should follow an iterative design‐and‐test process that includes routine usability evaluation. User testing should include the study of objective measures and not rely only on self‐reported measures of satisfaction.     

An innovative concept map approach for improving students' learning performance with an instant feedback mechanism

Concept maps have been widely employed for helping students organise their knowledge as well as evaluating their knowledge structures in a wide range of subject matters. Although researchers have recognised concept maps as being an important educational tool, past experiences have also revealed the difficulty of evaluating the correctness of a concept map. It usually takes days or weeks for teachers to manually evaluate the concept maps developed by students; consequently, the students cannot receive timely feedback from the teachers, which not only affects their learning schedules, but also significantly influences the students' learning achievements. In this paper, a computer‐based concept map‐oriented learning strategy with real‐time assessment and feedback is proposed in order to cope with the problems mentioned above. Our approach provides immediate evaluation of concept maps and gives also real‐time feedback to the students. An experiment has been conducted to evaluate the effectiveness of this new strategy in comparison with the conventional computer‐based concept map approach. It is found that our innovative approach can be significantly beneficial to promote learning achievements as well as the learning attitudes of students.     

Was Hochschuldozierende aus Lehrevaluations-Rückmeldungen lernen k?nnen

The extent to which students profit from a university course depends on the lecturers?? approach to teaching, with a student-focused approach being more supportive for student gains in competences than a teacher-focused approach. Little is known, however, about how to foster a student-focused approach to teaching. In this paper, we want to investigate how to frame the feedback lecturers obtain from students?? course evaluations in such a manner that a student-focused approach to teaching is encouraged. In two studies using either a qualitative or a quantitative methodology it can be shown that providing lecturers with a feedback on students?? subjective gain in competences (output-focused teaching evaluation) is more effective in promoting a student-focused approach to teaching than a feedback on how satisfied students were with the lecturer (process-focused teaching evaluation). Results are discussed in the context of the Bologna reforms which demand a continuous evaluation of all university courses.     

Leveraging new media skills in a peer feedback tool

Today's youth thrive in informal participatory communities where they not only consume but also act as contributors or producers. Through active participation they develop new media skills encompassing cultural competencies and social skills. In a participatory culture of learning, students' active contributions to their learning are stressed and peer feedback is considered an important component. In this paper we consider how to leverage aspects of these skills and the environments in which they are developed in the design of a peer feedback tool for students engaged in collaborative inquiry learning. The tool needs to be lightweight, and “playful” and draw on these participatory skills in order to entice and motivate students to give and receive feedback.     

什么是好的教育政策执行效果的评估

教育政策执行效果的评估是教育政策活动乃至教育系统有序运行的基本保障。好的教育政策执行效果的评估,不仅是教育政策目标实现的根本途径,也是强化教育政策操作与执行力度的重要措施,而且是"再造教育政策"或"生产新的教育政策"的基本依据。教育政策执行效果的评估是评估主体依据一定的价值标准,运用特定的方式方法,对教育政策执行活动作用于政策目标领域内的社会系统要素的影响变化进行事实描述与价值审判,并反馈于政策循环的活动,具有涉及复杂的利益相关者、目标模糊且广泛、过程不稳定与不确定并存、方法具有局限性、结果具有反馈性等特征。当前我国教育政策执行效果的评估存在着理论研究匮乏滞后、评估视域困守于实证范式、评估方法与技术单一、评估活动要素专业性有待提升、评估结果的反馈利用低效等问题。基于此,好的教育政策执行效果的评估需要从评估标准坚持事实与价值相统一、指标体系兼顾要素多元与联动协调、评估方法与技术切实可行、评估结果分析说理与反馈应用并重等方面重构分析框架,从而使教育政策执行实践中的政策虚置、政策悬置、政策失灵等现象尽量少出现乃至不出现。     

高职软件课程体系构建的探讨与实践

针对软件专业课程体系构建中存在的问题,通过研究与实践,本文提出了加大市场调研,凸现专业特色,以项目实训为平台,构建“三维一体”的专业课程体系的总体思路。强调坚持走校企合作之路,充分整合校企资源,互补长短,辅以多样化的考核评价标准、完善的反馈机制,运用先进的教学方法和手段提高课程改革的实践效果。     

An interdisciplinary analysis of microteaching evaluation forms: how peer feedback forms shape what constitutes “good teaching”

Microteaching, a standard method for developing teaching skills, places high importance on peer feedback, which is guided by post-session feedback forms. This paper focuses on how feedback forms can shape what becomes understood as important to teaching. A sample of 10 microteaching evaluation forms drawn from North American postsecondary education institutions were examined using both qualitative and quantitative methods. Through a mixed-methods, interdisciplinary approach combining quantitative form classification based on distinct teaching elements with qualitative analysis drawing on Foucauldian and post-structural feminisms, key challenges are identified in the way that peer feedback forms may shape perceptions of what constitutes “good teaching”. We interpret that the close attention paid to the management of the body and the disproportionate focus on presentation and style may foreclose other modes of teaching beyond a conventional lecture-based class. This leads to a discussion on the ways in which the evaluation forms can be enhanced to provide a more effective educational tool.     

The alignment of written peer feedback with draft problems and its impact on revision in peer assessment

Despite the wide use of peer assessment, questions about the helpfulness of peer feedback are frequently raised. In particular, it is unknown whether, how and to what extent peer feedback can help solve problems in initial texts in complex writing tasks. We investigated this research gap by focusing on the case of writing literature reviews in an academic writing course. The dataset includes two drafts from 21 students, sampled to represent a wide range of document qualities, and 84 anonymous peer reviews, involving 1,289 idea units. Our study revealed that: (1) at both substance and high prose levels, drafts of all quality levels demonstrated more common problems on advanced writing issues (e.g. counter-argument); (2) peer feedback was driven by difficulty of the problem rather than overall draft quality, peer comments were not well aligned with the relative frequency of problems, more comments were given to less difficult problems; (3) peer feedback had a moderate impact on revision, and importantly, receiving multiple comments on the same issue led to more repairs and improvement of draft quality, but consistent with the comments received, authors tended to fix basic problems more often. Implications for practice and research are drawn from these findings.     

Using Slack for computer-mediated communication to support higher education students’ peer interactions during Master’s thesis seminar

Our study contributes to the research on computer-mediated communication in higher education by experimenting a modern communication tool called Slack. In particular, we consider using Slack to support students’ peer interactions during Master’s thesis work. For this purpose, we designed a case study that was executed in a Master’s thesis seminar course. During the course, all out-of-class communication was carried out by using Slack, instead of e-mails or learning management systems. After the course, we used a questionnaire to investigate how the students perceived Slack for asking for assistance, their intention to use Slack, and Slack’s ease of use. Furthermore, the questionnaire asked feedback about challenges that the students found in slack. To examine the students’ peer support in Slack, we analysed the messages in the course’s public discussion channels. We investigated opportunities and challenges of Slack from instructional perspective by conducting an auto-ethnographic data collection. Our analysis revealed that the students perceived Slack as an easy-to-use communication tool with a low threshold of asking questions. The students also expressed high intentions to use Slack in the future. However, the students were worried of information overload in Slack, frustrated with decentralisation of communication tools in higher education, and cautious of using communication tools that are not officially supported by the university. The students’ interactions were assigned to three categories: practical, technical, and thesis-related. Analysis of these categories revealed that the students were able to explicate and solve issues in Slack, but the issues were only related to practical and technical problems, instead of actual Master’s thesis writing. The teacher perceived that Slack enhanced bi-directional communication with the students, but faced issues related to file management and user authentication. The results implicate that developing an alternative for Slack from educational premises could be more useful than a product that is not originally developed for pedagogical needs. Finally, we present recommendations that help educators to use Slack in their educational practices.     

360‐degree feedback: Going around in circles?

Multisource feedback (often called 360‐degree feedback) is, in itself, an excellent idea. In practice, though, multisource feedback is often implemented in a way that can do more harm than good. Many commercially available 360‐degree feedback programs seem to be poorly designed. They may provide reasonably accurate assessments but are inadequate in their delivery of those assessments. The problem lies primarily in the way they deliver the feedback. In this article, we discuss the problems and offer some remedies.     

The Development of the Manchester Motor Skills Assessment (MMSA): An initial evaluation

This article is an initial evaluation of a motor skills assessment for primary aged children. The Manchester Motor Skills Assessment (MMSA) is designed to be quick and easy for teaching assistants to complete, with the dual purposes of informing group programme planning and demonstrating an individual child’s progress following a period of intervention. Inter‐rater reliability checks were conducted during initial assessments of 37 children in 11 schools. Focus groups discussed practical relevance of the tool. The evidence from this small‐scale study suggests that a range of assessors can reliably complete the MMSA. The tool also taps into functionally relevant motor skills for primary age children and informs programme planning. The MMSA appears to be a promising tool for supporting primary schools in identifying children with motor skills difficulties. The contribution of EPs to the MMSA’s development and implementation is discussed.     

A Multilevel CFA–MTMM Approach for Multisource Feedback Instruments: Presentation and Application of a New Statistical Model

Multisource feedback instruments are a widely used tool in human resource management. However, comprehensive validation studies remain scarce and there is a lack of statistical models that account appropriately for the complex data structure. Because both peers and subordinates are nested within the target but stem from different populations, the assumption of traditional multilevel structural equation models that the sample on a lower level stems from the same population is violated. We present a multilevel confirmatory factor analysis multitrait–multimethod (ML–CFA–MTMM) model that considers this peculiarity of multisource feedback instruments. The model is applied to 2 scales of the Benchmarks® instrument and it is demonstrated how measures of reliability and of convergent and discriminant validity can be obtained using multilevel structural equation modeling software. We discuss the results as well as some implications and guidelines for the use of the model.