Meta-assessment in a project-based systems engineering course

Project-based learning (PBL) facilitates significant learning, but it poses a major assessment challenge for assessing individual content knowledge. We developed and implemented an assessment approach and tool for a mandatory undergraduate systems engineering PBL-based course. We call this type of assessment student-oriented meta-assessment. Research participants included 131 undergraduate engineering students who carried out team projects requiring conceptual modelling of complex systems. Next, individual students assessed their peer team projects. Finally, the course staff assessed students individually, based on the content knowledge that was reflected in students’ comments to their peers, by using the new meta-assessment tool. The research findings validated the meta-assessment tool which can serve for assessing various PBL courses. Our contribution is twofold: a new approach for assessing project-based undergraduate engineering courses, and classification of meta-assessment methods into three types, where a student-oriented meta-assessment as one of the three meta-assessment methods is our newly introduced and validated method.     

Curriculum assessment as a direct tool in ABET outcomes assessment in a chemical engineering programme

The chemical engineering programme at the United Arab Emirates University is designed to fulfil the Accreditation Board for Engineering and Technology (ABET) (A–K) EC2000 criteria. The Department of Chemical & Petroleum Engineering has established a well-defined process for outcomes assessment for the chemical engineering programme in order to ensure that its graduates achieve the programme educational objectives. Different direct and indirect tools are implemented in the assessment process. Among these tools, the greatest weight in the assessment process has been given to the curriculum (30%). The course and curriculum assessment usually plays an important role in improving the course content and quality; as it should provide considerable information on the effectiveness of an academic programme. This paper aims to discuss the methods used to assess the courses/curriculum of the chemical engineering programme as an important direct tool in the assessment process for ABET accreditation. Application of the curriculum assessment results in the continuous improvement of chemical engineering programmes is also addressed.     

Improving teaching and learning in engineering education through a continuous assessment process

This paper presents the effects of a continuous assessment process on mechanical engineering education at Kuwait University. This process has been in place for the last five academic years and is producing results. Efforts to educate faculty and students about the process are beginning to produce accurate and consistent measurements of student learning. Although another assessment cycle is needed to make a better judgement, there is evidence that assessment is effective in improving teaching and learning in the programme. There is also an increased constituent awareness of the utility of assessment, which helps in instilling an institutional culture and therefore assuring a sustainable process.     

Designing effective,contemporary assessment on a flipped educational sciences course

ABSTRACT

Evidence shows flipped learning increases academic performance and student satisfaction. Yet, often practitioners flip instruction but keep traditional curricula and assessment. Assessment in higher education is often via written exams. But these provide limited feedback and do not ask students to put knowledge into practice. This does not support the tenets of flipped learning. For two years, the author flipped instruction but retained traditional curricula and assessment. However, on the author’s current course, all three aspects were redesigned to better support flipped learning. The aim of this research is to test the effectiveness of this redesign regarding student engagement and satisfaction. Thus, it is asked: How, on this course, can meaningful, continuous assessment be provided as well as effective, personalized feedback, while staying in line with the philosophy of flipped learning? Action research took place from September 2016 to June 2017. Quantitative data from a student survey, and qualitative data from a research diary and student focus group were gathered. What emerged is: a little-and-often assessment approach is effective for learning and engagement; tasks must be authentic and test demonstration of knowledge, not memory; quality, not quantity, is key for student learning; and students desire individualized feedback.     

The content aspect of validity in a rubric-based assessment system for course syllabuses

The growing trend among universities to promote systems of programme and course evaluation entails more responsibility for faculties and departments. These systems require resources to ensure that they are not only valid and reliable but also effective and sustainable. The design of rubric-based assessment systems may provide a solution, but there is a gap in the research on curriculum evaluation concerning their use and validation. We examine the content aspect of validity in a rubric-based assessment system for course syllabuses using a mixed method that combines an analysis of the agreement among 23 experts with a phenomenographic study. With data gathered through a questionnaire linked to the Delphi technique, content validity indexes were calculated and the experts' different perspectives were identified. The content validity indexes (greater than .80) met the standards set out in literature, and the qualitative study of the experts' feedback showed three different perspectives on the system's use. Beyond providing evidence of the system's content validity, the study highlights the extent to which it is important to give appropriate consideration to experts' – and by extension final users' – experience in order to ensure the successful implementation of rubric-based assessment systems.     

Establishing the critical elements that determine authentic assessment

This study sought to determine the critical elements of an authentic learning activity, design them into an applicable framework and then use this framework to guide the design, development and application of work-relevant assessment. Its purpose was to formulate an effective model of task design and assessment. The first phase of the study identified from the literature critical elements that determined assessment as being authentic, and presented these to practitioners and experts for feedback. In phase two, it codified the elements into a framework that was then applied to the redesign of assessments in an army course. Phase three involved student evaluation of the redesigned assessment activities. This led to further review and revision of elements in phase four. The study outcomes suggest that it is possible, by identifying and codifying individual elements, to determine the ways in which the authenticity of an individual assessment activity might be enhanced. The paper concludes with a literature update on the framework elements that lead to suggestions for further research.     

Design,implementation, and evaluation of an innovative anatomy course

Starting in 2004, a medical school gross anatomy course faced with a 30% cut in hours went through an extensive redesign, which transformed a traditional dissection course into a course with a clinical focus, learning societies, and extensive on‐line learning support. Built into the redesign process was an extensive and ongoing assessment process, which included student focus groups, faculty development, surveys, and examinations. These assessments were used formatively, to enhance the course from year to year, and summatively, to determine how well the course was meeting the new learning objectives. The assessments from focus groups and faculty development prompted changes in support structures provided to students and the training and preparation of faculty. Survey results showed that, after student satisfaction declined the first year, satisfaction increased steadily through the fourth iteration as the course gained acceptance by students and faculty alike. There was a corresponding increase in the performance of students on course examinations. An additional examination given to students one and a half and three years after their anatomy course ended demonstrated the redesigned course's long‐term effectiveness for retaining anatomical knowledge and applying it to clinical cases. Compared to students who took the original course, students who took the shorter, more clinical course performed as well, or better, on each section of the examination. We attribute these positive results to the innovative course design and to the changes made based on our formative assessment program. Anat Sci Educ, 2010. © 2010 American Association of Anatomists.     

Applying cognitive load theory to the redesign of a conventional database systems course

Cognitive load theory (CLT) was used to redesign a Database Systems course for Information Technology students. The redesign was intended to address poor student performance and low satisfaction, and to provide a more relevant foundation in database design and use for subsequent studies and industry. The original course followed the conventional structure for a database course, covering database design first, then database development. Analysis showed the conventional course content was appropriate but the instructional materials used were too complex, especially for novice students. The redesign of instructional materials applied CLT to remove split attention and redundancy effects, to provide suitable worked examples and sub-goals, and included an extensive re-sequencing of content. The approach was primarily directed towards mid- to lower performing students and results showed a significant improvement for this cohort with the exam failure rate reducing by 34% after the redesign on identical final exams. Student satisfaction also increased and feedback from subsequent study was very positive. The application of CLT to the design of instructional materials is discussed for delivery of technical courses.     

Effective formative e-assessment of student learning: a study on a statistics course

The process of formative assessment in universities has the potential to engage students in reflection and to take greater ownership of their learning. We report on a study involving 104 students taking a statistics course in a degree programme in rehabilitation science in Hong Kong. The assessment strategy was redesigned to include a weekly online formative assessment activity for 10 weeks. In this case, it was observed that there was a high participation rate across weeks. An end-of-course online survey was conducted to explore the learning experiences during formative assessment of the cohort, and students reported high satisfaction with the process, particularly how this contributed to a change in their learning approach. Students not only showed regular commitment of time on tasks but also took active steps to find answers for themselves and with their peers.     

Evaluating a technology supported interactive response system during the laboratory section of a histology course

Monitoring of student learning through systematic formative assessment is important for adjusting pedagogical strategies. However, traditional formative assessments, such as quizzes and written assignments, may not be sufficiently timely for making adjustments to a learning process. Technology supported formative assessment tools assess student knowledge, allow for immediate feedback, facilitate classroom dialogues, and have the potential to modify student learning strategies. As an attempt to integrate technology supported formative assessment in the laboratory section of an upper‐level histology course, the interactive application Learning Catalytics^TM, a cloud‐based assessment system, was used. This study conducted during the 2015 Histology courses at Cornell University concluded that this application is helpful for identifying student misconceptions “on‐the‐go,” engaging otherwise marginalized students, and forming a new communication venue between students and instructors. There was no overall difference between grades from topics that used the application and grades from those that did not, and students reported that it only slightly helped improve their understanding of the topic (3.8 ± 0.99 on a five‐point Likert scale). However, they highly recommended using it (4.2 ± 0.71). The major limitation was regarding the image display and graphical resolution of this application. Even though students embrace the use of technology, 39% reported benefits of having the traditional light microscope available. This cohort of students led instructors to conclude that the newest tools are not always better, but rather can complement traditional instruction methods. Anat Sci Educ 10: 328–338. © 2016 American Association of Anatomists.     

Example of good practice of a learning environment with a classroom response system in a mechanical engineering bachelor course

Results of a successful pilot study are presented, in which quizzes are introduced in a second year bachelor course for mechanical engineering students. The pilot study course entailed the basic concepts of mechanical vibrations in complex, realistic structures. The quiz is held weekly using a SharePoint application. The purpose of the quizzes is to repeat important course material, give instantaneous feedback (i.e. formative assessment), stimulate peer instruction and, as a consequence, increase the students’ comprehension of the basic concepts taught in the course so that their deeper understanding of the subject matter improves. Students can earn half a point bonus, on a scale from 0 to 10, on top of their exam mark if they correctly answer 55% of all the quiz questions. The efficacy of the pilot study is determined by investigating the percentage of students that pass the course on their first attempt, i.e. the first time pass rate, and asking students for feedback through questionnaires. The first time pass rate of the students in the pilot study groups has, on average, increased significantly in comparison to groups in which the quizzes are not performed. Students indicated that the feedback from the quizzes helps them to identify gaps in their knowledge. Therefore, the pilot study is considered effective.     

Identifying threshold concepts: case study of an open catchment hydraulics course

The Threshold Concept Framework is used to initiate a dialogue on an empirically supported pedagogy that focuses on students’ conceptual understanding required for solving application-based problems. The present paper uses a triangulation approach to identify the threshold concept in a third-year undergraduate civil engineering course on open channel hydraulics. Evidence from teachers, students, and assessment data point to ‘critical flow’ as the threshold concept – a concept that is transformative, integrative, and troublesome. Identifying the threshold concept by engaging various course stakeholders in a dialogue about conceptual understanding and capabilities makes learning visible for all participants in the process. Implementing this approach can result in an empirically driven rationale for adjusting pedagogies and assessments to foster enhanced student learning outcomes.     

“信息光学”过程化考核的课堂教学建设与实践

“信息光学”是现代光学的重要分支,普遍被学生认为是比较难懂的一门光学专业课程。采用过程化考核模式,对信息光学的教学内容、方法与设计进行了课程建设与实践。在教学内容和方法上进行了探索,从绪论中引入苏州大学光学工程学科的光信息特色方向介绍、注重基础理论与现代光学应用结合、教学与科研结合。基础内容学习采用章考核方式、应用学习注重文献阅读和翻转课堂汇报。过程化考核的实际教学过程中,注重启发和引导学生的自主学习和思考,激发了学习的主动性和积极性,收到了良好的教学效果。     

Formative Assessment in EFL Writing: An Exploratory Case Study

In second‐language writing, assessment has traditionally focused on the written products and how well (or badly) students perform in writing. Teachers dominate the assessment process as testers, while students remain passive testees. Assessment is something teachers ‘do to’ rather than ‘with’ students, mainly for administrative and reporting purposes (i.e. summative). Such assessment, being more retrospective than prospective, holds little value for teaching and learning. In recent years, with a major paradigm shift in assessment and evaluation in English language teaching, writing assessment informed primarily by a product and summative orientation, is considered increasingly inadequate. Such assessment, which focuses on measurement – i.e. marking, monitoring and checking, fails to capture the formative potential of assessment for promoting learning. A formative approach to assessment, on the other hand, focuses more on inquiry – i.e. discovering, diagnosing and understanding, as well as the opportunities assessment provides for improving teaching and learning. To harness the potential of formative assessment in the writing classroom, it is axiomatic that classroom assessment practices be geared towards maximizing student learning. This provides the impetus for my study, which investigates an EFL teacher's attempt to implement formative assessment in her writing classroom and its impact on her classroom practice and students' beliefs and attitudes to writing.     

Interpretations of formative assessment in the teaching of English at two Chinese universities: a sociocultural perspective

Formative assessment is increasingly being implemented through policy initiatives in Chinese educational contexts. As an approach to assessment, formative assessment derives many of its key principles from Western contexts, notably through the work of scholars in the UK, the USA and Australia. The question for this paper is the ways that formative assessment has been interpreted in the teaching of College English in Chinese Higher Education. The paper reports on a research study that utilised a sociocultural perspective on learning and assessment to analyse how two Chinese universities – an urban-based Key University and a regional-based Non-Key University – interpreted and enacted a China Ministry of Education policy on formative assessment in College English teaching. Of particular interest for the research were the ways in which the sociocultural conditions of the Chinese context mediated understanding of Western principles and led to their adaptation. The findings from the two universities identified some consistency in localised interpretations of formative assessment which included emphases on process and student participation. The differences related to the specific sociocultural conditions contextualising each university including geographical location, socioeconomic status, and teacher and student roles, expectations and beliefs about English. The findings illustrate the sociocultural tensions in interpreting, adapting and enacting formative assessment in Chinese College English classes and the consequent challenges to and questions about retaining the spirit of formative assessment as it was originally conceptualised.     

Learning-oriented e-assessment: the effects of a training and guidance programme on lecturers’ perceptions

Various frameworks that acknowledge the importance of assessment as a core aspect of the learning process have been proposed to enhance life-long learning and promote participative strategies. In this context, learning-oriented e-assessment was developed to enhance learning through assessment in a technology-mediated context. Using a quantitative–qualitative mixed research method, the current study aimed to analyse the effects of a learning-oriented e-assessment training and guidance programme on university lecturers’ perceptions of the importance of assessment, their competence in assessment and their actual use of assessment, and to consider lecturers’ experience and perception when implementing learning-oriented e-assessment in one of their courses. The results of the quantitative research showed a significant difference between the pre-test and post-test measures in the competence and use criteria for the following four categories: ‘assessment planning and design’, ‘monitoring of student learning’, ‘participation of students in the assessment process’ and ‘improvement and changes to the assessment process’. The qualitative results support the quantitative outcomes, providing some insight into lecturers’ perception of the experience.     

Renovation of a mechanical engineering senior design class to an industry-tied and team-oriented course

In this work, an industry-based and team-oriented education model was established based on a traditional mechanical engineering (ME) senior design class in order to better prepare future engineers and leaders so as to meet the increasing demand for high-quality engineering graduates. In the renovated curriculum, industry-sponsored projects became the most important course component and critical assessment tool, from which problem-solving skills as well as employability skills of the ME students can be fully developed. Hands-on experiences in finite element analysis (FEA) modelling and simulation were also added into the renovated curriculum to promote the application of FEA on engineering design and assessment. Evaluation of the renovated course was conducted using two instruments and the results have shown that the course made the ME senior students more prepared for their future career and a win–win model was created between the industry partner and the ME programme through it. Impact of the renovated syllabus on Accreditation Board for Engineering Technology goals was discussed. Based on the current progress, a more substantial change is being planned to further improve the effectiveness and practicability of this design course. The renovated course was started to offer to the ME senior students at Mississippi State University.     

高职铁道工程技术专业课评价方式改革的实践——以铁路桥梁施工与维护课程为例

课程评价方式改革是高职教育课程改革的重要内容之一。通过总结近年来高职课程评价方式改革的成果与不足,分析过程性评价和终结性考核的特点,建议高职铁道工程技术专业课的评价要过程性评价与终结性考核相结合。以铁路桥梁施工与维护课为例,介绍了上述评价理念在课程评价中的应用以及评价内容和方法。     

Reverse engineering course at Philadelphia University in Jordan

Reverse engineering (RE) is the process of testing and analysing a system or a device in order to identify, understand and document its functionality. RE is an efficient tool in industrial benchmarking where competitors’ products are dissected and evaluated for performance and costs. RE can play an important role in the re-configuration and redesign of legacy and/or undocumented systems. It can also play a key role in helping students understand engineering products. This paper presents the Philadelphia University experience in constructing a RE course and integrating it within the engineering curricula. This paper can be used as a guide to other universities that wish to introduce RE to their curricula. The information provided in this paper covers the RE methodology for a system level, as well as mechanical, electronics and software levels. Finally, samples of student projects are presented in order to show the learning capabilities provided throughout the course.