Written feedback provided by first-year engineering students,undergraduate teaching assistants,and educators on design project work

ABSTRACT

Feedback is an important part of design education. To better understand how feedback is provided to students on their engineering design work, we characterised and compared first-year engineering students’, undergraduate teaching assistants’, and educators’ written feedback on sample student design work. We created a coding scheme including two domains: Substance and Focus of feedback. Educators made more and longer comments than undergraduate teaching assistants, and undergraduate teaching assistants made more and longer comments than first-year students. The first-year students focused on giving specific directions in their feedback while educators and undergraduate teaching assistants asked thought-provoking questions. Students tended to make more comments about the ways that their peers had communicated their design work while educators and undergraduate teaching assistants made more comments about the design ideas presented in the sample work. This study offers implications for practice for supporting educators, undergraduate teaching assistants, and first-year engineering students to be able to provide feedback on design work.     

Student feedback to improved retention: using a mixed-methods approach to extend specific feedback to a generalisable concept

ABSTRACT

Across the Higher Education (HE) sector student feedback is used to feed into university processes and guide decision making. In this study, data gathered allowed the authors to investigate a hitherto neglected, but important, cohort of successful students – those who succeeded when all the odds were stacked against them. The identified group of students had relatively low predictive probabilities of passing their module, and yet despite all the odds passed. These identified students were interviewed by phone which yielded a much richer data set than is often available from student surveys. The advice they considered to be important to future students included specific guidance to get ahead, to plan and to contact their own academic tutor. Based on this advice an opportunity for future students to have a flexible early start on a specific module was created. A 4% increase in the number of registered students who subsequently started the module was maintained through to module completion.     

Pursuing Fluency: A Curricular Intervention in Tanakh Education

ABSTRACT

This study set out to design and implement an approach to Tanakh education that would help students become expert decoders of the Biblical Hebrew text as they became expert interpreters of it. The goal, following existing, research-based best instructional practices from literacy, was to create a curriculum in which language skills and meaning making were intimately connected. This paper describes the curriculum and its implementation.     

The relationship between design reflectivity and conceptions of informed design among high school students

ABSTRACT

Although reflection is a key behaviour of expert designers, it is often a challenging task for new designers. In addition, research on the reflectivity of student designers is limited. The purpose of this study is twofold: (1) to identify the levels of reflectivity while designing, and (2) to study the relationship between reflectivity and conceptions of informed design. We collected data from high school students engaged in an engineering design project. We developed a coding protocol to score levels of reflectivity in student reflections at three levels (low, medium, and high), and used the conceptions of design test to assess changes in student understanding of design activities. Using Wilcoxon signed-rank tests, we determined if students tended to select more ‘key’ design activities and fewer ‘distractors’ within each reflection group. We also performed McNemar’s tests to determine which specific design activities were important within each reflection group after the design project. The results show moderately reflective students had higher gains in understanding of informed design activities compared to those with high or low reflectivity. Results also indicate that different design activities became important for students within each of the three reflective groups. Implications from this research indicate that groups of students experience changing conceptions of design in different ways. An understanding of what students deem important while designing would better allow teachers to encourage behaviours that are like those of informed designers.     

‘It has a purpose beyond justifying a mark’: examining the alignment between the purpose and practice of feedback

Abstract

Research has shown that written feedback is important for student learning and development. However, the messages embedded in feedback may lead to students being misled about what they need to learn or how they need to develop. This article reports on a small-scale investigation into the messages embedded in feedback. Legitimation Code Theory was used to first conceptualise the often-hidden purpose of a discipline (English Studies), and concomitantly of feedback within the discipline, and second to analyse actual comments given to first-year students on their assignments. It was found that there is a clear misalignment between the purpose and practice of feedback, thereby suggesting that students are receiving misleading messages about what they need for success within the discipline. This may have implications beyond merely passing the module. A suggestion is made to actively consider, and develop, feedback as a discipline-specific literacy.     

Involving older people in the design,development, and delivery of an innovative module on aging for undergraduate students

ABSTRACT

As the number of older people in society increases, gaining an awareness of the needs of an aging population is important for university students from all academic backgrounds. Using a multidisciplinary approach to aging, we developed a new teaching module (NU-AGE [Newcastle University Aging Generations Education]) aimed at students enrolled in all undergraduate degree programs at our university. In acknowledgement of the important role that members of the public can play in higher education, we recruited a team of older adults to help design and deliver the module. The curriculum was constructed through collaboration with older members of the public and students to outline both the pertinent challenges facing an aging society, and the positive aspects of aging. The team of older adults contributed to the development and delivery of a series of interactive teaching sessions for a cohort of undergraduate students and collaborated with evaluation of the module. Early analysis of evaluation data suggested that students and members of the public highly valued the opportunity to interact and exchange ideas about the importance of aging in society in a multidisciplinary environment.     

Affective-motivational effects of performance feedback in computer-based assessment: Does error message complexity matter?

The impact of computer-based performance feedback on students’ affective-motivational state may be very different, depending on the positive or negative direction of the feedback message and its specific content. This experiment investigated whether more elaborated error messages improve students’ affective-motivational response to negative (i.e., corrective) feedback. We systematically varied the presence and complexity of corrective feedback messages (1 × 4 between-subjects design) and analyzed the effects of the provided feedback on students’ emotions, task-related perceived usefulness, and expectancy-value beliefs. University students (N = 439) worked on a low-stakes test with 12 constructed-response geometry tasks. They received either no feedback or different complexities of immediate corrective feedback after incorrect responses (i.e., Knowledge of Results [KR], Knowledge of Correct Response [KCR], or Elaborated Feedback [EF]), paired with immediate confirmatory KCR feedback after correct responses (i.e., confirming their response). Our data showed that students’ task-level performance moderated the emotional impact of feedback (i.e., beneficial effects after correct responses; detrimental effects after incorrect responses). Students’ performance further moderated several feedback effects on students’ expectancy-value beliefs. Regarding error message complexity, we found that students reported higher levels of positive emotions after receiving EF or KCR compared to KR, while only EF decreased students' level of negative emotions compared to KR and increased students' task-related perceived usefulness compared to all other groups. Overall, our results suggest that performance feedback is likely to improve students’ affective-motivational state when the feedback confirms a correct response. Moreover, when reporting an error, EF (or KCR messages) were more beneficial to affective-motivational outcomes than simple KR notifications.     

Case study: what supports students to improve their grades?

Abstract

This case study considers students who achieved ‘borderline’ (40–45%) grades in their first assignment on a module, but went on to markedly improve their grades over the course of the module. The students were studying nursing and social work at the UK Open University, and they were all sponsored by employers. Semi-structured telephone interviews were used to explore the experiences of the students, including the way in which they reacted to assessment feedback and how this contributed to their learning and development. Other themes that emerged from the study include the emotions experienced by the students, sometimes challenging their self-confidence and evoking feelings associated with emotional vulnerability, and the tendency for students to initially approach assessment independently, but later to create and take advantage of opportunities for social learning. The role of the tutor in helping students to prepare for assessment appears to be significant, as does the fact that students are sponsored by their employers, which provides an additional incentive to complete the module.     

Feedback sessions as mediation spaces: empowering teacher candidates to deepen instructional knowledge and engage in the construction and transformation of theory in practice

ABSTRACT

The process of engaging in classroom research provides teacher candidates with opportunities to enter into the kind of deep, self-reflective work that we believe is an important capacity to build in teachers as they begin to engage in the construction and transformation of theory and knowledge within their own unique contexts. Our Action Research (AR) process involves multiple opportunities throughout their coursework, advising, and feedback sessions to try to promote the development of this capacity. These opportunities serve as a holding space where candidates are both supported and challenged through interactions with their research advisors, other faculty members, their peers, and panelists consisting of educators from the field. In this paper, we present what we believe to be important elements of AR that emerged from our final feedback sessions with 13 teacher candidates in our program as they prepared for the professional presentations. We found that considering these feedback sessions as mediation spaces (1) empowers teacher candidates to externalize and deepen personal understandings of their research through dialogical discourse with expert others, and (2) negotiate their power and emerging practitioner researcher identities.     

Investigating the impact of automated feedback on students’ scientific argumentation

ABSTRACT

This study investigates the role of automated scoring and feedback in supporting students’ construction of written scientific arguments while learning about factors that affect climate change in the classroom. The automated scoring and feedback technology was integrated into an online module. Students’ written scientific argumentation occurred when they responded to structured argumentation prompts. After submitting the open-ended responses, students received scores generated by a scoring engine and written feedback associated with the scores in real-time. Using the log data that recorded argumentation scores as well as argument submission and revisions activities, we answer three research questions. First, how students behaved after receiving the feedback; second, whether and how students’ revisions improved their argumentation scores; and third, did item difficulties shift with the availability of the automated feedback. Results showed that the majority of students (77%) made revisions after receiving the feedback, and students with higher initial scores were more likely to revise their responses. Students who revised had significantly higher final scores than those who did not, and each revision was associated with an average increase of 0.55 on the final scores. Analysis on item difficulty shifts showed that written scientific argumentation became easier after students used the automated feedback.     

Peer review in the classroom: student perceptions,peer feedback quality and the role of assessment

Abstract

Peer review in the classroom can enhance numerous employability skills such as critical appraisal, writing skills, reflection practices and collaborative experiences. This study takes place over two years and discusses the implementation of a repeating blind peer review cycle across a single semester for final year chemistry students enrolled on a compulsory employability module. The feedback cycle promotes personal reflection through the use of mini-reflective questionnaires. The process was assessed by academic tutors at the resubmission stage and/or the peer feedback stage where the quality of peer feedback was directly assessed. The research investigates the quality of peer feedback, the importance of assessment and student perceptions of what is most useful. Methods include directed content analysis of feedback produced, student opinions and a focus group. Students were capable of offering useable feedback across a range of assessment criteria but tend to focus on the important criteria best aligned to the particular assignment. A range of motivational factors and tactics were noticed but students tended to find reviewing the most beneficial.     

A GEARed peer feedback model and implications for learning facilitation

This study proposes a model to view and analyse college students’ written peer feedback after they assess each other’s formative task assignment and provide comments. The model was developed by: (a) revisiting learning-oriented assessment and feedback theories, (b) reviewing and summarising existing peer feedback analysis approaches as detailed in published empirical studies, and (c) analysing a small set of primary learner peer feedback data. In the proposed GEARed model of peer feedback, each complete piece of comment on one improvable unit may have four cognitive elements, i.e. identification of the Gap between performance and goal, Explanation of the gap, gap-bridging Aspirations, and being Resourceful to reach beyond the current knowledge/skill level. Two social-affective-interactive elements also play a part, i.e. being encouraging to the peer feedback receiver and doubtful toward the provider self. Each feedback comment may contain any or all of these incremental elements rather than being assigned only to one category. It is argued that the absent or unresolved aspects in feedback leave room for the receiver to reflect and may suggest instructional actions for teachers. Finally, different patterns of GEARed component combinations are hypothesised and their implications for learning and teaching are discussed.     

Teaching aspects of the interrelationship between science and technology: a research report on the design,enactment and evaluation of a teaching proposal

Background: Helping upper elementary and lower secondary school students develop an awareness of various aspects of the nature of science (NOS) and nature of technology (NOT) is a widely recognized goal of science teaching. In this study, we focus on the connections between science and technology (S&T).

Purpose: We report on the design, development, enactment and evaluation of a teaching-learning sequence (TLS) that combines hands-on activities in geometrical optics with explicit epistemological discourse for reflection purposes. The design of the TLS draws on perspectives from the inquiry-oriented and design-based teaching and learning frameworks.

Sample: The enactment of the TLS involved a class of 17 sixth-grade students, aged 10–11 years old, of a public elementary school in Cyprus.

Design and methods: We present findings from written responses to both closed and open-ended tasks as well as follow-up semi-structured interviews that probed students’ understanding of the difference between the main goals of S&T.

Results: The results illustrate elementary students’ readiness to engage with epistemic issues and demonstrate the potential of prompting young learners’ ability to develop informed awareness of the NOS and NOT. The results also provided feedback for the revision of the TLS so as to further enhance its effectiveness in achieving the stated learning objectives.

Conclusion: We discuss the implications of our findings for the teaching of the NOS and NOT and for the design and validation of TLSs. It is possible for students of this age group to develop an awareness of issues related to the NOS and NOT. TLSs can be improved through design-based research approaches to serve as productive tools to this end.     

The Sheffield Industrial Project Scheme: team teaching for Engineering students

Abstract

Practical work is an important element in undergraduate engineering courses and laboratory‐based classes provide valuable experience. It is also important students should experience real industrial problems preferably through industrially related projects. A scheme was started in 1969 to provide students in the Department of Electronic & Electrical Engineering with industry‐based, group project work. The Sheffield Industrial Project Scheme (SHIPS) is an intensive 8‐day exercise in which groups of students are taken to a company, given a real problem and then asked to provide some possible solutions which they present orally and as a written report. It has hitherto been regarded as a valuable exercise in itself, developing students' personal skills and experience, and no mark is awarded at present. A student‐based marking scheme has been tried, but it was not well received.     

A distributed collaborative science learning laboratory on the internet

Abstract

A Distributed Collaborative Science Learning Laboratory (DCSLL) was designed, prototyped, and pilot‐tested as the “Electrical Circuit Simulator.” This laboratory was part of a module on electricity within an introductory distance course for postsecondary students on the scientific method. The concept of DCSLL emerged from work in distance education and new technologies, cooperative/collaborative learning, and science education. Instructional design principles derived from these areas are presented, and their implementation in the DCSLL is described, followed by results from the pilot test. Analysis of the results led to the articulation of six instructional design guidelines, identified as being key to the development of such learning environments.     

Interactive Robotics Using the Internet

We report on a project currently under way within the department of Computer Science at the University of Reading exploiting networking and multimedia technology to provide a shared robotics laboratory on the Internet. The laboratory is for use by our own students and students at other Higher Education (HE) institutions in the UK. We present a mobile robot system we have developed to support teaching, and we describe how the Internet is being used to provide and control access to the robot. A case study is presented illustrating the use of the mobile robot in student project work. The key goal of the work we are pursuing is the provision of robots as networked resources similar to other computing resources, in this way making them accessible to many more students than would otherwise be able to gain access to this advanced technology.     

Instructional Efficiency of Tutoring in an Outreach Gene Technology Laboratory

Our research objective focused on examining the instructional efficiency of tutoring as a form of instructional change as opposed to a non-tutoring approach in an outreach laboratory. We designed our laboratory based on cognitive load (CL) theory. Altogether, 269 twelfth-graders participated in our day-long module Genetic Fingerprinting. In a quasi-experimental design, the control group (n?=?121) followed the non-tutoring approach previously used, while the treatment group (n?=?148) followed the newly developed tutoring approach. Each tutor was in charge of two student work groups and recorded the tutoring activities requested by the tutees throughout the day. We measured the students’ invested mental effort (as an index of CL), cognitive achievement (in a pre-post-follow-up design), and the students’ cooperation in their work groups as well as calculated the student instructional involvement (as a motivational variable). Additionally, we examined which aspects of the hands-on phases were of particular relevance to the students’ invested mental effort. Unexpectedly, the combined mental effort and cognitive achievement data indicated that our implemented tutoring approach resulted in a lower instructional efficiency despite the relevance of tutoring for students’ mental effort invested during the experimental phases. Most of the tutor assistance was unnecessarily requested for performing the procedural steps and using the equipment. Our results indicate an assistance dilemma and consequently underscore the necessity for effective tutor preparation in outreach laboratories.     

University physics students’ ideas of thermal radiation expressed in open laboratory activities using infrared cameras

Background: University physics students were engaged in open-ended thermodynamics laboratory activities with a focus on understanding a chosen phenomenon or the principle of laboratory apparatus, such as thermal radiation and a heat pump. Students had access to handheld infrared (IR) cameras for their investigations.

Purpose: The purpose of the research was to explore students’ interactions with reformed thermodynamics laboratory activities. It was guided by the research question: How do university physics students make use of IR cameras in open-ended investigation of thermal radiation as a phenomenon?

Sample: The study was conducted with a class of first-year university physics students in Sweden. The interaction with the activities of four of the students was selected for analysis. The four students are males.

Design and methods: We used a qualitative, interpretive approach to the study of students’ interaction. The primary means of data collection was video recording of students’ work with the laboratory activities and their subsequent presentations. The analysis focused on how IR cameras helped students notice phenomena relating to thermal radiation, with comparison to previous research on students’ conceptions of thermal radiation.

Results: When using the IR camera, students attended to the reflection of thermal radiation on shiny surfaces, such as polished metals, windows or a whiteboard and emissive properties of surfaces of different types. In this way, they went beyond using the technology as a temperature probe. Students were able to discuss merits and shortcomings of IR cameras in comparison with digital thermometers.

Conclusions: With the help of IR cameras, university physics students attend to thermal phenomena that would otherwise easily go unnoticed.     

Exploring the Impact of Assessment on Medical Students’ Learning

Abstract

What and how students learn depend largely on how they think they will be assessed. This study aimed to explore medical students’ perception of the value of assessment and feedback on their learning, and how this relates to their examination performance. A mixed methods research design was adopted in which a questionnaire was developed and administered to the students to gain their perceptions of assessments. Perceptions were further explored in focus group discussions. Survey findings were correlated with students’ performance data and academic coordinators’ perceptions. Students’ perceptions of the level of difficulty of different assessments mirrored their performance in examinations, with an improvement observed in clinical assessments as students progressed through their degree. Students recognised that feedback is important to allow improvements and seek more timely, better quality and personalised feedback. Academic coordinators identified that some of the students’ suggestions are more realistic than others. Students had a positive attitude towards assessment, but emphasised the need for educators to highlight the relevance of assessment to clinical practice.