To risk or not to risk it: student (non‐)engagement with seen examination questions

This case study explores students’ perceptions of seen examination questions about topics not covered by the formal curriculum of a final‐year economics module and of the associated group support sessions. Eight semi‐structured interviews with a total of 13 students were analysed. Contrary to expectations, learners taking a strategic approach to the module were not attracted by the seen questions. The uncertainty of an unfamiliar assessment format and the prospect of undertaking independent research and group work were perceived as involving more risks than the familiar unseen examination. Take‐up for the seen examination questions was low, and the students who did not answer a seen question tended to make workload considerations as well as concerns about group work for assessment purposes responsible for their decision. Despite not participating in the group support sessions, a few students researched the seen question in conjunction with trusted fellow students or on their own. The students who answered a seen question enjoyed the autonomy which the seen exam questions provided, while other students were critical of the way in which their autonomy might be externally controlled. The study provides insight into the impact of assessment on risk taking and students’ perception of risk associated with this type of assessment.     

NEWS AND INFORMATION

The purpose of the investigation was to determine whether and how the quality of students’ explanations of chemical phenomena was affected by changing the method of giving the question and answer, respectively, between the spoken and written formats. It also focused on these effects for different questions and different students. Two experiments used the same population of first‐year university students who attempted the same set of questions, variously presented and answered in each of the four different ways of combining these two methods of questioning and answering. From the results, it was concluded that, for these students and questions, there was no observable difference between the performance of the students using any of these combinations of formats.

     

Gender Differences in the Use and Effectiveness of Personal Response Devices

The use of personal response devices (or “clickers”) in the classroom has increased in recent years. While few quantitative studies on the effectiveness of clickers have been published, it is generally reported that clickers have been well-received by the students who use them. Two separate populations (Winter 2006 and Spring 2006) of engineering students were given clickers to use during a general chemistry class. Clicker use was compared to student grades for each course. During both terms, a higher percentage of female students than male students “actively participated” in the lectures, where active participation was defined as answering more than 75% of the clicker questions over the course of the term. Active male students earned final grades about 10 points higher than non-active male students. Active female students, however, scored only about 5 points higher than non-active female students. Student learning was assessed by comparing performance on exam questions and clicker questions with similar content. Students who answered clicker questions correctly were 11–13% more likely to answer the corresponding exam questions correctly than were students who did not answer the clicker question. In this paper, we demonstrate the effectiveness of clicker use in the classroom and examine gender differences associated with this use.     

Assessment of learning gains associated with independent exam analysis in introductory biology

This study evaluates the impact of an independent postmidterm question analysis exercise on the ability of students to answer subsequent exam questions on the same topics. It was conducted in three sections (～400 students/section) of introductory biology. Graded midterms were returned electronically, and each student was assigned a subset of questions answered incorrectly by more than 40% of the class to analyze as homework. The majority of questions were at Bloom's application/analysis level; this exercise therefore emphasized learning at these higher levels of cognition. Students in each section answered final exam questions matched by topic to all homework questions, providing a within-class control group for each question. The percentage of students who correctly answered the matched final exam question was significantly higher (p < 0.05) in the Topic Analysis versus Control Analysis group for seven of 19 questions. We identified two factors that influenced activity effectiveness: 1) similarity in topic emphasis of the midterm-final exam question pair and 2) quality of the completed analysis homework. Our data suggest that this easy-to-implement exercise will be useful in large-enrollment classes to help students develop self-regulated learning skills. Additional strategies to help introductory students gain a broader understanding of topic areas are discussed.     

How Individual Differences Interact With Task Demands in Text Processing

Reading is affected by both situational requirements and one’s cognitive skills. The current study investigated how individual differences interacted with task requirements to determine reading behavior and outcome. We recorded the eye movements of college students, who differed in reading efficiency, while they completed a multiple-choice (MC) comprehension test in two within-subject conditions: one in which they read passages and answered MC questions as in a typical reading test and one in which they wrote a summary before answering the MC questions. We found students spent longer time reading the text in the summary-writing condition, resulting in a benefit in the time they spent when answering MC questions. This time benefit was larger for students who had relatively low reading efficiency. These results demonstrated that both task requirements and individual differences can interact to affect reading behavior and performance. Implications for reading practice and assessment are discussed.     

To see or not to see? Comparing the effectiveness of examinations and end of module assessments in online distance learning

Research comparing continuous assessment (e.g. coursework) with examinations generally reveals a student preference for the former. The perceived increased use of continuous assessment periodically captures media attention, with claims of greater numbers of higher degree classifications being awarded. This paper takes a case-study approach to investigate the extent to which different types of assessment allow students to effectively demonstrate and apply their learning. By considering data gathered from second-level, undergraduate students completing 60 point, online psychology modules, this paper investigates assessment effectiveness in terms of student satisfaction, pass rate and level of pass rate. Findings reveal that modules with an end of module assessment (EMA), rather than an examination, have higher completion and pass rates. Whilst students who took a seen examination also performed well, those who completed an unseen examination recorded the lowest overall pass and completion rates, despite high ratings of student satisfaction. Findings are discussed in terms of implications for improved pedagogy and student experience.     

How do people read the passages during a reading comprehension test? The effect of reading purpose on text processing behavior

In this study, we investigate the effect of reading purpose on students’ processing behavior during a reading comprehension test. In a repeated measures design, sixty undergraduates answered multiple-choice (MC) reading comprehension questions in a condition with no overarching goal for reading and in an alternate condition where the same students were first provided with the goal of summarizing the text before answering MC questions. Results from eye tracking analysis showed that when students read and answered questions without an overarching goal, they spent much less time reading the passages before answering the questions, more time re-reading the texts while answering the questions, and more time on parts of the text that were not necessary to answer the questions. We conclude that providing examinees without an explicit goal for reading may inadvertently encourage a “search for the answer” reading process, rather than on building a coherent mental model of text content.     

The role of paraphrased adjunct questions in facilitating learning from prose

Twenty-six college student read a 5000-word prose passage and answered either verbatim or paraphrased inserted questions while reading about international communication. Students either were or were not permitted to review the text while answering the inserted questions. On a subsequent post-test containing new paraphrases of the adjunct questions, students who received paraphrased adjunct questions outperformed students who received verbatim adjunct questions. This result supported the contention that paraphrased adjunct questions could facilitate meaningful learning from prose.     

Group in‐course assessment promotes cooperative learning and increases performance

The authors describe and evaluate a method to motivate medical students to maximize the effectiveness of dissection opportunities by using In‐Course‐Assessments (ICAs) to encourage teamwork. A student's final mark was derived by combining the group dissection mark, group mark for questions, and their individual question mark. An analysis of the impact of the ICA was performed by comparing end of module practical summative marks in student cohorts who had, or had not, participated in the ICAs. Summative marks were compared by two‐way ANOVA followed by Dunnets test, or by repeated measures ANOVA, as appropriate. A cohort of medical students was selected that had experienced both practical classes without (year one) and with the new ICA structure (year two). Comparison of summative year one and year two marks illustrated an increased improvement in year two performance in this cohort. A significant increase was also noted when comparing this cohort with five preceding year two cohorts who had not experienced the ICAs (P <0.0001). To ensure that variation in the practical summative examination was not impacting on the data, a comparison was made between three cohorts who had performed the same summative examination. Results show that students who had undertook weekly ICAs showed significantly improved summative marks, compared with those who did not (P <0.0001). This approach to ICA promotes engagement with learning resources in an active, team‐based, cooperative learning environment. Anat Sci Educ 7: 224–233. © 2013 American Association of Anatomists.     

Gender Inequality in Biology Classes in China and Its Effects on Students' Short-Term Outcomes

This study investigated gender inequality in biology lessons and analysed the effects of the observed inequality on students' short-term knowledge achievement, situational interest and students' evaluation of teaching (SET). Twenty-two biology teachers and 803 7th-grade students from rural and urban classrooms in China participated in the study. Each teacher was videotaped for 1 lesson on the topic blood and circulatory system. Before and after the lessons, the students completed tests and questionnaires. Chi-square analysis was conducted to compare the boys' and girls' participation rates of answering teachers' questions in the lessons. The findings revealed that in the urban classrooms the boys had a significantly higher rate of participation than did the girls, and hence also a higher situational interest. However, no such gender inequity was found among the rural students. The study also revealed that urban students answered more complicated questions compared with the rural students in general. The findings of this study suggest that the teachers should try to balance boys' and girls' participation and involve more students in answering questions in their lessons. The study also raises questions about long-term effects of students' participation in answering teachers' questions on their outcomes-knowledge achievement, situational interest and SET.     

Impact of text availability and question format on reading comprehension processes

We conducted two experiments to analyze how text availability and question format affect readers’ processes and performance on measures of expository text reading comprehension. Junior high school students read expository texts and answered both multiple choice and open-ended questions on a computer that recorded reading times and readers’ actions with Read&Answer software. The results showed that readers reread prior text segments during initial reading of the text more often when they knew that the text would be unavailable when answering questions than when they knew that the text would be available. In addition, readers made more search decisions in the text- available condition when answering open-ended questions than when answering multiple-choice questions. Regarding performance, we repeatedly found an interaction effect between availability and question format: text availability benefited the open-ended but not the multiple-choice format. We concluded that the two availability conditions are useful in assessing different discourse processes. We discuss theoretical and practical implications for the development of models of reading and new ways to assess reading literacy skills that emphasize purposeful reading.     

Does formative feedback on search behaviour help students in answering comprehension questions from an available text? / ¿Ayuda la retroalimentación formativa sobre el comportamiento de búsqueda a contestar preguntas de comprensión en lecturas con texto disponible?

This study analyses the efficacy of formative feedback to boost students’ search behaviour when answering comprehension questions in a with-text reading situation, which is a common reading situation in instructional and assessment settings. In these reading situations search strategies play an important role to predict students’ performance. Sixty-five high school students read two texts and answered eight multiple-choice comprehension questions per text using the software Read&Answer, which recorded all the students’ actions. After answering each question, students received either global-search-feedback or specific-search-feedback, which differed in the specificity of their information, or no-feedback. Participants who received any feedback had a second chance to correct their wrong answers. Specific-search-feedback increased students’ search decisions and improved their use of relevant information to repair wrong answers over global-search-feedback. Consequently, specific-search-feedback improved students’ performance when they corrected wrong answers over global-search-feedback. These results have implications for the design and implementation of formative feedback in computer-based systems aimed at improving students’ performance and teaching reading literacy skills.     

Using student-generated questions for student-centred assessment

In small groups, medical students were involved in generating questions to contribute to an online item bank. This study sought to support collaborative question‐writing and enhance students’ metacognitive abilities, in particular, their ability to self‐regulate learning and moderate understanding of subject material. The study focused on supporting students to write questions requiring higher order cognitive processes. End‐of‐year formal examinations comprised 25% student‐generated questions (SGQs), while mid‐year examination items were completely unseen. Data were gathered from repeated administration of a questionnaire and from examination results. No statistically significant changes were identified in self‐rated monitoring of understanding and regulation of learning. The activity of generating questions supported students to work collaboratively in developing questions and answers. The bank of questions was appreciated by students as a source of revision material, even though it was not strongly focused on higher order processes. Based on scores, it would appear that many students chose to memorise the question bank as a ‘high‐yield’ strategy for mark inflation, paradoxically favouring surface rather than deep learning. The study has not identified directly improvements in metacognitive capacity and this is an area for further investigation. Continual refinement of the study method will be undertaken, with an emphasis on education of students in developing questions addressing higher order cognitive processes. Although students may have memorised the questions and answers, there is no evidence that they do not understand the information.     

Can pictorial attributes serve mathemagenic functions?

Using mathemagenic-based research, the authors attempted to determine if the location or type of a picture affects reading comprehension. College students read a passage and then answered multiple-choice questions that assessed different types of incidental learning. Generally, subjects who viewed a picture after reading the passage scored higher than those who viewed before reading.     

Early narrative skills in Chilean preschool: Questions scaffold the production of coherent narratives

This study examined whether or not question answering aided the construction of coherent narratives in pre-readers. Sixty Chilean preschoolers completed two tasks using a wordless picture-book: 30 children answered questions about the story and then produced a narrative using the book; 30 children completed the tasks in reverse order. Elements of coherence were assessed in both tasks, namely problem, resolution, and mental-states. The findings indicate that questions can scaffold the production of more coherent narratives. Narratives elicited after questions were judged to be more coherent than those produced before the question–answering task. In contrast, there were no differences between scores for the question answers in the different order conditions. The results are discussed regarding the interactional role of questions and the facilitative effect they have on focusing attention to the narrative task.     

Clay modeling versus written modules as effective interventions in understanding human anatomy

The effectiveness of clay modeling to written modules is examined to determine the degree of improvement in learning and retention of anatomical 3D relationships among students with different learning preferences. Thirty‐nine undergraduate students enrolled in a cadaver dissection course completed a pre‐assessment examination and the VARK questionnaire, classifying learning preference as visual, auditory, read/write, or kinesthetic. Students were divided into clay, module, and control groups with preference for learning style distributed among groups. The clay and module groups participated in weekly one‐hour classes using either clay models or answering written questions (modules) about anatomical relationships, respectively. The control group received no intervention. Post‐assessment and retention examinations were administered at the end of the semester, and three months later, respectively. Two variables (Δ1, Δ2) represented examination score differences between pre‐ and post‐assessment and between post‐assessment and retention examinations, respectively. The Δ1 for clay and module groups were each significantly higher than controls (21.46 ± 8.2 vs. 15.70 ± 7.5, P ≤ 0.05; and 21.31 ± 6.9 vs. 15.70 ± 7.5, P ≤0.05, respectively). The Δ2 for clay and module groups approached but did not achieve significance over controls (?6.09 ± 5.07 vs. ?8.80 ± 4.60, P = 0.16 and ?5.73 ± 4.47 vs. ?8.80 ± 4.60, P = 0.12, respectively). No significant differences were seen between interventions or learning preferences in any group. However, students of some learning styles tended to perform better when engaging in certain modalities. Multiple teaching modalities may accommodate learning preferences and improve understanding of anatomy. Anat Sci Educ. © 2013 American Association of Anatomists.     

Coursework Marks High,Examination Marks Low: Discuss

It is commonly believed that the standard of student performance in coursework tends to be higher than that achieved in formal examinations. This view was tested by analysing undergraduate performances in six subjects at four UK universities. Two measures of relative coursework performance were employed. The first is the difference between the mean coursework and examination marks for each module. The second considers the proportion of students in each module who achieve a higher mark in the coursework than in the examination. The measures showed that in English and History coursework performances are slightly higher, equivalent to one-third of one honours class (or division) while, in Biology, Business Studies, Computer Studies and Law, coursework performances are higher by as much as two-thirds of one honours class (or division). The differences observed in the latter subjects are very significant and have serious implications for parity of treatment in degree programmes where students may choose modules with contrasting modes of assessment.     

The relative effects of experimenter and subject generated questions on learning from museum case exhibits

Fifty-five undergraduate education majors were randomly assigned to one of three treatment conditions as they entered the “walk-through” cave exhibit at The Florida State Museum. Subjects in treatment I received set induction materials directing them to enter the cave, to observe the cave carefully, and finally to read the information in five case exhibits after exiting the cave. Subjects in treatment II observed the cave exhibit, read the information in the five case exhibits, and answered two experimenter generated questions about each case exhibit immediately after reading the information in it. Subjects in treatment III observed the cave exhibit, read the information in the five case exhibits, and generated two questions about each case exhibit and answered them immediately after reading the information in each exhibit. All subjects received an immediate 20-item posttest. Seven of these items focused on observations from the “walk-through” portion of the cave exhibit. The remaining 13 items covered the information in the case exhibits. Analysis of variance indicated no significant differences between groups on the observation questions; however, a significant difference was detected for the case exhibit questions (p < 0.02). Follow up analyses indicated that subjects who answered the experimenter generated questions performed significantly better than subjects who did not answer any questions. Although no other differences were detected at the 0.05 level, the trend of results indicated that subjects who answered experimenter generated questions performed better than subjects who generated their own questions. These latter subjects performed better than subjects who read the case exhibits without answering any questions.     

Exploring the effect of background knowledge and text cohesion on learning from texts in computer science

In this study, we examine the effect of background knowledge and local cohesion on learning from texts. The study is based on construction–integration model. Participants were 176 undergraduate students who read a Computer Science text. Half of the participants read a text of maximum local cohesion and the other a text of minimum local cohesion. Afterwards, they answered open-ended and multiple-choice versions of text-based, bridging-inference and elaborative-inference questions. The results showed that students with high background knowledge, reading the low-cohesion text, performed better in bridging-inference and in elaborative-inference questions, than those who read the high-cohesion text. Students with low background knowledge, reading the high-cohesion text, performed better in all types of questions than students reading the low-cohesion text only in elaborative-inference questions. The performance with open-ended and multiple-choice questions was similar, indicating that this type of question is more difficult to answer, regardless of the question format.     

Medical student retention of embryonic development: Impact of the dimensions added by multimedia tutorials

The purpose of this project was to develop Web‐based learning modules that combine (1) animated 3D graphics; (2) 3D models that a student can manipulate independently; (3) passage of time in embryonic development; and (4) animated 2D graphics, including 2D cross‐sections that represent different “slices” of the embryo, and animate in parallel. These elements were presented in two tutorials, one depicting embryonic folding and the other showing development of the nervous system after neural tube formation. The goal was to enhance the traditional teaching format—lecture combined with printed diagrams, text, and existing computer animations—with customized, guided, Web‐based learning modules that surpassed existing resources. To assess module effectiveness, we compared quiz performance of control groups who attended lecture and did not use a supporting module, with study groups who used a module in addition to attending lecture. We also assessed our students' long‐term retention of the material, comparing classes who had used the module with students from a previous year that had not seen the module. Our data analysis suggests that students who used a module performed better than those given only traditional resources if they used the module after they were already somewhat familiar with the material. The findings suggest that our modules—and possibly computer‐assisted‐instruction modules in general—are more useful if used toward the later stages of learning, rather than as an initial resource. Furthermore, our data suggest that the animation aids in long‐term retention. Both medical students at the University of Cincinnati and medical faculty from across the country commented favorably on their experiences with the embryonic development modules. Anat Sci Ed 1:252–257, 2008. © 2008 American Association of Anatomists.