What are Middle School Students Talking About During Clicker Questions? Characterizing Small-Group Conversations Mediated by Classroom Response Systems

There is a growing interest in using classroom response systems or clickers in science classrooms at both the university and K-12 levels. Typically, when instructors use this technology, students are asked to answer and discuss clicker questions with their peers. The existing literature on using clickers at the K-12 level has largely focused on the efficacy of clicker implementation, with few studies investigating collaboration and discourse among students. To expand on this work, we investigated the question: Does clicker use promote productive peer discussion among middle school science students? Specifically, we collected data from middle school students in a physical science course. Students were asked to answer a clicker question individually, discuss the question with their peers, answer the same question again, and then subsequently answer a new matched-pair question individually. We audio recorded the peer conversations to characterize the nature of the student discourse. To analyze these conversations, we used a grounded analysis approach and drew on literature about collaborative knowledge co-construction. The analysis of the conversations revealed that middle school students talked about science content and collaboratively discussed ideas. Furthermore, the majority of conversations, both ones that positively and negatively impacted student performance, contained evidence of collaborative knowledge co-construction.     

The benefits of using clickers in small-enrollment seminar-style biology courses

Although the use of clickers and peer discussion is becoming common in large-lecture undergraduate biology courses, their use is limited in small-enrollment seminar-style courses. To investigate whether facilitating peer discussion with clickers would add value to a small-enrollment seminar-style course, we evaluated their usefulness in an 11-student Embryology course at the University of Colorado, Boulder. Student performance data, observations of peer discussion, and interviews with students revealed that adding clickers to a small-enrollment course 1) increases the chance students will do the required reading before class, 2) helps the instructor engage all students in the class, and 3) gives students a focused opportunity to share thinking and to learn from their peers.     

A Campus-Wide Investigation of Clicker Implementation: The Status of Peer Discussion in STEM Classes

At the University of Maine, middle and high school teachers observed more than 250 university science, technology, engineering, and mathematics classes and collected information on the nature of instruction, including how clickers were being used. Comparisons of classes taught with (n = 80) and without (n = 184) clickers show that, while instructional behaviors differ, the use of clickers alone does not significantly impact the time instructors spend lecturing. One possible explanation stems from the observation of three distinct modes of clicker use: peer discussion, in which students had the opportunity to talk with one another during clicker questions; individual thinking, in which no peer discussion was observed; and alternative collaboration, in which students had time for discussion, but it was not paired with clicker questions. Investigation of these modes revealed differences in the range of behaviors, the amount of time instructors lecture, and how challenging the clicker questions were to answer. Because instructors can vary their instructional style from one clicker question to the next, we also explored differences in how individual instructors incorporated peer discussion during clicker questions. These findings provide new insights into the range of clicker implementation at a campus-wide level and how such findings can be used to inform targeted professional development for faculty.     

Does Displaying the Class Results Affect Student Discussion during Peer Instruction?

The use of personal response systems, or clickers, is increasingly common in college classrooms. Although clickers can increase student engagement and discussion, their benefits also can be overstated. A common practice is to ask the class a question, display the responses, allow the students to discuss the question, and then collect the responses a second time. In an introductory biology course, we asked whether showing students the class responses to a question biased their second response. Some sections of the course displayed a bar graph of the student responses and others served as a control group in which discussion occurred without seeing the most common answer chosen by the class. If students saw the bar graph, they were 30% more likely to switch from a less common to the most common response. This trend was more pronounced in true/false questions (38%) than multiple-choice questions (28%). These results suggest that observing the most common response can bias a student''s second vote on a question and may be misinterpreted as an increase in performance due to student discussion alone.     

Classroom Voting Questions to Stimulate Discussions in Precalculus

Classroom voting can be an effective way to stimulate student discussions. In this pedagogy, the instructor poses a multiple-choice question to the class, and then allows a few minutes for consideration and small-group discussion before students vote, either with clickers, cell phones, or a non-electronic method. After the vote the instructor guides a class-wide discussion. Here we report on a study of precalculus voting questions that includes data from 25 classes taught by eight instructors at five institutions over the course of 7 years. The goal of this study is to explore ways of identifying the questions most likely to provoke good student discussions. We recorded the percentage of each class voting for each option on each question posed, a total of 851 votes. We have 60 questions on which we recorded the results from at least five classes. We identified the five questions with the most widely dispersed votes, a method that has a history of being helpful in identifying good discussion questions. We present these five questions here, four of which we found to be examples of questions which are particularly good at stimulating student discussions. We include notes about how we used the questions in class.     

Do clickers work for students with poorer grades and in harder courses?

We studied the impact of clickers, also known as electronic student response systems, on the performance of students on two undergraduate finance courses. Consistent with some of the recent literature, we found that clickers have very little impact on student performance, as measured by final course grades. Further, we found that clickers do not have a significant impact on course grades for students in relation to their designated performance ability (weak versus strong) or whether the course in question is less or more difficult. However, after simultaneously controlling for course difficulty and student aptitude, we found that clickers have a meaningfully positive impact on the performance of poorly performing students on more challenging quantitative courses. Our results suggest that the impact of clickers on student performance may depend on the type of student (academically weak, average or strong) and the type of course (average or difficult). This finding has particular implications for curriculum planners at the post-secondary level, although the findings may also have application at the secondary school level.     

Assessing Multimedia Influences on Student Responses Using a Personal Response System

To date, research to date on personal response systems (clickers) has focused on external issues pertaining to the implementation of this technology or broadly measured student learning gains rather than investigating differences in the responses themselves. Multimedia learning makes use of both words and pictures, and research from cognitive psychology suggests that using both words and illustrations improves student learning. This study analyzed student response data from 561 students taking an introductory earth science course to determine whether including an illustration in a clicker question resulted in a higher percentage of correct responses than questions that did not include a corresponding illustration. Questions on topics pertaining to the solid earth were categorized as illustrated questions if they contained a picture, or graph and text-only if the question only contained text. For each type of question, we calculated the percentage of correct responses for each student and compared the results to student ACT-reading, math, and science scores. A within-groups, repeated measures analysis of covariance with instructor as the covariate yielded no significant differences between the percentage of correct responses to either the text-only or the illustrated questions. Similar non-significant differences were obtained when students were grouped into quartiles according to their ACT-reading, -math, and -science scores. These results suggest that the way in which a conceptest question is written does not affect student responses and supports the claim that conceptest questions are a valid formative assessment tool.     

Gender Differences in the Use and Effectiveness of Personal Response Devices

The use of personal response devices (or “clickers”) in the classroom has increased in recent years. While few quantitative studies on the effectiveness of clickers have been published, it is generally reported that clickers have been well-received by the students who use them. Two separate populations (Winter 2006 and Spring 2006) of engineering students were given clickers to use during a general chemistry class. Clicker use was compared to student grades for each course. During both terms, a higher percentage of female students than male students “actively participated” in the lectures, where active participation was defined as answering more than 75% of the clicker questions over the course of the term. Active male students earned final grades about 10 points higher than non-active male students. Active female students, however, scored only about 5 points higher than non-active female students. Student learning was assessed by comparing performance on exam questions and clicker questions with similar content. Students who answered clicker questions correctly were 11–13% more likely to answer the corresponding exam questions correctly than were students who did not answer the clicker question. In this paper, we demonstrate the effectiveness of clicker use in the classroom and examine gender differences associated with this use.     

Using clickers in nonmajors- and majors-level biology courses: student opinion, learning, and long-term retention of course material

Student response systems (clickers) are viewed positively by students and instructors in numerous studies. Evidence that clickers enhance student learning is more variable. After becoming comfortable with the technology during fall 2005-spring 2006, we compared student opinion and student achievement in two different courses taught with clickers in fall 2006. One course was an introductory biology class for nonmajors, and the other course was a 200 level genetics class for biology majors. Students in both courses had positive opinions of the clickers, although we observed some interesting differences between the two groups of students. Student performance was significantly higher on exam questions covering material taught with clickers, although the differences were more dramatic for the nonmajors biology course than the genetics course. We also compared retention of information 4 mo after the course ended, and we saw increased retention of material taught with clickers for the nonmajors course, but not for the genetics course. We discuss the implications of our results in light of differences in how the two courses were taught and differences between science majors and nonmajors.     

Online Discussions Improve Student Perceptions of Instructor Efforts to Relate Faith to Learning in Graduate Occupational Therapy Courses

This article presents the results of a retrospective controlled study conducted in a graduate occupational therapy program. The study examined the effect that an online discussion targeting integration of faith and learning had on student perceptions of instructor effectiveness in relating faith to learning. This study addresses the following question: Does the addition of a single online discussion targeting integration of faith and learning in graduate occupational therapy courses significantly increase student ratings on a course evaluation question addressing faith–learning integration? This study also asked two secondary questions:

1. Do online students and face-to-face students both respond favorably to the addition of an online discussion targeting faith–learning integration?

2. Do students in three different courses all respond favorably to the addition of an online discussion targeting the integration of faith and learning?

Data were collected from three different courses taught by the same professor between Fall 2009 and Fall 2013 (four sections of each course; N = 138). The ordinal data were analyzed using nonparametric tests to determine significant differences and effect sizes. The results indicated that the addition of a single online discussion addressing faith–learning integration can significantly increase student perceptions of instructor effectiveness in such integration within graduate occupational therapy coursework, both in face-to-face and online learning environments. These findings provide support for the use of online discussions to challenge students to integrate Christian faith beliefs with what they are learning in their area of study.     

Influence of study approaches and course design on academic success in the undergraduate anatomy laboratory

Many pre‐health professional programs require completion of an undergraduate anatomy course with a laboratory component, yet grades in these courses are often low. Many students perceive anatomy as a more challenging subject than other coursework, and the resulting anxiety surrounding this perception may be a significant contributor to poor performance. Well‐planned and deliberate guidance from instructors, as well as thoughtful course design, may be necessary to assist students in finding the best approach to studying for anatomy. This article assesses which study habits are associated with course success and whether course design influences study habits. Surveys (n = 1,274) were administered to students enrolled in three undergraduate human anatomy laboratory courses with varying levels of cooperative learning and structured guidance. The surveys collected information on potential predictors of performance, including student demographics, educational background, self‐assessment ability, and study methods (e.g., flashcards, textbooks, diagrams). Compared to low performers, high performers perceive studying in laboratory, asking the instructor questions, quizzing alone, and quizzing others as more effective for learning. Additionally, students co‐enrolled in a flipped, active lecture anatomy course achieve higher grades and find active learning activities (e.g., quizzing alone and in groups) more helpful for their learning in the laboratory. These results strengthen previous research suggesting that student performance is more greatly enhanced by an active classroom environment that practices successful study strategies rather than one that simply encourages students to employ such strategies inside and outside the classroom. Anat Sci Educ 11: 496–509. © 2018 American Association of Anatomists.     

Student-Generated Content: Enhancing learning through sharing multiple-choice questions

The relationship between students' use of PeerWise, an online tool that facilitates peer learning through student-generated content in the form of multiple-choice questions (MCQs), and achievement, as measured by their performance in the end-of-module examinations, was investigated in 5 large early-years science modules (in physics, chemistry and biology) across 3 research-intensive UK universities. A complex pattern was observed in terms of which type of activity (writing, answering or commenting on questions) was most beneficial for students; however, there was some evidence that students of lower intermediate ability may have gained particular benefit. In all modules, a modest but statistically significant positive correlation was found between students' PeerWise activity and their examination performance, after taking prior ability into account. This suggests that engaging with the production and discussion of student-generated content in the form of MCQs can support student learning in a way that is not critically dependent on course, institution, instructor or student.     

Promoting discussion in peer instruction: Discussion partner assignment and accountability scoring mechanisms

Peer instruction (PI) involves students answering questions and peer discussion learning activities. PI can enhance student performance and engagement in classroom instruction. However, some students do not engage in the discussions. This study proposes two mechanisms, discussion partner assignment and accountability scoring mechanisms, to form positive interdependence among students and individual accountability for each student to promote student discussions in PI. The evaluation results show that the student discussion rate with the promoting mechanisms is significantly higher than the rate without the mechanisms. The results from the questionnaire reveal that the mechanisms form positive interdependence and individual accountability to promote students to engage in the discussions.     

Effects of student‐generated questions as the source of online drill‐and‐practice activities on learning

This study investigated the effects of online drill‐and‐practice activities using student‐generated questions on academic performance and motivation as compared with online drill‐and‐practice using teacher‐generated questions and no drill‐and‐practice in a student question‐generation (SQG) learning context. A quasi‐experimental research method was adopted for the purposes of this study. Six fifth‐grade classes (n = 145) participated in a weekly online activity for 5 weeks. Analysis of covariance results showed significant differences among the different treatment groups with regard to both academic performance and motivation. Post hoc comparisons using simultaneous confidence intervals, however, did not demonstrate the use of online SQG combined with answering peer‐generated questions to be more conducive to learning with regard to any of the observed variables as compared with the use of online SQG combined with answering teacher‐generated questions. Furthermore, answering student‐generated questions in addition to online SQG did not lead to added gains in learning as compared with the online SQG‐alone group. Some reasons for the unexpected findings are proposed, and the significance of this study, as well as suggestions for instructional implementations and future research, are provided.     

Instructor perspectives of multiple-choice questions in summative assessment for novice programmers

Learning to program is known to be difficult for novices. High attrition and high failure rates in foundation-level programming courses undertaken at tertiary level in Computer Science programs, are commonly reported. A common approach to evaluating novice programming ability is through a combination of formative and summative assessments, with the latter typically represented by a final examination. Preparation of such assessment is driven by instructor perceptions of student learning of programming concepts. This in turn may yield instructor perspectives of summative assessment that do not necessarily correlate with student expectations or abilities. In this article, we present results of our study around instructor perspectives of summative assessment for novice programmers. Both quantitative and qualitative data have been obtained via survey responses from programming instructors with varying teaching experience, and from novice student responses to targeted examination questions. Our findings highlight that most of the instructors believed that summative assessment is, and is meant to be, a valid measure of a student's ability to program. Most instructors further believed that Multiple-choice Questions (MCQs) provide a means of testing a low level of understanding, and a few added qualitative comments to suggest that MCQs are easy questions, and others refused to use them at all. There was no agreement around the proposition that if a question was designed to test a low level of skill, or a low level in a hierarchy of a body of knowledge, that such a question should or would be found to be easy by the student. To aid our analysis of assessment questions, we introduced four measures: Syntax Knowledge; Semantic Knowledge; Problem Solving Skill and the Level of Difficulty of the Problem. We applied these measures to selected examination questions, and have identified gaps between the instructor perspectives of what is considered to be an easy question and also in what is required to be assessed to determine whether students have achieved the goals of their course.     

From Gatekeeping to Engagement: A Multicontextual, Mixed Method Study of Student Academic Engagement in Introductory STEM Courses

The lack of academic engagement in introductory science courses is considered by some to be a primary reason why students switch out of science majors. This study employed a sequential, explanatory mixed methods approach to provide a richer understanding of the relationship between student engagement and introductory science instruction. Quantitative survey data were drawn from 2,873 students within 73 introductory science, technology, engineering, and mathematics (STEM) courses across 15 colleges and universities, and qualitative data were collected from 41 student focus groups at eight of these institutions. The findings indicate that students tended to be more engaged in courses where the instructor consistently signaled an openness to student questions and recognizes her/his role in helping students succeed. Likewise, students who reported feeling comfortable asking questions in class, seeking out tutoring, attending supplemental instruction sessions, and collaborating with other students in the course were also more likely to be engaged. Instructional implications for improving students’ levels of academic engagement are discussed.     

Hofstra's New College: The Student View

The authors examined student reactions to an activity in computer science, psychology, women's studies, and education courses. The reciprocal interview involves the instructor gathering information about the students, followed by students collectively asking questions of the instructor. The interview aims to make students more comfortable in interactions with the instructor by soliciting student input. Student responses indicated they enjoyed the activity and recommend faculty use it in various courses. Participants also reported the interview process helped create a more comfortable classroom environment and helped clarify instructors' expectations of students.     

Using clickers to facilitate development of problem-solving skills

Classroom response systems, or clickers, have become pedagogical staples of the undergraduate science curriculum at many universities. In this study, the effectiveness of clickers in promoting problem-solving skills in a genetics class was investigated. Students were presented with problems requiring application of concepts covered in lecture and were polled for the correct answer. A histogram of class responses was displayed, and students were encouraged to discuss the problem, which enabled them to better understand the correct answer. Students were then presented with a similar problem and were again polled. My results indicate that those students who were initially unable to solve the problem were then able to figure out how to solve similar types of problems through a combination of trial and error and class discussion. This was reflected in student performance on exams, where there was a statistically significant positive correlation between grades and the percentage of clicker questions answered. Interestingly, there was no clear correlation between exam grades and the percentage of clicker questions answered correctly. These results suggest that students who attempt to solve problems in class are better equipped to solve problems on exams.     

Written justifications to multiple-choice concept questions during active learning in class

Increasingly, instructors of large, introductory STEM courses are having students actively engage during class by answering multiple-choice concept questions individually and in groups. This study investigates the use of a technology-based tool that allows students to answer such questions during class. The tool also allows the instructor to prompt students to provide written responses to justify the selection of the multiple-choice answer that they have chosen. We hypothesize that prompting students to explain and elaborate on their answer choices leads to greater focus and use of normative scientific reasoning processes, and will allow them to answer questions correctly more often. The study contains two parts. First, a crossover quasi-experimental design is employed to determine the influence of asking students to individually provide written explanations (treatment condition) of their answer choices to 39 concept questions as compared to students who do not. Second, we analyze a subset of the questions to see whether students identify the salient concepts and use appropriate reasoning in their explanations. Results show that soliciting written explanations can have a significant influence on answer choice and, when it does, that influence is usually positive. However, students are not always able to articulate the correct reason for their answer.     

Clickers in college classrooms: Fostering learning with questioning methods in large lecture classes

What can be done to promote student–instructor interaction in a large lecture class? One approach is to use a personal response system (or “clickers”) in which students press a button on a hand-held remote control device corresponding to their answer to a multiple choice question projected on a screen, then see the class distribution of answers on a screen, and discuss the thinking that leads to the correct answer. Students scored significantly higher on the course exams in a college-level educational psychology class when they used clickers to answer 2 to 4 questions per lecture (clicker group), as compared to an identical class with in-class questions presented without clickers (no-clicker group, d = 0.38) or with no in-class questions (control group, d = 0.40). The clicker treatment produced a gain of approximately 1/3 of a grade point over the no-clicker and control groups, which did not differ significantly from each other. Results are consistent with the generative theory of learning, which predicts students in the clicker group are more cognitively engaged during learning.