Enhancing the effectiveness of concept inventories using textual analysis: investigations in an electrical engineering subject

ABSTRACT

Concept inventories (CIs) are assessment instruments designed to measure students’ conceptual understanding of fundamental concepts in particular fields. CIs utilise multiple-choice questions (MCQs), and specifically designed response selections, to help identify misconceptions. One shortcoming of this assessment instrument is that it fails to provide evidence of the causes of the misconceptions, or the nature of students’ conceptual understanding. In this article, we present the results of conducting textual analysis on students’ written explanations in order to provide better judgements into their conceptual understanding. We compared students’ MCQ scores in Signals and Systems Concept Inventory questions, with the textual analysis utilising vector analysis approaches. Our analysis of the textual data provided the ability to detect answers that students identified as a ‘guessed’ response. However, the analysis was unable to detect if conceptually correct ideas existed within the ‘guessed’ responses. The presented approach can be used as a framework to analyse assessment instruments that utilise textual, short-answer responses. This analysis framework is best suited for the restricted conditions imposed by the short-answer structure.     

Heuristic value of the notion of zone of proximal development in the study of child and adolescent construction of concepts in physics

The heuristic value of the concept ofzone of proximal development in the field of scientific instruction is demonstrated by means of a study on the construction of the concept offorce by students aged 14 and 15. Based on an analysis of the differences between the students’ intuitive concepts and those taught in school, a sequence of learning steps are proposed to allow students to achieve the “shifts” needed to receive the concept of force, defined as an interaction between systems. Acting on the zone of proximal development means taking advantage of what students already know in order to help them construct precursory concepts in preparation for new conceptual propositions. The original teaching sequence presented is based on experimental problem situations designed to promote modelling skills. Students are asked to make predictions about measurement variations. To perform these prediction tasks, they must construct new concepts and use specific representation techniques. The study enabled us to show that it is possible in ten sessions to lead students to construct a precursor for the formal concept of force.     

Effect of conceptual change approach accompanied with concept mapping on understanding of solution concepts

The purpose of this study was to investigate the effect of conceptual change texts accompanied with concept mapping instruction, compared to traditional instruction (TI), on 8th grade students understanding of solution concepts and their attitudes toward science as a school subject. Solution Concept Test was developed as a result of examination of related literature and interviews with teachers regarding their observations of students difficulties. The test was administered to a total of 64 eighth grade students from two classes of a general science course, taught by the same teacher. The experimental group received the conceptual change texts accompanied with concept mapping in a lecture by the teacher. This instruction explicitly dealt with students misconceptions. It was designed to suggest conditions in which misconceptions could be replaced by scientific conceptions and new conceptions could be integrated with existing conceptions. The control group received TI in which the teacher provided instruction through lecture and discussion methods. The results showed that conceptual change text accompanied with concept mapping instruction caused a significantly better acquisition of scientific conceptions related to solution concept and produced significantly higher positive attitudes toward science as a school subject than the TI. In addition, logical thinking ability and prior learning were strong predictors for the concept learning related to solution.     

The impact of concept mapping and visualization on the learning of secondary school chemistry students

The aim of this study was to examine whether the construction of integrated knowledge structures by students can be stimulated by concept mapping and by better visualization of concepts and their interrelationships. The investigation was carried out in regular teaching settings: chemistry courses in secondary schools in Flanders, in the domain of electrochemistry. A significant positive effect of extra attention to visualization on the learning achievement of students was found. However, significant effects of concept mapping as an instruction method could not be detected under the given research conditions.     

Secondary Students’ Stable and Unstable Optics Conceptions Using Contextualized Questions

This study focuses on elucidating and explaining reasons for the stability of and interrelationships between students’ conceptions about Light Propagation and Visibility of Objects using contextualized questions across 3 years of secondary schooling from Years 7 to 9. In a large-scale quantitative study involving 1,233 Korean students and 1,149 Singaporean students, data were analyzed from responses to the Light Propagation Diagnostic Instrument consisting of four pairs of items, each of which evaluated the same concept in two different problem situations. Findings show that only about 10–45 % of students could apply their conceptions of basic optics in contextualized problem situations giving rise to both stable and unstable alternative conceptions. Students’ understanding of Light Propagation concepts compared with Visibility of Objects concepts was more stable in different problem situations. The concepts of Light Propagation and Visibility of Objects were only moderately correlated. School grade was not a strong predictive variable, but students’ school achievement correlated strongly with their conceptual understanding in optics. The teaching and learning approach and education systems in the two countries may have had some influence on students’ conceptual understanding.     

Comparing Classroom Enactments of an Inquiry Curriculum: Lessons Learned From Two Teachers

Abstract

Examining how teachers structure the activities in a unit and how they facilitate classroom discussion is important to understand how innovative technology-rich curricula work in the context of classroom instruction. This study compared 2 enactments of an inquiry curriculum, then examined students' learning outcomes in classes taught by 2 teachers. The quantitative data show that there were significant differences in the learning outcomes of students in classes of the 2 teachers. This study then examined classroom enactments by the 2 teachers to understand the differences in the learning outcomes. This research specifically focused on how teacher-led discussions (a) helped connect the activities within a curriculum unit and (b) enabled deeper conceptual understanding by helping students make connections between science concepts and principles. This study examined the role that teacher facilitation played in helping students focus on the relations between the various activities in the unit and the concepts that they were learning. The results point to important differences in the 2 enactments, helping to understand better what strategies might enable a deeper conceptual understanding of the science content.     

Teaching Nonverbal Communication

Abstract

Lectures remain the primary method of instruction in higher education despite several limitations: Students typically lose interest during hour-long lectures, lectures lead to rote learning by some students, and lectures do not lead to development of higher-level conceptual thinking skills. As an alternative to a lecture on the topic, an active learning exercise was developed to teach students about nonverbal communication. The exercise was used with several classes, and students reported it was motivating, educational, and thought-provoking. Advantages of implementing active learning exercises in lecture courses are discussed.     

Systems learning with a conceptual representation: a quasi-experimental study

In this paper, we share results from a classroom intervention that used a conceptual representation to support reasoning about ecosystems. Engaging students in modeling allows them to make their ideas visible while being malleable and available for discussion, which enables students to make meaning out of systems. Further, the Components-Mechanisms-Phenomena (CMP) conceptual representation was designed to enable students to construct coherent mental models. Following our intervention, students deepened their understanding of ecosystem dynamics when compared to students who engaged in traditional instruction without use of the CMP conceptual representation. We discuss our results in terms of data that helped guide the design of the intervention and we describe a theoretical perspective that can be used to guide future instruction.     

The Contribution of Conceptual Change Texts Accompanied by Concept Mapping to Eleventh-Grade Students Understanding of Cellular Respiration Concepts

The present study conducted to investigate the contribution of conceptual change texts, accompanied by concept mapping instruction to eleventh-grade students’ understanding of cellular respiration concepts, and their retention of this understanding. Cellular respiration concepts test was developed as a result of examination of related literature and interviews with teachers regarding their observations of students’ difficulties. The test was administrated as pre-test, post-test, and delayed post-test to a total of 70 eleventh-grade students in two classes of the same high school in an urban area, taught by the same teacher. The experimental group was a class of 34 students who received conceptual change texts accompanied by concept mapping instruction. A class of 36 students comprised the control group who received traditional instruction. Besides treatment, previous understanding and logical thinking ability were other independent variables involved in this study. The results showed that logical thinking, treatment, previous understanding of cellular respiration concepts each made a statistically significant contribution to the variation in students’ understanding of cellular respiration concepts. The result also showed that conceptual change texts accompanied by concept mapping instruction was significantly better than traditional instruction in retention of this understanding.     

Concepts before coding: non-programming interactives to advance learning of introductory programming concepts in middle school

ABSTRACT

Background and Context: Learners struggle with conceptual understanding of introductory programming concepts such as variables, expressions, and loops.

Objective: We examine whether and how designed activities for conceptual exploration support preliminary engagement with and learning of foundational and often hard-to-grasp programming concepts for students in grades 6–8.

Method: Drawing on principles from dynamic mathematics, we developed a suite of non-programming digital and unplugged activities embedded in a curriculum before students engage in Scratch block-based programming. We conducted empirical research in three middle school classrooms in diverse urban US schools and examined student performance through mixed qualitative and quantitative methods.

Findings: Learning gains were significant and not predicted by grade, gender or prior academic preparation. Free-choice projects of students showed statistically greater (correct) use of key concepts compared to those not in the study.

Implications: Our work demonstrates the promise of novel approaches such as interactive non-programming activities for deeper understanding of programming concepts.     

Using structured examples and prompting reflective questions to correct misconceptions about thermodynamic concepts

This paper explores the effectiveness of using ‘structured examples in concert with prompting reflective questions’ to address misconceptions held by mechanical engineering students about thermodynamic principles by employing pre-test and post-test design, a structured questionnaire, lecture room observation, and participants’ interviews. Students’ misconceptions were identified through pre-tests that evaluated students’ understanding of the chosen concepts, while conceptual change was assessed in pre-test–post-test design that revealed students’ ability to apply the concepts and transfer skills from a worked example to satisfactorily undertake a fairly complex similar problem. The use of worked examples in concert with prompting reflective questions is effective for inducing correct conceptual change and effective problem-solving skills. However, it is recommended that engineering tutors should incorporate inquiry-based learning approach and computer simulations alongside the use of worked examples with prompting reflective questions in order to enhance students’ conceptual understanding of thermodynamic concepts.     

High school students' understanding of resistance in simple series electric circuits

In view of the lack of work on the understanding of resistance concept, we studied the understanding that high school students (from the eighth to the twelfth grades) were able to develop with regard to the interrelationships between this concept, and the potential difference and current concepts (Ohm's law). In addition, we explored the immediate effects of exposure to electricity courses on the intuitive mastery of these relationships. The participants were presented with information on potential difference and intensity of current and asked to predict the corresponding resistance values. In this current-potential difference context, the resistance concept was difficult to understand. For the majority of participants, resistance was a direct function of both current and potential difference, which is more reminiscent of the concept of power than of the concept of resistance. The systematic teaching of electricity concepts and Ohm's law had only limited, positive as well as negative, effects on the understanding of these relationships. The implications for education are discussed.     

Developing a technical vocabulary on teacher planning: Preservice teachers' concept maps

Recent literature has focused on a need to develop a technical vocabulary in the teaching profession. Concept mapping, a technique of graphically representing concepts and their hierarchical interrelationships along two dimensions, was used to examine the growth of student teachers' knowledge concerning planning over three sequenced courses. Quantitative and content analysis of the maps indicated that students developed a more shared, technical vocabulary reflecting course content and evidencing greater hierarchical organization. The study provides information about the nature of and growth of student teachers' thinking, and demonstrates a means of monitoring the growth of student teachers' conceptual development over a course.     

The effectiveness of propositional manipulation as a lecturing method in the statistics knowledge domain

The current experiment examined the potential effects of the method of propositional manipulation (MPM) as a lecturing method on motivation to learn and conceptual understanding of statistics. MPM aims to help students develop conceptual understanding by guiding them into self-explanation at two different stages: First, at the stage of propositions (statements referring to single statistical concepts and ideas), and subsequently, at the stage of more complex problems that comprise a set of relevant propositions. A total of 71 bachelor students in psychology who were preparing for the re-sit of their inferential statistics exam participated in one of two possible lectures. Topic, content, lecturer, and duration of both lectures were the same, and in both lectures five true/false hypotheses were presented. Students in the first lecture (control group) discussed interactively the truth or falsity of each hypothesis. In the second lecture (MPM group), this interactive discussion was structured by presenting a number of short open-ended questions along with each hypothesis. Conceptual understanding was measured by means of a twelve items multiple choice test. Further, the intrinsic motivation inventory was administered to examine motivation to learn. The results indicate that MPM does not lead to enhanced motivation to learn but can facilitate conceptual understanding development among students.     

Concept maps about chemical equilibrium and students' achievement scores

The purpose of this study was to examine relationships between structural characteristics of students' concept maps about chemical equilibrium and independent measures of their achievement in chemistry. Fifty students in 1991 and seventy students in 1992 completed a concept-mapping task using twenty-four specified concepts. Using similarities in concept map structure, based on the presence or absence of linked pairs of concepts, non-metric multidimensional scaling (MDS) was used to plot the location of the concept maps in coordinate space. The distribution of maps was based on differences in their structure, but also reflected levels of student achievement on independent tests. The relationship between the coordinate location of each student's maps and his or her test scores on independent chemistry achievement tests was sought by canonical correlation analysis of the 1991 data set. This revealed significant relationships between the MDS coordinates and test scores of recall of knowledge and its application. Multiple regression analysis of sixty-one students' maps from the 1992 data set against their percentile rank scores on a national chemistry quiz revealed significant relationships. The results are interpreted as revealing structural differences in conceptual organisation about chemical equilibrium among students with different levels of achievement and thus relative expertise in the domain. The significant relationship between map structure and cognitive process scores in chemistry also supports the view that the organisation of declarative knowledge influences its accessibility in cognitive tasks.     

Analysing Concept Maps as an Assessment Tool in Teaching Physics and Comparison with the Achievement Tests

Concept mapping is a technique that paves the way to represent knowledge schematically. In this research, concept mapping was used as an assessment method on the impulse–momentum topic. The purpose of this study was to determine teacher candidates’ knowledge about understanding of the concepts of impulse and momentum by comparing and contrasting two different methods; namely, students’ concept maps and an achievement test. The mean of teacher candidates’ concept map scores are extremely low when compared with the scores of the achievement test. In addition, it was seen that although a great number of concepts were written down, not many relationships were established between these concepts. There is a weak correlation between the achievement test and the concept map scores since concept maps assess the students’ knowledge from a conceptual perspective while the achievement tests measure the level of students’ knowledge on the topic and his/her ability to apply this knowledge on different occasions.     

E–Learning in Hong Kong: comparing learning outcomes in online multimedia and lecture versions of an introductory computing course

This paper evaluates the effectiveness of Web–based, highly interactive, and multimedia–rich e–learning materials by comparing students' learning outcomes in the lecture and online versions of an introductory computing course. The course versions differed only in that face–to–face lectures were replaced with e–learning modules in the online course; the other course elements (laboratory sessions, use of computer–mediated communications, examinations) were the same. The e–learning trial took place at the Hong Kong University of Science and Technology, where the first author taught the lecture course to 105 students, and the online course to 180 and 129 students in the following semesters. The lecture and online students achieved comparable factual learning outcomes and the online students outperformed the lecture students in applied–conceptual learning. Findings suggest that the use of carefully designed interactive e–learning modules fosters higher–order learning outcomes.     

Evaluation of Students’ Understanding of Thermal Concepts in Everyday Contexts

The aims of this study were to determine the underlying conceptual structure of the thermal concept evaluation (TCE) questionnaire, a pencil-and-paper instrument about everyday contexts of heat, temperature, and heat transfer, to investigate students’ conceptual understanding of thermal concepts in everyday contexts across several school years and to analyse the variables—school year, science subjects currently being studied, and science subjects previously studied in thermal energy—that influence students’ thermal conceptual understanding. The TCE, which was administered to 515 Korean students from years 10–12, was developed in Australia, using students’ alternative conceptions derived from the research literature. The conceptual structure comprised four groups—heat transfer and temperature changes, boiling, heat conductivity and equilibrium, and freezing and melting—using 19 of the 26 items in the original questionnaire. Depending on the year group, 25–55% of students experienced difficulties in applying scientific concepts in everyday contexts. Years of schooling, science subjects currently studied and physics topics previously studied correlated with development of students’ conceptual understanding, especially in topics relating to heat transfer, temperature scales, specific heat capacity, homeostasis, and thermodynamics. Although students did improve their conceptual understandings in later years of schooling, they still had difficulties in relating the scientific concepts to their experiences in everyday contexts. The study illustrates the utility of using a pencil-and-paper questionnaire to identify students’ understanding of thermal concepts in everyday situations and provides a baseline for Korean students’ achievement in terms of physics in everyday contexts, one of the objectives of the Korean national curriculum reforms.