Self-Reported Learning Gains: A Theory and Test of College Student Survey Response

Recent studies have asserted that self-reported learning gains (SRLG) are valid measures of learning, because gains in specific content areas vary across academic disciplines as theoretically predicted. In contrast, other studies find no relationship between actual and self-reported gains in learning, calling into question the validity of SRLG. I reconcile these two divergent sets of literature by proposing a theory of college student survey response that relies on the belief-sampling model of attitude formation. This theoretical approach demonstrates how students can easily construct answers to SRLG questions that will result in theoretically consistent differences in gains across academic majors, while at the same time lacking the cognitive ability to accurately report their actual learning gains. Four predictions from the theory are tested, using data from the 2006–2009 Wabash National Study. Contrary to previous research, I find little evidence as to the construct and criterion validity of SRLG questions.     

Analysis of assessment practice and subsequent performance of third year level students in natural sciences

Abstract

Summative assessment qualifies the achievement of a student in a particular field of specialization at a given time. Questions should include a range of cognitive levels from Bloom's taxonomy and be consistent with the learning outcomes of the module in question. Furthermore, a holistic approach to assessment, such as the application of the principles of the Herrmann Whole Brain Model, needs to be used to accommodate learning style diversity. The purpose of this study was to analyse, assess and compare the summative assessment of two third year level modules in the Bachelor of Science degree programme, namely Biochemistry and Zoology as part of action research with a view to enhancing the professional development of the lecturers involved. The questions posed in summative assessments were classified in terms of Bloom's differentiation of cognitive levels and the four different learning styles determined by Herrmann. Spearman's non-parametric analysis indicated that no correlation existed in this study between cognitive level and student performance based on achievement. In addition, there was not much difference between the cognitive levels and student performance between the two disciplines. Although the students seemed to do better at application level questions, the authors need to reflect on whether the assessments were valid with respect to the learning outcomes, methods of facilitating learning, and the assessments based on cognitive levels and learning style preferences. We conclude that continuous action research must be taken to improve the formulation of learning outcomes and students' achievement of these outcomes and quality of student learning – the main aim being the successful completion of the modules.     

The effects of instructional methods on students' learning outcomes requiring different cognitive abilities: context-aware ubiquitous learning versus traditional instruction

The purpose of this study was to explore the effects of the context-aware ubiquitous learning (u-learning) approach versus traditional instruction on students' ability to answer questions that required different cognitive skills, using the framework of Bloom's taxonomy of educational objectives, including knowledge, comprehension, application, analysis, and synthesis. In this study, 230 third- and fourth-grade students in 8 classes were counterbalanced and assigned to the u-learning approach and traditional instruction for learning different topics in two separate plant-observing activities. The results showed that the students who learned with traditional instruction performed better than those who used the u-learning approach in terms of answering questions that required their cognitive abilities of knowledge, comprehension, application, analysis, and synthesis. Moreover, an in-depth analysis of the students' learning behaviors in the u-learning context revealed that most of their learning behaviors recorded in the u-learning system were not significantly related to their cognitive abilities.     

Adopting an active learning approach to teaching in a research-intensive higher education context transformed staff teaching attitudes and behaviours

The conventional lecture has significant limitations in the higher education context, often leading to a passive learning experience for students. This paper reports a process of transforming teaching and learning with active learning strategies in a research-intensive educational context across a faculty of 45 academic staff and more than 1000 students. A phased approach was used, involving nine staff in a pilot phase during which a common vision and principles were developed. In short, our approach was to mandate a move away from didactic lectures to classes that involved students interacting with content, with each other and with instructors in order to attain domain-specific learning outcomes and generic skills. After refinement, an implementation phase commenced within all first-year subjects, involving 12 staff including three from the pilot group. The staff use of active learning methods in classes increased by sixfold and sevenfold in the pilot and implementation phases, respectively. An analysis of implementation phase exam questions indicated that staff increased their use of questions addressing higher order cognitive skills by 51%. Results of a staff survey indicated that this change in practice was caused by the involvement of staff in the active learning approach. Fifty-six percent of staff respondents indicated that they had maintained constructive alignment as they introduced active learning. After the pilot, only three out of nine staff agreed that they understood what makes for an effective active learning exercise. This rose to seven out of nine staff at the completion of the implementation phase. The development of a common approach with explicit vision and principles and the evaluation and refinement of active learning were effective elements of our transformational change management strategy. Future efforts will focus on ensuring that all staff have the time, skills and pedagogical understanding required to embed constructively aligned active learning within the approach.     

The relationship of deep and surface study approaches on factual and applied test‐bank multiple‐choice question performance

With the advent of online test banks and large introductory classes, instructors have often turned to textbook publisher‐generated multiple‐choice question (MCQ) exams in their courses. Multiple‐choice questions are often divided into categories of factual or applied, thereby implicating levels of cognitive processing. This investigation examined the influence student study approaches have on performance of shallow compared to deep, cognitive process outcomes measured by scores on factual versus application MCQ test bank questions respectively. Fifty‐six introduction to psychology students, aged 18–52 years, completed a modified version of the Revised Two‐Factor Study Process Questionnaire (R‐SPQ‐2F) to access deep or surface study approaches. Mid‐term and final exam MCQ were equally divided between factual and application questions from the textbook publisher test bank. Overall, students performed significantly better on factual MCQ, with older students achieving higher factual and applied scores. Results suggest younger students tend to use a surface approach to study, with reproduction of what was taught to meet minimum requirements. With age statistically controlled, analyses demonstrated that the surface learning approach negatively impacts MCQ performance on questions categorised as factual and applied more than a deep learning approach benefits MCQ performance.     

Improving teacher questioning in science using ICAP theory

Abstract

The purpose of this research was to investigate the efficacy of professional development in changing two middle school science teachers’ questioning to include more questions that require deeper student responses. The professional development was based on ICAP theory which proposes a framework for identifying cognitive engagement based on what is required of students. ICAP hypothesizes that Interactive and Constructive questions, such as those requiring inferential thought and collaboration, lead to deeper thinking and therefore stronger learning gains than questions that are more Active, Passive. Teacher questioning before and after the PD showed a marked improvement in increasing the number of questions that required students to make inferences about the materials. Student gain scores from tests on the lesson’s content suggests a positive relationship between constructive questions and student learning. This paper also details a coding scheme based on ICAP that can be used to provide feedback on teachers’ questioning.     

Will media influence learning? Reframing the debate

This article addresses the position taken by Clark (1983) that media do not influence learning under any conditions. The article reframes the questions raised by Clark to explore the conditions under which media will influence learning. Specifically, it posits the need to consider the capabilities of media, and the methods that employ them, as they interact with the cognitive and social processes by which knowledge is constructed. This approach is examined within the context of two major media-based projects, one which uses computers and the other, video. The article discusses the implications of this approach for media theory, research and practice.     

Cognitive load in percentage change problems: unitary,pictorial, and equation approaches to instruction

Eighth grade students in Australia (N = 60) participated in an experiment on learning how to solve percentage change problems in a regular classroom in three conditions: unitary, pictorial, and equation approaches. The procedure involved a pre-test, an acquisition phase, and a post-test. The main goal was to test the relative merits of the three approaches from a cognitive load perspective. Experimental results indicated superior performance of the equation approach over the unitary or pictorial approach especially for the complex tasks. The unitary approach required students not only to process the interaction between numerous elements within and across solution steps, but also to search for critical information, thus imposing high cognitive load. The pictorial approach did not provide a consistent approach to tackling various percentage change problems. Coupled with the need to coordinate multiple elements within and across solution steps, and the need to search for relevant information in the diagram, this approach imposed high cognitive load. By treating the prior knowledge of percentage quantity as a single unit, the equation approach required students to process two elements only. Empirical evidence and theoretical support favor the equation approach as an instructional method for learning how to solve percentage change problems for eighth graders.     

Alignment of Assessment Objectives with Instructional Objectives Using Revised Bloom's Taxonomy—The Case for Food Science and Technology Education

Abstract: Nine food science and technology (FST) subjects were assessed for alignment between the learning outcomes and assessment using revised Bloom's taxonomy (RBT) of cognitive knowledge. Conjoint analysis was used to estimate the utilities of the levels of cognitive, knowledge, and the attribute importance (cognitive process and knowledge dimension) for learning outcomes and assessments. Lecturers for these subjects produced learning outcomes for ability of students to Understand (4.935) Procedural (3.316) as well as Apply (4.491) Conceptual (3.083) knowledge. Lecturers’ expected students’ to move beyond mere recall and recognition of knowledge to higher order cognitive knowledge of apply, analyze, evaluate, and create. However, the assessments tested students’ ability to Understand (4.791) Conceptual (4.168) as well as Remember (3.217) Procedural (0.581) knowledge resulting in a misaligned teaching and learning exercise. For all the subjects, emphasis was more (52.9% to 72.9%) on the cognitive dimension than on forms of knowledge in formulating the learning outcomes, whereas emphasis placed on the cognitive (33.3% to 62.5%) dimension and the knowledge (37.5% to 66.7%) forms in the questions differed from subject to subject. The cognitive weight in the assessment was more for Understand (1.781)/Remember (0.787) Conceptual (1.416) knowledge. RBT provides an assessment framework that can be used to assist instructors in going beyond factual knowledge and comprehension to include academic skills such as application, analysis, evaluation, and creation.     

Learning by doing? Prospective elementary teachers' developing understandings of scientific inquiry and science teaching and learning

This study examined prospective elementary teachers' learning about scientific inquiry in the context of an innovative life science course. Research questions included: (1) What do prospective elementary teachers learn about scientific inquiry within the context of the course? and (2) In what ways do their experiences engaging in science investigations and teaching inquiry‐oriented science influence prospective elementary teachers' understanding of science and science learning and teaching? Eleven prospective elementary teachers participated in this qualitative, multi‐participant case study. Constant comparative analysis strategies attempted to build abstractions and explanations across participants around the constructs of the study. Findings suggest that engaging in scientific inquiry supported the development more appropriate understandings of science and scientific inquiry, and that prospective teachers became more accepting of approaches to teaching science that encourage children's questions about science phenomena. Implications include careful consideration of learning experiences crafted for prospective elementary teachers to support the development of robust subject matter knowledge.     

Reggio Emilia Inspired Learning Groups: Relationships,Communication, Cognition,and Play

A key aspect of the Reggio Emilia inspired curriculum is a learning group approach that fosters social and cognitive development. The purpose of this paper is to investigate how a Reggio Emilia inspired learning group approach works for children with and without disabilities. This study gives insight into how to form an appropriate learning group including the effective ratio between children with and without disabilities, the teacher expertise required and other supporting strategies for children with special needs to function more positively in a learning group approach. This study employed both qualitative and quantitative data analysis to code selected videotaped learning group sessions. Findings indicated that across the five children, four children developed in the areas of relationship, communication, and play skills. Children within the learning group exhibited increased interest, inclusion, friendship, and empathy toward both children with special needs. In addition, facilitating high quality learning groups requires intentional planning, reflection, and engagement from children and teachers.     

The effectiveness of two methods for teaching a constraint-seeking questioning strategy to students with learning disabilities

This study investigated the relative effectiveness of two methods of teaching an efficient questioning strategy to students with learning disabilities. The methods compared were (a) cognitive modeling alone in the use of constraint-seeking questions, and (b) cognitive modeling and verbalization in the use of constraint-seeking questions, involving explicit and consistent instruction and feedback. Sixty subjects with learning disabilities were assigned randomly to one of these two conditions (n = 30 in each group). A pretreatment and posttreatment design was employed, with the criterion variable being number of questions required to obtain the correct answers to two 20 questions games. Results indicated that both groups showed significant reductions from pretreatment to posttreatment in the number of questions required. The two methods of instruction did not differ significantly in effectiveness.     

Mathematics achievement and mathematics learning strategies: Cognitive competencies and construct differentiation

Responding to a survey requires cognitive abilities similar to those for any other linguistic task. We argue that, when faced with a cognitively challenging task, individuals respond by simplifying the task, declining it, or performing it poorly. Using the 2003 Program for International Student Assessment (PISA 2003) data sets, we document that the extent of construct differentiation in students’ mathematics learning strategies is strongly associated with their level of mathematics achievement. Likewise, the higher the level of mathematics achievement, the lower is the probability of a non-response to the mathematics learning strategy questions. Finally, the higher the students’ level of mathematics achievement, the more varied were their responses to the learning strategies items. These associations were examined across 40 participating countries.     

Neurodevelopmental Variation as a Framework for Thinking About the Twice Exceptional

Developmental exceptionalities span the range of learning abilities and encompass children with both learning disorders and learning gifts. The purpose of this article is to stimulate thinking about these exceptionalities, particularly the complexities and variations within and across people. Investigators tend to view learning disabilities or abilities, and gifts or high-end exceptionalities, as if they were necessarily and completely independent. This approach has led many in the field to look upon only limited aspects of the exceptional child, culminating in an inability to resolve the great variation and covariation that exists within and across children. Although there are a number of cognitive differences models that correctly advocate for an appreciation of profiles of strengths and weaknesses in the exceptional child, there remains a need for a neuroscientific approach that can help us better understand and accommodate the twice-exceptional individual—one with developmental disorders but also with high skills in the talent, creativity, or intellectual domains. We propose a model that will help us to fully appreciate that the brain that produces developmental learning abilities across the spectrum must be viewed as an integrated and multifaceted organ that is more than a simple reflection of its separate parts or domain-specific symptoms. We use developmental reading disability or dyslexia and the twice-exceptional individual as a means to illustrate how this model can aid in our thinking about these conditions.     

Teachers’ awareness of the semio-cognitive dimension of learning mathematics

While many semiotic and cognitive studies on learning mathematics have focused primarily on students, this study focuses mainly on teachers, by seeking to bring to light their awareness of the semiotic and cognitive aspects of learning mathematics. The aim is to highlight the degree of awareness that teachers show about: (1) the distinction between what the institution (school, university, society, etc.) proposes as a mathematical object (not in itself but as the content to be learned) and one of its semiotic representations; (2) the different aspects of a semiotic representation that the student able to handle the representation and the student who handles the representation with difficulty may focus on; (3) the semiotic conflicts generated by the contents of semiotic representations that are similar to each other in some respect. For this purpose, in this study, the semio-cognitive approach introduced by Raymond Duval was complemented with the semiotic-interpretative approach of the Peircean tradition. By embracing the pragmatist research paradigm, the methodology was based on the research questions, which guided the selection of the research methods within a qualitatively driven mixed methods design. The research results clearly show the need for a review of professional teacher training programs, as regards the role the semiotic handling plays in the cognitive construction of the mathematical objects and the learning assessment.     

Can creative podcasting promote deep learning? The use of podcasting for learning content in an undergraduate science unit

This paper examines the effect of a podcasting task on the examination performance of several hundred first‐year chemistry undergraduate students. Educational researchers have established that a deep approach to learning that promotes active understanding of meaning can lead to better student outcomes, higher grades and superior retention of knowledge over time. We attempted to promote such an approach by setting a task that involved student collaboration, contextualisation of content, and communication through new media, specifically creative podcasting. Examination results were used as a source of empirical evidence of changes in understanding and retention that occurred for students who completed this task. In comparing results across 2 years on similar questions related to the podcasting topics, we found a statistically significant improvement after introduction of the podcasting task on the questions related to one of the topics on which students had actively created their own podcasts (“acids & bases”), but not on the other (“oxidisation and reduction”). Improved learning outcomes in the form of better understanding and retention over time in at least one case suggest that under some circumstances creative podcasting may indeed help to promote a deep learning approach.     

Student teachers’ knowledge integration across conceptual borders: the role of study approaches,learning strategies,beliefs, and motivation

There is widespread agreement that student teachers need to construct an integrated knowledge base across multiple domains. This study examined the contributions of intraindividual factors of self-regulated learning to explaining student teachers’ (a) integration of knowledge across topics and domains (i.e., integrative learning) and (b) disjointed processing of potentially domain-specific learning content (i.e., separative learning). The factors considered were study approaches; cognitive, metacognitive, and resource-related learning strategy use; epistemological and pedagogical beliefs; and career choice motivation. The study applied a cross-sectional survey design and examined separative and integrative learning in N = 103 student teachers by way of multiple regression analyses with backward eliminations. A key finding is that deep and strategic study approaches and certain cognitive learning strategies contributed significantly to explaining integrative learning in student teachers. Epistemological and pedagogical beliefs were not able to predict integrative learning. Regarding separative learning, the study identified the surface study approach, specific epistemological and pedagogical beliefs, and the “usefulness” motive for career choice as positive predictors and critical thinking as a negative predictor. The study demonstrates differences in how integrative and separative learning are shaped by distinct intraindividual factors. Implications are discussed with regard to student teachers’ self-regulated learning and pre-service teacher education.

     

多媒体学习的认知体系

本文对主要的认知体系进行了概述,据此可以形成多媒体教学设计的理论基础.这些认知体系包括对记忆存储、记忆编码以及认知操作的描述.与多媒体学习相关的认知体系有Paivio的双重编码理论、Baddeley的工作记忆模型、Engelkamp的多模块理论、Sweller的认知负荷理论、Mayer的多媒体学习理论以及Nathan的动画理论.本文重点讨论传统研究和教学应用之间的交互作用,教学应用主要涉及如何增加回忆率、减少干扰、降低认知负荷以及加强理解等方面.得出的初步结论是:虽然各自有所侧重,不同的理论之间仍有一些共性;这些模型中缺乏对学习者多元编码整合作用的研究;在心理模拟足够的情况下并不需要动画教学;动作必须有意义才能有效果;多模块教学比单个不同的特定模块更加有效.     

Unravelling the influence of cognitive style on Chinese students’ classroom behaviours: the mediating effects of the structure-oriented/depth-oriented learning approach

This study was designed to investigate how cognitive style affects Chinese students’ learning behaviours in the classroom. A concept labelled as the structure-oriented vs. depth-oriented learning approach was constructed, and its mediating effects in the link between cognitive style and learning behaviour were proposed and examined in this study. Three measures assessing cognitive style, learning approach and learning behaviour in the classroom were administered to 332 undergraduates recruited from two large public universities in China. The results of data analyses revealed that Chinese students tended to be more holistic in cognitive style, and holistic style significantly predicted learning behaviour. It was also found that the structure-oriented learning approach significantly mediated the association between holistic style and learning behaviour. The findings were, in general, supportive of the major assumption of this study that cognitive style plays a role in explaining the behavioural traits presented by Chinese students in the classroom.     

Directed-inquiry approach to learning science process skills: treatment effects and aptitude-treatment interactions

The directed-inquiry approach to learning science process skills and scientific problem solving [DIAL(SPS)2] was developed to help high-school students gain the critical thinking skills required to solve problems in the biology lab. This curriculum integrated several learning strategies into a single approach: advance organizers, the learning cycle, concept maps, Vee diagrams, a focusing strategy, and writing. Two general questions were addressed. First, was the DIAL(SPS)2 treatment more effective than a conventional treatment? Second, was there evidence of an aptitude-treatment interaction? Four high-school biology classes taught by this investigator were used to test the DIAL(SPS)2 curriculum. Scheduling of students involved ability grouping. To test the curriculum in the most rigorous way, the experimental group consisted of average ability students and the comparison group consisted of above-average students. Both the groups were pretested in August and posttested in May. In the intervening time, the experimental group received the DIAL(SPS)2 treatment while the comparison group received a more traditional approach. Analysis of covariance revealed that the DIAL(SPS)2 curriculum had no significant effect on the learning of science process skills or on cognitive development. Aptitude-treatment interaction analyses revealed an interaction of DIAL(SPS)2 treatment and cognitive development.