Rapid Assessment of Learners’ Proficiency: A cognitive load approach

To tailor instruction to levels of learner proficiency in a domain in adaptive learning environments, rapid on‐line methods of assessing learner knowledge and skills are required. Cognitive studies indicate that major factors in expert performance are organised structures in the person’s knowledge base. Measuring different levels of acquisition of such knowledge structures should be a major purpose of cognitive diagnostic assessment. Traditional tests may not provide reliable evidence for diagnostic purposes and are not suitable for dynamic learning environments. This paper discusses an alternative diagnostic approach based on monitoring immediate traces of students’ knowledge structures in working memory. Results of an experimental study applying this method to assessment of user reading skills are described. The study demonstrated a sufficiently high level of correlation (.66) between obtained measures and traditional test scores, and a substantial increase in reliability, with testing time reduced by a factor of 3.8.     

Teaching Environmental Education through PBL: Evaluation of a Teaching Intervention Program

If our chosen aim in science education is to be inclusive and to improve students’ learning achievements, then we must identify teaching methodologies that are appropriate for teaching and learning specific knowledge. Karagiorgi and Symeo (2005) remind us that instructional designers are thus challenged to translate the philosophy of constructivism into current practice. Thus, research in science education must focus on evaluating intervention programs which ensure the effective construction of knowledge and development of competencies. The present study reports the elaboration, application and evaluation of a problem-based learning (PBL) program with the aim of examining its effectiveness with students learning Environmental Education. Prior research on both PBL and Environmental Education (EE) was conducted within the context of science education so as to elaborate and construct the intervention program. Findings from these studies indicated both the PBL methodology and EE as helpful for teachers and students. PBL methodology has been adopted in this study since it is logically incorporated in a constructivism philosophy application (Hendry et al. 1999) and it was expected that this approach would assist students towards achieving a specific set of competencies (Engel 1997). On the other hand, EE has evolved at a rapid pace within many countries in the new millennium (Hart 2007), unlike any other educational area. However, many authors still appear to believe that schools are failing to prepare students adequately in EE (Walsche 2008; Winter 2007). The following section describes the research that was conducted in both areas so as to devise the intervention program.     

Ask Systems: interrogative access to multiple ways of thinking

The purpose of this paper is to familiarize instructional designers and researchers with a useful design and research paradigm known as “Ask Systems.” Ask Systems are interrogative interfaces to information and learning environments that model conversations with a skilled, reflective practitioner (Sch?n, The reflective practitioner, 1983) or teacher. By selecting questions to answer, users access information or scaffolds that help them to solve a problem, perform a task, or comprehend domain knowledge. Ask Systems are potentially effective for designing interactive learning systems because question asking is one of the most fundamental skills in learning that affects comprehension and problem solving (Graesser et al., in: Britton and Graesser (eds.) Models of understanding text, 1996b; Graesser et al., Journal of Artificial Intelligence in Education, 3:235–241, 1992). By selecting questions to answer, students may also learn how to generate their own meaningful questions, because the quality of student-generated questions is strong predictor of domain knowledge and problem solving (Graesser and Olde, Journal of Educational Psychology, 95(3):524–536, 2003). Ask Systems are demonstrated that represent a variety of learning and task structures, including frequently asked questions, problem solving, metacognition, causal understanding, and multiple perspectives. Although empirical research on Ask Systems is very limited, they appear to represent a potentially effective paradigm for designing instruction and conducting research in a variety of learning environments.     

Constructing knowledge: An effective use of educational technology for teaching Islamic studies in the UK

The 21st century as a digital age is characterized by the increased accessibility of information and knowledge through the medium of sophisticated technological tools. The main aim of this article is to show how educational technology can be used effectively to help students construct knowledge when teaching Islamic studies in the UK. The first part of this paper summarizes the differences between information sharing and knowledge construction with reference to the essence of knowledge as Aristotelian ‘episteme’ (theoretical knowledge) and technê (practical knowledge: know-how); and the extent which the former process is created by the use of Information Technology (IT) while the latter is enhanced by Educational Technology (ET). The second part explores how ET can be used effectively to ‘provide training in critical and creative thinking skills of students’ as an integral part of producing useful tools and generating practical benefit during their learning process (Felder et al. 2000, p. 26; Callaos 2009, p. 3). The third, then, explains why a student centred and research-based teaching is preferred to traditional research-led method in order to support the construction of knowledge. The paper concludes by presenting some reflections and limitations on how effective use of ET and research-based teaching can help students to become critical thinkers while studying Islam and Middle Eastern politics as part of international politics curriculum in the UK.     

An Innovative Course Featuring Action Research Integrated with Unifying Science Themes

In this article, we describe an innovative capstone course for preservice K-8 teachers integrating action research and a unifying theme in science (AAAS in Science for all Americans. Oxford University Press, New York, 1989; NRC in National science education standards. National Academy Press, Washington, DC, 1996). The goals of the capstone course are to increase student knowledge of the unifying theme, improve written communication skills, and introduce students to educational research. We provide evidence that each of these goals is met. Student growth in theme knowledge is demonstrated through concept maps, questionnaires, and previously reported assessments. Improved writing ability is demonstrated using the spelling and grammar checking feature of Microsoft Word. The analysis of action research reports demonstrates that students are able to connect their action research project results to previous research.     

The CLIA-model: A framework for designing powerful learning environments for thinking and problem solving

A major challenge for education and educational research is to build on our present understanding of learning for designing environments for education that are conducive to fostering in students self-regulatory and cooperative learning skills, transferable knowledge, and a disposition toward competent thinking and problem solving. Taking into account inquiry-based knowledge on learning and recent instructional research, this article presents the CLIA-model (Competence, Learning, Intervention, Assessment) as a framework for the design of learning environments aimed to be powerful in eliciting in students learning processes that facilitate the acquisition of productive knowledge and competent learning and thinking skills. Next, two intervention studies are described that embody major components of this framework, one focussing on mathematical problem solving in primary school, and a second one relating to self-regulatory skills in university freshmen. Both studies were carried out in parallel with the development of the framework, and were instrumental in identifying and specifying the different components of the model. They yielded both promising initial support for the model by showing that CLIA-based learning environments are indeed powerful in facilitating in students the acquisition of high-literacy learning results, especially the acquisition and transfer of self-regulation skills for learning and problem solving.     

Curriculum-based measurement in the content areas: vocabulary matching as an indicator of progress in social studies learning

The purpose of this study was to examine the reliability and validity of curriculum-based measures as indicator of growth in content-area learning. Participants were 58 students in 2 seventh-grade social studies classes. CBM measures were student- and administrator-read vocabulary-matching probes. Criterion measures were performance on a knowledge test, the social studies subtest of the Iowa Test of Basic Skills (ITBS), and student grades. Both the student- and examiner-read measures reflected change in performance; however, only the student-read measure resulted in interindividual differences in growth rates. Significant relations were found between the growth rates generated by the student-read vocabulary measure and student course grades, ITBS scores, and growth on the knowledge test. These results support the validity of a vocabulary-matching measure as an indicator of student learning in the content areas. The results are discussed in terms of the use of CBM as a system for monitoring performance and evaluating interventions for students with learning disabilities in content-area classrooms.     

A Comparative Study of Problem-Based and Lecture-Based Learning in Junior Secondary School Science

The purpose of the study is to compare problem based learning (PBL) and lecture-based learning (LBL) in Hong Kong secondary students’ science achievement. Secondary One students were divided into two groups: group A (n = 37), was taught two topics: “Human Reproduction” and “Density” through PBL; group B (n = 38) was taught the same topics by LBL. Multiple choice questions and short structured response items were used to assess students’ academic performance. Pre and post tests were categorized into three domains: knowledge, comprehension and application according to Bloom’s Taxonomy (Bloom 1956). The results of this study suggest first that PBL is at least as effective as LBL in gaining the knowledge required to achieve the syllabus’ learning objectives; secondly, the PBL group shows a significant improvement in students’ comprehension and application of knowledge over an extended time. Seemingly, PBL is favored for knowledge retention compared to a more conventional teaching approach, by these early adolescent children in Hong Kong. An ongoing longitudinal study on students’ interactions will further determine whether students taught through PBL develop improved learning in relation to high order skills, in a local situation which still tends to focus on factual recall but where higher skills are being demanded by systemic reform.     

Making inquiry-based science learning visible: the influence of CVS and cognitive skills on content knowledge learning in guided inquiry

ABSTRACT

Many science curricula and standards emphasise that students should learn both scientific knowledge and the skills associated with the construction of this knowledge. One way to achieve this goal is to use inquiry-learning activities that embed the use of science process skills. We investigated the influence of scientific reasoning skills (i.e. conceptual and procedural knowledge of the control-of-variables strategy) on students’ conceptual learning gains in physics during an inquiry-learning activity. Eighth graders (n?=?189) answered research questions about variables that influence the force of electromagnets and the brightness of light bulbs by designing, running, and interpreting experiments. We measured knowledge of electricity and electromagnets, scientific reasoning skills, and cognitive skills (analogical reasoning and reading ability). Using structural equation modelling we found no direct effects of cognitive skills on students’ content knowledge learning gains; however, there were direct effects of scientific reasoning skills on content knowledge learning gains. Our results show that cognitive skills are not sufficient; students require specific scientific reasoning skills to learn science content from inquiry activities. Furthermore, our findings illustrate that what students learn during guided inquiry activities becomes visible when we examine both the skills used during inquiry learning and the process of knowledge construction. The implications of these findings for science teaching and research are discussed.     

The effect of the biology critical thinking project on the development of critical thinking

This article describes the Biology Critical Thinking (BCT) project in which carefully designed activities for developing specific critical thinking skills are incorporated into the biology curriculum. The objectives were to find out whether the BCT project contributes to the development of critical thinking skills in various biological and nonbiological topics and how it affects students' biological knowledge and classroom learning environment. The study consisted of 678 seventh graders who were assigned randomly into two groups that studied the same seventh-grade biology textbook. Only one group, the experimental, completed the BCT activities. The results indicate that the students in the experimental group improved their critical thinking skills compared to their own initial level and compared to their counterparts in the control group. Improved critical thinking skills were observed in a new biological context and nonbiological everyday topics, suggesting generalization of thinking skills across domains. The experimental students scored significantly higher than the control on a knowledge test, suggesting that “knowledge of facts” as one educational goal and “learning to think” as another, need not conflict, but rather can interact with each other. Finally, the results show that BCT involvement decreased the frequency of teacher-centered teaching and enhanced student-centered, more active learning.     

The impact of the pre-instructional cognitive profile on learning gain and final exam of physics courses: a case study

The case study described in this paper investigates the relationship among some pre-instructional knowledge, the learning gain and the final physics performance of computing engineering students in the introductory physics course. The results of the entrance engineering test (EET) have been used as a measurement of reading comprehension, logic and mathematics skills and basic physics knowledge of a sample of 47 Computing Engineering freshmen at the University of Palermo (Italy). These data give a significant picture of the initial knowledge status of a student choosing engineering studies. The students' physics learning gain has been calculated using a standardized tool in mechanics: the force concept inventory (FCI). The analysis shows that mathematical and physical background contribute to achieve a good final preparation in physics courses of engineering faculties; however the students' learning gain in physics is independent of students' initial level of mathematics skills and physics knowledge. Initial logic skills and reading comprehension abilities are not significant factors for the learning physics gain and the performance on physics courses.     

Levels of students’ “conception” of fractions

In this paper, we examine sixth grade students’ degree of conceptualization of fractions. A specially developed test aimed to measure students’ understanding of fractions along the three stages proposed by Sfard (1991) was administered to 321 sixth grade students. The Rasch model was applied to specify the reliability of the test across the sample and cluster analysis to locate groups by facility level. The analysis revealed six such levels. The characteristics of each level were specified according to Sfard’s framework and the results of the fraction test. Based on our findings, we draw implications for the learning and teaching of fractions and provide suggestions for future research.     

Using strategy instruction and confidence judgments to improve metacognitive monitoring

Current models of self-regulated learning emphasize the pervasive need for metacognitive monitoring skills at all phases of the learning process (Winne and Hadwin in Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 227–304). Mahwah, NJ: Erlbaum, 1998). In this investigation, we examined the impact of teaching 5th grade students how to self-monitor their comprehension and make confidence judgments. One treatment class (N = 21) engaged in process-oriented comprehension monitoring training while the other (N = 24) engaged in both comprehension monitoring training and response-oriented monitoring accuracy training. Findings revealed that students in both treatment classes improved their calibration accuracy and showed higher confidence on test performance than students in two comparison classes (N = 47, N = 26) after 2 weeks of instruction. However, students in the monitoring accuracy training class also showed significant gains in overconfidence in comparison to those in the other three classes. Implications for integrating comprehension-monitoring training at the elementary school level are discussed.

Reading characteristics of Chinese-English adolescents: knowledge and application of strategic reading

Integrating both quantitative and qualitative approaches, this study examined the knowledge and application of strategic reading among Taiwanese adolescents learning to read English as a foreign language. A total of 144 Grade 11 students participated in this study. Correlational analysis and t-tests were applied to analyzing quantitative data, including an English reading comprehension test and two sets of reading strategy questionnaires; the constant comparative method (Glaser and Strauss 1967) was used for analyzing qualitative data, comprising semi-structured interviews and think-aloud protocols. Results from quantitative analysis show a relatively strong link between the perceived use of first language (L1, Chinese) and second language (L2, English) strategies. Nonetheless, metacognitive and cognitive strategies were reportedly used more frequently for L1 reading, whereas support strategies were more often used for L2 reading. One salient theme that emerged from the qualitative analysis is that the reading of the English texts more closely resembled the reading of the texts written in the classic literary style of Chinese than the reading of the texts written in contemporary Chinese. These findings to some extent support the reading universals hypothesis (Goodman in Journal of Typographic Reseach, 103–110, 1970); yet, skills transfer is not uniformly automatic (Bell in TESOL Quarterly, 687–704, 1995; Urquhart and Weir 1998).     

Web‐delivered,problem‐based learning in organisational behaviour: a new form of CAOS

Recent calls have been made for a fundamental reorientation in higher education from a curriculum that simply transmits information (learner as receiver) to one that encourages students actively to construct their own knowledge and practice using new skills and concepts (learner as constructor). In response, the Case Analysis in Organisational Situations (CAOS) assessment tool was developed for use in teaching organisational behaviour. It follows the principles of problem‐based learning and group work, using an unfolding case design. A pre‐test (n=365), immediate post‐test (n=323) and delayed post‐test (n=129) design was implemented to evaluate the effectiveness of this new learning tool. The assessment of student approaches to learning revealed that students were more likely to adopt a deep approach to learning at the end of CAOS than at the beginning. In general, the results show that CAOS is a useful web‐based tool that promotes learning in organisational behaviour. The benefits of CAOS and some suggested improvements to the existing CAOS format are discussed.     

The Development of an Instrument to Measure the Project Competences of College Students in Online Project-Based Learning

This study sought to develop a self-report instrument to be used in the assessment of the project competences of college students engaged in online project-based learning. Three scales of the KIPSSE instrument developed for this study, namely, the knowledge integration, project skills, and self-efficacy scales, were based on related theories and the analysis results of three project advisor interviews. Those items of knowledge integration and project skill scales focused on the integration of different disciplines and technological skills separately. Two samples of data were collected from information technology-related courses taught with an online project-based learning strategy over different semesters at a college in southern Taiwan. The validity and reliability of the KIPSSE instrument were confirmed through item analysis and confirmatory factor analysis using structural equation modeling of two samples of students’ online response sets separately. The Cronbach’s alpha reliability coefficient for the entire instrument was 0.931; for each scale, the alpha ranged from 0.832 to 0.907. There was also a significant correlation (r = 0.55, p < 0.01) between the KIPSSE instrument results and the students’ product evaluation scores. The findings of this study confirmed the validity and reliability of the KIPSSE instrument. The confirmation process and related implications are also discussed.     

Individual differences in memory-monitoring accuracy

The reliability of memory-monitoring accuracy was examined in two studies. In Study 1, college students recalled answers to general knowledge questions and rated confidence in the answers. Odd-even reliability of monitoring accuracy was not significantly different from zero; students with at least 20 correct and 20 incorrect answers had higher reliability when the odd and even subsets of questions were matched on recall accuracy. The best reliability was observed when the distribution of confidence ratings was equated across odd and even question subsets, and students had at least 20 confidence judgments at each level. In Study 2, a 100-item mutiple-choice general knowledge test led to odd-even reliability for confidence-recognition accuracy that was fairly high, relative to previous research, but well below desired levels. For FOK-recognition, odd-even reliability was significantly different from zero, but low. These data indicate that current methods allow an individuals' monitoring accuracy to be assessed with low to moderate reliability. Suggestions for further research are offered.     

Effect of a Problem Based Simulation on the Conceptual Understanding of Undergraduate Science Education Students

A study of the effect of science teaching with a multimedia simulation on water quality, the “River of Life,” on the science conceptual understanding of students (N = 83) in an undergraduate science education (K-9) course is reported. Teaching reality-based meaningful science is strongly recommended by the National Science Education Standards (National Research Council, 1996). Water quality provides an information-rich context for relating classroom science to real-world situations impacting the environment, and will help to improve student understanding of science (Kumar, 2005a; Kumar and Chubin, 2000). The topics addressed were classes of organisms that form river ecosystem, dissolved oxygen, macroinvertebrates, composition of air, and graph reading skills. Paired t-test of pre- and post-tests, and pre- and delayed post-tests showed significant (p < 0.05) gains. The simulation had a significant effect on the conceptual understanding of students enrolled in a K-9 science education course for prospective teachers in the following areas: composition of air, macroinvertebrates, dissolved oxygen, classes of organisms that form a river ecosystem, and graph reading skills. The gain was more in the former four areas than the latter one. A paired t-test of pre- and delayed post-tests showed significant (p < 0.05) gains in the water quality and near transfer subsets than the dissolved oxygen subset. Additionally students were able to transfer knowledge acquired from the multimedia simulation on more than one concept into teachable stand-alone lesson plans.