Web‐based learning environments: developing a framework for evaluation

With the widespread use of web‐based learning environments in the tertiary sector it is important to establish the usability of such environments for the target audience and their effectiveness in terms of meeting the educational objectives. However, a search of the literature has shown a scarcity of systematic evaluative studies of web‐based learning environments. Furthermore, the literature did not reveal a consistent starting position on appropriate methodologies with which to carry out such evaluations. This paper presents a general methodology for evaluating complex systems that is particularly appropriate for web‐based learning systems. Using what is called a trailing methodology (Finne et al., 1995 Finne, H, Levin, M and Nilssen, T. (1995). Trailing research: a model for useful program evaluation. Evaluation, 1(1): 11–31. [Crossref] [Google Scholar]), an evaluation was carried out of a web site that was used with student industrial experience projects. A key element in this evaluation was that the process was adaptive and collaborative; another was that it involved a team with expertise in evaluation, knowledge of the functional aspects of the web site and the educational purpose of the site. The evaluation process pointed to the importance of a flexible approach that utilizes the skills of the key stakeholders.     

Introducing a fifth pedagogy: experience‐based strategies for facilitating learning in natural environments

Educators have identified four categories of ‘productive pedagogies’ that are considered to lead to authentic student engagement and learning in the classroom. This study was designed to explore and extend these pedagogies in the context of learning in natural environments, in particular, through the programmes of Queensland environmental education centres. In‐depth interview and observation data were collected from students, classroom teachers and centre teachers who had participated in 12 environmental education programmes across Queensland, in order to identify the strategies that are most effective in facilitating learning in the natural environment. A fifth productive pedagogy category, ‘experience‐based learning’, is proposed. Experience‐based learning is particularly important in addressing students’ environmental attitudes and actions. The implications for the delivery of environmental education programmes both within and outside the classroom are discussed.     

Development of the human interaction dimension of the Self‐Regulated Learning Questionnaire in asynchronous online learning environments

Two studies focusing on the development and validation of the Online Self‐Regulated Learning Inventory (OSRLI) were conducted. The OSRLI is a self‐report instrument assessing the human interaction dimension of online self‐regulated learning. It consists of an affect/motivation scale and an interaction strategies scale. In Study 1, exploratory factor analysis of an initial affect/motivation item pool yielded four factors: enjoyment of human interaction, self‐efficacy for interaction with instructors, concern for interaction with students, and self‐efficacy for contributing to the online community. Exploratory factor analysis of an initial learning strategies item pool revealed three factors: writing strategies, responding strategies, and reflection strategies. In Study 2, confirmatory factor analysis was conducted in order to evaluate the stability of multidimensional factor structures. These exploratory and confirmatory factor analyses showed the OSRLI to be statistically moderate in terms of reliability and validity.     

Understanding community in synchronous online learning: do perceptions match behaviours?

ABSTRACT

The community of inquiry (CoI) framework is becoming increasingly popular in online learning. The relationships among CoI constructs (cognitive, teaching and social presences) were generally investigated through surveys. However, some methodological challenges were noticed in CoI studies such as the limitations in data diversity or data verification. Hypothesising that learners’ perceptions about CoI constructs may not reflect their actual behaviours, this study matches students’ perceptions and actual behaviours reflecting the CoI constructs in an online learning setting in a synchronous online undergraduate programme. Data from CoI surveys, online discussion records and observations were comparatively discussed. The results indicated that relationships between the behaviours and perceptions about CoI constructs were quite positive and students’ perceptions about CoI constructs almost matches with their actual behaviours. Thus, the study confirmed the power of determining the interactions reflecting CoI constructs in online learning within CoI surveys. It is hoped that the study may shed a light with its contribution in providing some evidence for the question of whether conventional data collection techniques in CoI studies is required to be revisited or not.     

Exploring high school students' and teachers' preferences toward the constructivist Internet‐based learning environments in Taiwan

This paper explores high school students' and teachers' preferences towards constructivist Internet‐based learning environments. The study proposes a framework, including two dimensions and five aspects, to illustrate the features of the Internet‐based learning environments. Based upon this framework, the Constructivist Internet‐based learning environment survey improvement (CILESI) was developed, which includes the scales of ease of use, multiple sources, student negotiation, reflective thinking, critical judgement and epistemological awareness. Questionnaire responses gathered from 630 high school students in Taiwan suggested that the CILESI showed adequate reliability in assessing students' preferences. Male students placed more emphasis on the student negotiation, critical judgement and epistemological awareness enhanced by the Internet‐based learning environments than female students did. In addition, the teachers of the sampled students (n?=?78) were also surveyed by CILESI. The teachers tended to express stronger preferences on the ease of use of the Internet‐based learning environments than did their students. However, students, when compared with their teachers, seemed to express more preferences towards the features of student negotiation, reflective thinking, critical judgement and epistemological awareness of Internet‐based learning environments.     

Learner presence,perception, and learning achievements in augmented–reality–mediated learning environments

Although research has indicated that augmented reality (AR)–facilitated instruction improves learning performance, further investigation of the usefulness of AR from a psychological perspective has been recommended. Researchers consider presence a major psychological effect when users are immersed in virtual reality environments. However, most research on learner presence has focused on measuring it rather than the factors influencing it. This study explored the relationship between learner perception and presence in an AR-facilitated learning environment. A game-type AR learning process was developed and an experiment was implemented employing a one-group pretest–posttest method. Three sets of research instruments were developed, comprising a learning achievement test, a learner's presence inventory, and a questionnaire on learners’ perception of the AR-mediated environment. A total of 60 validated data samples were collected and statistically analysed. The researchers determined that learner presence closely relates to learning achievement and AR learning perception. These results indicate that a more satisfactory AR perception leads to a higher level of learner presence in the AR environment, thereby resulting in more beneficial learning outcomes.     

Understanding project‐based learning in Second Life with a pedagogy,training, and assessment trio

This paper presents an empirical study of how Second Life (SL) was utilized for a highly successful project‐based graduate interdisciplinary communication course. Researchers found that an integrated threefold approach emphasizing project‐based pedagogy, technical training and support, and assessment/research was effective in cultivating and understanding learning in SL. Based on research findings, a project‐based application of SL that fully accommodates student experiential learning is recommended. It is suggested that teachers who are new To Whom It May Concern: SL and are preparing to “take the plunge” adopt a systematic team approach to integrate this trio of components. The students participating in this study formed an interdisciplinary team for their required SL project in which they chose to initiate a collaboration with the Basic Initiative and a group of architecture students to create a virtual presence for two green, sustainable, urban housing designs called the Alley Flats. Preliminary survey results suggest that the use of SL substantially enhanced the quality and experiences of student learning. Specific areas for improvement in future use are also identified.

Comprendre l’Apprentissage basé sur les projets dans le cadre de Second Life avec le trio Pédagogie‐Formation et Evaluation.

Le présent article présente une étude empirique sur la façon dont Second Life (SL) a été utilisé dans le cadre d’un cours de communication interdisciplinaire avancé fondé sur la réalisation de projets, cours qui a connu un grand succès. Les chercheurs ont trouvé qu’une approche intégrée en trois points, mettant l’accent sur la pédagogie des projets, la formation et le soutien techniques et l’évaluation/recherche se révélait efficace pour cultiver et comprendre l’apprentissage dans le cadre de SL. Les résultats de la recherche conduisent à recommander une application de Second Life fondée sur les projets, application qui prend pleinement en compte l’apprentissage des étudiants par l’expérience. On suggère que les professeurs qui sont nouveaux dans SL et qui se préparent à “faire le plongeon”, adoptent de façon systématique un mode d’approche en équipe, ceci pour intégrer ce trio de composantes. Les étudiants qui ont participé à cette étude formaient une équipe interdisciplinaire pour le projet SL qu’ils devaient réaliser et dans ce cadre, ils ont choisi de mettre en route une collaboration avec “The Basic Initiative” et un groupe d’étudiants d’architecture dans le but de créer une présence virtuelle pour concevoir deux projets d’habitat urbain vert et gérable appelés les Appartements de l’Allée. Une enquête préliminaire permet de penser que l’utilisation de SL a fortement amélioré le vécu et la qualité de l’apprentissage chez les étudiants. On a également identifié des zones spécifiques pour les améliorations à apporter pour des usages futurs.

Verstehen Projektbasierten Lernens in Second Life im Kontext von Pädagogik, Training und Beurteilung

Mit diesem Beitrag wird eine empirische Studie präsentiert, die berichtet wie Second Life (SL) sehr erfolgreich in einem projektorientierten und interdisziplinären Kommunikationskurs für Studenten genutzt worden ist. Unsere Untersuchung ergab, dass ein integrierter dreistufiger Ansatz – projektbasierte Pädagogik, technische Ausbildung und Unterstützung, sowie Beurteilung – zum besseren Verständnis über das Lernen mit Hilfe von SL führte. Auf Grundlage von Forschungsbergebnissen wird empfohlen, dass einer projektbasierten Ausarbeitung wird auf Erfahrung beruhendes Lernen zur Anwendung von SL empfohlen. Es wird geraten, dass Lehrer, die noch keine Erfahrung mit SL haben, einen systematischen Teamansatz übernehmen, um diese drei Bestandteile zu integrieren. Die Studenten, die an dieser Studie teilnahmen, bildeten ein interdisziplinäres Team, damit ihr angestrebtes SL Projekt, worin sie eine Kollaboration mit der Grundinitiative und einer Gruppe von Architekturstudenten initiieren wollten, eine virtuelle Gegenwart für zwei “the Alley Flats” genannte grüne, nachhaltige, städtische Unterbringungspläne schaffe. In vorläufigen Umfrageergebnissen wird festgestellt, dass die Verwendung von SL die Qualität der Arbeit und die Möglichkeit der Erfahrungen der Studenten zu lernen, beträchtlich verbesserte. Bestimmte Bereiche zur Verbesserung zukünftiger Verwendungen wurden ebenfalls identifiziert.

Comprender el aprendizaje basado en proyectos dentro de Second Life con el trio Pedagogía, Formación/Evaluación.

Este artículo presenta un estudio empírico sobre la manera en que Second Life fue utilizado para un curso de comunicación avanzado, interdisciplinario y altamente exitoso basado en proyectos. Los investigadores han descubierto que un abordaje integrado triple con énfsis en la pedagogía basada en proyectos, en la formación y el soporte técnico y en evaluación/investigación, resultaba efectivo para cultivar y comprender el aprendizaje en SL. Basandose en los resultados de la investigación, se recomienda una aplicación de Second Life basada en proyectos integrando completamente la experiencia de aprendizaje por parte de los estudiantes. Nuestra sugestión es que los profesores que son nuevos en SL y listos a zambullirse, adopten una metodología de trabajo en equipo para integrar este trio de componentes. Los estudiantes que participaron en este proyecto formaban equipos interdisciplinarios para sus proyectos SL y decidieron de iniciar una colaboración con “The Basic Initiative” y un grupo de estudiantes de arquitectura de modo a crear una presencia virtual para dos diseños de viviendas urbanas verdes y sostenibles llamadas Los Apartamentos de la Alameda. Los resultados de las encuestas preliminarias indican que el uso de SL realzó fuertemente la calidad y la vivencia del aprendizaje estudiantil. Se identificó también sectores específicos para futuras mejoras.     

Integrating assessment tasks in a problem‐based learning environment

The purpose of this study was to get more insight in the effects of written assessment tasks integrated in a problem‐based learning environment. Both the influence on students' performances and students' perceptions were investigated. Students' final exam results were used to find out whether students who make the assessment tasks do better than students who do not. Answers from questionnaires and semi‐structured interviews were used to discover the most important concerns in students' and teachers' perceptions of the assessment tasks. The results indicate that making the assessment tasks had positive influence on the students' overall performance. From the questionnaires and interviews it appears that both the students and the teachers see the benefits of the assessment tasks. It is concluded that small steps in the change of the assessment system can result in relatively big changes in students' learning and results.     

A multimedia‐enhanced problem‐based learning experience in the Malaysian classroom

In this paper a multimedia project was incorporated into a problem‐based learning environment to examine how learning in multimedia could be enhanced through the use of problem‐based learning. Students in a second year course in the Faculty of Creative Multimedia (FCM) were assigned the problem‐based multimedia project, they worked in groups, solving the problem using the multimedia authoring process (MAP). This paper reports on the structuring of the student learning process, the impact of the problem‐based learning environment on student learning and the important skills such as creative and critical thinking, teamwork, communication, collaborative and problem‐solving skills that arose as the result of this intervention. In the process, a multimedia‐enhanced problem‐based learning (M‐PBL) model was created. The interrelationships between the teacher, students and technology in this learning mode were also established. A survey was conducted to assess the reactions and attitudes of the students towards this learning environment. The results obtained were positive and satisfactory.     

Characterising programme‐level assessment environments that support learning

This article outlines a methodology for characterising features of programme‐level assessment environments so that the relationship between features of the assessment environment and students’ learning response can be studied. The methodology was developed through the detailed case study of nine undergraduate degree programmes: one in each of three contrasting discipline areas in each of three contrasting universities. Each case study involved examination of course documentation, interviews with academics and interviews with students, following which each degree programme was coded in relation to a range of features of the assessment environment, such as the proportion of marks derived from examinations and the volume and timeliness of feedback on assignments. Programmes were found to differ profoundly in terms of variables that are known to have implications for student‐learning processes. They also differed widely in the extent to which they illustrated the application of conventional wisdom about curriculum design, embodied in national quality assurance guidelines and the Bologna Agreement. Programmes were found to have either a high volume of summative assessment or a high volume of formative‐only assessment, but never both at the same time. Programmes also differed in the mechanisms used to make goals and standards clear, having either highly explicit curriculum design or high volumes of written and oral feedback, but never both at the same time. The findings suggest that there are distinctive programme‐level assessment environments that operate in quite different ways despite all programmes studied being subject to the same quality assurance code of practice.     

The role of visualization in learning from computer‐based images

Among the sciences, the practice of geology is especially visual. To assess the role of spatial ability in learning geology, we designed an experiment using: (1) web‐based versions of spatial visualization tests, (2) a geospatial test, and (3) multimedia instructional modules built around QuickTime Virtual Reality movies. Students in control and experimental sections were administered measures of spatial orientation and visualization, as well as a content‐based geospatial examination. All subjects improved significantly in their scores on spatial visualization and the geospatial examination. There was no change in their scores on spatial orientation. A three‐way analysis of variance, with the geospatial examination as the dependent variable, revealed significant main effects favoring the experimental group and a significant interaction between treatment and gender. These results demonstrate that spatial ability can be improved through instruction, that learning of geological content will improve as a result, and that differences in performance between the genders can be eliminated.     

Critical social theory and transformative learning: evidence in pre‐service teachers’ service‐learning reflection logs

This paper reports on the opportunities for transformational learning experienced by a group of pre‐service teachers who were engaged in service‐learning as a pedagogical process with a focus on reflection. Critical social theory informed the design of the reflection process as it enabled a move away from knowledge transmission toward knowledge transformation. The structured reflection log was designed to illustrate the critical social theory expectations of quality learning that teach students to think critically: ideology critique and utopian critique. Butin’s lenses and a reflection framework informed by the work of Bain, Ballantyne, Mills and Lester were used in the design of the service‐learning reflection log. Reported data provide evidence of transformational learning and highlight how the students critique their world and imagine how they could contribute to a better world in their work as a beginning teacher.     

E‐learning and the transformation of social interaction in higher education

This article examines the way in which e‐learning is transforming the nature of social interaction in higher education. In this new educational environment, radical societal transitions and the opportunities afforded by modern communication technologies together produce formidable challenges. Significant as these challenges may be, concentration upon problems of a practical kind draws attention away from the more theoretical concerns in understanding e‐learning. By drawing together developments in social, educational and communicational theory and Gilly Salmon’s hands‐on approach to teaching and learning online, this article reveals some unintended consequences: e‐moderation and the use of e‐tivities may perpetuate the very conditions that limit our chances of dealing successfully with the challenges posed by e‐learning. While theory may muddle what might otherwise be communicated meaningfully to those in search of practical answers, theoretical developments provide concepts and frameworks that can be placed in the service of a critical understanding of e‐learning and the transformation of social interaction in higher education.     

Adult learners’ emotions in online learning

The aim of the research study reported in this article was to investigate how adult learners talk about their emotions in the context of a year‐long online course, the first online course these adults take, as part of a distance education program. The theoretical and methodological approach focused on formulating an account of how emotion discourses are used by learners, what role they play in online learning, and how they change over a one‐year period (if they do so). The findings of this study provide three insights: (1) they show how adult learners (who also happen to be novice online learners) respond emotionally and talk about their emotions in relation to online learning; (2) they call attention to the ways in which emotion talk changes from the beginning of the course to the end, always in response to specific demands and dimensions of online learning; and (3) they reveal the differential emotional responses between men and women in relation to their social and gender roles and responsibilities. Empirical and policy implications of this study are discussed at the end.     

The value simulation‐based learning added to machining technology in Singapore

This study seeks to understand the value simulation‐based learning (SBL) added to the learning of Machining Technology in a 15‐week core subject course offered to university students. The research questions were: (1) How did SBL enhance classroom learning? (2) How did SBL help participants in their test? (3) How did SBL prepare participants for workshop practice? The findings suggest that SBL enlivened the learning of Machining Technology, and promoted autonomous and mastery learning. SBL made a deep impression on the participants’ visual experience, helping them remember the machine processes. SBL also helped learners to conceptualize their answers and provided them with opportunities to become familiar with the conventional machines before workshop practice. An infusion of SBL has the potential to add value to the learning of Machining Technology.     

A computer‐based learning environment in the field of Newtonian mechanics

This paper describes a Computer‐based Learning Environment (CBLE) whose objective is to stimulate and support learning in the field of Newtonian mechanics through exploratory activities in various microworlds. The structure of the microworld TABLE is described in detail. A pilot study is presented: it concerns the modelling process of the learner knowledge in the topic involved and the evaluation of the impact of the exploratory activity on student learning. Some suggestions drawn from the results are discussed.     

A methodology for integrating computer‐based learning tools in science curricula

This paper demonstrates a methodology for effectively integrating computer‐based learning tools in science teaching and learning. This methodology provides a means of systematic analysis to identify the capabilities of particular software tools and to formulate a series of competencies relevant to physical science that could be developed by means of these capabilities. This analysis could inform the transition to technology‐rich learning environments by helping to deflect reliance on computer‐based tools for objectives they cannot meet, while encouraging the design of activity sequences that make the most of the educationally useful capabilities of available computer‐based tools. This methodology is demonstrated by means of two software tools widely used in science learning, namely modelling and simulation tools.     

Teachers’ socio-spatial practice in innovative learning environments

ABSTRACT

Understanding the relationship between innovative learning spaces and teachers’ pedagogical and social practices is essential given the increasing investment in innovative learning spaces worldwide. This study examined the socio-spatial practices of a group of teachers at a secondary school in an innovative learning space. Interview responses from 21 teachers revealed that space and social practices co-became through the interplay of the school culture, the physical, social and curriculum structures, and individual teachers’ sense-making of the practice. The study found that the cultural and structural systems drove the initial becoming of social practices, and that the individual systems shaped the diversified nature of the becoming. The findings suggest the importance of supporting teachers’ sense-making in terms of conceptions of subject teaching, professional identity, the affordances and constraints of the space and confidence in and capacity of constructing and managing effective learning experience in the space, when effecting educational changes with space.