Web‐based learning environments: developing a framework for evaluation

With the widespread use of web‐based learning environments in the tertiary sector it is important to establish the usability of such environments for the target audience and their effectiveness in terms of meeting the educational objectives. However, a search of the literature has shown a scarcity of systematic evaluative studies of web‐based learning environments. Furthermore, the literature did not reveal a consistent starting position on appropriate methodologies with which to carry out such evaluations. This paper presents a general methodology for evaluating complex systems that is particularly appropriate for web‐based learning systems. Using what is called a trailing methodology (Finne et al., 1995 Finne, H, Levin, M and Nilssen, T. (1995). Trailing research: a model for useful program evaluation. Evaluation, 1(1): 11–31. [Crossref] [Google Scholar]), an evaluation was carried out of a web site that was used with student industrial experience projects. A key element in this evaluation was that the process was adaptive and collaborative; another was that it involved a team with expertise in evaluation, knowledge of the functional aspects of the web site and the educational purpose of the site. The evaluation process pointed to the importance of a flexible approach that utilizes the skills of the key stakeholders.     

Introducing a fifth pedagogy: experience‐based strategies for facilitating learning in natural environments

Educators have identified four categories of ‘productive pedagogies’ that are considered to lead to authentic student engagement and learning in the classroom. This study was designed to explore and extend these pedagogies in the context of learning in natural environments, in particular, through the programmes of Queensland environmental education centres. In‐depth interview and observation data were collected from students, classroom teachers and centre teachers who had participated in 12 environmental education programmes across Queensland, in order to identify the strategies that are most effective in facilitating learning in the natural environment. A fifth productive pedagogy category, ‘experience‐based learning’, is proposed. Experience‐based learning is particularly important in addressing students’ environmental attitudes and actions. The implications for the delivery of environmental education programmes both within and outside the classroom are discussed.     

Development of the human interaction dimension of the Self‐Regulated Learning Questionnaire in asynchronous online learning environments

Two studies focusing on the development and validation of the Online Self‐Regulated Learning Inventory (OSRLI) were conducted. The OSRLI is a self‐report instrument assessing the human interaction dimension of online self‐regulated learning. It consists of an affect/motivation scale and an interaction strategies scale. In Study 1, exploratory factor analysis of an initial affect/motivation item pool yielded four factors: enjoyment of human interaction, self‐efficacy for interaction with instructors, concern for interaction with students, and self‐efficacy for contributing to the online community. Exploratory factor analysis of an initial learning strategies item pool revealed three factors: writing strategies, responding strategies, and reflection strategies. In Study 2, confirmatory factor analysis was conducted in order to evaluate the stability of multidimensional factor structures. These exploratory and confirmatory factor analyses showed the OSRLI to be statistically moderate in terms of reliability and validity.     

Exploring high school students' and teachers' preferences toward the constructivist Internet‐based learning environments in Taiwan

This paper explores high school students' and teachers' preferences towards constructivist Internet‐based learning environments. The study proposes a framework, including two dimensions and five aspects, to illustrate the features of the Internet‐based learning environments. Based upon this framework, the Constructivist Internet‐based learning environment survey improvement (CILESI) was developed, which includes the scales of ease of use, multiple sources, student negotiation, reflective thinking, critical judgement and epistemological awareness. Questionnaire responses gathered from 630 high school students in Taiwan suggested that the CILESI showed adequate reliability in assessing students' preferences. Male students placed more emphasis on the student negotiation, critical judgement and epistemological awareness enhanced by the Internet‐based learning environments than female students did. In addition, the teachers of the sampled students (n?=?78) were also surveyed by CILESI. The teachers tended to express stronger preferences on the ease of use of the Internet‐based learning environments than did their students. However, students, when compared with their teachers, seemed to express more preferences towards the features of student negotiation, reflective thinking, critical judgement and epistemological awareness of Internet‐based learning environments.     

Learner presence,perception, and learning achievements in augmented–reality–mediated learning environments

Although research has indicated that augmented reality (AR)–facilitated instruction improves learning performance, further investigation of the usefulness of AR from a psychological perspective has been recommended. Researchers consider presence a major psychological effect when users are immersed in virtual reality environments. However, most research on learner presence has focused on measuring it rather than the factors influencing it. This study explored the relationship between learner perception and presence in an AR-facilitated learning environment. A game-type AR learning process was developed and an experiment was implemented employing a one-group pretest–posttest method. Three sets of research instruments were developed, comprising a learning achievement test, a learner's presence inventory, and a questionnaire on learners’ perception of the AR-mediated environment. A total of 60 validated data samples were collected and statistically analysed. The researchers determined that learner presence closely relates to learning achievement and AR learning perception. These results indicate that a more satisfactory AR perception leads to a higher level of learner presence in the AR environment, thereby resulting in more beneficial learning outcomes.     

Understanding project‐based learning in Second Life with a pedagogy,training, and assessment trio

This paper presents an empirical study of how Second Life (SL) was utilized for a highly successful project‐based graduate interdisciplinary communication course. Researchers found that an integrated threefold approach emphasizing project‐based pedagogy, technical training and support, and assessment/research was effective in cultivating and understanding learning in SL. Based on research findings, a project‐based application of SL that fully accommodates student experiential learning is recommended. It is suggested that teachers who are new To Whom It May Concern: SL and are preparing to “take the plunge” adopt a systematic team approach to integrate this trio of components. The students participating in this study formed an interdisciplinary team for their required SL project in which they chose to initiate a collaboration with the Basic Initiative and a group of architecture students to create a virtual presence for two green, sustainable, urban housing designs called the Alley Flats. Preliminary survey results suggest that the use of SL substantially enhanced the quality and experiences of student learning. Specific areas for improvement in future use are also identified.

Comprendre l’Apprentissage basé sur les projets dans le cadre de Second Life avec le trio Pédagogie‐Formation et Evaluation.

Le présent article présente une étude empirique sur la façon dont Second Life (SL) a été utilisé dans le cadre d’un cours de communication interdisciplinaire avancé fondé sur la réalisation de projets, cours qui a connu un grand succès. Les chercheurs ont trouvé qu’une approche intégrée en trois points, mettant l’accent sur la pédagogie des projets, la formation et le soutien techniques et l’évaluation/recherche se révélait efficace pour cultiver et comprendre l’apprentissage dans le cadre de SL. Les résultats de la recherche conduisent à recommander une application de Second Life fondée sur les projets, application qui prend pleinement en compte l’apprentissage des étudiants par l’expérience. On suggère que les professeurs qui sont nouveaux dans SL et qui se préparent à “faire le plongeon”, adoptent de façon systématique un mode d’approche en équipe, ceci pour intégrer ce trio de composantes. Les étudiants qui ont participé à cette étude formaient une équipe interdisciplinaire pour le projet SL qu’ils devaient réaliser et dans ce cadre, ils ont choisi de mettre en route une collaboration avec “The Basic Initiative” et un groupe d’étudiants d’architecture dans le but de créer une présence virtuelle pour concevoir deux projets d’habitat urbain vert et gérable appelés les Appartements de l’Allée. Une enquête préliminaire permet de penser que l’utilisation de SL a fortement amélioré le vécu et la qualité de l’apprentissage chez les étudiants. On a également identifié des zones spécifiques pour les améliorations à apporter pour des usages futurs.

Verstehen Projektbasierten Lernens in Second Life im Kontext von Pädagogik, Training und Beurteilung

Mit diesem Beitrag wird eine empirische Studie präsentiert, die berichtet wie Second Life (SL) sehr erfolgreich in einem projektorientierten und interdisziplinären Kommunikationskurs für Studenten genutzt worden ist. Unsere Untersuchung ergab, dass ein integrierter dreistufiger Ansatz – projektbasierte Pädagogik, technische Ausbildung und Unterstützung, sowie Beurteilung – zum besseren Verständnis über das Lernen mit Hilfe von SL führte. Auf Grundlage von Forschungsbergebnissen wird empfohlen, dass einer projektbasierten Ausarbeitung wird auf Erfahrung beruhendes Lernen zur Anwendung von SL empfohlen. Es wird geraten, dass Lehrer, die noch keine Erfahrung mit SL haben, einen systematischen Teamansatz übernehmen, um diese drei Bestandteile zu integrieren. Die Studenten, die an dieser Studie teilnahmen, bildeten ein interdisziplinäres Team, damit ihr angestrebtes SL Projekt, worin sie eine Kollaboration mit der Grundinitiative und einer Gruppe von Architekturstudenten initiieren wollten, eine virtuelle Gegenwart für zwei “the Alley Flats” genannte grüne, nachhaltige, städtische Unterbringungspläne schaffe. In vorläufigen Umfrageergebnissen wird festgestellt, dass die Verwendung von SL die Qualität der Arbeit und die Möglichkeit der Erfahrungen der Studenten zu lernen, beträchtlich verbesserte. Bestimmte Bereiche zur Verbesserung zukünftiger Verwendungen wurden ebenfalls identifiziert.

Comprender el aprendizaje basado en proyectos dentro de Second Life con el trio Pedagogía, Formación/Evaluación.

Este artículo presenta un estudio empírico sobre la manera en que Second Life fue utilizado para un curso de comunicación avanzado, interdisciplinario y altamente exitoso basado en proyectos. Los investigadores han descubierto que un abordaje integrado triple con énfsis en la pedagogía basada en proyectos, en la formación y el soporte técnico y en evaluación/investigación, resultaba efectivo para cultivar y comprender el aprendizaje en SL. Basandose en los resultados de la investigación, se recomienda una aplicación de Second Life basada en proyectos integrando completamente la experiencia de aprendizaje por parte de los estudiantes. Nuestra sugestión es que los profesores que son nuevos en SL y listos a zambullirse, adopten una metodología de trabajo en equipo para integrar este trio de componentes. Los estudiantes que participaron en este proyecto formaban equipos interdisciplinarios para sus proyectos SL y decidieron de iniciar una colaboración con “The Basic Initiative” y un grupo de estudiantes de arquitectura de modo a crear una presencia virtual para dos diseños de viviendas urbanas verdes y sostenibles llamadas Los Apartamentos de la Alameda. Los resultados de las encuestas preliminarias indican que el uso de SL realzó fuertemente la calidad y la vivencia del aprendizaje estudiantil. Se identificó también sectores específicos para futuras mejoras.     

Characterising programme‐level assessment environments that support learning

This article outlines a methodology for characterising features of programme‐level assessment environments so that the relationship between features of the assessment environment and students’ learning response can be studied. The methodology was developed through the detailed case study of nine undergraduate degree programmes: one in each of three contrasting discipline areas in each of three contrasting universities. Each case study involved examination of course documentation, interviews with academics and interviews with students, following which each degree programme was coded in relation to a range of features of the assessment environment, such as the proportion of marks derived from examinations and the volume and timeliness of feedback on assignments. Programmes were found to differ profoundly in terms of variables that are known to have implications for student‐learning processes. They also differed widely in the extent to which they illustrated the application of conventional wisdom about curriculum design, embodied in national quality assurance guidelines and the Bologna Agreement. Programmes were found to have either a high volume of summative assessment or a high volume of formative‐only assessment, but never both at the same time. Programmes also differed in the mechanisms used to make goals and standards clear, having either highly explicit curriculum design or high volumes of written and oral feedback, but never both at the same time. The findings suggest that there are distinctive programme‐level assessment environments that operate in quite different ways despite all programmes studied being subject to the same quality assurance code of practice.     

The role of visualization in learning from computer‐based images

Among the sciences, the practice of geology is especially visual. To assess the role of spatial ability in learning geology, we designed an experiment using: (1) web‐based versions of spatial visualization tests, (2) a geospatial test, and (3) multimedia instructional modules built around QuickTime Virtual Reality movies. Students in control and experimental sections were administered measures of spatial orientation and visualization, as well as a content‐based geospatial examination. All subjects improved significantly in their scores on spatial visualization and the geospatial examination. There was no change in their scores on spatial orientation. A three‐way analysis of variance, with the geospatial examination as the dependent variable, revealed significant main effects favoring the experimental group and a significant interaction between treatment and gender. These results demonstrate that spatial ability can be improved through instruction, that learning of geological content will improve as a result, and that differences in performance between the genders can be eliminated.     

Critical social theory and transformative learning: evidence in pre‐service teachers’ service‐learning reflection logs

This paper reports on the opportunities for transformational learning experienced by a group of pre‐service teachers who were engaged in service‐learning as a pedagogical process with a focus on reflection. Critical social theory informed the design of the reflection process as it enabled a move away from knowledge transmission toward knowledge transformation. The structured reflection log was designed to illustrate the critical social theory expectations of quality learning that teach students to think critically: ideology critique and utopian critique. Butin’s lenses and a reflection framework informed by the work of Bain, Ballantyne, Mills and Lester were used in the design of the service‐learning reflection log. Reported data provide evidence of transformational learning and highlight how the students critique their world and imagine how they could contribute to a better world in their work as a beginning teacher.     

E‐learning and the transformation of social interaction in higher education

This article examines the way in which e‐learning is transforming the nature of social interaction in higher education. In this new educational environment, radical societal transitions and the opportunities afforded by modern communication technologies together produce formidable challenges. Significant as these challenges may be, concentration upon problems of a practical kind draws attention away from the more theoretical concerns in understanding e‐learning. By drawing together developments in social, educational and communicational theory and Gilly Salmon’s hands‐on approach to teaching and learning online, this article reveals some unintended consequences: e‐moderation and the use of e‐tivities may perpetuate the very conditions that limit our chances of dealing successfully with the challenges posed by e‐learning. While theory may muddle what might otherwise be communicated meaningfully to those in search of practical answers, theoretical developments provide concepts and frameworks that can be placed in the service of a critical understanding of e‐learning and the transformation of social interaction in higher education.     

Adult learners’ emotions in online learning

The aim of the research study reported in this article was to investigate how adult learners talk about their emotions in the context of a year‐long online course, the first online course these adults take, as part of a distance education program. The theoretical and methodological approach focused on formulating an account of how emotion discourses are used by learners, what role they play in online learning, and how they change over a one‐year period (if they do so). The findings of this study provide three insights: (1) they show how adult learners (who also happen to be novice online learners) respond emotionally and talk about their emotions in relation to online learning; (2) they call attention to the ways in which emotion talk changes from the beginning of the course to the end, always in response to specific demands and dimensions of online learning; and (3) they reveal the differential emotional responses between men and women in relation to their social and gender roles and responsibilities. Empirical and policy implications of this study are discussed at the end.     

The value simulation‐based learning added to machining technology in Singapore

This study seeks to understand the value simulation‐based learning (SBL) added to the learning of Machining Technology in a 15‐week core subject course offered to university students. The research questions were: (1) How did SBL enhance classroom learning? (2) How did SBL help participants in their test? (3) How did SBL prepare participants for workshop practice? The findings suggest that SBL enlivened the learning of Machining Technology, and promoted autonomous and mastery learning. SBL made a deep impression on the participants’ visual experience, helping them remember the machine processes. SBL also helped learners to conceptualize their answers and provided them with opportunities to become familiar with the conventional machines before workshop practice. An infusion of SBL has the potential to add value to the learning of Machining Technology.     

A methodology for integrating computer‐based learning tools in science curricula

This paper demonstrates a methodology for effectively integrating computer‐based learning tools in science teaching and learning. This methodology provides a means of systematic analysis to identify the capabilities of particular software tools and to formulate a series of competencies relevant to physical science that could be developed by means of these capabilities. This analysis could inform the transition to technology‐rich learning environments by helping to deflect reliance on computer‐based tools for objectives they cannot meet, while encouraging the design of activity sequences that make the most of the educationally useful capabilities of available computer‐based tools. This methodology is demonstrated by means of two software tools widely used in science learning, namely modelling and simulation tools.     

A computer‐based learning environment in the field of Newtonian mechanics

This paper describes a Computer‐based Learning Environment (CBLE) whose objective is to stimulate and support learning in the field of Newtonian mechanics through exploratory activities in various microworlds. The structure of the microworld TABLE is described in detail. A pilot study is presented: it concerns the modelling process of the learner knowledge in the topic involved and the evaluation of the impact of the exploratory activity on student learning. Some suggestions drawn from the results are discussed.     

Synchronous collaboration competencies in web‐conferencing environments – their impact on the learning process

Based on a three‐semester design‐based research study examining learning and teaching in a web‐conferencing environment, this article identifies types of synchronous collaboration competencies and reveals their influence on learning processes. Four levels of online collaborative competencies were observed – operational, interactional, managerial, and design. The relative importance of students and teachers possessing the different levels of competencies depended on the degree of interactivity in the learning designs being applied. Both misunderstandings and misuses impacted on learning and collaborative processes, with misuses occurring more persistently throughout semesters than misunderstandings. The distinction between developing students’ technical skills and their collaborative capabilities is drawn. Strategies for developing each are recommended.     

Authentic arts‐based learning in teacher education: a musical theatre experience

There is increasing concern over disengagement of teacher candidates during on‐campus periods of pre‐service programs due to the perceived disconnect between educational theory and practice. In response, teacher education has been called to engage candidates in praxis through authentic learning. In this paper, we bring together the fields of teacher education, authentic learning, and arts‐based learning to articulate a means of enhancing pre‐service programs. Specifically, this paper documents and analyzes the process of engaging teacher candidates in an authentic learning experience of a musical theatre production. Data collected from 30 teacher candidates through rehearsal observations, a post‐production questionnaire, and focus group interviews indicated that participation created a sense of community and belonging, as well as developed skills necessary to carry forward art‐based learning activities into their teaching careers.     

Evaluating computer‐based assessment in a risk‐based model

There are three purposes for evaluation: evaluation for action to aid the decision making process, evaluation for understanding to further enhance enlightenment and evaluation for control to ensure compliance to standards. This article argues that the primary function of evaluation in the ‘Catherine Wheel’ computer‐based assessment (CBA) cyclic model is evaluation for action and secondary functions are evaluation for understanding and control. By studying segment and cyclic dependencies, conducting a risk analysis and identifying key stakeholders it is possible to identify where action evaluation has the most effect, what is required to be evaluated, who will perform the action and how it will be controlled. Each segment and cycle in the CBA model must therefore be monitored and the observations made must have meaning for the elimination or reduction of pedagogic, operational, technical non web‐based, web‐based and financial risks. It is important to consider stakeholders because they have differing views on what is most important in the evaluation process and inevitably will be involved in the evaluation process itself. CBA system stakeholders include management, academic, support and administrative staff together with external verifiers and students. The information gained from these stakeholders provides quality enhancement and assurance and forms the basis for change.     

Students’ experience of synchronous learning in distributed environments

This article reports on a two-year ethnographic study of learners participating in multi-site, graduate-level education classes. Classes sometimes met face-to-face in the same physical location; at other times part of the class met physically elsewhere. Yet all were linked through the virtual space. Ethnographic analysis of four data types explored how the instructor and students were able to interact through videoconferencing technologies. Most of the interaction occurred between the local and distance learners by way of cultural guides, local students assigned to host a distance learner through Google Video chat. The distance learners were able to receive real-time attention from the instructor and were able to share differing perspectives that contributed to increased satisfaction in the course. These interactions allowed for a dynamic collaborative effort among a diverse set of actors in the field of education.