Design‐focused evaluation

In this paper an approach to the writing of evaluation questions is outlined and developed which focuses attention on the question of the effectiveness of an educational design for bringing about the learning it is intended to facilitate. The approach develops from the idea that all educational designs rely on instructional alignment, implicitly or explicitly, and succeed or fail to the extent to which the implementation of that alignment is effective. The approach to evaluation that is described, design‐focused evaluation, utilises students’ experiences of instructional designs and strategies and focuses questions on students’ awareness of the effectiveness of those strategies for facilitating the intended learning outcomes. Detailed advice is given on the construction of items that fit with the approach described and that maximise the generalisability of the approach to most if not all educational settings and designs.     

Can an Analysis of the Contrast Between pre-Galilean and Newtonian Theoretical Frameworks Help Students Develop a Scientific Mindset

This study examines a course in which students use two writing activitiesand collaborative group activities to examine the conceptual structure ofthe calculus-based introductory Physics course. Students are presented withtwo alternative frameworks; pre-Galilean Physics and Newtonian Physics. Theidea of the course design is that students would at first view the frameworksalmost in a theatrical sense as a view of a drama involving a conflict of actors;Aristotle, Galileo, Newton and others occurring a long time ago. As participantspassing through a series of interventions, the students become aware that theframeworks relate concepts from different parts of the course and learn to evaluatethe two alternative frameworks. They develop a scientific mindset changing theiroutlook on the course material from the viewpoint that it consists of a tool kit ofassorted practices, classified according to problem type, to the viewpoint that itcomprises a connected structure of concepts.     

The relationship between epistemological beliefs and the propensity for lifelong learning

The characteristics of lifelong learners have been extensively discussed in the literature and generally encapsulate two broad dimensions; skills and abilities related to learning, and beliefs about learning and knowledge. This study examined the factors that may predict such characteristics and thus an individual's propensity to engage in lifelong learning in a sample of university students. Together, openness to experience, change readiness, approaches to learning, self-efficacy and epistemological beliefs significantly predicted lifelong learning characteristics. In particular, the unique contribution of epistemological beliefs to the profile of a lifelong learner was supported. Results indicate that these beliefs may be a key predictor of lifelong learning.     

Who is listening? An examination of gender effects and employment choice in sustainability education in an undergraduate business school

Whilst universities acknowledge the importance of sustainability education, numerous problems exist in relation to the nature, delivery and outcomes of sustainability instruction. Many of these problems arise due to a lack of understanding about students’ perception towards, and knowledge about business sustainability. This article examines gender-specific perceptions of business sustainability issues, sustainability education and the importance students attach to sustainability in their choice of future employer. A paper-based self-complete survey was used to collect data from a convenience sample of 224 undergraduate marketing students at an Australian University. Factor analysis, T-tests, and multiple regression were used to test the hypothesised relationships. Results show significant differences between the female and male cohorts, supporting all but one of the hypothesised relationships. In addition, gender differences were observed across the constructs predicting employment choice. Implications for teaching practice and future research are provided.     

Ombuds processes and disputant reconciliation

Effective dispute resolution systems are critical to the functioning of large universities and organizations. However, the design of dispute resolution systems is often inadequate to achieving the stated goals of the system. Analysis of the cases of 45 disputants reveals how the design of a university dispute system and the social context can prevent an ombudsperson from successfully reconciling relationships. This study shows how a dispute system that generally failed to utilize face-to-face meetings or mediation techniques between disputants in a social context of fear and limited social ties was unable to reconcile or preserve working relationships. Analysis from normative pragmatic and social contextual approaches to disputing provides a mechanism for understanding the problems encountered by this ombudsperson and provides a tool for designing more effective dispute resolution systems.     

Randomized trial of a statewide home visiting program to prevent child abuse: impact in reducing parental risk factors

OBJECTIVES: To assess the impact of a home visiting program in reducing malleable parental risk factors for child abuse in families of newborns identified, through population-based screening, as at-risk of child abuse. METHODS: This randomized trial focused on Healthy Start Program (HSP) sites operated by three community-based organizations on Oahu, HI, USA. From 11/94 to 12/95, 643 families were enrolled and randomly assigned to intervention and control groups. Mothers in both groups were interviewed annually for 3 years (88% follow-up each year) to measure mental health, substance use, and partner violence. HSP records were reviewed to measure home visiting services provided. Home visitors were surveyed annually to measure their perceived competence. RESULTS: Malleable parental risks for child abuse were common at baseline. There was no significant overall program effect on any risk or on at-risk mothers' desire for and use of community services to address risks. There was a significant reduction in one measure of poor mental health at one agency and a significant reduction in maternal problem alcohol use and repeated incidents of physical partner violence for families receiving > or =75% of visits called for in the model. Home visitors often failed to recognize parental risks and seldom linked families with community resources. HSP training programs were under-developed in preparing staff to address risks and to link families with community resources. CONCLUSIONS: Overall, the home visiting program did not reduce major risk factors for child abuse that made families eligible for service. Research is needed to develop and test strategies to improve home visiting effectiveness in reducing parental risks for child abuse.     

The Role of the Learning Community in the Development of Discipline Knowledge and Generic Graduate Outcomes

In this paper we describe a study of learning outcomes at a research-intensive Australian university. Three graduate outcome variables (discipline knowledge and skills, communication and problem solving, and ethical and social sensitivity) are analysed separately using OLS regression and comparisons are made of the patterns of unique contributions from four independent variables (the CEQ Good Teaching and Learning Communities Scales, and two new, independent, scales for measuring Teaching and Program Quality). Further comparisons of these patterns are made across the Schools of the university. Results support the view that teaching and program quality are not the only important determinants of students’ learning outcomes. It is concluded that, whilst it continues to be appropriate for universities to be concerned with the quality of their teaching and programs, the interactive, social and collaborative aspects of students’ learning experiences, captured in the notion of the Learning Community, are also very important determinants of graduate outcomes, and so should be included in the focus of attempts at enhancing the quality of student learning.     

Evaluation of a university‐wide strategy providing staff development for tutors: effectiveness,relevance and local impact

In this article we report on the results of an evaluation of a two‐tiered strategic approach to the development of sessional teaching staff in one Australian university. Analysis of the data reveals a tension between the institutional imperatives underpinning the strategy and its local implementations. This tension ultimately undermines the effectiveness of such development strategies. We identify and describe various dimensions of this tension and its impact on programme quality and effectiveness and make recommendations for future strategic initiatives and their implementations.