Enhancing environmental educators' evaluation competencies: insights from an examination of the effectiveness of the My Environmental Education Evaluation Resource Assistant (MEERA) website

To conduct evaluations that can benefit individual programs as well as the field as a whole, environmental educators must have the necessary evaluation competencies. This exploratory study was conducted to determine to what extent a self‐directed learning resource entitled My Environmental Education Evaluation Resource Assistant (MEERA) can enhance environmental educators' evaluation competencies. The multiple case studies relied on data from eight environmental educators with limited evaluation experience who used MEERA to evaluate one of their programs. Results suggest that MEERA can (1) increase environmental educators' perceived evaluation competencies, (2) help environmental educators produce quality evaluation outputs, and (3) foster their use of evaluation results. Perceived benefits of using MEERA included obtaining evidence of program success, insights into how to improve programs, and alternative ways of thinking about programs. Perceived challenges included varying difficulties with evaluation tasks such as prioritizing evaluation questions and designing data collection instruments and, in line with this, desiring personal expert assistance for context‐specific advice and reassurance. This research contributes to expanding understanding of how to enhance environmental educators' evaluation competencies and practices.     

Engaging youth in public lands monitoring: opportunities for enhancing ecological literacy and environmental stewardship

Abstract

The US Forest Service has a long history of youth conservation education. We investigated U.S. Forest Service citizen science programs that involve secondary school students in field collection of monitoring data to understand (1) how the programs integrated science and environmental education and (2) whether these programs advance ecological literacy and environmental stewardship. We conducted semi-structured interviews with the program leads, teachers, and students. Program leads and students said programs produced reliable data and met monitoring and other U.S. Forest Service stewardship objectives. Although these programs varied in design and objectives, our findings suggest these programs were incorporating both science and environmental education, and there is some indication they are creating ecological literacy among participants. Students exhibited environmental stewardship to some degree as a result of all programs, but the extent of this is tied to programs’ objectives and design.     

Book Reviews

Abstract

Educators have invested considerable effort in developing environmental education programs that address students' knowledge, attitudes, and action competence regarding environmental issues. The authors explore the effectiveness of such programs in terms of both student learning outcomes and the intergenerational influence that results when students discuss their learning experiences with their parents and other community members. Six environmental education programs involving 284 students in Queensland schools, from Grades 5-12, were investigated. Students and their parents were surveyed and interviewed regarding their perceptions about the program, the program's influence on their environmental learning, and the extent and nature of discussions that the program stimulated between students and their parents. The authors draw conclusions about key features that should be incorporated into environmental education programs to encourage and empower students to bring about environmental change in their homes and communities.     

The Impact of a Professional Development Program on Teachers' Understandings about Watersheds,Water Quality,and Stream Monitoring

Abstract

Professional development programs should provide teachers with experiences that develop their knowledge and skills to integrate environmental field studies into their school curriculum. This article reports on a professional development model that engaged teachers in designing and conducting local environmental science research projects. An overview of the professional development model is provided and evaluation data are reported, detailing the impact of the professional development model on teachers' understandings about watersheds, water quality, and stream monitoring.     

Field News

Abstract

The recent (1990) establishment of a new campus of the University of Washington at Tacoma (UWT) has led to the development of an upper-division environmental curriculum within an interdisciplinary arts and sciences framework. All of the natural sciences at UWT were founded under the rubric of environmental science, which was integrated with a range of other disciplines' perspectives on the environment, to form a more comprehensive environmental education program. The community-oriented mission, evolving structure, and flexibility of this relatively new urban commuter campus allowed the authors to create a multitiered environmental education program that broadly integrates environmental issues into interdisciplinary undergraduate education and is consistent with national and international environmental education goals. In this study, the authors compare the curricula developed at UWT with other new and established environmental programs nationally.     

It Says in The JER

Abstract

This study identified the outstanding teachers in 10 randomly chosen elementary schools. Very high concordance was found among the ten methods of identification, some of which were objective while others depended upon subjective opinions of respondents.

Three methods chosen to represent salary determinants (training and experience), formal evaluation (principal judgments), and informal nonadministrative evaluation (selections by peer teachers, non-teaching staff members and school-associated parents), proved to have moderate statistical correlation.

Apparent general inadequacies in the formal evaluation process should be critically analyzed, especially if merit pay programs based on the evaluation process are being considered.     

Developing and implementing a robust and flexible framework for the evaluation and impact of educational development in higher education in Chile

ABSTRACT

This article describes the development of an evaluation and impact framework to assess the effectiveness of educational innovations. It can be utilized within a single program, as well as at institutional and national levels. While it is contextualized in a Chilean university, it is argued that it is widely applicable as it is informed by international best practice.

The rationale that informed the development of the evaluation framework is described and is illustrated using two programs: Faculty Learning Communities; and Student Learning Assistants. These demonstrate how the framework can be customized utilizing indicators and outcomes relevant to specific programs and stakeholders.     

Encouraging transformation and action competence: A Theory of Change evaluation of a sustainability leadership program for high school students

Abstract

Make a Difference (MAD) is a sustainability leadership program for high school students in Auckland, New Zealand. It offers a residential camp and follow-up activities. This article documents a participatory, utilization-focused evaluation of MAD, based on a Theory of Change approach. We ran workshops with MAD coordinators and students and conducted a survey with previous participants. This identified the core developmental and action outcomes of MAD, and the role of transformation and action competence in inspiring change. We discuss the benefits and limitations of both MAD and our evaluation approach and offer recommendations for others working on youth sustainability programs.     

Evaluation of retirement planning programs: A qualitative analysis of methodologies and efficacy

ABSTRACT

The objective of this integrative literature review was to evaluate the quality of retirement planning programs described in the extant literature. This was accomplished through a qualitative analysis of methodological and efficacy criteria as described by Flay et al. (2005), Kazdin (2010, 2011 and Murta (2005). Several databases were consulted in searching for retirement program articles including: Academic Search Premier, Medline, PsycInfo, and Web of Science, among others. Retirement planning intervention articles published in English, Portuguese, and Spanish were considered, with a focus on their evaluation methods and results. Eleven studies were identified that described the procedures for both program implementation and intervention evaluation. Results revealed methodological shortcomings in the papers reviewed, with concerns being related to a lack of experimental or quasiexperimental approaches, a failure to use previously validated measurement instruments and longitudinal assessments, and insufficiently robust data analysis procedures. That said, however, there was evidence from multiple investigations that the intervention programs examined led to increases in knowledge, positive changes in attitudes linked to retirement, and an increase in retirement-linked planning and preparation behaviors. Identification of strengths and weaknesses in the methods used and efficacy of these interventions could facilitate the construction of a research agenda aimed at promoting more favorable research designs. Use of more rigorous designs would stand to improve the internal validity of these retirement programs and, consequently, progress in this field.     

Environmental Sensitivity and the Mass Media

Abstract

The traditional methods of interpretation have placed most of the emphasis on presentation, form, and accuracy of content. These points remain important, but if the audience to whom the program is directed is not understood, the program will be a failure. There is a diversity of people found in recreation areas. Recent efforts in interpretive research have emphasized the need to include the audience as a factor in interpretive planning. This paper has been concerned with one such audience, the handicapped, and its purpose is to provide interpretive suggestions based on knowledge of their individual requirements.

One cannot conclude that because people have some physical or mental impairment, their needs for outdoor recreation, or abilities to benefit from it, are impaired. Rather, variations in interpretive approaches are required to match these unique visitors with a message. In many cases very little modification will be required.

Blind: As only five to ten percent of blind people read Braille, information can be best presented by way of sound, such as by use of cassette tape players. No special guiding apparatus is necessary; they move throughout the rest of the world without guide ropes or kickrails, and the interpretive setting is no different.

Deaf: If interpreters would use fingerspelling or some sign language, they could make some of their programs available to deaf persons.

Deaf-Blind: Because deaf-blind visitors would usually be accompanied by someone who is not handicapped, no special accommodations are necessary.

Mentally Retarded: Interpretation that is geared to their particular level of comprehension and which allows total personal involvement is the most satisfactory.

Ambulatory Limitations: Facilities made accessible and navigable is all that is necessary.     

Evaluation of the 2000 and 2001 Rhode Island early childhood summer institutes

Abstract

This article presents the theoretical basis and methods for the process and outcome evaluations of the Rhode Island Early Childhood Summer Institutes held in 2000 and 2001. The article also discusses the evaluation results that demonstrated the institutes were received as high‐quality inservice professional development programs in which participants made significant knowledge gains. Additionally, the influence of the evaluations on the institute programs held in 2000 and 2001 as well as the impacts on the planning of future institutes are described. These evaluation studies can serve as models to others planning evaluations of professional development programs. © 2004 Elsevier Inc. All rights reserved.     

Practice change in environmental education: lessons from professional development

Abstract

The field of environmental education (EE) strives to develop innovative practices to address emergent issues such as equity, climate change, and urbanization. Through facilitating workshops and ongoing networks for the exchange of ideas, professional development programs may foster innovation or practice change among environmental educators. This study investigates change in practice among environmental educators who participated in one of three online and face-to-face professional development programs. Drawing from practice theory, we measured elements of EE practice including goals, audiences, settings, activities, resources, and ideas. The results showed that across all three programs participants incorporated new resources and ideas into their practices whereas changes in other practice elements varied among programs. Participants in all three programs produced eBooks or ongoing networks that can be used as indicators of practice innovation. This study suggests that practice theory can be used to inform studies of professional development outcomes and examine EE practice, but that practice innovations may be more readily measured at the group rather than individual level.     

Using Student-Determined Needs to Evaluate Environmental Education Programs

Abstract

All too often, evaluation is not used as the important management tool it could be, because educators feel inadequately prepared to use complex evaluation techniques. “Direct evaluation,” as a research concept, is based on the idea of “research as an intermediate technology,” developed by Steven and Rachel Kaplan in Cognition and Environment (1982: Praeger). Direct evaluation means you ask what you need to know of those who are participants in a program you wish to evaluate. The results, when collated, can be used for justification as well as modification of programs.     

Environmental Education in the United States: A Survey of Preservice Teacher Education Programs

Abstract

The status of the environmental education component of preservice teacher education programs is unknown nationally. This study surveyed 715 institutions of teacher education using a mail questionnaire. The response rate was approximately 63%. The results indicate that most schools have few requirements related to environmental education, and in the majority of schools environmental education is not institutionalized.     

Engaged environmental science for underserved youth

Abstract

The National Institute for Environmental Health Sciences has called for targeted efforts to engage underserved youth in environmental education programs that support environmental literacy and contribute to the development of a diverse workforce pipeline for environmental science-related occupations. Evidence suggests that career knowledge among low income and minority youth is more likely to be incompatible with post-secondary educational opportunities than other racial and ethnic groups. One approach to attenuating discordant college and career expectations among underserved youth is building networks for information sharing between secondary and post-secondary students. The purpose of this commentary is to describe the development and implementation of a high school curriculum on environmental science and environmental justice by Texas A&M University in collaboration with community engagement partners, students, and teachers at Furr High School, an innovative XQ Super School in Houston, Texas.     

Turning Minds On and Faucets Off: Water Conservation Education in Jordanian Schools

Abstract

An evaluation was conducted to measure the impact of a curriculum implementation through the Jordan Water Conservation Education Project funded by USAID. This study examined the effect of recommending water conservation at the household level and the impact of using interactive teaching methods to promote conservation behaviors among students and their families. The evaluation used a postintervention design with random selection of participants. Comparisons were made among 671 students (424 experimental, 247 control) belonging to high school eco-clubs in central Jordan. Most students were girls in rural settings. The experimental group consisted of students whose teachers implemented an interactive curriculum and promoted household water-conservation behaviors. Teachers of students in the control group did not participate in the curriculum implementation, but those students were exposed to lectures about biodiversity issues. The results indicate that students who were exposed to the new curriculum demonstrated a higher level of knowledge about water conservation and performed recommended behaviors more often than students in the control group.     

Analytic considerations in distance education research

Abstract

All articles submitted to The American Journal of Distance Education are subjected to a multi‐stage review process. One of the stages of review is an evaluation of methodology, including the appropriateness of any analytic procedures that are used. For this article, a number of empirical research reports submitted over the past several years were reviewed to identify common analytic problems and errors often overlooked by distance education researchers.     

Perspectives of lifelong education in Portuguese higher education: a critical analysis of learning outcomes

ABSTRACT

Nurturing students’ continuous learning is a current trend in the higher education agenda. Curricula and academic contents should enable students to take part in stimulating learning experiences, as well as promoting both developing and training opportunities in the course of their lives and careers. Despite the relevance given to lifelong learning in the educational system, there are still some open questions: how this concept is understood and put into practice by higher education institutions? The paper aims to analyse the conceptions of lifelong learning as reflected on the learning outcomes proposed in a sample of study programs. A qualitative methodology and a data-driven approach are adopted to explore the content of the learning outcomes proposed in 10% of total study programs submitted to quality accreditation, since 2009. Generally, results reveal that higher education institutions are committed to the lifelong learning paradigm, particularly in master and PhD degrees. Students are expected to ‘invest in personal and professional development through life’, to ‘develop learning competences through life’, as well as to ‘foster lifelong learning’. This study provides a better understanding of the range of perspectives and the relevance given to lifelong learning as a valuable learning outcome.     

‘Come in with an open mind’: changing attitudes towards mathematics in primary teacher education

Background: The relationship between attitudes and behaviour has led to a focus on the role played by attitudes in the teaching and learning of mathematics.

Purpose: This paper reports on an investigation into student teachers’ self-reported attitudes towards mathematics in the context of a mathematics education programme. The programme had been developed by the authors in response to a new, reformed four-year undergraduate initial teacher education (ITE) for primary teachers in Ireland. The authors aimed to use the feedback to evaluate and develop the course further.

Sample: We report on the attitudes of 345 pre-service primary teachers enrolled in a four-year undergraduate ITE programme. 281 of these participants had matched survey data before and after participating in the mathematics education programme.

Method and Design: Four evaluation criteria were used as a conceptual framework for the evaluation of the mathematics education programme. A multi-method approach was taken in investigating the students’ attitudes to mathematics by using a scale as a quantitative measure of attitude alongside open-ended survey items that provided qualitative insights into changes in attitudes.

Results and conclusions: Results from the survey data suggested overall that the participants’ attitudes towards mathematics were more positive after they had experienced the mathematics education programme. Overall, the written responses suggested that according to self-reportage, participants perceived that participation in the mathematics education programme had affected their attitudes towards mathematics positively. Comments indicated that the ‘small group’ tutorial structure was particularly well received and was associated with opportunities to actively engage, explore, interact and reflect. Suggestions offered by the participants will be used to inform planning and development of the mathematics programme.