Recommendations for Implementing an Effective,Efficient, and Enduring Outcomes-Based Assessment Program

To propose recommendations to implement outcomes-based assessment program review in a California Community College District, a grounded theory analysis was performed within the district. The findings generated were cross-referenced with findings from a multi-institutional case study analysis. A key finding from the cross-reference was the need for the leadership to clarify across all levels of the organization who the audience is for the data generated from the program review process; to clarify the indicators that each audience requires; and to clarify how that audience intends to use results generated from the process prior to designing the outcomes-based assessment process.     

Challenges in the Implementation of Outcomes-Based Assessment Program Review in a California Community College District

The purpose of this study was to explore the manner in which a community college district planned to implement a systematic outcomes-based assessment program review process in order to have results inform institutional, district, and state policy discussions. Data derived from this grounded theory study indicated that there was not a shared conceptualization across institutional and district leadership for the identification of the audience that would use the data generated from the program review process. This finding is typical for many institutions, particularly as leadership desires to honor the individual uniqueness of each college.     

An Estimate of the Reliability of a Technique of Increasing Educational Accountability through Goal Analysis Involving Community,Staff, and Students

Abstract

This study investigated whether data generated via the use of a school needs assessment model can provide local school boards and other district personnel with reliable decision-making information for program and policy revision. Each member of a randomly selected sample from three Iowa school districts was requested to rate and rank the 18 educational goal statements of the Phi Delta Kappa Educational Planning Model (16), both as individuals and as members of a small group. Initially these small groups were randomly divided into two halves. The mean goal ratings of die two groups were then correlated with each other. Three societal groups-faculty, students, and community members-represented each school’s sample. The relationship of a person’s status to the movement of his individual goal preference ratings during the small group consensus goal rating process was examined. Results indicated that in all participating school districts, there was a high degree of correlation between goal preference ratings generated by both halves of the randomly split sample. High correlation values linked with a high level of significance tends to support the hypothesis that this type of model is reliable, in that it does produce consistency of result in providing for the intended measure. Further analysis revealed that differences in movement of individual goal preference ratings in achieving small group consensus, due to person status, were statistically significant, thus indicating that a person’s status has a direct relationship to movement on consensus.     

Senior secondary school assessment and standard-setting in Queensland,Australia: social context and paradigmatic change

ABSTRACT

This article draws on three assessment paradigms – psychometrics, outcomes-based and curriculum-based assessment – to discuss paradigmatic changes in senior school assessment and achievement standard-setting in Queensland, Australia, over the last 50 years. These include radical reforms in 1970 from university-controlled examinations to school-based assessments applying normative standard-setting, to subsequent reforms in 1978 introducing competence(curriculum)-based assessment and standards. From 2019, a new reform introduces a combination of school-based and external assessment with procedures for establishing standards still in progress.

Changes to Queensland assessment and standard-setting are discussed in terms of three preconditions for paradigm change – dissatisfaction, an alternative acceptable paradigm, and majority acceptance of change. Influence of paradigmatic origins of reformers is discussed. The amalgam of curriculum-based assessment and psychometric paradigms in the new Queensland system is considered in terms of theoretical compatibility and potential impact on the new standards.     

Exploring the challenge of developing student teacher data literacy

Abstract

A number of trends are converging to drive the need for more informed teacher data use. These include advocacy for formative assessment and the need for teachers to account for student learning. In this context, assessment literacy and data literacy have emerged as a focus in research and professional development. Problematically, research signals that developing assessment/data literacy is challenging with evidence that teachers may not have mastered relevant aspects of mathematics and statistics. This paper reports lecturer, student teacher and school leader views of the role and requirements of data literacy using data from a larger study into how to foster student teacher mathematical thinking for the breadth of teacher professional work. Data were generated via interviews, surveys and document analysis. Findings suggest a concern and opportunity to develop assessment/data literacy as this calls on mathematical and statistical understanding.     

Beliefs and attitudes about assessment of a sample of student teachers in South Africa

Abstract

The purpose of this study was to take the first steps in a long-term approach to helping South African teachers understand and respond to government demands that they change their assessment practices. Specifically, it attempted to identify the beliefs, perceptions and attitudes about assessment that student teachers bring with them to courses that are designed to equip them to teach in ways that are consistent with current curriculum trends in South Africa. The study attempted to answer the following research questions: are student teachers’ beliefs about assessment consistent with the approaches to assessment advocated in the South African Revised National Curriculum Statement? Are student teachers’ beliefs about assessment consistent with the basic principles of outcomes-based education? The results indicated broad general agreement between student teachers’ beliefs about assessment and the principles of assessment espoused in the Revised National Curriculum Statement. However, there was evidence that the beliefs of some of the subjects were inconsistent with these principles.     

A Model for Wholesale and Retail Assessment Centres in Regulated Occupational Learning in South Africa

ABSTRACT

South Africa's new occupational and vocational qualifications require assessment centres with a national footprint to conduct national summative assessments which are convenient and cost effective for learners. This new assessment process requires specialised facilities, but few current facilities can provide these. The research project proposed a model which is able to serve the needs of the quality assurance body and industry and be appropriate to the retail sector occupational qualifications. A qualitative method was used, involving a literature review and secondary data analysis; small focus groups; semi-structured interviews; and follow-up workshops to peer review the findings. The participants included industry experts, education providers and quality assurance experts. Two models were identified and considered, namely: Model A, which involves private businesses that can be accredited to become assessment centres; and Model B, which uses Technical Vocational Education and Training (TVET) colleges for all national assessments. The overall finding was that Model B is more suitable.     

Relative Importance of the Institute of Food Technologists (IFT) Core Competencies—A Case Study Survey

ABSTRACT: Nearly 90% of graduating seniors from the Food Science program at Purdue University gain employment in the food industry. Ensuring the program is meeting employers' needs is one mission of our curriculum committee with input from an Industrial Advisory Board (IAB). In light of the recent outcomes-based assessment requirements outlined by IFT, feedback from our IAB members regarding the relative importance of the core competencies was needed. The objective of this study was to survey the IAB members regarding relative importance of each of the 46 core competencies. Each competency was ranked using a Likert scale from 1 = not important to 5 = very important. Statistical analysis was conducted using the Mann-Whitney nonparametric test. Significant differences (P = 0.05) were found between several of the five content categories with Success Skills scoring highest (4.35 out of 5) on relative importance followed by Applied Food Science Knowledge (4.02 out of 5). Also, significant differences (P = 0.05) were found between competencies within each content category. Overall, the results suggest that the 46 core competencies should not be given the same instructional weight within a curriculum. One option for addressing the variation in importance could be to include some planned redundancy within the curriculum for those core competencies that were ranked the highest. In addition, part of the outcomes-based assessment process should include measures of the level of competency being achieved by graduates. By considering relative importance and competency achieved by graduates, important areas for improvement can be identified.     

A systematic literature review of student engagement in software visualization: a theoretical perspective

ABSTRACT

Background and Context: In spite of the decades spent developing software visualization (SV), doubts still remain regarding their effectiveness. Furthermore, student engagement plays an important role in improving SV effectiveness as it is correlated with many positive academic outcomes. It has been shown that the existing SV has failed to engage students effectively.

Objective: Therefore, there is a need to understand the theories behind SV design from the engagement perspective to produce a roadmap for future tool construction. The aim of this study was to identify the theories have been used in literature to explain or construct student engage- ment with SV in computer science courses for novices.

Method: We performed a systematic literature review that identified a total of 58 articles published between 2011 and 2017, which were then selected for the study. However, among them, only 18 articles had discussed their theoretical framework.

Findings: The results of this study show a richness in the theoretical framework obtained from different disciplines, however, constructivism is still dominant in the computing education research (CER) domain. It is evidently clear from the findings that the theories generated from the CER domain are needed to enhance the effectiveness of SV.

Implications: As a result of this review, we suggest several design principles and engagement attributes to be considered while creating an engaging SV.     

An empirical review of anonymity effects in peer assessment,peer feedback,peer review,peer evaluation and peer grading

Abstract

Peer assessment has proven to have positive learning outcomes. Importantly, peer assessment is a social process and some claim that the use of anonymity might have advantages. However, the findings have not always been in the same direction. Our aims were: (a) to review the effects of using anonymity in peer assessment on performance, peer feedback content, peer grading accuracy, social effects and students’ perspective on peer assessment; and (b) to investigate the effects of four moderating variables (educational level, peer grading, assessment aids, direction of anonymity) in relation to anonymity. A literature search was conducted including five different terms related to peer assessment (e.g., peer feedback) and anonymity. Fourteen studies that used a control group or a within group design were found. The narrative review revealed that anonymous peer assessment seems to provide advantages for students’ perceptions about the learning value of peer assessment, delivering more critical peer feedback, increased self-perceived social effects, a slight tendency for more performance, especially in higher education and with less peer assessment aids. Some conclusions are that: (a) when implementing anonymity in peer assessment the instructional context and goals need to be considered, (b) existent empirical research is still limited, and (c) future research should employ stronger and more complex research designs.     

Early field experience: A question of effectiveness

Abstract

This paper offers a historical perspective of field‐based experiences in teacher preparation. The literature review presents information from studies that support the educative value of early field experiences and studies reporting results that do not lend empirical support for endorsing early field experiences.

Based on the information reported in the literature review, the writer offers guidelines for program development designed to increase the likelihood that an early field experience program will have the beneficial results that educators intend. The key issues for program development are: (a) determine the purpose of the experience, (b) determine the administrative feasibility of the experience, and (c) determine the method of evaluation for the experience.

In summary, the writer expresses a need for congruence between the university's teacher education program and the classroom sites for early field experiences. He further states that the goals, methods, and philosophy of university and classroom teachers need to be mutually understood and endorsed.     

Contemporary perspectives on the Australian doctorate: framing insights to guide development

ABSTRACT

The growth of the doctorate in recent decades has spawned concerns about programme quality and external relevance. This article reports findings from research conducted in Australia that was shaped by the view that the doctorate remains in need of educational clarification and design. The research generated insights that can be used to modernise and realign the doctorate. First, this article discusses the development of a doctoral education framework. Next, it presents results from a questionnaire developed using the framework and administered in a national survey to both five-year-out doctoral graduates and to doctoral experts. This identifies the need to design programmes that better orient and align research training with students’ professional aspirations, incorporate more real-world forms of learning, enhance traditional aspects of research training and provide structured reports of doctoral outcomes, experiences and preparations. The results underpin the need for more specific analyses of outcomes and assessment, the nature and distribution of doctoral experiences and the design and management of the resources that go into doctoral education.     

Longitudinal Relations among Maternal Self-Efficacy,Maternal Warmth,and Early Adolescents’ Prosocial Behavior

SYNOPSIS

Objective. This article examined two-wave longitudinal paths among maternal self-efficacy, maternal warmth, and early adolescents’ prosocial behavior. Design . Participants were 194 Italian mother/child dyads from 9–13 years at the first assessment and 12–15 years at the second assessment. Using a two-wave longitudinal design, the present study tested cross-lagged paths among maternal self-efficacy, warmth, and early adolescents’ prosocial behavior controlling for the stability of these variables and their concurrent associations. Results . Maternal warmth mediated the link between maternal self-efficacy and adolescents’ prosocial behavior, and bi-directional effects between maternal self-efficacy and maternal warmth were found. Conclusions . These findings support a parent-driven model in the promotion of prosocial behaviors across early adolescence and a general need to consider simultaneously parental cognitions (maternal self-efficacy) and actions (maternal warmth) in explaining adolescents’ prosocial behaviors. Implications for parenting program interventions are discussed.     

Keeping the Focus on the Child: Supporting and Reporting on Teaching and Learning With a Classroom-Based Performance Assessment System

Abstract

This article describes the findings of studies conducted on a large-scale, classroom-based performance assessment of literacy for the early grades designed to provide information that is useful for reporting, as well as teaching. Technical studies found the assessment to be a promising instrument that is reliable and valid. Follow-up studies of the assessment's use point to its positive impact on teachers' practice and on school and district policies. The studies' findings suggest that classroom-based performance assessment can be a viable accountability, as well as instructional, tool, capturing a range of students' abilities in a range of formats, and that use of such an assessment has the potential to enhance teachers' knowledge about literacy and their abilities to effectively support students' learning.     

Teacher evaluation standards in practice: A standards‐based assessment tool for diversity‐responsive teaching

Abstract

Although there is no question about the need for preservice and inservice teachers to have the skills to meet the diverse needs of students in todays classrooms, the issue of how to assess those same skills has proved to be problematic. This article reports on a joint effort by faculty from an urban university and a large public school district to create an observation tool that will assess and mentor preservice and inservice teachers’ abilities to address issues of diversity in their classrooms meaningfully. The authors will first report on missions and background information representing both the university and the school district and then on the development of a standards‐based assessment tool. Lastly, suggestions that compliment a comprehensive body of evidence of teacher evaluation will be discussed.     

A Survey of Teen Museum Education Participants and their Parents

ABSTRACT

In this article, we discuss a museum program for teens located in an urban environment. The participants were high school students from public, private, religious and home schools. The program allowed learning to occur in an informal setting and united teens from one city through a common interest in visual art. Also, it was an opportunity for the museum to build a teen audience and strengthen community involvement. As part of the program, the participants and their parents were surveyed to study what aspects of the program they found to be meaningful and relevant. The results underline the importance of developing and maintaining museum education for teens.     

Students’ perceptions on the recognition of holistic competency achievement: A systematic mixed studies review

Holistic competency development is rarely certified in practice, and the assessment of holistic competencies remains an under-researched topic in the education literature. The primary aim of this systematic mixed studies review is to explore students’ perceptions on the recognition of holistic competencies. The review found very few studies that specifically focused on or directly addressed student perceptions in this area. Based on the analysis of 31 studies identified from existing literature, three themes were generated: evidence of overall achievement, assessment of holistic competencies in learning tasks, and challenges. The findings show that there is a lack of attention to holistic competency recognition and is a major gap in the current empirical literature. The current review highlights the need for a systematic process of documenting, reporting, and certifying holistic competency in higher education that takes into consideration the broader level of overall recognition and the assessment of competencies.     

Secondary school students who drop out of school in rural Pakistan: The perspectives of fathers

ABSTRACT

Background: The problem of students dropping out of school is one of the barriers to improving educational outcomes in poor and developing countries.

Purpose: This small scale, in-depth study sought to explore the phenomenon of students who drop out from secondary education, through the perspectives of fathers of such students in a rural district of Pakistan.

Method: Primary data were collected through detailed, individual interviews with 14 fathers whose sons had dropped out from secondary education. The study adopted a framework involving push and pull factors to analyse qualitatively the processes and examine the reasons for dropping out that prevailed inside and outside school.

Findings: The analysis suggested that, from the perspectives of the fathers, family poverty, poor academic performance and issues relating to teachers’ engagement with teaching at school were main factors involved in a student dropping out of school.

Conclusions: The findings from this small scale, detailed analysis of data draw attention to the issues arising from a tension in mainstream teachers’ educational roles in and beyond the school. It further highlights the need for greater investment in secondary education, in order to help to reduce rates of student drop out in the remote rural areas of Pakistan.