Measuring the Impact of Professional Accreditation on Student Experiences and Learning Outcomes

This research uses a conceptual model to examine the influence of a change in accreditation standards on a representative national sample of 203 engineering programs at 40 institutions. Based on data collected from more than 140 program chairs, 1,200 faculty, 4,300 graduates of 2004, and 5,400 graduates of 1994, the study investigates the differential impact of the change in accreditation standards on programs reviewed in different years during the period of transition. Despite significant variation across most of the measures and groups in 1994 (before the introduction of the new accreditation criteria), the 2004 evidence demonstrates a surprisingly uniform level of student experiences and outcomes. These findings suggest that engineering accreditation is beginning to accomplish its quality assurance goals. This is a revised version of a research paper presented at the meeting of the Association for Institutional Research, May 17, 2006, Chicago, IL.     

The PhD – is it out of alignment?

ABSTRACT

Globally there is recognition that doctoral programmes need to offer the opportunity for career pathways beyond academia, as PhD graduates are entering a range of careers. Consequently, some doctoral programmes now provide opportunities to develop a broader skill set. However, most PhD programmes in the United Kingdom and Australasia still concentrate on disciplinary knowledge and developing research skills. Moreover, the assessment has remained narrowly focused on a written thesis and an oral examination. Here we apply the notion of ‘constructive alignment’ to doctoral education and find that, even in terms of preparing PhD graduates for academic and research careers, the curriculum lacks opportunities to develop and assess relevant skills. The situation is even more extreme for wider careers beyond academia. To achieve alignment, we suggest providing more structured learning opportunities and personalised professional development plans with formative portfolios. If wanting true alignment in the PhD, a portfolio would be the summative assessment, but this is likely a step too far for a conservative academy.     

Literacy as a social practice in the early years and the effects of the arts: a case study

Literacy as a social practice has a fundamental role in children’s lives especially in the early years context, in which social interactions are in the centre of knowledge achievement. Several pieces of research investigate the positive contribution of the arts in children’s literacy development in the early years settings. However, most of them focus on the aspect of emergent literacy and phonological awareness, with some indirect arguments about literacy as a social practice. Having this in mind and the importance of literacy as a social practice, this project was designed. The purpose of this paper is to examine the effects of the arts in the development of literacy as a social practice in the early years settings. The intervention used the ‘Play and Learn through the Arts’ (PLA) programme for a full school year in a case study with 5–6-year-old children in Greece. The outcomes were measured using authentic assessment techniques and a semi-structure interview. The findings showed the positive contribution of the arts in the development of literacy as a social practice in the early years setting.     

Students’ perceptions of vocabulary knowledge and learning in a middle school science classroom

This study investigated eighth-grade science students’ (13–14-year-olds) perceptions of their vocabulary knowledge, learning, and content achievement. Data sources included pre- and posttest of students’ perceptions of vocabulary knowledge, students’ perceptions of vocabulary and reading strategies surveys, and a content achievement test. Students’ perceptions of vocabulary knowledge were compared before and after instruction to see whether students believed they gained knowledge and the ability to explain categories of technical science terms. Students’ perceptions of vocabulary knowledge increased as a result of instruction. The participants had favorable views of the vocabulary and reading strategies implemented and believed the literacy approaches were important for their developing science knowledge. In addition, students’ content achievement was compared to a national data set. Students in this study outperformed a national data set on all content knowledge items assessed. Students’ perceptions of their knowledge and vocabulary and reading strategies were congruent with their content achievement. This study is one of the first to highlight the pivotal role students’ perception of vocabulary knowledge and vocabulary and reading strategies plays in science content learning.     

Scaffolding strategies in a rubric-based intervention to promote engineering students’ ability to address wicked problems

ABSTRACT

In recent years, there has been increasing interest within the engineering education research community to prepare engineering students to address wicked problems (WPs) such as climate change, resource scarcity and violent conflict. Previous research suggests that engineering students are able to address WPs if they are given adequate support, but there is a lack of research on what kinds of support are needed. This paper aims to reduce this gap by reporting on students’ performance in, and approaches to, addressing WPs when different scaffolding strategies were used in different parts of a rubric-based intervention. The intervention aimed to provide undergraduate engineering students with an understanding of the nature of WPs and with a structured way of addressing them. For each part of the intervention, we discuss affordances for learning provided by the different scaffolding strategies. The results suggest that strong cognitive scaffolding can support students’ understanding of the nature of WPs and students’ performance in written responses to WPs, but possibly also limits deep engagement with WPs and transfer of learning to other contexts.     

A Second-Order Longitudinal Model for Binary Outcomes: Item Response Theory Versus Structural Equation Modeling

Measuring academic growth, or change in aptitude, relies on longitudinal data collected across multiple measurements. The National Educational Longitudinal Study (NELS:88) is among the earliest, large-scale, educational surveys tracking students’ performance on cognitive batteries over 3 years. Notable features of the NELS:88 data set, and of almost all repeated measures educational assessments, are (a) the outcome variables are binary or at least categorical in nature; and (b) a set of different items is given at each measurement occasion with a few anchor items to fix the measurement scale. This study focuses on the challenges related to specifying and fitting a second-order longitudinal model for binary outcomes, within both the item response theory and structural equation modeling frameworks. The distinctions between and commonalities shared between these two frameworks are discussed. A real data analysis using the NELS:88 data set is presented for illustration purposes.     

A proposed framework and tool for non-economic research impact measurement

ABSTRACT

Research impact features heavily in debates about ‘the measured university’ and is now formally assessed by governments in the UK and Australia. Yet clear guidance on how impact can be measured in non-monetary ways is often lacking because of confused thinking and the context-specific nature of outcomes. To help resolve this, we first propose a general impact model of inputs, processes, outputs and outcomes framework. Then, using a survey approach of research users, we measure outcomes at different levels of abstraction using usefulness as the central construct and impact categories from the European (EU) and Association to Advance Collegiate Schools of Business (AACSB). The survey measures are simple, comparable between different impact cases, cost-effective, externally verifiable, and easily administered by those for whom impact measurement is new and puzzling. They can also be combined to form an impact index to address criticism that there is little standardisation in impact measurement. To improve the standardisation of context-specific measures, we suggest a common methodology for deriving these (SROI). The article discusses limitations of using surveys including administration, self-report data, and impact timescale problems, and suggests ways to reduce these. Implications for researchers, research managers, and assessors interested in measuring impact, such as who should do it and pay for it, are discussed.     

Evidencing student success and career outcomes among business and creative industries graduates

ABSTRACT

The teaching performance of higher education institutions is increasingly gauged by graduate employment outcomes. Measuring outcomes in full-time employment terms does not capture the complexities of underemployment, the rise of portfolio careers, the constraints of the labour market and graduate motivations for working arrangements that can allow greater flexibility and work-life balance. This study explores the career outcomes of Business and Creative Industries graduates using both traditional measures (full-time employment outcomes) and a suite of broader measures that examine career satisfaction, perceived employability, perceived career success, underemployment, and graduate motivations for seeking new roles. Findings confirm disciplinary differences in graduate experience, and raise some broad concerns about the quality of graduate employment, particularly given the lack of improvement in outcomes over time since course completion. Findings suggest graduates are optimistic about their career futures, despite unmet expectations – particularly on income.     

AILEM Programme: a long‐term intervention to promote literacy learning in low‐performing primary schools in Chile

After one year of implementation, this paper describes a programme designed to support literacy learning in low performing schools from poor sectors in Santiago, Chile. School‐based intensive training on theory and practice of a literacy learning framework for classroom instruction and long‐term support were used to impact the achievement of students from disadvantaged families in 16 kindergarten classrooms and 16 1st‐grade classrooms in nine primary schools. Preliminary results showed an improvement in learning achievement positively related to level of implementation of the literacy instruction framework in the classrooms and teachers' perception of an improvement in their knowledge and classroom practices. Although findings from this study cannot support definite conclusions, because of the lack of experimental conditions, they provide relevant information for future experimental studies on how to meet the literacy learning needs of children from disadvantaged backgrounds.