DETERMINING THE QUALITY OF COMPETENCE ASSESSMENT PROGRAMS: A SELF-EVALUATION PROCEDURE

As assessment methods are changing, the way to determine their quality needs to be changed accordingly. This article argues for the use Competence Assessment Programs (CAPs), combinations of traditional tests and new assessment methods which involve both formative and summative assessments. To assist schools in evaluating their CAPs, a self-evaluation procedure was developed, based on 12 quality criteria for CAPs developed in earlier studies. A self-evaluation was chosen as it is increasingly used as an alternative to external evaluation. The CAP self-evaluation is carried out by a group of functionaries from the same school and comprises individual self-evaluations and a group interview. The CAP is rated on the 12 quality criteria and a piece of evidence is asked for to support these ratings. In this study, three functionaries from eight schools (N = 24) evaluated their CAP using the self-evaluation procedure. Results show that the group interview was very important as different perspectives on the CAP are assembled here into an overall picture of the CAP's quality. Schools seem to use mainly personal experiences to support their ratings and need to be supported in the process of carrying out a self-evaluation.     

Transfer of learning in Swedish technical vocational education: student experiences in the energy and industry programmes

One of the most important issues in vocational education is the interaction between learning at school and at different workplaces. Students need to transfer what they have learned in and between these different learning arenas. However, little agreement exists among researchers on how to conceptualise and empirically study transfer of learning. This article aims to enhance our understanding of this process in technical vocational education by adding a new perspective on transfer: the phenomenological lifeworld approach. Eleven interviews with students enrolled in the Energy and Industry programmes at a Swedish upper-secondary school were conducted. Three transfer themes emerged from the data: learning for new situations, variation and integration of theory and practice. The student perspective, based on the lifeworld approach shows that the different learning arenas – the school and workplaces – can complement each other. The learning arenas offer different opportunities for learning, and learning in both arenas is necessary in order to prepare for an unknown future, according to the students in this study.     

Entertainment Education and social change: Evaluating a children's soap opera in Kenya

This article discusses audience responses to a children's soap opera produced and broadcast in Kenya. It examines the evaluation process in relation to qualitative audience research within Cultural Studies. It challenges an Entertainment–Education model of media as vehicles for messages and links strongly with Communication for Social Change priorities in which media are the facilitators. In such an approach this soap opera should not only fit into the cultural and social contexts of its child audience but should actively promote discussion and the interactive negotiation of meaning. The narratives address post conflict concerns of democracy, rights, peace and reconciliation, encouraging dialogue with peers and family about extremely difficult issues. The question is how we evaluate and assess such processes.     

Exploring the role of assessment criteria during teachers’ collaborative judgement processes of students’ portfolios

Student portfolios are increasingly used for assessing student competences in higher education, but results about the construct validity of portfolio assessment are mixed. A prerequisite for construct validity is that the portfolio assessment is based on relevant portfolio content. Assessment criteria, are often used to enhance this condition. This study aims to identify whether assessment criteria can improve content, argumentation and communication during teacher moderation while judging student portfolios. Six teachers scored 32 student portfolios in dyads with and without assessment criteria. Their judgement processes were qualitatively analysed. Results indicated that the quality of their judgement processes was low, since teachers based their judgements mainly on their own personal opinion and less on evidence found in the portfolio. Teachers barely paid attention to quality checks and easily agreed with each other. When teachers used assessment criteria, the quality of their judgements slightly improved. They based their judgements more on relevant evidence, used less personal experiences and more often checked the quality of their judgement processes. It is concluded that the quality of teacher portfolio judgement is low, and that the use of assessment criteria can enhance its quality.     

Explicating Students' Personal Professional Theories in Vocational Education through Multi-method Triangulation

Students in competence-based vocational education are expected to actively construct a personal professional theory, in which they integrate different types of knowledge and beliefs. Students' personal professional theories are seen as an important learning outcome of competence-based vocational education. However, it is unknown how personal professional theories can be measured. This study focused on measuring the content and nature of students' personal professional theories using a multi-method triangulation approach, in which 16 students in the domain of Social Work constructed a concept map, an interview and a self-report. The results show that the relatively structured methods (i.e., interviews and concept maps) reveal more insight into students' personal professional theories than less structured methods (i.e., self-reports). It is concluded that both structure as well as adequate prompts are important in the process of explicating personal professional theories.     

Promoting science and technology in primary education: a review of integrated curricula

Integrated curricula seem promising for the increase of attention on science and technology in primary education. A clear picture of the advantages and disadvantages of integration efforts could help curriculum innovation. This review has focused on integrated curricula in primary education from 1994 to 2011. The integrated curricula were categorised according to a taxonomy of integration types synthesised from the literature. The characteristics that we deemed important were related to learning outcomes and success/fail factors. A focus group was formed to facilitate the process of analysis and to test tentative conclusions. We concluded that the levels in our taxonomy were linked to (a) student knowledge and skills, the enthusiasm generated among students and teachers, and the teacher commitment that was generated; and (b) the teacher commitment needed, the duration of the innovation effort, the volume and comprehensiveness of required teacher professional development, the necessary teacher support and the effort needed to overcome tensions with standard curricula. Almost all projects were effective in increasing the time spent on science at school. Our model resolves Czerniac’s definition problem of integrating curricula in a productive manner, and it forms a practical basis for decision-making by making clear what is needed and what output can be expected when plans are being formulated to implement integrated education.     

Trust and perception related to information about biofuels in Belgium

Public knowledge regarding green energy is limited and there is a perceived lack of information, particularly in the case of biofuels. This study investigates the information seeking behaviour of fuel consumers with regard to biofuels, their preferences for particular elements and channels of information and their trust in information sources. The majority of respondents were interested in many elements of information, and particularly in the tax (dis)advantages associated with biofuels. They prefer to obtain information via newspapers and brochures. Scientists, environmental and consumer organizations and the government are perceived as the most trustworthy sources of information whilst word of mouth sources, journalists and the fuel sector are less well trusted. Four consumer clusters could be distinguished, on the basis of the perceived relative trustworthiness of information sources, and these clusters have significantly different socio-demographic characteristics and preferences for elements and channels of information.     

VET students’ integration of knowledge engaged with in school-based and workplace-based learning environments in the Netherlands

In vocational education, students learn in different school-based and workplace-based learning environments and engage with different types of knowledge in these environments. Students are expected to integrate these experiences and make meaning of them in relation to their own professional knowledge base. This study focuses both on what types of knowledge students learn in these environments and how they integrate these different types of knowledge. Individual and group interviews were conducted with students, teachers and workplace supervisors in a vocational programme in the Netherlands. Results show that students recognise the importance of vocational knowledge learned in school-based learning environments while they are in the workplace and vice versa, and continuously contextualise knowledge to make it applicable for new circumstances. Also, students learn differently at school due to their experiences in the workplace.     

Signal detection theory as a tool for successful student selection

Prediction accuracy of academic achievement for admission purposes requires adequate sensitivity and specificity of admission tools, yet the available information on the validity and predictive power of admission tools is largely based on studies using correlational and regression statistics. The goal of this study was to explore signal detection theory as a tool to extend the available information; signal detection theory allows for comparisons of selection outcomes on both group and individual levels and the development of tailor-made criteria for specific programmes and admission goals. We investigated who would or would not have been admitted applying specific criteria for each of three common admission tools, how many admitted students would fail and how many applicants who would have been successful would be rejected. Both comparisons at an individual level and the receiver operating characteristic curves at a group level revealed that scores obtained in a programme-specific matching programme and non-cognitive factors appear more valuable than regression statistics suggest when it comes to admitting applicants who will become successful students. Signal detection theory allows not only for admission-goal-specific and programme-specific fine-tuning of the content of admission tools, it also informs about the effects of criteria and thus allows the setting of criteria.