Editorial: Kompetenzmodelle zur Erfassung individueller Lernergebnisse und zur Bilanzierung von Bildungsprozessen

This special issue comprises eleven articles from the DFG priority program “Competence Models for Assessing Individual Learning Outcomes and Evaluating Educational Processes”. The special issue aims at showing the priority program’s present state of discussion concerning essential concepts and questions related to the modeling of competencies. The first article describes the structure and research areas of the program and gives an overview on the current state of research. The short articles two to eleven discuss questions and problems which proved to be of particular importance during the work of the priority program in the last years. These articles constitute the result of a workshop of the participating scientists of the priority program in terms of position papers.     

The academic progress of foreign students: study achievement and study behaviour

In this study we investigated associations between studying in a foreign language and students' academic achievement and study behaviour. Data were gathered in the Netherlands, at the Delft University of Technology (DUT). The results show that the success rate of foreign students was about the same as that of the Dutch students, but that foreign students needed more time to pass the examination. Language proficiency played a major part in explaining differences in study achievement as did the presence or absence of a selection procedure in the student's country of origin. Furthermore, the results show that the study behaviour of foreign students was significantly different from the Dutch students: their average grades on examinations were lower, they made more attempts before passing an examination, they were more likely to postpone examinations and to follow a different order from the recommended one. Suggested measures to improve foreign students' study success include requiring higher levels of language proficiency and stricter selection of foreign students in the country of origin. It is suggested that a system of mentors which provides for intensive coaching of foreign students in their first year may prevent a slow start.     

How to use preconceptions? The contact strategy dismantled

The effects of the contact strategy (a computer-assisted instructional strategy aimed at conceptual change in text processing) were investigated by dismantling the strategy. An experiment with 86 Ss (5th/6th-graders) was conducted in which the number of instructional steps was cumulatively varied from 0 steps (no activation) over 1 step (search for old idea) and 3 steps (1. search for old idea; 2. compare and contrast with new information; 3. formulate new idea) to all 5 contact steps (1. search for old idea; 2. compare and contrast with new information; 3. formulate new idea; 4. apply new idea; 5. evaluate new idea). A design with 2 between-subjects factors (instructional strategy and students familiarity with the central concepts from the 7 instructional texts used) and 2 within-subjects factors (type of learning performance test item and time of testing) was used. Dependent variables concerned quality of final conceptions and learning performance. Results indicated that the complete contact strategy was the most effective variant. It seemed to be the case, however, that students mainly focused their attention on the central concepts from the texts. In our view, instructional strategies to foster conceptual change should both support knowledge restructuring processes and offer a solution for this problem of selective attention.     

In search of quality criteria in peer assessment practices

With the increasing popularity of peer assessment as an assessment tool, questions may arise about its measurement quality. Among such questions, the extent peer assessment practices adhere to standards of measurement. It has been claimed that new forms of assessment, require new criteria to judge their validity and reliability, since they aim for specific goals and address different aspects of learning. But although new criteria have been formulated, little is known about how they are deployed and operate in actual peer assessment practices. This study intends to explicate the quality criteria relevant to the context of peer assessment. We conducted a survey in vocational education on peer assessment quality criteria applied in classrooms. Fifty-six teachers using peer assessment rated several quality criteria as relevant to their teaching practice. The findings suggest that peer assessment practices entail many of the quality criteria recognized in measurement and evaluation, although in an embedded way: the generic quality criteria are tuned or adapted to the peer assessment setting. The findings also show that peer assessment is very much in the hands of the teacher, who organizes and orchestrates the appraisal process in which students receive clear guidelines to appraise one another's work. Our results indicate that quality criteria hardly differ from the function peer assessment serves in classrooms, either within a summative or formative evaluation context. By gaining insight on the specific criteria of peer assessment and their precise purposes, we believe it becomes possible to help teachers improve its measurement quality.     

Developer,teacher, student and employer evaluations of competence-based assessment quality

This study examines how different stakeholders experience the quality of a nationally developed assessment framework for summative, competence-based assessment (CBA) in AVET, which aims to reflect theoretical characteristics of high quality CBAs. The quality of two summative CBAs, based on this national framework, is evaluated along an extensive, validated set of quality criteria for CBA evaluation and through involving key stakeholders (i.e., students, teachers, developers, and employers). By triangulating quantitative and qualitative evaluations and argumentations of key stakeholders, this study gives insight into the processes and characteristics that determine CBA quality in VET educational practice in relation to theoretical notions of high quality CBAs. Results support many theoretical characteristics and refine them for reaching quality in actual assessment practice. Strikingly, developers and teachers are more critical about the assessment quality than students and employers. The discussion reflects on the theoretical CBA characteristics in the light of the empirical findings and deduces practical implications for the national assessment framework as well as other summative CBAs in VET.     

überzeugungen zur Verwendung des Wissens aus Lernstandserhebungen Eine professionstheoretische Analyse

Mit Lernstandserhebungen wird in das Bildungssystem ein outputorientiertes Steuerungsinstrument implementiert, das Steuerungswissen dezentral auf Schulen und Lehrkr?fte verteilt. Damit wird beabsichtigt, empirisches Wissen für professionelles Handeln und Schulentwicklung bereit zu stellen. Die Verwendung der Lernstandserhebungen ist dabei ihrerseits von Professionalit?t abh?ngig. Wie die Moderationsfunktion der Professionalit?t zwischen wissenschaftlichem und praktischem Wissen von den Lehrkr?ften ausgefüllt wird, ist empirisch noch weitgehend unbeantwortet. Diesen Prozessen n?her zu kommen, ist Ziel des hier vorgestellten explorativen Pilotprojekts. Mit einem qualitativen Fallstudiendesign wird ein heuristisches Auswertungsinstrument zur Erfassung der professionellen überzeugungen von Lehrkr?ften zur Verwendung von Lernstandserhebungen vorgestellt. Anhand beispielhafter Interviewausschnitte erfolgt eine Plausibilisierung, um es für weiterführende Studien nutzbar zu machen. Im theoretischen Rahmen finden systemtheoretische Professions-und Organisationstheorien Verwendung.     

Argumentation-Based Computer Supported Collaborative Learning (ABCSCL): A synthesis of 15 years of research

Learning to argue is an essential objective in education; and online environments have been found to support the sharing, constructing, and representing of arguments in multiple formats for what has been termed Argumentation-Based Computer Supported Collaborative Learning (ABCSCL). The purpose of this review is to give an overview of research in the field of ABCSCL and to synthesize the findings. For this review, 108 publications (89 empirical studies and 19 conceptual papers) on ABCSCL research dating from 1995 through 2011 were studied to highlight the foci of the past 15 years. Building on Biggs’ (2003) model, the ABCSCL publications were systematically categorized with respect to student prerequisites, learning environment, processes, and outcomes. Based on the quantitative and qualitative findings, this paper concludes that ABCSCL environments should be designed in a systematic way that takes the variety of specific conditions for learning into account. It also offers suggestions for educational practice and future research.     

Does Self-directedness in Learning and Careers Predict the Employability of Low-Qualified Employees?

Employability has become a key element in sustaining successful vocational careers. The role of self-directedness is considered paramount in maintaining one’s employability. However, it also requires certain competences on part of employees to invest in learning and career development. This study examines the influence of self-directedness in learning and career of low-qualified employees on their employability. In a follow-up study of 284 low-qualified employees, we find that higher levels of self-directedness in learning and career of employees corresponds with higher chance to be promoted to higher-level job positions (vertical job mobility). However, no relationship was found between different formats of self-directedness and job retention or horizontal job mobility of lower qualified personnel.