Separating PIAAC competencies from general cognitive skills: A dimensionality and explanatory analysis

This study aims to investigate how test scores from PIAAC (Programme for the International Assessment of Adult Competencies) can be interpreted, by comparing the PIAAC competencies literacy and numeracy to reasoning and perceptual speed. Dimensionality analyses supported, that the PIAAC competencies can be separated into a common factor overlapping with reasoning and perceptual speed, and domain-specific factors. For the common and specific factors, relations to other variables were analyzed. The nested factor for PIAAC literacy was as expected unrelated to age, positively related to learning opportunities during one’s lifetime, and positively related to literacy skill use. The nested factor for PIAAC numeracy was also as expected unrelated to age, against expectation unrelated to learning opportunities during one’s lifetime, and as expected positively related to numeracy skill use. Results support the validity of the intended test score interpretation for PIAAC literacy, while results for PIAAC numeracy were less clear.     

Gender differences in adult numeracy skills: what is the role of education?

Previous research suggests that as societies empower women educationally, gender differences in numeracy skills will decline. Using direct measures of 56,142 adults’ numeracy skills from the Programme for International Assessment for Adult Competencies (PIAAC), this article studies whether this claim is evidenced across 20 Organisation for Economic Co-operation and Development (OECD) countries. Results show that in many countries, as education has equalised over generations, gender differences in adult numeracy skills have indeed declined. However, women’s advances in education have not always been matched by a reduction in the gender difference in adult numeracy. Contrary to expectations, gender differences in fields of study in further and higher education cannot systematically explain gender differences in adult numeracy. This suggests that to achieve gender equality in numeracy skills, societies must do more than empower women educationally. More research is needed on the educational policies, contextual, and life course factors contributing to gender differences in adult numeracy in post-industrial societies.     

Assessing the basic skills of the highly educated in 21 OECD countries: an international benchmark study of graduates’ proficiency in literacy and numeracy using the PIAAC 2012 data

This study compares the literacy and numeracy proficiencies of higher education (HE) degree holders in 21 OECD countries based on primary analysis of the national data sets collected via the OECD’s Programme for the International Assessment of Adult Competencies (PIAAC) 2012 study. The differences in the graduates’ average literacy and numeracy proficiencies amongst the OECD countries are substantial. Depending on the country, a smaller or greater proportion of a young highly educated age group does not have sufficient skills in literacy or numeracy to cope with many of the everyday tasks requiring the use of that skill. The PIAAC study challenges existing evaluation practices of the effectiveness of HE in fostering individual skills and puts into perspective the attempts to lift national average skill levels by increasing the HE sector’s intake.     

Adult numeracy and the totally pedagogised society: PIAAC and other international surveys in the context of global educational policy on lifelong learning

This paper aims to discuss the emergence, form and likely effects of international surveys of adults’ skills by locating them in the global context of policies on education and Life Long Learning (LLL). It focuses on adults’ numeracy and discusses its conceptualisation and assessment in the Project for the International Assessment of Adult Competencies (PIAAC), which is the most recent survey. Drawing on critical theoretical resources about new forms of governance in education and transformations in the pedagogic discourse, the paper further substantiates existing critiques of global policy trends, namely that they are motivated by human capital approaches to education and LLL. In particular, we show that the apparently commonsensical appeal of evaluative instruments like PISA and PIAAC is based on a competency model of knowledge, which embodies a narrow notion of competence. Relatedly, the notional curricula promoted by such surveys potentially articulate a more radical idea of LLL, captured by Bernstein’s conception of trainability as the mode of socialisation into a Totally Pedagogised Society. The paper presents a dual approach to understanding international adult performance surveys in general—in that, besides deploying the theoretical resources already indicated, it also raises a number of methodological issues relevant to the valid interpretation of these studies’ results. Ultimately, it argues for the importance of mobilising resources from critical educational perspectives to support the development of potentially powerful knowledge like numeracy and to prevent its being reduced to a narrow competency.     

Getting closer to the digital divide: An analysis of impacts on digital competencies based on the German PIAAC sample

This paper takes an intersectional perspective to investigate the effect of socio-demographic variables that may constitute to digital divide. The concept of digital divide emerged from a perspective on unequal access to digital technology and relates nowadays primarily the differences in the competencies necessary to handle this technology. To investigate digital divide, the present paper uses the PIAAC framework of digital competencies which is called problem solving in technology-rich environments (PS-TRE). It introduces the approach of intersectionality that describes persons impaired by multiple inequalities.The paper analyzes the impact of these factors on PS-TRE for three subsamples of the German study: (1) employed people who use computers at work and at home, (2) employed people who use computers only at home, and (3) people that are out of the labor force. It analyzes furthermore contributions to digital divide by a comparison of these impacts with literacy and numeracy scores.While employed people with computer use at work and home only had generation as a factor for constituting digital divide, employed people with computer use only at home had migration background as a further factor. Education and cultural capital showed lower impacts on PS-TRE than on literacy and numeracy.     

Returns on vocational education over the life cycle: Between immediate labour market preparation and lifelong employability

An important issue in the design of secondary-level education is the balance between conveying general and occupation-specific (vocational) skills. On the one hand, vocationally oriented programmes, providing occupation-specific skills with immediate labour market relevance, have repeatedly been shown to secure safe pathways into employment. On the other hand, these programmes tend to put less emphasis on developing general knowledge, skills and competencies, including numeracy and literacy, which are foundational to lifelong learning. Hence, when the needs of the labour market change, employees who opted for a vocational track when they were at secondary school risk being less flexible in adapting to such changes later in their career. The authors of this article examine whether this results in a trade-off between short-term gains and long-term losses by considering differences in the labour market careers of vocationally and generally educated respondents in the 2012 Programme for the International Assessment of Adult Competencies (PIAAC). Their results suggest that early labour market benefits of vocational specialisation decrease over time; the authors relate this to its lower ability to equip secondary school students – future employees – with skills for lifelong learning.     

A comparative analysis of national media responses to the OECD Survey of Adult Skills: policy making from the global to the local?

The Organization for Economic Co-operation and Development’s (OECD) Programme of International Assessment of Adult Competencies (PIAAC) is put forward as a landmark development in the lifelong monitoring and international comparison of education. The first round of PIAAC’s Survey of Adult Skills compared performance in literacy, numeracy and problem solving in technology-rich environments across 24 countries. However, the translation of any OECD agenda into national policies is mediated by many actors, including the media. This paper examines and compares how the national media of Japan, England and France reported on the PIAAC results of their countries and the extent to which these reports mirror key messages from the OECD’s Country Notes. It begins to trace how the OECD PIAAC agendas materialise into national policies. Although their role in this initial period was limited, we argue the roles of the media together with other policy actors must be monitored as they interact to shape possibilities for sustainable adult education policies.     

Can competencies assessed by PISA be considered the fundamental school knowledge 15-year-olds should possess?

The OECD Programme for International Student Assessment (PISA), launched by governments of the Organisation for Economic Co-operation and Development (OECD) in 1997, aims at assessing some of the key competencies that contribute to the success of 15-year-old individuals, on a regular basis and within an internationally accepted framework. PISA seeks to provide a basis for policy dialogue and for collaboration in defining and implementing educational goals, in innovative ways that reflect judgements about the skills that are relevant to adult life. PISA defines competence as the ability to successfully meet complex demands in varied contexts through the mobilisation of psychosocial resources, including knowledge and skills, motivation, attitudes, emotions, and other social and behavioural components. Measuring and comparing competencies across languages and cultures is a difficult challenge and is being pursued by PISA progressively. PISA focused its first three assessments on literacy skills, defined as the capacity of young adults to access, manage, integrate and evaluate information, to think imaginatively, to hypothesise and discover, and to communicate their thoughts and ideas effectively. The reasoning behind shifting the emphasis from assessing whether students can reproduce what they have learned towards whether they can extrapolate from what they have learned and apply their competencies in novel situations, derives from the nature of knowledge and skills required in modern life. For example, the tasks that can be solved through simple memorisation or with pre-set algorithms are those that are also easiest to digitise, automatise and offshore, and will thus be less relevant in a modern knowledge society. Since there is no overarching cross-national and cross-cultural agreement on what fundamental competencies 15-year-olds should possess, an international assessment such as PISA can only capture a selection of competencies. Moreover, since various methodological constraints limit the nature of competencies that are currently amenable to large scale assessment, PISA cannot capture the entirety of competencies that will make young people successful. However, the findings presented in this article suggest that the competencies that PISA does assess are highly predictive for the future success of students. In addition, PISA provides policy makers and practitioners with useful tools to improve quality, equity and efficiency in education, by revealing some common characteristics of students, schools and education systems that do well. In a modern world, comparative assessments are an essential tool for educational improvement and research shows that the existence of standardised assessments and examinations is one of the most powerful predictors for the success of an education system. That is not hard to understand, because without such assessments, all students, schools and education systems look the same, it is impossible for teachers and school administrators to detect institutional and systemic strengths and weaknesses, and to support and intervene where expectations are not met. Without reliable and comparable information on learning outcomes, teachers and governments alike rely on input-based incentives and policies that are all too often mirrored in large quality variation between schools as well as a strong dependency between learning success and the socio-economic context of students and schools. Last but not least, it is important to keep in mind that the absence of the reading, mathematical and scientific competencies measured by PISA does not automatically imply the presence of all those important competencies that have not been measured.     

Influence of firm size on the competencies required to management engineers in the Jordanian telecommunications sector

The objective of this study is to identify the competencies required to achieve success in the transition from higher education to the labour market based on the perceptions of employers. This paper analyses the assessments made by a group of engineering company employers. An item-battery of 20 competencies was grouped into 3 dimensions by using factor analysis. Subsequently, respondents’ scores were also clustered into three groups and characterised through contingency tables. The competencies demanded by employers were grouped into business and finance, problem-solving and strategic planning. Significant differences were found between responses from employers working in medium and small companies, who placed more importance on competencies related to problem-solving and strategic planning, and employers in big companies, who were more concerned about the difficulties of finding well-trained graduates. The findings from this paper have important implications for research in the areas of higher education and organisations that usually employ graduate engineers.     

Good Practice in Science Teaching: What Research has to Say (2nd edn)

The scientific competencies advocated by the Programme for International Student Assessment (PISA) focus on the abilities needed in students' adult lives. This study investigated how such scientific competencies could be improved by using online argumentation. One hundred and thirty-eight 8th grade high school students took part in the study, with 69 in the experimental group and 69 in the control group. A quasi-experimental design was adopted and qualitative and quantitative analyses were used. An online argumentation system served as an aid for argumentation instruction and activities among experimental group students during the experiment. The results showed that using online argumentation could improve the students' scores for the PISA scientific competencies. The experimental group students outperformed their counterparts in terms of overall mean scores for the scientific competencies. On the one hand, the individual competencies of ‘using scientific evidence’ and ‘identifying scientific issues’ of the experimental group were higher than those of the control group. On the other hand, the experimental group students did not outperform their counterparts in terms of competency in ‘explaining phenomena scientifically’. Using an online environment to complement argumentation instruction and organizing argumentation activities focused on related topics may be a potential direction to consider for improving students’ PISA scientific competencies.     

Education,age and skills: An analysis using PIAAC data

The main aim of this article is to analyse the change of adult skills, as captured by cognitive skills assessed in PIAAC, across age cohorts, taking into account that the quality of schooling may change from one cohort to another. We estimate a model that relates numeracy and literacy skills to age, schooling, gender and variables related to both family background and labour market performance. The specification allows us to control for changes in the efficiency of the transformation of schooling into skills when drawing age‐skill profiles. Our results show that the effect of ageing on skills, once isolated from cohort effects related to schooling, decreases monotonically across consecutive cohorts. The change of the efficiency of the transformation of schooling into both numeracy and literacy skills shows a remarkably similar pattern. Nonetheless, this change differs substantially between education levels, with the efficiency of the transformation of schooling into skills showing a steadier profile for intermediate than for higher education. Finally, empirical evidence is provided for the decomposition of the differences in the skill levels of the older vs. the prime age generations. The results suggest that the progressive expansion of schooling across younger generations partially offsets the negative effect of the irrepressible ageing of society on skills.     

Examining problem-solving skills in technology-rich environments as related to numeracy and literacy

ABSTRACT

This study sought to a better understanding of the construct of problem solving in technology-rich environments and the effect of literacy and numeracy on problem solving. Data used in this study were drawn from Programme for the International Assessment of Adult Competencies US data which includes 5010 completed cases and a total of 1326 variables. The assessment of literacy, numeracy and problem-solving competencies were administrated using computer-based approaches. The result of the study showed that adults with higher numeracy and literacy competencies were more likely to have higher level of problem-solving skills. The results of the analyses also revealed that solution latency (i.e. time) were an important factor influencing problem-solving skills. This study indicates that basic mathematical skills are essential for solving problems that require interpersonal communication, computer and software knowledge, planning, and organising. The findings from this study provide several implications for researchers, educators, teachers and policymakers.     

The effects of post-compulsory education and training systems on literacy and numeracy skills: A comparative analysis using PISA 2000 and the 2011 survey of adult skills

This article analyses the contribution of post-compulsory education and training systems to the development of literacy and numeracy skills across OECD countries. While there is extensive cross-country comparative research on the effects of primary and lower secondary education systems on aggregate skills levels, there has been little comparative analysis of system effects after the end of lower secondary education. This article uses a quasi-cohort analysis of the tested literacy and numeracy skills of 15-year-olds in PISA 2000 and 27-year-olds in the 2011 OECD Survey of Adult Skills (SAS) to estimate the gains in different countries in mean levels of competence in literacy and numeracy. We found that Nordic countries (Norway and Sweden) with comprehensive upper secondary education and training systems and German-speaking countries (Austria and Germany) with dual systems of apprenticeship were particular effective, whilst countries with mixed systems (England, Ireland, Northern Ireland and Spain) showed a relative decline in both literacy and numeracy. The education system characteristics that account for these differences are (a) the inclusiveness – as proxied by high rates of participation at 17/18 and low social gradients of level 3 completion; (b) the esteem of vocational programmes; and (c) curriculum standardisation with regard to the study of maths and the national language.     

Using PIAAC data to learn more about the literacy practices of adults

ABSTRACT

The present article connects a secondary analysis of quantitative data from the Programme for the International Assessment of Adult Competencies (PIAAC) with the theoretical approach of ‘literacy practices’ and related research results from the so-called New Literacy Studies (NLS) tradition, which follows a cultural practices paradigm.

According to the literacy as social practice approach, the analysis of adults’ literacy and numeracy practices could provide relevant policy information about how to address target groups in adult literacy and basic education. Thus, a Latent Class Analysis was carried out with the German PIAAC dataset in order to differentiate the adult population by their uses of literacy, numeracy and ICT.

As a result of this procedure, three subgroups of adults can be distinguished by the frequency in which they use selected skill-related activities. Surprisingly, an adult’s individual literacy level does not clearly predict group membership. A further interesting result is that participants in one of the groups seem to compensate for the few chances they have to use their skills at work by using them more often in their everyday life. Both results contribute to the need to draw a more differentiated picture of adults with lower literacy skills.     

The OECD and the expansion of PISA: new global modes of governance in education

This paper examines the expansion of the OECD's Programme for International Student Assessment (PISA) and associated growth in the influence of the OECD's education work. PISA has become one of the OECD's most successful ‘products’ and has both strengthened the role of the Directorate for Education within the organization and enhanced the significance of the organization in education globally. We provide an overview of the OECD, including organizational changes in response to globalization and the changing place of the Directorate for Education within the organization, particularly with the development of PISA in the late 1990s. We show how the OECD is expanding PISA by broadening the scope of what is measured; increasing the scale of the assessment to cover more countries, systems and schools; and enhancing its explanatory power to provide policy‐makers with better information. The OECD has also developed the Programme for International Assessment of Adult Competencies (PIAAC) and PISA‐based Tests for Schools, which draw on the PISA template to extend the influence of its education work to new sites. The paper draws on data from 33 interviews with past and present personnel from the OECD, the International Association for the Evaluation of Educational Achievement (IEA) and the English and Australian education systems, as well as analysis of relevant OECD documents. We argue that PISA, and the OECD's education work more broadly, has facilitated new epistemological and infrastructural modes of global governance for the OECD in education.     

经济合作与发展组织的三大国际教育测试研究

除了PISA(国际学生评估项目)以外,经济合作与发展组织还研发了关注教师专业发展和学校教学的TAMS项目(国际教学调查项目)、调查高等教育学生学习情况的AHELO项目(高等教育学习成果测评)和对16～65岁成人进行能力评估的PIAAC(国际成人能力测评项目)。这些国际教育测试分别针对不同的教育人群进行能力评估,以更好地使各国政府了解自身的教育状况,在世界范围内进行跨文化比较,为教育的发展制定更加科学合理的教育政策。     

‘To be numerate is to be someone…’: Tracing the doings of students labelled ‘at risk’

Scores from the Australian National Assessment Program—Literacy and Numeracy (NAPLAN) identify students ‘at risk’ of not meeting minimum standards deemed necessary for future success in school and employment. The NAPLAN tests include items related to numeracy but also mathematics content and skills. Research in the area of mathematics education examining the effectiveness of pedagogical interventions in improving student scores on NAPLAN and other international measures is not only shaped by the standardised testing regime, it also effectively corrals the problem within the school context. As such, it is unable to answer questions related to other factors implicated in the lives of those who continue to ‘fail’ in relation to numeracy outcomes. This paper critically examines the type of funded research being done in relation to numeracy and mathematics education, the ‘social’ turn and the disconnect between this research and the widening ‘gap’ in NAPLAN numeracy outcomes. It argues for a research approach informed by institutional ethnography that begins with the ‘doings’ of individual students labelled ‘at risk’.     

国际成人能力评估项目的述评——基于OECD首次成人技能调查结果的分析

国际成人能力评估项目（PIAAC）是世界经济合作与发展组织（OECD）2013年第一次在24个国家和地区开展的针对成人能力问题而进行的较大规模调查。其目的是了解当今各国成人能力（技能）的现状,成人能力之间的相互影响;工作、生活中成人能力的运用情况以及成人能力与经济、劳动力市场和社会的关系,从而为各国制定教育及经济政策提供依据。这项调查以经济全球化为背景,以信息通信技术为主线,提出了成人关键信息处理能力,即读写能力、计算能力、问题解决能力;通用能力,如合作能力、自我管理能力等,调查了这些能力的现状及分布情况,以及在工作中成人能力应用的状况。这项调查还从社会人口学的视角,研究了教育程度、性别、移民与成人能力的相互关系,探讨了成人能力与社会效益的问题。PIAAC不仅为我们提供了大量、翔实的调查数据和分析,也为国际成人教育研究提供了科学研究成人能力的量表。PIAAC的贡献对于国际成人能力研究具有奠基和里程碑式的作用。     

Linking via Pseudo-Equivalent Group Design: Methodological Considerations and an Application to the PISA and PIAAC Assessments

This article presents the pseudo-equivalent group approach and discusses how it can enhance the quality of linking in the presence of nonequivalent groups. The pseudo-equivalent group approach allows to achieve pseudo-equivalence using propensity score reweighting techniques. We use it to perform linking to establish scale concordance between two assessments. The article presents Monte-Carlo simulations and a real data application based on data from the Survey of Adult Skills (PIAAC) and the Programme for International Student Assessment (PISA). Monte-Carlo simulations suggest that the pseudo-equivalent group design is particularly useful whenever there is a large overlap across the two groups with respect to balancing variables and when the correlation between such variables and ability is medium or high. The example based on PISA and PIAAC data indicates that the approach can provide reasonable accurate linking that can be used for group-level comparisons.     

Association between mothers' education and grade six children numeracy and literacy in Kenya

The objective of the paper is to re-examine the mother–child education achievement hypothesis, by re-examining the effect of mother's education, on math and literacy test scores of children in Kenya. Data come from the classroom Education Research Programme at the African Population and Health Research Centre which was collected between January and March 2012. Since pupils are nested within schools, we fitted a two-level random intercept model. Our findings show that mothers' and fathers' education has a positive and significant independent association with literacy and numeracy achievement. After interacting mothers' and fathers' education and controlling for school and pupil characteristics, we observed two significant findings: (1) mother's education remains statistically significant but is negatively associated with the pupil's score in both literacy and numeracy; and (2) the interaction of both parents' education is significant and positively associated with pupil scores in literacy and numeracy. This study underscores the importance of the complementarity between mothers' and fathers' education in order for children to acquire and learn literacy and numeracy in schools. In as much as mothers' education is important in the children's literacy and numeracy, the importance of fathers in children's literacy and numeracy cannot be ignored.