The Measurement of Adult Literacy in Theory and in Practice

The International Adult Literacy Survey (IALS) was designed as a vehicle to directly measure the functional literacy skills of adults across different countries. This survey was conducted as a household-based survey whereby a random sample of adults in participating countries was selected and administered a literacy skill assessment. The IALS theoretical framework was founded upon a solid base of research and statistical theory. Proven statistical methods such as Item Response Theory and Plausible Values Methodology were used in the development and analysis of the literacy test. Likewise, from the practical viewpoint, the IALS has been successfully administered in the household-survey setting. Several quality assurance steps have shown to be effective in this regard. Finally, the resultant national literacy skill profiles have provided useful literacy indicators at both government and individual levels.     

International Adult Literacy Survey (IALS): An analysis of international comparisons of adult literacy

The International Adult Literacy Survey (IALS) raises a number of important issues that are inherent in all attempts to make comparisons of cognitive and behavioural attributes across countries. This article discusses both the statistical and interpretational problems. A detailed analysis of the survey instruments is carried out to demonstrate the cultural specificity involved. The data modelling techniques used in IALS are critiqued and alternative analyses performed. The article argues for extreme caution in interpreting results in the light of the weaknesses of the survey.     

The IALS approach to defining and measuring literacy skills

The term “illiterate” that is commonly used to describe someone who cannot read or write is an absolute term, which fails to recognize that those who are deemed “illiterate” are probably in fact “literate” to some degree. The approach taken to define and measure literacy in the International Adult Literacy Survey (IALS) acknowledges the limitation of viewing literacy in a dichotomous way. The IALS assesses literacy proficiency along a continuum. This chapter considers some of the conceptual issues surrounding the definition and measurement of literacy proficiency. In particular, it addresses what the IALS literacy proficiency measure is and what it is not.     

The International Adult Literacy Survey: What Does It Really Measure?

The paper evaluates the work of the International Adult Literacy Survey as reported in OECD 1997. It assesses its contribution to understanding literacy in terms of the perspective of the New Literacy Studies. It outlines this perspective as a basis for a critique that is mostly concerned with the validity of the test. Three criticisms of the survey are made: that it provides only a partial picture of literacy; that culture is treated as bias; and that the test items do not represent the real-life items as claimed. Finally, the paper concludes with an overall evaluation of what the IALS achieves in terms of its own aims.     

The International Adult Literacy Survey: What Does It Really Measure?

The paper evaluates the work of the International Adult Literacy Survey as reported in OECD 1997. It assesses its contribution to understanding literacy in terms of the perspective of the New Literacy Studies. It outlines this perspective as a basis for a critique that is mostly concerned with the validity of the test. Three criticisms of the survey are made: that it provides only a partial picture of literacy; that culture is treated as bias; and that the test items do not represent the real-life items as claimed. Finally, the paper concludes with an overall evaluation of what the IALS achieves in terms of its own aims.     

Knowledges of adult literacy: surveying for competitiveness

The International Adult Literacy Survey (IALS) is now the standard for assessing population literacy levels in industrialized countries. It uses elegant conceptual and methodological procedures to construct a `literacy' as universal across individuals and situations, and to assess individual levels of literacy ability. Central among these procedures is the conception of reading ability as information-processing skills underlying the performance of diverse documentary tasks in a `knowledge society' that ubiquitously demands these tasks. This conception builds into the knowledge of literacy a managerial or `competitive' standpoint, interested in abilities to perform prespecified tasks within ruling social relations. In further requiring that for individuals to be ranked at a given level, they must be able to `sight read' any task at that level of information-processing complexity, IALS makes tested literacy the counterpart of `flexibility' as a labour force attribute. Projects for literacy with democratic objectives require a broader conception, encompassing not only how courses of reading activity are done, but also how reading operates as moments of extended courses of action, including those aimed to develop people's capacities to control what they do.     

Accumuler le capital humain tout au long de la vie: Quels facteurs prédictifs de la participation à la formation permanente?

This article addresses the main predictors of adult education, using statistical methods different from those generally used by social science researchers. Its aim is twofold. First, it seeks to explain in a simple and comprehensible manner the methodological value of these methods (in relation to the use of structural models); secondly, it demonstrates the concrete usefulness of these methods on the basis of a recent piece of research on the data from the International Adult Literacy Survey (IALS).     

Smaller,quicker, cheaper: alternative strategies for literacy assessment in the UN Literacy Decade

In the context of the UN Literacy Decade (declared in February 2003), the present paper suggests three parameters that should be considered when new tools for assessment are considered in less developed countries (LDCs), each of which poses a special challenge to international comparative literacy assessment, such as in the International Adult Literacy Survey (IALS):

• •Smaller: Assessment methods do not need to be major entrepreneurial enterprises, but rather just robust enough to answer key policy questions at the national and local levels. International comparative studies often run counter to this perspective.
• •Quicker: Literacy assessments need to be completed in ‘real time’ so that results can affect policy and spending in the ‘lifetime’ of current ministerial appointments. Studies that take 3–5 years to generate results, even if robust, nonetheless fail to meet the test of timeliness.
• •Cheaper: LDCs cannot afford either the fiscal or human resources costs of deep involvement in highly technical assessment exercises. The higher the cost, the more difficult to get to an initial ‘yes’ to participate in such an exercise, and the more difficult to gather time-series data to follow policy decisions.

In sum, this paper finds that there is a very important need for improving literacy assessment methodologies and the empirical database in developing countries, especially in light of the new UN Literacy Decade. While the IALS presents interesting and important options for methodological consideration, it also has a number of inherent limitations as discussed herein. Other options exist which should also be considered, especially for poor countries, such as the smaller/quicker/cheaper (SQC) approach.     

A “Nordic model” of adult education: What might be its defining parameters?

Judging by their literacy proficiency scores, Nordic countries stand out from others. Their consistently high scores are intriguing and make their populations interesting benchmarks for other countries that participated in the International Adult Literacy Survey. This article addresses the question of whether there are any specific ‘Nordic’ ways of planning and implementing adult education policies. Are there any features that define a common approach to adult education, one that sets the Nordic countries apart from other advanced regions in Europe and North America?Beyond the general pattern, specific sub-groups of the population are explored, and especially the groups ‘at-risk’, i.e. those that score low on literacy proficiency scales, have the least education, are old or unemployed. All Nordic countries are included in the analysis: Denmark, Finland, Norway, and Sweden participated in the original IALS survey, whereas Iceland collected comparable data on adult education participation in a separate survey.That Nordic countries have a comparatively high level of participation in adult education is a fact that leaves no room for discussion. However, if not only the rate of participation but also volume is considered then the Nordic countries appear more similar to others. What sets the Nordic countries apart is the level of public support for adult education for the low-skilled population. More generally, it would seem that public support for disadvantaged groups is the main defining characteristic of Nordic countries.     

Adult Literacy: Monitoring and Evaluation for Practice and Policy

With the advent of the UN Literacy Decade launched in 2003, there is increased emphasis on monitoring and evaluation of adult literacy around the globe. The present paper presents an overview of the main approaches that have been taken to measure adult literacy within and across countries. A particular focus of the present review is to compare and contrast three models of literacy assessment: (1) the “traditional” model based largely on census enumeration, which has been used over recent decades to collect national and regional rates of illiteracy; (2) the large-scale survey techniques employed with the International Adult Literacy Survey and similar models; and (3) an intermediate type of assessment that borrows from the previous two models, and attempts to “tailor” the size and complexity of the survey methodology to the policy questions needing answers (called the SQC model). The present paper suggests that there is no single model or set of methods that are universally appropriate to monitoring and measuring in adult literacy around the world, but that blending both programmatic and comparative models through the SQC approach may bring greater involvement in, and insight into, adult literacy evaluations.     

Using PIAAC data to learn more about the literacy practices of adults

ABSTRACT

The present article connects a secondary analysis of quantitative data from the Programme for the International Assessment of Adult Competencies (PIAAC) with the theoretical approach of ‘literacy practices’ and related research results from the so-called New Literacy Studies (NLS) tradition, which follows a cultural practices paradigm.

According to the literacy as social practice approach, the analysis of adults’ literacy and numeracy practices could provide relevant policy information about how to address target groups in adult literacy and basic education. Thus, a Latent Class Analysis was carried out with the German PIAAC dataset in order to differentiate the adult population by their uses of literacy, numeracy and ICT.

As a result of this procedure, three subgroups of adults can be distinguished by the frequency in which they use selected skill-related activities. Surprisingly, an adult’s individual literacy level does not clearly predict group membership. A further interesting result is that participants in one of the groups seem to compensate for the few chances they have to use their skills at work by using them more often in their everyday life. Both results contribute to the need to draw a more differentiated picture of adults with lower literacy skills.     

The framework used in developing and Interpreting the International Adult Literacy Survey (IALS)

This paper offers a framework that has been used for both developing the tasks used to measure literacy in the International Adult Literacy Survey and for understanding the meaning of what has been reported with respect to the comparative literacy proficiencies of adults in participating countries. The framework consists of six parts that represent a logical sequence of steps from needing to define and represent a particular domain of interest, to identifying and operationalizing characteristics used to construct items, to providing an empirical basis for interpreting results. The various parts of the framework are seen as important in that they help to provide a deeper understanding of the construct of literacy and the various processes associated with it. A processing model is proposed and variables associated with performance on the literacy tasks are identified and verified through regression analyses. These variables are shown to account for between 79 and 89 percent of the variance in task difficulty. Collectively, these process variables provide a means for moving away from interpreting performance on large-scale surveys in terms of discrete tasks or a single number towards identifying levels of performance that have generalizability across pools of tasks and towards what Messick has called a higher level of measurement.     

Revisiting major determinants of participation in adult education with a direct measure of literacy skills

The purpose of this article is to investigate major determinants of participation in adult education. Specifically, a direct measure of literacy skills available in the International Adult Literacy Survey is included. Interpreted as a measure of human capital, it is expected that literacy skills are at least as important a predictor of participation in adult education and training as educational attainment. The findings however do not support this expectation. Instead educational attainment remains the most important factor predicting participation in adult education and training. The models in this article are based on the idea that readiness to learn is formed early in life and further developed through educational and work experiences. Factors that are hypothesised to influence participation in adult education and training are separated into factors associated with the long arm of the family and the long arm of the job. The findings indicate the long arm of the family plays an important role, which supports early intervention, especially during the formal schooling years. The results also highlight the strong link between the use of literacy skills at work and participation in adult education and training.     

Lessons from past literacy campaigns: A critical assessment

Former Director of the Centre for International Programmes, University of Guelph, Ontario (1974–85); of the Division of Adult Education (1972–74) and of the Literacy Division (1968–72), Unesco. He has considerable experience with adult education and literacy programmes in less developed countries, notably in South and South-East Asia.     

The Collective that Didn't Quite Collect: Reflections on the IALS

The concept and methodology of IALS obviously represent progress for literacy research. Nevertheless, this survey presents serious limitations. It prioritises the handling of data-based information as opposed to experiential and contextualized information, ranking at a higher position those who handle the first kind of information. The survey ignores how people solve problems in their life context. The competence of people cannot be understood merely through outside observation and a questionnaire situation. In the case of Poland, the interpretation of data manifests an ethnocentric Western attitude, tending to reduce cultural differences to a cultural deficit. Two IALS-inspired surveys conducted in Spain and Portugal mixed quantitative and qualitative methods in order to take into account the social relations in their cultural context. This article questions the role of research, which cannot be limited to the objectivation of reality, but should help to transform and improve it. The alternative model proposed is named "investigation action."     

The Vocational Turn in Adult Literacy Education and the Impact of the International Adult Literacy Survey

The authors critically examine some of the underlying epistemological and theoretical assumptions of the IALS. In doing so, they distinguish among two basic orientations towards literacy. First, the standard approach (of which IALS is an example) subscribes to the possibility of measuring literacy as abstract, cognitive skills, and endorses the claim that there is an important relationship between literacy skills and economic success in the so-called 'knowledge society.' The second, called a socio-cultural approach, insists on the contextual and power-related character of people's literacy practices. The authors further illustrate that the assumptions of the IALS are rooted in a neo-liberal ideology that forces all members of society to adjust to the exigencies of the globalised economy. In the current, contingent conditions of the risk society, however, it does not seem very wise to limit the learning of adults to enhancing labour-market competencies. Adult education should relate to the concrete literacy practices people already have in their lives. It should make its learners co-responsible actors of their own learning process and participants in a democratic debate on defining the kind of society people want to build.     

Using evidence and implementation experience for advocacy and policy influence: the Rwanda Emergent Literacy and Maths Initiative (ELMI) case study

This paper discusses applied research initiatives led by Save the Children in Rwanda that were used to inform advocacy and policy efforts surrounding the expansion of high quality early childhood education (ECE) services in Rwanda. Qualitative and quantitative studies were carried out around a new program focused on improving early literacy and math competencies (the Emergent Literacy and Maths Initiative (ELMI)) from 2013–2017. They include a Political Economy Analysis in 2013–2014 and longitudinal impact evaluations in 2014–2015 and 2015–2016. Together, the mixed-methods data helped to identify strengths and gaps in the Rwandan education system, potential solutions to improving ECE for all children in the country, and insights into best practices for taking successful pilot programs to scale in a low-income country. Changes in national policy and ECE service provision are discussed.     

Effects of early literacy environments on the reading attitudes, behaviours and values of veteran teachers

Research has linked early literacy environments to the attitudes, behaviours and instructional values of reading teachers, but most prior research has addressed preservice or early inservice teachers. This mixed-methods, hypothesis-generating, Q methodology-based study explored the relationship between early literacy environments and the subsequent reading experiences, attitudes and behaviours of a purposive sample of veteran literacy educators. By-person factor analysis identified three empirical profiles of shared reading experiences, attitudes and behaviours: Early Readers Supported by Nurturing Early Literacy Environments; Early Readers Not Supported by Nurturing Early Literacy Environments; and Later Readers Not Supported by Nurturing Early Literacy Environments. Conceptual variables found across all models using qualitative analysis included: Nature of Early Literacy Environments, Early and Later Competence and Appreciation of Reading, Attitudes about the Value of Reading, and Instructional Values and Behaviors in Literacy Education. These models suggest further evidence for the importance of early literacy environments (both home and classroom environments) and provide empirically-derived testable hypotheses for future research.     

The Collective that Didn't Quite Collect: Reflections on the IALS

The concept and methodology of IALS obviously represent progress for literacy research. Nevertheless, this survey presents serious limitations. It prioritises the handling of data-based information as opposed to experiential and contextualized information, ranking at a higher position those who handle the first kind of information. The survey ignores how people solve problems in their life context. The competence of people cannot be understood merely through outside observation and a questionnaire situation. In the case of Poland, the interpretation of data manifests an ethnocentric Western attitude, tending to reduce cultural differences to a cultural deficit. Two IALS-inspired surveys conducted in Spain and Portugal mixed quantitative and qualitative methods in order to take into account the social relations in their cultural context. This article questions the role of research, which cannot be limited to the objectivation of reality, but should help to transform and improve it. The alternative model proposed is named "investigation action."