Validation of a touch screen tablet assessment of early literacy skills and a comparison with a traditional paper-based assessment

ABSTRACT

Touch screen tablets are being increasingly used in schools for learning and assessment. However, the validity and reliability of assessments delivered via tablets are largely unknown. The present study tested the psychometric properties of a tablet-based app designed to measure early literacy skills. Tablet-based tests were also compared with traditional paper-based tests. Children aged 2–6 years (N?=?99) completed receptive tests delivered via a tablet for letter, word, and numeral skills. The same skills were tested with a traditional paper-based test that used an expressive response format. Children (n?=?35) were post-tested 8 weeks later to examine the stability of test scores over time. The tablet test scores showed high internal consistency (all α’s?>?.94), acceptable test-retest reliability (ICC range?=?.39–.89), and were correlated with child age, family SES, and home literacy teaching to indicate good predictive validity. The agreement between scores for the tablet and traditional tests was high (ICC range?=?.81–.94). The tablet tests provides valid and reliable measures of children’s early literacy skills. The strong psychometric properties and ease of use suggests that tablet-based tests of literacy skills have the potential to improve assessment practices for research purposes and classroom use.     

学生创新和实践能力考核在大学招生中的价值：美国“彩虹项目”的启示

“彩虹项目”是由美国大学委员会资助的大学招生研究项目,其目的是探索研究大学招生中考核学生创新和实践能力的必要性。研究结果表明测评这两项能力在大学招生中具有极其重要的意义和价值。增加这项评价有利于对申请学生的能力进行更加全面的考核,不仅可以提高对大学GPA的预测效度,而且可以削减由于弱势群体和其他学生在大学入学标准考试中的不同表现所造成的招生偏差,使招生更具有卓越性和公平性。本文扼要介绍和阐述“彩虹项目”研究的理论基础、考核创新和实践能力的内容、考试结果对大学GPA的预测效度以及在美国塔夫茨大学的实践经验。     

The Effect of Test Translation on the Performance of Second Grade English Learners on the KeyMath‐3

This study was conducted to determine whether Spanish‐enhanced administration of a standardized math assessment would result in improved scores for English Learners who used Spanish as a heritage language. Twenty‐one typically developing second‐graders (English Learners) were administered the traditional KeyMath‐3. If the child made an error on an item, a Spanish version of the item was presented. Difference scores were calculated to determine whether the Spanish‐enhanced version resulted in improved scores. Data were analyzed using paired t‐tests and simple regression. The data results showed that all children significantly benefited from the Spanish‐enhanced administration of items answered incorrectly in English. The amount of benefit was predicted by a child's degree of Spanish dominance. It was concluded that standardized math tests that do not accommodate second‐language learners may be inadvertently testing language skills in addition to math skills. Implications for assessment and interpretations of assessments are discussed.     

The assessment of learning skills in nursing student selection: A scoping review

Abstract

Higher education student selection has significant societal, institutional and individual impacts. Thousands of applicants apply only for nursing, one of the major higher education disciplines. As the nursing profession is characterised by cognitive requirements, higher education institutions assess the learning skills of nursing applicants. However, there has been no comprehensive analysis of learning skills assessment for nursing student selection. The purpose of this scoping review was to describe the assessment of learning skills in undergraduate nursing student selection. Five databases were systematically searched, and 24 studies published between 2006 and 2016 were included. Learning skills were most commonly assessed using standardised tests in the areas of language and communication, reasoning, mathematics and natural sciences. Overall scores of onsite selection methods were found to best predict future academic performance. The results indicate that higher education institutions may benefit from comprehensive assessment of learning skills in their selection processes. This assessment should focus on a wider range of cognitive aptitudes, including reasoning skills. This review focussed on nursing education, but the results may benefit other higher education disciplines due to the generic nature of learning skills and similar cognitive requirements of higher education studies. The results support the development of more comprehensive and valid methods for assessing learning skills.     

The Application of the Multiple Assessment to Oral English Teaching

Instruction and assessment are two important aspects of modern education.Based on the rationales of multiple means of assessment,this paper explores the necessity of a shift from oral tests to the assessment of speaking skills through multiple measures.It also discusses how to implement multiple means of assessment in the regular oral English teaching.     

Assessing comprehension during reading with the Reading Strategy Assessment Tool (RSAT)

Comprehension emerges as the results of inference and strategic processes that support the construction of a coherent mental model for a text. However, the vast majority of comprehension skills tests adopt a format that does not afford an assessment of these processes as they operate during reading. This study assessed the viability of the Reading Strategy Assessment Tool (RSAT), which is an automated computer-based reading assessment designed to measure readers’ comprehension and spontaneous use of reading strategies while reading texts. In the tool, readers comprehend passages one sentence at a time, and are asked either an indirect (“What are your thoughts regarding your understanding of the sentence in the context of the passage?”) or direct (e.g., why X?) question after reading each pre-selected target sentence. The answers to the indirect questions are analyzed on the extent that they contain words associated with comprehension processes. The answers to direct questions are coded for the number of content words in common with an ideal answer, which is intended to be an assessment of emerging comprehension. In the study, the RSAT approach was shown to predict measures of comprehension comparable to standardized tests. The RSAT variables were also shown to correlate with human ratings. The results of this study constitute a “proof of concept” and demonstrate that it is possible to develop a comprehension skills assessment tool that assesses both comprehension and comprehension strategies.     

Educational assessment in Norway

Norway has seen major changes in the field of educational assessment over the past decade, following the 2001 ‘PISA shock’ that stimulated reform of the entire primary and secondary education systems: new outcome-based curricula with cross-disciplinary basic skills were accompanied by major revision of assessment regulations, comprehensive government projects promoting formative assessment, national tests as a main component in a new national quality assessment system and new regulations for examinations and teacher reporting of overall achievement marks. The paper provides a historical context to the country’s prohibition of formal marking in primary education and the recent tensions determining how assessment criteria should be stated and used for formative and summative purposes. It is argued that Norwegian primary and secondary education is riddled with unresolved tensions as to the role of assessment criteria and national tests, sparked by incremental implementation of assessment policies and principles accompanying the new outcomes-based curricula.     

大学英语教学表现性评价的测评新取向

近年来在大学英语教学领域,对教学效果的考评,已经从传统的纸笔测试方式,转变为科学的、优异的表现性评价。这种评价方式,从静态上说它全面地涉及受业者的理论、知识、技能的各个层面;从动态上说它具体地展示了受业者结合实战、运用策略、解决问题、掌握理论知识,形成技能的状态。实践表明,表现性评价是科学的优异的测评方式。     

Visual stress theory and its application to reading and reading tests

This paper presents a theory of visual stress. The theory is applied to the assessment of symptoms of visual stress and its treatment with coloured filters. The theory has implications for standard reading assessments that relate both to the visual skills and the age of the children taking the tests. These implications are reviewed, with the conclusion that insufficient attention is paid to visual factors in reading, particularly in the design of reading tests.     

电大护理学专业毕业实习考核方法初探

成功的护理人才是学历、技能皆具备的护理人员。电大开放教育专科护理学专业毕业实习成绩的评定分医院成绩和终结性考核成绩两部分。医院和学校联合对技能操作与护理运用双重测评,对于强化学生理论知识,加强护理技能,提高护理水平有非常重要的意义。终结考核在医院展开,可以带动临床护理的规范性,促使学校教学相长     

Test-taking skills: a missing component of deaf students' curriculum

Often, conclusions about what students have learned in school are based on their performance on standardized or informal tests. For several reasons, these tests may not accurately reflect deaf students' learning. The author discusses the limitations of tests and other "products" administered to students during or after reading that are interpreted as reflecting comprehension of what is read. The author also reviews documented differences in test-taking abilities of deaf and hearing students, and describes specific compensatory test-taking strategies used by deaf readers. A rationale is provided for including a formal test-taking skills component in the curriculum for deaf students. Finally, the author discusses portfolio assessment and contemporary societal forces working against academic testing.     

Trends in the assessment of infants and toddlers with disabilities.

Public Law 99-457 requires a multidisciplinary assessment and evaluation of children with disabilities in five domains: cognitive development, physical development, language and speech development, psychological development, and self-help skills; and this assessment must include information required in the Individualized Family Service Plan. A variety of assessment instruments are available, and some of the most notable are surveyed in this article. Traditional tests, as well as more current, naturalistic assessments, are reviewed. The author cautions against using one test for all purposes, or using a test that has an insufficient research base.     

Rapid Assessment of Learners’ Proficiency: A cognitive load approach

To tailor instruction to levels of learner proficiency in a domain in adaptive learning environments, rapid on‐line methods of assessing learner knowledge and skills are required. Cognitive studies indicate that major factors in expert performance are organised structures in the person’s knowledge base. Measuring different levels of acquisition of such knowledge structures should be a major purpose of cognitive diagnostic assessment. Traditional tests may not provide reliable evidence for diagnostic purposes and are not suitable for dynamic learning environments. This paper discusses an alternative diagnostic approach based on monitoring immediate traces of students’ knowledge structures in working memory. Results of an experimental study applying this method to assessment of user reading skills are described. The study demonstrated a sufficiently high level of correlation (.66) between obtained measures and traditional test scores, and a substantial increase in reliability, with testing time reduced by a factor of 3.8.     

Adapting the Assessing British Sign Language Development: Receptive Skills Test into American sign language

Signed languages continue to be a key element of deaf education programs that incorporate a bilingual approach to teaching and learning. In order to monitor the success of bilingual deaf education programs, and in particular to monitor the progress of children acquiring signed language, it is essential to develop an assessment tool of signed language skills. Although researchers have developed some checklists and experimental tests related to American Sign Language (ASL) assessment, at this time a standardized measure of ASL does not exist. There have been tests developed in other signed languages, for example, British Sign Language, that can serve as models in this area. The purpose of this study was to adapt the Assessing British Sign Language Development: Receptive Skills Test for use in ASL in order to begin the process of developing a standardized measure of ASL skills. The results suggest that collaboration between researchers in different signed languages can provide a valuable contribution toward filling the gap in the area of signed language assessment.     

Designing a formative e‐assessment: Latent class analysis of early reading skills

Computer‐based testing, or e‐assessment, has the potential to deliver immediate results for the benefit of schools. This paper describes a project that aimed to exploit this potential by designing e‐assessments where the results were intended for use by teachers in planning the next steps in teaching and learning: low‐stakes, formative assessment. A pair of tests assessing early reading was developed, with the test items based on a range of distinct skills, including phonological segmentation, rhyming and word recognition. The tests were administered on a screen with the questions presented aurally and visually. In trials, a total of 1345 test results were obtained from pupils aged 5–7 years from 26 schools. Latent class analysis was used to identify patterns of performance within the data. Four latent classes were distinguished, each characterised by a pattern of responses related to the different test items. The strengths and weaknesses in early reading skills implied by each of these latent classes were described in terms of formative ‘profiles’ provided for teachers in an online reporting package together with indicators for the next steps in teaching. The research resulted in an automated marking and analysis system that can be genuinely formative.     

Advantages of authentic assessment for improving the learning experience and employability skills of higher education students: A systematic literature review

This study is a systematic review of literature that investigates the advantages of implementing authentic assessment for improving two major categories that are relevant to academic and professional success of higher education students: learning experience and employability skills. Authentic assessment involves students in challenging tasks that closely resemble those of the workplace settings. 26 papers from 2010 to 2019 that were relevant to the topic of this paper were selected for the review process. Findings of this review indicate that authentic assessment can play a role in improving the learning experience of higher education students through enhancing their engagement in learning and improving their satisfaction as well as positively influencing their efforts to achieve educational goals. We also discuss the benefits of authentic assessment for equipping students with essential skills for their future professional life, such as communication skills, collaboration skills, critical-thinking and problem-solving skills, self-awareness, and self-confidence.     

Effects of explicit instruction on the acquisition of students’ science inquiry skills in grades 5 and 6 of primary education

In most primary science classes, students are taught science inquiry skills by way of learning by doing. Research shows that explicit instruction may be more effective. The aim of this study was to investigate the effects of explicit instruction on the acquisition of inquiry skills. Participants included 705 Dutch fifth and sixth graders. Students in an explicit instruction condition received an eight-week intervention of explicit instruction on inquiry skills. In the lessons of the implicit condition, all aspects of explicit instruction were absent. Students in the baseline condition followed their regular science curriculum. In a quasi-experimental pre-test–post-test design, two paper-and-pencil tests and three performance assessments were used to examine the acquisition and transfer of inquiry skills. Additionally, questionnaires were used to measure metacognitive skills. The results of a multilevel analysis controlling for pre-tests, general cognitive ability, age, gender and grade level indicated that explicit instruction facilitates the acquisition of science inquiry skills. Specifically on the performance assessment with an unfamiliar topic, students in the explicit condition outperformed students of both the implicit and baseline condition. Therefore, this study provides a strong argument for including an explicit teaching method for developing inquiry skills in primary science education.     

Does calibration reduce variability in the assessment of accounting learning outcomes?

Reliable, consistent assessment process that produces comparable assessment grades between assessors and institutions is a core activity and an ongoing challenge with which universities have failed to come to terms. In this paper, we report results from an experiment that tests the impact of an intervention designed to reduce grader variability and develop a shared understanding of national threshold learning standards by a cohort of reviewers. The intervention involved consensus moderation of samples of accounting students’ work, with a focus on three research questions. First, what is the quantifiable difference in grader variability on the assessment of learning outcomes in ‘application skills’ and ‘judgement’? Second, does participation in the workshops lead to reduced disparity in the assessment of the students’ learning outcomes in ‘application skills’ and ‘judgement’? Third, does participation in the workshops lead to greater confidence by reviewers in their ability to assess students’ skills in application skills and judgement? Our findings suggest consensus moderation does reduce variability across graders and also builds grader confidence.