Skill and will: test‐taking motivation and assessment quality

An achievement test score can be viewed as a joint function of skill and will, of knowledge and motivation. However, when interpreting and using test scores, the ‘will’ part is not always acknowledged and scores are mostly interpreted and used as pure measures of student knowledge. This paper argues that students’ motivation to do their best on the assessment – their test‐taking motivation – is important to consider from an assessment validity perspective. This is true not least in assessment contexts where the assessment outcome has no consequences for the test‐taker. The paper further argues that the quality of assessment of test‐taking motivation also needs attention. Theoretical and methodological issues related to the assessment of test‐taking motivation are presented from a validity perspective, and findings from empirical studies on the relation between test stakes, test‐taking motivation and test performance are presented.     

Change in test-taking motivation and its relationship to test performance in low-stakes assessments

Since the turn of the century, an increasing number of low-stakes assessments (i.e., assessments without direct consequences for the test-takers) are being used to evaluate the quality of educational systems. Internationally, research has shown that low-stakes test results can be biased due to students’ low test-taking motivation and that students’ effort levels can vary throughout a testing session involving both cognitive and noncognitive tests. Thus, it is possible that students’ motivation varies throughout a single cognitive test and in turn affects test performance. This study examines the change in test-taking motivation within a 2-h cognitive low-stakes test and its association with test performance. Based on expectancy-value theory, we assessed three components of test-taking motivation (expectancy for success, value, and effort) and investigated its change. Using data from a large-scale student achievement study of German ninth-graders, we employed second-order latent growth modeling and structural equation modeling to predict test performance in mathematics. On average, students’ effort and perceived value of the test decreased, whereas expectancy for success remained stable. Overall, initial test-taking motivation was a better predictor of test performance than change in motivation. Only the variability of change in the expectancy component was positively related to test performance. The theoretical and practical implications for test practitioners are discussed.     

The Validity of Value-added Estimates from Low-Stakes Testing Contexts: The Impact of Change in Test-Taking Motivation and Test Consequences

Accountability mandates often prompt assessment of student learning gains (e.g., value-added estimates) via achievement tests. The validity of these estimates have been questioned when performance on tests is low stakes for students. To assess the effects of motivation on value-added estimates, we assigned students to one of three test consequence conditions: (a) an aggregate of test scores is used solely for institutional effectiveness purposes, (b) personal test score is reported to the student, or (c) personal test score is reported to faculty. Value-added estimates, operationalized as change in performance between two testing occasions for the same individuals where educational programming was experienced between testing occasions, were examined across conditions, in addition to the effects of test-taking motivation. Test consequences did not impact value-added estimates. Change in test-taking motivation, however, had a substantial effect on value-added estimates. In short, value-added estimates were attenuated due to decreased motivation from pretest to posttest.     

多元智能理论下普职融通试点班学生评价研究

普职融通试点班的学生如果达到一定的标准,可以分流到与其结对的普通高中进行学习和生活。目前,分流的标准是依据学生的考试成绩,但单一的纸笔测验并不能真正反映出学生的综合素质和能力。为了促进学生的全面发展以及个性化发展,运用访谈法与文献分析相结合的方法,从多元智能理论的视角出发,得出普职融通试点班应采用可以量化的纸笔测验和基于学生表现的质性评价即档案袋评价相结合的方法进行学生评价的结论。     

The Effects of Motivational Instruction on College Students' Performance on Low-Stakes Assessment

Assessments of student learning outcomes (SLO) have been widely used in higher education for accreditation, accountability, and strategic planning purposes. Although important to institutions, the assessment results typically bear no consequence for individual students. It is important to clarify the relationship between motivation and test performance and identify practical strategies to boost students' motivation in test taking. This study designed an experiment to examine the effectiveness of a motivational instruction. The instruction increased examinees' self-reported test-taking motivation by .89 standard deviations (SDs) and test scores by .63 SDs. Students receiving the instruction spent an average of 14 more seconds on an item than students in the control group. Score difference between experimental and control groups narrowed to .23 SDs after unmotivated students identified by low response time were removed from the analyses. The findings provide important implications for higher education institutions which administer SLO assessments in a low-stakes setting.     

Enhancing language learning through creation: the effect of digital storytelling on student learning motivation and performance in a school English course

This study explored how a free-space digital storytelling approach that advocates autonomy and creativity can be implemented in a formal elementary classroom and how it impacts students’ language learning motivation and performances. Participants of the study were 64 sixth grade students in Taiwan. Following an experimental design, the data collected from three data sources, including motivation surveys, achievement test scores, and digital stories, were analyzed and triangulated. Two performance indicators of the digital storytelling, levels of language usage and levels of creativity, were found to have significant but different impacts on language learning. While the students’ language usage performance in digital storytelling was significantly related to their achievement test scores, their creativity performance were significantly related to multiple motivation components, including extrinsic motivation, task value and elaboration. It was also found that the proposed digital storytelling approach had a positive impact on students’ language performance and contributed to an increase to students’ motivation in two dimensions: extrinsic goal orientation and elaboration, rather than intrinsic goal orientation. The results suggest that the positive impact of the proposed storytelling pedagogy resides in allowing students to stretch their creativity while demonstrating their language productivity, with the leverage of a holistic assessment scheme.     

Measuring value‐added in higher education: conditions and caveats – results from using the Measure of Academic Proficiency and Progress (MAPP™)

The Commission on the Future of Higher Education in the USA emphasises accountability in higher education as one of its key areas of interest. A programme, called the Voluntary System of Accountability, was developed to evaluate the effectiveness of general public college education. This study examines how students progress in college, indicated by the performance difference between freshmen and seniors after controlling for admission scores, can be measured using the Measure of Academic Proficiency and Progress? (MAPP?) test. A total of 6196 students from 23 institutions was included in this study. Results indicated that MAPP was able to differentiate between the performance of freshmen and seniors after controlling for SAT^®/ACT scores. The institutions were classified into 10 groups on the basis of the difference in the actual versus expected MAPP performance. The assumptions and implications of linking student learning growth to institutional effectiveness are discussed. Methodological issues on value‐added calculation and student motivation in taking standardised tests are also noted.     

Strategies to Motivate Students for Program Assessment

Accountability in higher education has increased, with more institutions requiring standardized tests. These tests are high stakes for institutions, but low-stakes test for students, who seldom experience consequences for their performance. This study describes how one psychology department improved students' scores on the Psychology Area Concentration Achievement Test. Results were compared between three motivation conditions: no incentive, a monetary incentive, and a motivational Microsoft PowerPoint presentation. The presentation gave students information about the assessment, encouraged them to do well, and informed them that faculty would discuss scores while evaluating the psychology program. Results showed that test scores were significantly higher and correlated significantly with grade point average for students exposed to the motivational presentation. The motivational PowerPoint presentation seemed to have reduced the number of underachieving students and provided more accurate assessment data, with minimal investment in time and effort on the part of faculty.     

关于以学生体质测试促进体育教学的思考——以阜新高等专科学校为例

根据国家教育部对在校大学生体质测试的统一要求,各高校都进行了体质测试,但就目前的情况来看,在测试中存在很多问题。以阜新高等专科学校为例,最关键的问题是体质测试冲击和影响了正常的体育教学,这一问题亟待解决。要想使两者相互促进、协调发展,必须对学生体质测试进行统筹组织和安排,并保证科学实施,同时学校要高度重视体质测试工作。     

试论正确运用测试反拨作用的重要性

应试教育正向素质教育转轨,这就要求人们对考试在教育中的作用有一个正确的理解,使考试回归到它应有的位置上去．本文通过对一次考试成绩的分析,探讨了考试反拨作用的正确利用对指导学生学法、改进教学方法以及提高测试质量等方面的积极作用．     

ANXIETY AND ACHIEVEMENT IN A MULTICULTURAL SITUATION

Oral reading comprehension test scores are generally determined by a variety of factors, not only by task difficulty and examinee proficiency, but also by stimuli extraneous to the testing task itself — such as group vs. individual testing situation, scorer method and bias, and examinees’ sociolinguistic background, motivational disposition, and anxiety during the test.

By giving the same examination under the same testing conditions to students, teachers and administrators generally believe that they are thereby being fair in their testing practice. However, if the same test situation debilitates the motivation and performance of some groups while increasing the motivation and performance of others, the assumption of fairness in standardised test procedure may be only illusory. Indeed, while stimuli in the test situation appear to be held constant from the examiner's point of view, they may interact with background variables to differentially affect levels of test performance (Cole and Bruner, 1970).

Two types of anxiety effects have been differentiated in the literature, namely debilitative and facilitative effects (Alpert and Haber, 1960).

The present study sets out to examine the effects of the following factors on oral reading comprehension test scores: (a) anxiety under varying test conditions, (b) students’ gender, (c) students’ native language, (d) the testing situation (individual vs. group), and (e) the relative difficulty of the English language texts.

Two studies were conducted during an advanced English reading course at Haifa University to examine these issues. Study One, based on a sample of 170 students, was designed to examine the affective, cognitive, and sociolinguistic correlates within the test situation. No meaningful relationship was observed between students’ attitudes and test performance. Study Two, using 112 students, was undertaken to examine the roles of anxiety, sociolinguistic background, and text difficulty within the oral testing situation in greater depth. Test scores were affected by gender and ethnic affiliation but not by text difficulty, reported level of anxiety, or group vs. individual testing situation.     

Asian medical students: quality of life and motivation to learn

Issues linked with the notions of quality of life (QOL) and motivation to learn among Asian medical students have not been well documented. This is true in both the international and the New Zealand contexts. Our paper addresses this lack of research by focusing on the QOL of international and domestic Asian students studying in New Zealand, where Asian students form a significant proportion of tertiary students. Although there is evidence to suggest that Asian students do well academically, it was felt that an investigation into their QOL would be instructive as QOL will likely have an impact on cognition, behavior, general well-being, and motivation. The present study surveyed fourth- and fifth-year medical students to examine the relationship between QOL and motivation to learn and to consider how Asian medical students compare against European medical and non-medical student peers. The study utilized the World Health Organization—Quality of Life questionnaire (BREF version) and a shortened version of the Motivated Strategies for Learning Questionnaire. The results show that the Asian medical students in this study generated significantly lower scores in terms of their satisfaction with social relationships compared with their non-Asian peers. In addition, international Asian medical students appear to be more at risk than domestic Asian students with respect to test anxiety. The paper considers the findings and the implications for quality of life, motivation to learn, medical education, and the Asian student community.     

Do the Demographic Differences Manifest in Motivation to Learn Science and Impact on Science Performance? Evidence from Sri Lanka

The present study examined whether gender, ethnicity, instructional medium and school category differences manifest in science performance and motivation to learn science among secondary school students in Sri Lanka. The mean of five successive term test scores was used as the measure of science performance. Level of motivation in terms of six dimensions was measured by using the Science Motivation Questionnaire. A sample of 1316 grade 11 students representing Sinhala, Tamil, and Muslim ethnic groups from the three categories of public schools, which provide instructions in the vernacular languages of Sinhala or Tamil, participated in the study. Girls showed significantly higher performance in science compared to boys, and there was a significant gender difference in the levels of motivational dimensions in favor of girls. Although Tamil medium students possessed a higher level of motivation to learn science, Sinhala medium students outperformed their Tamil medium counterparts in science performance. Significant differences in science performance between Sinhala, Tamil, and Muslim students were also observed. However, motivation towards learning science between Tamils and Muslims was not significantly different. Highly significant differences in both motivation to learn science and performance in science were found between three categories of schools. The present study provides information to education officials who have to achieve equity across gender, ethnicity, medium of instruction, and school category, teachers who deliver the subject and school principals who design academic support programs.     

Effort Analysis: Individual Score Validation of Achievement Test Data

Whenever the purpose of measurement is to inform an inference about a student’s achievement level, it is important that we be able to trust that the student’s test score accurately reflects what that student knows and can do. Such trust requires the assumption that a student’s test event is not unduly influenced by construct-irrelevant factors that could distort his score. This article examines one such factor—test-taking motivation—that tends to induce a person-specific, systematic negative bias on test scores. Because current measurement models underlying achievement testing assume students respond effortfully to test items, it is important to identify test scores that have been materially distorted by non-effortful test taking. A method for conducting effort-related individual score validation is presented, and it is recommended that measurement professionals have a responsibility to identify invalid scores to individuals who make inferences about student achievement on the basis of those scores.     

A preliminary study of multiple college admission criteria in Taiwan: the relationship among motivation,standardized tests,high school achievements,and college success

ABSTRACT

A new college admission policy will be implemented in Taiwan in 2022. The purpose of this study was to understand the relationship between admission criteria and college success. Data was obtained from the Taiwan Higher Education Database; a sample size of 8443 students from 156 universities was used in this study. By using the structural equation model, this study tested a research model that included factors such as motivation, standardized test scores, high school achievements, and college success. The findings revealed that the General Scholastic Ability Test scores (in Chinese, English, Social Studies) and high school average academic grades are significantly associated with college success. A student’s motivation to complete a certain major can significantly predict the quality of student effort and influence college success. These findings highlight the importance of some admission criteria and provide practical implications for educational policy-makers, school administrators, students, and parents.     

An Examination of College Students' Problem‐Solving Self‐Efficacy,Academic Self‐Efficacy,Motivation, Test Performance,and Expected Grade in Introductory‐Level Economics Courses

This study investigated whether self‐efficacy influenced students’ educational outcomes in introductory‐level economics courses. First, this study investigated the correlations between problem‐solving self‐efficacy, academic self‐efficacy, and motivation. Second, this study investigated whether problem‐solving and academic self‐efficacy served as predictors of students’ motivation, test performance, and expected grade. Correlational analyses suggest that problem‐solving and academic self‐efficacy are correlated with student motivation. Results show that problem‐solving self‐efficacy was a predictor of student motivation and test performance. Academic self‐efficacy and problem‐solving self‐efficacy were also predictors of their expected grade. Implications and future directions for economics education are also discussed.     

Test motivation in the assessment of student skills: The effects of incentives on motivation and performance

There is widespread concern that assessments which have no direct consequences for students, teachers or schools underestimate student ability, and that the extent of this underestimation increases as the students become ever more familiar with such tests. This issue is particularly relevant for international comparative studies such as the IEA’s Third International Mathematics and Science Study (TIMSS) and the OECD’s Programme for International Student Assessment (PISA). In the present experimental study, a short form of the PISA mathematical literacy test is used to explore whether the levels of test motivation and test performance observed in the context of the standard PISA assessment situation can be improved by raising the stakes of testing. The impact of (1) informational feedback, (2) grading, and (3) performance-contingent financial rewards on the personal value of performing well, perceived utility of participating in the test, intended and invested effort, task-irrelevant cognitions, and test performance are investigated. The central finding of the study is that the different treatment conditions make the various value components of test motivation equally salient. Consequently, no differences were found either with respect to intended and invested effort or to test performance.     

More than just test scores

Around the world we hear considerable talk about creating world-class schools. Usually the term refers to schools whose students get very high scores on the international comparisons of student achievement such as PISA or TIMSS. The practice of restricting the meaning of exemplary schools to the narrow criterion of achievement scores is usually premised on the view that test scores are closely linked to the provision of a capable labour force and competitive economy. In fact, the measured relationships between test scores and earnings or productivity are modest and explain a relatively small share of the larger link between educational attainment and economic outcomes. What has been omitted from such narrow assessments are the effects that education has on the development of interpersonal and intrapersonal skills and capabilities that affect the quality and productivity of the labour force. This article provides evidence on some of these relationships, on the degree to which the non-cognitive effects of schooling contribute to adult performance, and on the evidence that deliberate school interventions can influence non-cognitive outcomes. It concludes with the view that the quest for world-class schools must encompass a range of human development characteristics that extend considerably beyond test scores.     

Raising the Stakes of Test Administration: The Impact on Student Performance on the National Assessment of Educational Progress

This article briefly reviews the current discussion of the effects of test administration conditions (i.e., testing stakes), and the motivational levels associated with them, on achievement test performance. The non-experimental study presented here investigates whether differences in test administration conditions and presumed levels of motivation engendered by different testing environments affect student performance on National Assessment of Educational Progress (NAEP) administrations. The testing conditions under study are the "low-stakes" environment of the current NAG administration and a higher stakes environment typified by many state assessment programs. The results suggest that in comparison to a "moderate-stakes" testing environment NAEP does not seriously underestimate achievement levels. However, the results cannot lead to the conclusion that student achievement is unrelated to testing stakes. Nor can one conclude that substantially raising the stakes of NAEP would not be accompanied by an increase in achievement scores.     

Chinese Education Examined via the Lens of Self-Determination

Chinese education is controversial: it is not only lauded for Chinese students’ high test achievements but also criticized for curbing students’ deep learning and development into well-rounded individuals. In the current paper, we propose that self-determination theory (SDT) serves as a useful framework for anatomizing Chinese educational ecology, especially understanding the fundamental developmental costs behind Chinese students’ high test scores. In the first part, we provide an up-to-date overview of SDT, which proposes that a growth-oriented motivation fueled by basic psychological needs underlies human development; hence, the role of education is to provide environmental support for these needs. After reviewing research evidence, we conclude that SDT serves as a valid theoretical framework for analyzing Chinese education. In the second part, we apply the lens of SDT to better understand the motivational dynamics that prevail in Chinese education. In doing so, we first primarily focus on the distal institutional level, thereby examining in detail how the high-stakes testing system headed by Gaokao fails to support—and may even thwart—basic psychological needs; we also address counterarguments favoring Gaokao, such as heightened involvement and alignment. We then scrutinize the pros and cons at the proximal level of the student environment—i.e., teachers and parents. Finally, we discuss existing reform attempts, which seemingly have very limited effectiveness. We propose that awareness of the problem and more holistic change are needed to realize more effective and sustainable change in Chinese education.