Affordances of Item Formats and Their Effects on Test‐Taker Cognition under Uncertainty

The current study investigated how item formats and their inherent affordances influence test‐takers’ cognition under uncertainty. Adult participants solved content‐equivalent math items in multiple‐selection multiple‐choice and four alternative grid formats. The results indicated that participants’ affirmative response tendency (i.e., judge the given information as True) was affected by the presence of a grid, type of grid options, and their visual layouts. The item formats further affected the test scores obtained from the alternatives keyed True and the alternatives keyed False, and their psychometric properties. The current results suggest that the affordances rendered by item design can lead to markedly different test‐taker behaviors and can potentially influence test outcomes. They emphasize that a better understanding of the cognitive implications of item formats could potentially facilitate item design decisions for large‐scale educational assessments.     

Automated Test‐Form Generation

In automated test assembly (ATA), the methodology of mixed‐integer programming is used to select test items from an item bank to meet the specifications for a desired test form and optimize its measurement accuracy. The same methodology can be used to automate the formatting of the set of selected items into the actual test form. Three different cases are discussed: (i) computerized test forms in which the items are presented on a screen one at a time and only their optimal order has to be determined; (ii) paper forms in which the items need to be ordered and paginated and the typical goal is to minimize paper use; and (iii) published test forms with the same requirements but a more sophisticated layout (e.g., double‐column print). For each case, a menu of possible test‐form specifications is identified, and it is shown how they can be modeled as linear constraints using 0–1 decision variables. The methodology is demonstrated using two empirical examples.     

Rater Agreement in Test‐to‐Curriculum Alignment Reviews

During the development of large‐scale curricular achievement tests, recruited panels of independent subject‐matter experts use systematic judgmental methods—often collectively labeled “alignment” methods—to rate the correspondence between a given test's items and the objective statements in a particular curricular standards document. High disagreement among the expert panelists may indicate problems with training, feedback, or other steps of the alignment procedure. Existing procedural recommendations for alignment reviews have been derived largely from single‐panel research studies; support for their use during operational large‐scale test development may be limited. Synthesizing data from more than 1,000 alignment reviews of state achievement tests, this study identifies features of test–standards alignment review procedures that impact agreement about test item content. The researchers then use their meta‐regression results to propose some practical suggestions for alignment review implementation.     

A Flexible Latent Class Approach to Estimating Test‐Score Reliability

The latent class reliability coefficient (LCRC) is improved by using the divisive latent class model instead of the unrestricted latent class model. This results in the divisive latent class reliability coefficient (DLCRC), which unlike LCRC avoids making subjective decisions about the best solution and thus avoids judgment error. A computational study using large numbers of items shows that DLCRC also is faster than LCRC and fast enough for practical purposes. Speed and objectivity render DLCRC superior to LCRC. A decisive feature of DLCRC is that it aims at closely approximating the multivariate distribution of item scores, which might render the method suited when test data are multidimensional. A simulation study focusing on multidimensionality shows that DLCRC in general has little bias relative to the true reliability and is relatively accurate compared to LCRC and classical lower bound methods coefficients α and λ2 and the greatest lower bound.     

项目反应理论测验信度及其研究述评

项目反应理论下的测验信度能够评价潜在特质估计的可靠性与稳定性,由于具有宏观性的特点,项目反应理论信度的作用并不能被测验信息函数所取代,是IRT测验的一个重要指标。本文参考国内外文献,首先介绍国内外学者关于IRT信度作用的观点,并介绍和评价了多种IRT信度估计方法,然后简要介绍IRT信度的影响因素,最后展望了IRT信度领域后续研究尚可着力之处。     

The Benefits of Fixed Item Parameter Calibration for Parameter Accuracy in Small Sample Situations in Large‐Scale Assessments

Large‐scale assessments such as the Programme for International Student Assessment (PISA) have field trials where new survey features are tested for utility in the main survey. Because of resource constraints, there is a trade‐off between how much of the sample can be used to test new survey features and how much can be used for the initial item response theory (IRT) scaling. Utilizing real assessment data of the PISA 2015 Science assessment, this article demonstrates that using fixed item parameter calibration (FIPC) in the field trial yields stable item parameter estimates in the initial IRT scaling for samples as small as n = 250 per country. Moreover, the results indicate that for the recovery of the county‐specific latent trait distributions, the estimates of the trend items (i.e., the information introduced into the calibration) are crucial. Thus, concerning the country‐level sample size of n = 1,950 currently used in the PISA field trial, FIPC is useful for increasing the number of survey features that can be examined during the field trial without the need to increase the total sample size. This enables international large‐scale assessments such as PISA to keep up with state‐of‐the‐art developments regarding assessment frameworks, psychometric models, and delivery platform capabilities.     

Benefits of omega‐3 supplementation for schoolchildren: Review of the current evidence

Despite the potential impact nutrition may have on learning, there have been surprisingly few papers published directed towards the educational research community. In contrast, omega‐3 supplementation studies are being frequently cited in the media, leading to parents asking for advice and guidance. The purpose of this article is to review the evidence to date for any effect of using omega‐3 supplementation in school‐aged children. This article focuses on the research that has been undertaken, particularly in relation to behaviour, education and cognitive development, in both typically developing populations as well as in children with specific learning difficulties and developmental disorders. Recommendations for future studies in this area have been highlighted in view of current knowledge. In conclusion, it was found that there is a shortage of properly controlled omega‐3 supplementation trials, particularly with typically developing children, to advocate the supplementation of all children with omega‐3 fatty acids, but due to the known importance of omega‐3 fatty acids in the brain and early development, further research is required.     

The Benefits of Part‐Time Undergraduate Study and UK Higher Education Policy: a Literature Review

Part‐time study in the UK is significant: nearly 40 per cent of higher education students study part‐time. This article reports on a literature review that sought to understand the economic and social benefits of part‐time study in the UK. It concludes that there are substantial and wide‐ranging benefits from studying part‐time. The article also aims to place the discussion in the current policy context by drawing attention to the fact that while part‐time study is seen as important for increasing the global competitiveness of the UK economy, expansion of higher education has tended to focus on the young, full‐time student; furthermore, part‐time study is less generously resourced compared to full‐time study. New policy pronouncements made in 2009 appear to recognise these policy contradictions, which state that most future growth will be in provision other than the full‐time, 3‐year undergraduate degree. Indeed, the Government's independent review of fees has recognized that parity of funding is an issue and its recommendations on part‐time study have been endorsed by the government.     

思想政治教育过程基本矛盾研究现状与发展探索

思想政治教育过程基本矛盾的研究大致有＂教育者与受教育者＂说、＂社会要求与受教育者＂说和＂施教系统和受教系统＂说等几种代表性观点,它们各有其理论价值,但也不同程度地存在不足。探寻思想政治教育过程的基本矛盾,需要正确认识其理论地位,明确思想政治教育过程基本矛盾研究的思路和方法,并实现对德育过程基本矛盾的理论范式的超越。     

Modeling Student Test‐Taking Motivation in the Context of an Adaptive Achievement Test

This study examined the utility of response time‐based analyses in understanding the behavior of unmotivated test takers. For the data from an adaptive achievement test, patterns of observed rapid‐guessing behavior and item response accuracy were compared to the behavior expected under several types of models that have been proposed to represent unmotivated test taking behavior. Test taker behavior was found to be inconsistent with these models, with the exception of the effort‐moderated model. Effort‐moderated scoring was found to both yield scores that were more accurate than those found under traditional scoring, and exhibit improved person fit statistics. In addition, an effort‐guided adaptive test was proposed and shown by a simulation study to alleviate item difficulty mistargeting caused by unmotivated test taking.     

投票悖论研究

逻辑悖论没有被哲学家们禁锢在理论的藩篱中,相反,它成为我们日常生活中亟需解决的难题。从逻辑悖论的概念和类型入手,分析投票悖论中的公认正确的背景知识,运用自然推理演绎系统进行严密无误的逻辑推导并建立矛盾等价式,从而确证投票悖论是严格的逻辑悖论。     

投票悖论研究

逻辑悖论没有被哲学家们禁锢在理论的藩篱中,相反,它成为我们日常生活中亟需解决的难题。从逻辑悖论的概念和类型入手,分析投票悖论中的公认正确的背景知识,运用自然推理演绎系统进行严密无误的逻辑推导并建立矛盾等价式,从而确证投票悖论是严格的逻辑悖论。     

对双生子佯谬问题的探讨

双生子佯谬的提出,是假定地球和火箭是两个完全等价的惯性系,这个前提是明显错误的,其实无论从地球,还是火箭角度出发利用狭义相对论的理论,都推导出火箭上的孪生兄弟比地球上的孪生兄弟要年轻。     

Detecting Item Drift in Large‐Scale Testing

The early detection of item drift is an important issue for frequently administered testing programs because items are reused over time. Unfortunately, operational data tend to be very sparse and do not lend themselves to frequent monitoring analyses, particularly for on‐demand testing. Building on existing residual analyses, the authors propose an item index that requires only moderate‐to‐small sample sizes to form data for time‐series analysis. Asymptotic results are presented to facilitate statistical significance tests. The authors show that the proposed index combined with time‐series techniques may be useful in detecting and predicting item drift. Most important, this index is related to a well‐known differential item functioning analysis so that a meaningful effect size can be proposed for item drift detection.     

Test–retest reliability of the parent behavior importance questionnaire‐revised and the parent behavior frequency questionnaire‐revised

This study evaluated the test–retest reliability of two parenting measures: the Parent Behavior Importance Questionnaire‐Revised (PBIQ‐R) and Parent Behavior Frequency Questionnaire‐Revised (PBFQ‐R). These self‐report parenting behavior assessment measures may be utilized as pre‐ and post‐parent education program measures, with parents as well as nonparent respondents. The questionnaires are based on the parent development theory, with the parenting behaviors corresponding to theory and current parenting literature. Thus, respondents' relative weighting of importance (PBIQ‐R) or frequency (PBFQ‐R) of positive, supportive parenting as well as negative behaviors may be determined through questionnaire responses. Test–retest reliability estimates suggest psychometric strength. Results are discussed relative to parenting theory and research, as well as school psychology policy and practice. © 2011 Wiley Periodicals, Inc.     

Investigating the Effect of Item Position in Computer‐Based Tests

Computer‐based tests (CBTs) often use random ordering of items in order to minimize item exposure and reduce the potential for answer copying. Little research has been done, however, to examine item position effects for these tests. In this study, different versions of a Rasch model and different response time models were examined and applied to data from a CBT administration of a medical licensure examination. The models specifically were used to investigate whether item position affected item difficulty and item intensity estimates. Results indicated that the position effect was negligible.     

Extended Mixed‐Effects Item Response Models With the MH‐RM Algorithm

A mixed‐effects item response theory (IRT) model is presented as a logical extension of the generalized linear mixed‐effects modeling approach to formulating explanatory IRT models. Fixed and random coefficients in the extended model are estimated using a Metropolis‐Hastings Robbins‐Monro (MH‐RM) stochastic imputation algorithm to accommodate for increased dimensionality due to modeling multiple design‐ and trait‐based random effects. As a consequence of using this algorithm, more flexible explanatory IRT models, such as the multidimensional four‐parameter logistic model, are easily organized and efficiently estimated for unidimensional and multidimensional tests. Rasch versions of the linear latent trait and latent regression model, along with their extensions, are presented and discussed, Monte Carlo simulations are conducted to determine the efficiency of parameter recovery of the MH‐RM algorithm, and an empirical example using the extended mixed‐effects IRT model is presented.     

The Multiple True-False Item Format: A Status Review

How do multiple true-false items differ from other item formats? What does past research indicate about the quality of multiple true-false items? What additional research is needed?     

Scale Alignment in Between‐Item Multidimensional Rasch Models

Scores estimated from multidimensional item response theory (IRT) models are not necessarily comparable across dimensions. In this article, the concept of aligned dimensions is formalized in the context of Rasch models, and two methods are described—delta dimensional alignment (DDA) and logistic regression alignment (LRA)—to transform estimated item parameters so that dimensions are aligned. Both the DDA and LRA methods are applied to real and simulated data, and it is demonstrated that both methods are broadly effective for achieving aligned scales. The routine use of scale alignment methods is recommended prior to comparing scores across dimensions.     

The Test‐Taking Strategy Intervention for College Students with Learning Disabilities

Limited research exists related to empirically validated strategies to assist college students with learning disabilities (LD). Given that students with LD demonstrate both fewer test‐taking skills and higher levels of test anxiety than their peers without LD, and poor test‐taking skills contribute to higher levels of test anxiety, such research is critical. The present study examines the effectiveness of the test‐taking strategy on test performance (timed/untimed), degree of strategy usage, and time on test‐taking task, with a sample of university students with LD. This strategy has been successful with adolescents with LD, but has not been studied with postsecondary populations. Results of a multiple baseline design suggested that the strategy was an effective intervention for these students. Implications are discussed.