When Does Scale Anchoring Work? A Case Study

Providing information to test takers and test score users about the abilities of test takers at different score levels has been a persistent problem in educational and psychological measurement. Scale anchoring, a technique which describes what students at different points on a score scale know and can do, is a tool to provide such information. Scale anchoring for a test involves a substantial amount of work, both by the statistical analysts and test developers involved with the test. In addition, scale anchoring involves considerable use of subjective judgment, so its conclusions may be questionable. We describe statistical procedures that can be used to determine if scale anchoring is likely to be successful for a test. If these procedures indicate that scale anchoring is unlikely to be successful, then there is little reason to perform a detailed scale anchoring study. The procedures are applied to several data sets from a teachers’ licensing test.     

Is It Necessary to Make Anchor Tests Mini‐Versions of the Tests Being Equated or Can Some Restrictions Be Relaxed?

It is a widely held belief that anchor tests should be miniature versions (i.e., minitests), with respect to content and statistical characteristics, of the tests being equated. This article examines the foundations for this belief regarding statistical characteristics. It examines the requirement of statistical representativeness of anchor tests that are content representative. The equating performance of several types of anchor tests, including those having statistical characteristics that differ from those of the tests being equated, is examined through several simulation studies and a real data example. Anchor tests with a spread of item difficulties less than that of a total test seem to perform as well as a minitest with respect to equating bias and equating standard error. Hence, the results demonstrate that requiring an anchor test to mimic the statistical characteristics of the total test may be too restrictive and need not be optimal. As a side benefit, this article also provides a comparison of the equating performance of post-stratification equating and chain equipercentile equating.     

How Often Is the Misfit of Item Response Theory Models Practically Significant?

Standard 3.9 of the Standards for Educational and Psychological Testing ( 1999 ) demands evidence of model fit when item response theory (IRT) models are employed to data from tests. Hambleton and Han ( 2005 ) and Sinharay ( 2005 ) recommended the assessment of practical significance of misfit of IRT models, but few examples of such assessment can be found in the literature concerning IRT model fit. In this article, practical significance of misfit of IRT models was assessed using data from several tests that employ IRT models to report scores. The IRT model did not fit any data set considered in this article. However, the extent of practical significance of misfit varied over the data sets.     

A Note on Assessing the Added Value of Subscores

Brennan ( 2012 ) noted that users of test scores often want (indeed, demand) that subscores be reported, along with total test scores, for diagnostic purposes. Haberman ( 2008 ) suggested a method based on classical test theory (CTT) to determine if subscores have added value over the total score. According to this method, a subscore has added value if the corresponding true subscore is predicted better by the subscore than by the total score. In this note, parallel‐forms scores are considered. It is proved that another way to interpret the method of Haberman is that a subscore has added value if it is in better agreement than the total score with the corresponding subscore on a parallel form. The suggested interpretation promises to make the method of Haberman more accessible because several practitioners find the concept of parallel forms more acceptable or easier to understand than that of a true score. Results are shown for data from two operational tests.     

Prediction of Essay Scores From Writing Process and Product Features Using Data Mining Methods

Analysis of keystroke logging data is of increasing interest, as evident from a substantial amount of recent research on the topic. Some of the research on keystroke logging data has focused on the prediction of essay scores from keystroke logging features, but linear regression is the only prediction method that has been used in this research. Data mining methods such as boosting and random forests have been found to improve over traditional prediction methods such as linear regression in various scientific fields, but have not been used in the prediction of essay scores from keystroke logging features. This article first provides a review of boosting, which is a popular data mining method. The article then applies boosting to predict essay scores from a large number of keystroke logging features and other predictor variables from two real data sets.     

On the Choice of Anchor Tests in Equating

The choice of anchor tests is crucial in applications of the nonequivalent groups with anchor test design of equating. Sinharay and Holland (2006, 2007) suggested “miditests,” which are anchor tests that are content‐representative and have the same mean item difficulty as the total test but have a smaller spread of item difficulties. Sinharay and Holland (2006, 2007), Cho, Wall, Lee, and Harris (2010), Fitzpatrick and Skorupski (2016), Liu, Sinharay, Holland, Curley, and Feigenbaum (2011a), Liu, Sinharay, Holland, Feigenbaum, and Curley (2011b), and Yi (2009) found the miditests to lead to better equating than minitests, which are representative of the total test with respect to content and difficulty. However, these findings recently came into question as Trierweiler, Lewis, and Smith (2016) concluded, based on a comparison of correlation coefficients of miditests and minitests with the total test, that making an anchor test a miditest does not generally increase the anchor to total score correlation and recommended the continuation of the practice of using minitests over miditests. Their recommendation raises the question, “Should miditests continue to be considered in practice?” This note defends the miditests by citing literature that favors miditests and then by showing that miditests perform as well as the minitests in most realistic situations considered in Trierweiler et al. (2016), which implies that miditests should continue to be seriously considered by equating practitioners.     

Antioxidant activity of ethanol extract of rhizome ofPicrorhiza kurroa on indomethacin induced gastric ulcer during healing

Oral administration of ethanol extract of the rhizome ofPirorhiza kurroa at a dose of 20mg/kg body weight, for 10 consecutive days, was found to enhance the rate of healing on Indomethacin-induced gastric ulcer in rats, compared to the ulcerated group without treatment. The level of peroxidised lipid, in terms of thiobarbituric acid reactive species (TBARS), in gastric tissue, was increased in ulcerated rats which was restored to near normalcy on treatment with ethanol extract. The specific activity ofin vivo antioxidant enzymes, viz SOD and catalase and total tissue sulfhydryl (thiol) group, which were markedly decreased in ulcerated group, were found to be significantly elevated (p<0.05), on treatment with the above extract, at the specified dose, compared to the indomethacin—induced ulcerated group without any supporting treatment. The present study thus suggests that the ethanol extract of rhizome ofPicrorhiza kurroa, at the dose of 20mg/kg body weight, accelerated the healing of stomach wall of indomethacin induced gastric ulcerated rats by anin vivo free radical scavenging action.     

Too Simple to Be Useful: A Comment on Feinberg and Wainer (2014)

Feinberg and Wainer (2014) provided a simple equation to approximate/predict a subscore's value. The purpose of this note is to point out that their equation is often inaccurate in that it does not always predict a subscore's value correctly. Therefore, the utility of their simple equation is not clear.