EDUCATIONAL DIAGNOSTIC ASSESSMENT

A strong demand currently exists for testing instruments that are capable of providing more informative and diagnostic results than typical tests offer. This paper reviews approaches that have been proposed for educational diagnostic assessment. Two major approaches are identified: (a) deficit assessment, which focuses on weaknesses of the student, and (b) error analysis, which focuses on the kinds of errors the student commits. This paper also reviews recent work related to diagnostic assessment that is based on the integration of methods from cognitive psychology and artificial intelligence. It is concluded that the development of powerful diagnostic instruments may require a reexamination of existing psychometric models and possibly the development of alternative ones. It is also pointed out that the traditional approach to the specification of content in terms of static taxonomies may not be appropriate given the dynamic and sequential nature of diagnostic assessment. Finally, it is noted that the psychometric and content demands of diagnostic assessment all but require test admininstration by computer.     

大学生对英语学习的根本需求与课堂教学

通过分析调研的结果,可看出学生对大学英语学习的根本需求是提高实际运用语言的能力,现行的测试和测试引导下的教学模式与学生的根本需求存在一定矛盾。因此提出几点尝试性的课堂教学设想。     

Special educational needs,disability and school accountability: an international perspective

Standards-based accountability approaches in education are increasingly being adopted in many industrialised countries. This tends to involve the scrutiny of student performance in national academic tests and then holding schools accountable for subsequent progress. International assessments such as Programme for International Student Assessment (PISA) are not accountability measures in themselves, but are often utilised in a similar way and national policies are judged against performance in these tests. Drawing upon the examples of PISA and national assessments in the USA, this paper explores how students with special educational needs and disability (SEND) are included in, and served by, these assessment policies. The analysis suggests that while these approaches are inclusive in their intent (most obviously through efforts to include students with SEND in testing) they may poorly represent the progress of these students. It is argued that more equitable inclusion of students with SEND must find a balance between accountability measures which measure performance in traditional attainment-focused tests and other relevant assessments which are meaningful to the given student and/or SEND sub-group.     

Sobolev方程的各向异性非协调类Wilson有限元逼近

本文讨论了Sobolev方程的非协调类Wilson有限元逼近。通过新的技巧和方法在各向异性网格下给出了与传统有限元方法完全相同的最优误差估计。     

An international comparison investigating the relationship between national culture and student achievement

The purpose of this study was to examine the connections between national culture and student achievement. Using Hofstede’s six cultural dimensions and the two dimensions from the World Values survey, we conducted multiple regressions to determine the most significant predictors of student achievement as measured by the 2009 Program for International Student Assessment in reading, mathematics, and science. Our analyses found that the most significant predictors of student achievement on all three student outcome included the following cultural dimensions based on two different frameworks: (1) a culture’s focus on fostering long-term orientation to include emphasis on perseverance to achieve future-oriented results and (2) a culture’s focus on secular-rational values vs. traditional values. In addition, findings indicate that when mapped geographically, similar patterns emerge among the two cultural dimensions.     

The effects of faking on non-cognitive predictors of academic performance in University students

This study examined the extent to which students could fake responses on personality and approaches to studying questionnaires, and the effects of such responding on the validity of non-cognitive measures for predicting academic performance (AP). University students produced a profile of an ‘ideal’ student using the Big-Five personality taxonomy, which yielded a stereotype with low scores for Neuroticism, and high scores for the other four traits. A sub-set of participants were allocated to a condition in which they were instructed to fake their responses as University applicants, portraying themselves as positively as possible. Scores for these participants revealed higher scores than those in a control condition on measures of deep and strategic approaches to studying, but lower scores on the surface approach variable. Conscientiousness was a significant predictor of AP in both groups, but the predictive effect of approaches to studying variables and Openness to Experience identified in the control group was lower in the group who faked their responses. Non-cognitive psychometric measures can be valid predictors of AP, but scores on these measures can be affected by instructional set. Further implications for psychometric measurement in educational settings are discussed.     

Rapid Assessment of Learners’ Proficiency: A cognitive load approach

To tailor instruction to levels of learner proficiency in a domain in adaptive learning environments, rapid on‐line methods of assessing learner knowledge and skills are required. Cognitive studies indicate that major factors in expert performance are organised structures in the person’s knowledge base. Measuring different levels of acquisition of such knowledge structures should be a major purpose of cognitive diagnostic assessment. Traditional tests may not provide reliable evidence for diagnostic purposes and are not suitable for dynamic learning environments. This paper discusses an alternative diagnostic approach based on monitoring immediate traces of students’ knowledge structures in working memory. Results of an experimental study applying this method to assessment of user reading skills are described. The study demonstrated a sufficiently high level of correlation (.66) between obtained measures and traditional test scores, and a substantial increase in reliability, with testing time reduced by a factor of 3.8.     

Baseline assessment,value-added and the prediction of reading

Value-added approaches aim to make fair comparisons of the academic progress of pupils in different settings. They rest on measurements at two time points and it has been suggested that the assessment of 4-year-olds cannot provide a sufficiently strong base from which to make comparisons at the end of Key Stage 1. Data on a range of potential predictors from the Performance Indicators in Primary Schools (PIPS) are related to measures of reading and mathematics in Year 2. The questions addressed include: a) To what extent is it possible to predict the reading of 7-year-olds? b) Which measures of 4-year-olds are the most predictive of later success? c) What are the implications for baseline assessment nationally? The results suggest that on-entry assessments, which last about 20 minutes, can predict the reading and mathematics scores of seven-year-olds with correlations around 0.7. Several measures are identified as potential predictors. The quickest to administer reliably include digit and letter identification, counting, writing and doing simple sums. The analysis suggests that value-added measures are possible for schools but that individual pupil predictions remain problematic.     

Examination of objective and subjective financial factors in predicting financial and retirement satisfaction in retirees

The present study examined predictors of retirement and financial satisfaction and contributed to existing literature by examining the incremental prediction offered by subjective estimates of financial resources and the impact of the Recession over more traditional objective measures. We hypothesized that objective financial resources, subjective financial resources, and the economic recession would predict retirement satisfaction and financial satisfaction, but that subjective estimates of financial well-being would be superior predictors of the dependent variables as compared to objective indices of financial resources.

A total of 245 retirees participated in the study, and results suggested that subjective and objective measures of finances were relatively independent. Further, consistent with hypotheses, most of the subjective financial measures provided significant incremental prediction of retirement and financial post-retirement satisfaction over that offered by objective financial indices. Subjective estimates of the impact of the Recession significantly predicted financial satisfaction post-retirement. Practical implications, limitations, and suggestions for future research are discussed.     

Critical modeling principles when testing for gender equity in faculty salary

Multiple regression procedures are commonly used to investigate gender equity in faculty salary. However, a review of 19 case studies indicates that many of them fail to (a) adequately develop a regression model that examines the linear and nonlinear (i.e., interactive and curvilinear) relations between predictors and the criterion and (b) appropriately apply regression diagnostic statistics throughout salary model development. A seven-step process is presented as a comprehensive framework for testing allegations of gender discrimination in faculty salary. Steps include (a) identifying predictors of faculty salary, (b) identifying and establishing criteria for interpreting statistical tests and diagnostic procedures, (c) determining the criterion variable used in the salary model, (d) developing a salary model, (e) testing for gender discrimination in pay, (f) conducting diagnostic procedures to confirm the appropriateness of the final salary model, and (g) testing the assumptions of the regression model. An application of this model is presented using a case study (N = 725 faculty).     

Machine Learning–Based Profiling in Test Cheating Detection

In recent years, machine learning (ML) techniques have received more attention in detecting aberrant test-taking behaviors due to advantages when compared to traditional data forensics methods. However, defining “True Test Cheaters” is challenging—different than other fraud detection tasks such as flagging forged bank checks or credit card frauds, testing organizations are often lack of physical evidences to identify “True Test Cheaters” to train ML models. This study proposed a statistically defensible method of labeling “True Test Cheaters” in the data, demonstrated the effectiveness of using ML approaches to identify irregular statistical patterns in exam data, and established an analytical framework for evaluating and conducting real-time ML-based test data forensics. Classification accuracy and false negative/positive results are evaluated across different supervised-ML techniques. The reliability and feasibility of operationally using this approach for an IT certification exam are evaluated using real data.     

高维空间基于样式相似性的最大子空间聚类

研究了基于样式相似性的子空间聚类问题,使用样式相似性作为相似性度量.与在所有维或者子维集上聚集距离相近的对象的传统聚类方法不同的是,样式相似性寻找的是这样一种有趣的样式:对象在子维上呈现出相同起伏的一致变化.提出了一种挖掘基于样式相似性的最大子空间聚类的方法EMaPle.一般情况下数据集属性数目远小于对象数目,因此仅在属性计数空间查找簇,然后运用一些修剪策略.该方法能够找到同时满足一致性约束、大小约束和被MaPle忽视了的符号约束的聚类.在合成和实际数据集上的实验结果表明该算法优于原来的MaPle算法.     

Using rasch measurement to score,evaluate, and improve examinations in an anatomy course

Any examination that involves moderate to high stakes implications for examinees should be psychometrically sound and legally defensible. Currently, there are two broad and competing families of test theories that are used to score examination data. The majority of instructors outside the high‐stakes testing arena rely on classical test theory (CTT) methods. However, advances in item response theory software have made the application of these techniques much more accessible to classroom instructors. The purpose of this research is to analyze a common medical school anatomy examination using both the traditional CTT scoring method and a Rasch measurement scoring method to determine which technique provides more robust findings, and which set of psychometric indicators will be more meaningful and useful for anatomists looking to improve the psychometric quality and functioning of their examinations. Results produced by the more robust and meaningful methodology will undergo a rigorous psychometric validation process to evaluate construct validity. Implications of these techniques and additional possibilities for advanced applications are also discussed. Anat Sci Educ 7: 450–460. © 2014 American Association of Anatomists.     

Insights into the microbial diversity and community dynamics of Chinese traditional fermented foods from using high-throughput sequencing approaches

Chinese traditional fermented foods have a very long history dating back thousands of years and have become an indispensable part of Chinese dietary culture. A plethora of research has been conducted to unravel the composition and dynamics of microbial consortia associated with Chinese traditional fermented foods using culturedependent as well as culture-independent methods, like different high-throughput sequencing (HTS) techniques. These HTS techniques enable us to understand the relationship between a food product and its microbes to a greater extent than ever before. Considering the importance of Chinese traditional fermented products, the objective of this paper is to review the diversity and dynamics of microbiota in Chinese traditional fermented foods revealed by HTS approaches.     

comparison of three growth modeling techniques in the multilevel analysis of longitudinal academic achievement scores: Latent growth modeling, hierarchical linear modeling, and longitudinal profile analysis via multidimensional scaling

This study introduces three growth modeling techniques: latent growth modeling (LGM), hierarchical linear modeling (HLM), and longitudinal profile analysis via multidimensional scaling (LPAMS). It compares the multilevel growth parameter estimates and potential predictor effects obtained using LGM, HLM, and LPAMS. The purpose of this multilevel growth analysis is to alert applied researchers to selected analytical issues that are required for consideration in decisions to apply one of these three approaches to longitudinal academic achievement studies. The results indicated that there were no significant distinctions on either mean growth parameter estimates or on the effects of potential predictors to growth factors at both the student and school levels. However, the study also produced equivocal findings on the statistical testing of variance and covariance growth parameter estimates. Other practical issues pertaining to the three growth modeling methods are also discussed.     

Longitudinal stability of latent means and individual differences: A unified approach

We examined the stability of responses to a multi‐item self‐esteem scale collected on five occasions over an 8‐year period. A wide variety of approaches were critically examined that considered the stability of means, individual differences (i.e., test‐retest correlations), and factor structures using traditional approaches (e.g., ANOVA and correlations) and structural equation models. Structural equation models based on multiple indicators provided a unified analytic approach for evaluating different aspects of stability and offered important advantages over traditional approaches. We describe a hierarchy of invariances and the nature of interpretations that are justified by different patterns of factor structure invariance associated with each level. We conclude that the assumptions underlying the typical repeated‐measures ANOVA approach to testing mean differences in longitudinal data are far more restrictive, less easily tested, and less likely to be met than those in the structural equation modeling approach advocated here, and that the use of ANOVA for this purpose requires a huge leap of faith that can rarely be justified on logical or empirical grounds.     

What Satisfies Students? Mining Student-Opinion Data with Regression and Decision Tree Analysis

To investigate how students' characteristics and experiences affect satisfaction, this study uses regression and decision tree analysis with the CHAID algorithm to analyze student-opinion data. A data mining approach identifies the specific aspects of students' university experience that most influence three measures of general satisfaction. The three measures have different predictors and cannot be used interchangeably. Academic experiences are influential. In particular, faculty preparedness, which has a well-known relationship to student achievement, emerges as a principal determinant of satisfaction. Social integration and pre-enrollment opinions are also important. Campus services and facilities have limited effects, and students' demographic characteristics are not significant predictors. Decision tree analysis reveals that social integration has more effect on the satisfaction of students who are less academically engaged.     

Risk assessment with actuarial and clinical methods: Measurement and evidence-based practice

Child welfare agencies have adopted assessment tools and instruments to inform the level of risk and guide the agency’s level of intervention with the family. Actuarial assessments may be more uniform but inflexible with respect to practice wisdom whereas clinical or consensus-based assessments are more comprehensive and intuitive but lack objectivity. The purpose of the current study is to compare clinical and actuarial methods of risk assessment used by child welfare workers to make decisions about substantiation and services. The current study examined the (1) association between clinical and actuarial dimensions, (2) association between actuarial dimensions and outcomes, (3) association between clinical dimensions and outcomes, (4) caseworker primary use of actuarial dimensions, and (5) caseworker supplementary use of actuarial dimensions. Findings indicated that the actuarial may not be solely predictive of agency intensity with respect to case decision and service provision. Our findings suggest that dual-measurement does inform intensity, and we speculate from these findings that the measures may be involved with decision-making in a complex way. This study may be best viewed as a means by which researchers begin to parse how decisions are made; with this information, instruments may be better tailored to facilitate clinical, critical thought.     

关于学校弱势儿童的若干基本问题初探

学校弱势儿童是指在学校教育领域中,所拥有的教育资源相对匮乏,学校生活质量相对低劣,人格或身心结构存在一定问题或缺陷,处于发展的不利境地的在校儿童。构成其现实地位的各种弱势因素表现为原因源、权重性和置换率等三个分析维度。应该对学校弱势儿童进行概念整合和类别分析,研制简便可靠的评定测量手段,进一步研究其人格发展特征及其与教育内外环境的关系,寻求教育力量逆转其发展的途径与方法。     

Renovation of Teacher Training by the New Media

For the past fifteen years, new techniques and methods, most particularly audio-visual ones, have made it possible to foresee a profound renewal in the training and retraining of teachers. To name some, the use of tape recorders, and then closed-circuit television, the production of anthologies on films, radio and television programmes, which make up an integral part of a system of training at a distance. In industrialised countries, these methods are very unequally used; in some of them, they do not even have as yet a system of retraining at a distance. Also conditions vary greatly according to the training institutes. Some only use the most traditional methods more or less perfected; others, well equipped with projectors of all sorts, laboratories and closed-circuit television, try to introduce really new methods and mediafor teaching. And lastly, some new procedures have hardly gone outside the research centres.