Organization and visualization for initial analysis of forced-choice ipsative data

Forced-choice ipsative data are common in personality, philosophy and other preference-based studies. However, this type of data inherently contains dependencies that are challenging for usual statistical analysis. In order to utilize the structure of the data as a guide for analysis rather than as a challenge to manage, a visualisation tool was developed to emphasize the established framework and enhance analysis tools. This visualisation has many strength, including the display of significant amounts of data at once in three dimensions with little loss of detail, incorporating the theoretical framework for the data into the structure of the visualisation and organizing the data for use with common statistical tests and analysis tools.     

The instructor self-evaluation form: Development and validation of an ipsative forced-choice measure of self-perceived faculty performance

An ipsative forced-choice college faculty self-evaluation questionnaire was developed. The instrument was validated by comparing self-evaluation scores to student rating scores and by comparing superior and nonsuperior instructors on the basis of student ratings. Both analyses yielded positive, but modest, indications of validity. Implications are discussed relative to the questionnaire's utility in faculty development activities.     

Ordinal Factor Analysis of Graded-Preference Questionnaire Data

We introduce a new comparative response format, suitable for assessing personality and similar constructs. In this “graded-block” format, items measuring different constructs are first organized in blocks of 2 or more; then, pairs are formed from items within blocks. The pairs are presented 1 at a time to enable respondents expressing the extent of preference for 1 item or the other using several graded categories. We model such data using confirmatory factor analysis (CFA) for ordinal outcomes. We derive Fisher information matrices for the graded pairs, and supply R code to enable computation of standard errors of trait scores. An empirical example illustrates the approach in low-stakes personality assessments and shows that similar results are obtained when using graded blocks of size 3 and a standard Likert format. However, graded-block designs might be superior when insufficient differentiation between items is expected (due to acquiescence, halo, or social desirability).     

Developing student research capability for a ‘post-truth’ world: three challenges for integrating research across taught programmes

ABSTRACT

Research-based learning in taught courses develops the skills needed to judge knowledge sources and think critically in a post-truth world. In viewing research skills as threshold concepts, the paper argues that transforming a student cannot be a one-off event. Research capacity must build over a programme and this requires coherent research skill development and assessment that is progressive (ipsative). A study of five programmes each with a different design of research ‘throughline’ showed that such integrated research-based learning generates three challenges. Firstly, conceptualising the research skills and progression is not easy. Secondly, the accumulation and enrichment of research skills is not readily visible to students. Finally, providing a clear support system across the programme is not straightforward. The paper concludes that these challenges need to be addressed if the potential of research-based education to enable future citizens to interrogate populist claims and reject misinformation is to be realised.