Universal Design and Multimethod Approaches to Item Review

Test items undergo multiple iterations of review before states and vendors deem them acceptable to be placed in a live statewide assessment. This article reviews three approaches that can add validity evidence to states' item review processes. The first process is a structured sensitivity review process that focuses on universal design considerations for items. The second method is a series of statistical analyses intended to increase the limited amount of information that can be derived from analyses on low-incidence populations (such as students who are blind, deaf, or have cognitive disabilities). Finally, think aloud methods are described as a method for understanding why particular items might be problematic for students .     

Characteristics of character usage in Chinese Web searching

The use of non-English Web search engines has been prevalent. Given the popularity of Chinese Web searching and the unique characteristics of Chinese language, it is imperative to conduct studies with focuses on the analysis of Chinese Web search queries. In this paper, we report our research on the character usage of Chinese search logs from a Web search engine in Hong Kong. By examining the distribution of search query terms, we found that users tended to use more diversified terms and that the usage of characters in search queries was quite different from the character usage of general online information in Chinese. After studying the Zipf distribution of n-grams with different values of n, we found that the curve of unigram is the most curved one of all while the bigram curve follows the Zipf distribution best, and that the curves of n-grams with larger n (n = 3–6) had similar structures with β-values in the range of 0.66–0.86. The distribution of combined n-grams was also studied. All the analyses are performed on the data both before and after the removal of function terms and incomplete terms and similar findings are revealed. We believe the findings from this study have provided some insights into further research in non-English Web searching and will assist in the design of more effective Chinese Web search engines.     

Evaluating the Power of Latent Growth Curve Models to Detect Individual Differences in Change

We evaluated the statistical power of single-indicator latent growth curve models to detect individual differences in change (variances of latent slopes) as a function of sample size, number of longitudinal measurement occasions, and growth curve reliability. We recommend the 2 degree-of-freedom generalized test assessing loss of fit when both slope-related random effects, the slope variance and intercept-slope covariance, are fixed to 0. Statistical power to detect individual differences in change is low to moderate unless the residual error variance is low, sample size is large, and there are more than four measurement occasions. The generalized test has greater power than a specific test isolating the hypothesis of zero slope variance, except when the true slope variance is close to 0, and has uniformly superior power to a Wald test based on the estimated slope variance.     

A comparative judgement approach to the large-scale assessment of primary writing in England

ABSTRACT

Writing assessment is a key feature of most education systems, yet there are limitations with traditional methods of assessing writing involving rubrics. In contrast, comparative judgement appears to overcome the reliability issues that beset the assessment of performance assessment tasks. The approach presented here extends previous work on comparative judgement by directly involving teachers in a large number of schools in the judging of young pupils’ writing. To ensure quality control the process incorporated a process of ‘anchoring’ that ensured that teachers could not artificially inflate their own pupils’ scores. The approach was used to assess the writing of 55,599 primary pupils in England in 2017–2018. Overall, the results showed that a comparative judgement approach to writing incorporating anchoring shows promise in providing a fair and robust large-scale method to assess writing.     

Perceptions of challenge in science learning

Prior research has indicated that decisions made by students about their learning are influenced by perceptions of various factors related to the task and classroom context. This influence seems to be mediated by the extent to which the learner perceives personal challenge in what is done. This concept of personal challenge comprises both cognitive (demand) and affective (motivation) components. In the current paper, response trends of almost 4000 secondary science students to a questionnaire structured according to factors associated with challenge are considered by school, year level and gender. These trends indicate that challenge in science diminishes as students move from Years 7 to 10, due to increasingly negative perceptions of many of these factors. Results also indicate however that, through purposeful change in teaching perspectives and practices, teachers can establish and sustain students’ sense of challenge in classroom learning.     

Time for review: supporting the work of an advisory group

This paper raises methodological issues about the challenges and dilemmas of inclusive research practices reflecting on the work of an advisory group carrying out research on using Information and Communication Technology (ICT) to enhance community participation. The interests of three parties can be identified – the commissioning agent, the researchers and the researched – and these interplayed throughout the course of the research determining the outcomes. Given the relationship between inclusive research and advocacy, there were particular gains in enabling the voice of the advisory group to shape the way in which the research was conducted and to disseminate the findings both within the organisation and beyond. However, the fragility of these new structures required organisational changes for the research to be truly empowering. The experience of the group suggests the need for their involvement at all stages of the research process.     

Converting a Face-To-Face Introductory Research Methods Course to an Online Format: Pedagogical Issues and Technological Tools

ABSTRACT

This paper presents a step-by-step conversion of a face-to-face (F2F) entry level social work research methods course to a computer-mediated format. An overview of teaching practices and the important pedagogical foundations associated with this curricular effort are discussed followed by an introduction to the electronic platform utilized in the course. Five new teaching practices that guide the F2F-to-online course conversion of this introductory research methods course are presented with corresponding teaching activities and technological tools. A conversion matrix aligns the teaching practices, classroom activities, and provides references for the identified tools.     

Trajectories,institutions, and re-locations: a conversation on Inter-Asia outside Asia

ABSTRACT

This conversation explores the resonances of the Inter-Asia project outside the geographical boundaries of Asia. The participants, who represent diverse national, institutional, and professional experiences, discuss the following topics: how Asia and the Inter-Asia project has affected their intellectual trajectories; navigating academic institutions and formations; the changing meaning of diaspora and migration and their effects on language and communication; and the relationship between the academy and social movements. Particular attention is paid to reframing Australia and Canada from an Inter-Asia perspective.