Stability of psychometric questionnaires

In 1999, Wilson and Batterham proposed a new approach to assessing the test-retest stability of psychometric questionnaires. They recommended assessing the proportion of agreement - that is, the proportion of participants that record the same response to an item - using a test-retest design. They went on to use a bootstrapping technique to estimate the uncertainty of the proportion of agreement. The aims of this short communication are (1) to demonstrate that the sampling distribution of the proportion of agreement is well known (the binomial distribution), making the technique of 'bootstrapping' redundant, and (2) to suggest a much simpler, more sensitive method of assessing the stability of a psychometric questionnaire, based on the test-retest differences (within-individuals) for each item. Adopting methods similar to Wilson and Batterham, 97 sport students completed the Social Physique Anxiety Scale on two occasions. Test-retest differences were calculated for each item. Our results show that the proportion of agreement ignores the nature of disagreement. Items 4 and 11 showed similar agreement (44.3% and 43.3% respectively), but 89 of the participants (91.8%) differed by just +/-1 point when responding to item 4, indicating a relatively stable item. In contrast, only 78 of the participants (80.4%) recorded a difference within +/- 1 point when responding to item 11, suggesting quite contrasting stability for the two items. We recommend that, when assessing the stability of self-report questionnaires using a 5-point scale, most participants (90%) should record test-retest differences within a reference value of +/- 1.     

The Roles of Intuition and Informants’ Expertise in Children's Epistemic Trust

This study examined how children's intuitions and informants’ expertise influence children's trust in informants’ claims. Three‐ to 8‐year‐olds (N = 192) watched videos in which experts (animal/biology experts or artifact/physics experts) made either intuitively plausible or counterintuitive claims about obscure animals or artifacts. Claims fell either within or beyond experts’ domains of expertise. Children of all ages were more trusting of claims made by informants with relevant, as opposed to irrelevant, expertise. Children also showed greater acceptance of intuitive rather than counterintuitive claims, a differentiation that increased with age as they developed firmer intuitions about what can ordinarily happen. In summary, children's trust in testimony depends on whether informants have the relevant expertise as well as on children's own developing intuitions.     

Digital media as a reflective tool: creating appropriate spaces for students to become introspective

Global citizenship and competency development have become integral parts of political, economic and education discourses. This is particularly true in the USA, where higher education institutions are answering the global citizenship call through the promotion of short-term study abroad. Unfortunately, the ‘just do it’ analogy too often influences the ideology of study abroad, whereby simply providing students with an international experience is perceived to be sufficient in shaping global citizenship. Global citizenship, as an educational outcome, is optimally facilitated when educational experiences are married with appropriate pedagogy, including the shaping of subsequent understandings and actions with critical reflection. We postulate that reflective experiences can be further enhanced by meeting students on the platforms and forums where they live, communicate and already engage. Specifically, this communication, using the context of short-term study abroad programmes, will argue that: (1) critical reflection is an essential step to fostering global citizenship and (2) digital story telling (mobile pedagogy) can be a powerful tool for enriching the critical reflection process.     

Rain influences the physiological and metabolic responses to exercise in hot conditions

Outdoor exercise often proceeds in rainy conditions. However, the cooling effects of rain on human physiological responses have not been systematically studied in hot conditions. The present study determined physiological and metabolic responses using a climatic chamber that can precisely simulate hot, rainy conditions. Eleven healthy men ran on a treadmill at an intensity of 70% VO_2max for 30 min in the climatic chamber at an ambient temperature of 33°C in the presence (RAIN) or absence (CON) of 30 mm · h^?1 of precipitation and a headwind equal to the running velocity of 3.15 ± 0.19 m · s^?1. Oesophageal temperature, mean skin temperature, heart rate, rating of perceived exertion, blood parameters, volume of expired air and sweat loss were measured. Oesophageal and mean skin temperatures were significantly lower from 5 to 30 min, and heart rate was significantly lower from 20 to 30 min in RAIN than in CON (P < 0.05 for all). Plasma lactate and epinephrine concentrations (30 min) and sweat loss were significantly lower (P < 0.05) in RAIN compared with CON. Rain appears to influence physiological and metabolic responses to exercise in heat such that heat-induced strain might be reduced.     

Are the Multidimensional Body Self-Relations Questionnaire Scales stable or transient?

Body image is an important concept when assessing mental health and, thus, a target for intervention work. The common approach to assess body image is via self-report and, consequently, it is important to assess stability and susceptibility of items to random change. Participants (N = 99, aged: 18–30 years) completed the 34-item Multidimensional Body Self-Relations Questionnaire-Appearance Scales (MBSRQ-AS) twice, separated by two weeks. Stability was quantified by calculating the test–retest differences for each item, with a stable item showing 90% of respondents’ differences within a criterion of ≤±1. This method was used in preference to correlation, which is a test of association rather than agreement. Analyses identified 29 stable items. Five items were found to be unstable including the entire “Appearance Evaluation” scale. One particular unstable question, Q9 = “Most people would consider me good-looking”, revealed strong associations with both gender and weight status. Forty-four per cent of “overweight” respondents changed their minds by more than ±1 compared with only 10% of “normal-weight” participants, suggesting that being overweight is associated with uncertainty and instability. We suggest that future research is needed to investigate why such factors as being overweight appear to be linked to instability in self-report measures of body image.     

Has Motivational Interviewing Fallen into its Own Premature Focus Trap?

Since the initial conception of the behaviour change method Motivational Interviewing, there has been a shift evident in epistemological, methodological and practical applications, from an inductive, process and practitioner-focussed approach to that which is more deductive, research-outcome, and confirmatory-focussed. This paper highlights the conceptual and practical problems of adopting this approach, including the consequences of assessing the what (deductive outcome-focussed) at the expense of the how (inductively process-focussed). We encourage a return to an inductive, practitioner and client-focussed MI approach and propose the use of Computer Assisted Qualitative Data Analysis Systems such as NVivo in research initiatives to support this aim.     

Problem-solving in a case-based course: Strategies for facilitating coached expertise

This paper proposes the use of specific coaching strategies to facilitate student use of expert-like problem-solving strategies while analyzing and solving instructional design case studies. Findings from an exploratory study, designed to examine changes in students' problem-solving skills as they analyzed case studies, suggested that students could show expert characteristics at times, under some circumstances, but did not perform like experts on a regular basis. At two midwestern universities, 37 students analyzed 6 to 10 case studies both in class and in on-line discussions. Comparisons were made both within and across students, as well as across time, to examine patterns and changes in student problem-solving approaches. Findings suggested that primary influences on the incidence of expert performance were more external than internal and might be more aptly characterized as “coached expertise”. Specific suggestions are included for coaching the development of student problem-solving skills within a case-based course.