Test-retest stability of the Task and Ego Orientation Questionnaire

Establishing stability, defined as observing minimal measurement error in a test-retest assessment, is vital to validating psychometric tools. Correlational methods, such as Pearson product-moment, intraclass, and kappa are tests of association or consistency, whereas stability or reproducibility (regarded here as synonymous) assesses the agreement between test-retest scores. Indexes of reproducibility using the Task and Ego Orientation in Sport Questionnaire (TEOSQ; Duda & Nicholls, 1992) were investigated using correlational (Pearson product-moment, intraclass, and kappa) methods, repeated measures multivariate analysis of variance, and calculating the proportion of agreement within a referent value of +/-1 as suggested by Nevill, Lane, Kilgour, Bowes, and Whyte (2001). Two hundred thirteen soccer players completed the TEOSQ on two occasions, 1 week apart. Correlation analyses indicated a stronger test-retest correlation for the Ego subscale than the Task subscale. Multivariate analysis of variance indicated stability for ego items but with significant increases in four task items. The proportion of test-retest agreement scores indicated that all ego items reported relatively poor stability statistics with test-retest scores within a range of +/-1, ranging from 82.7-86.9%. By contrast, all task items showed test-retest difference scores ranging from 92.5-99%, although further analysis indicated that four task subscale items increased significantly. Findings illustrated that correlational methods (Pearson product-moment, intraclass, and kappa) are influenced by the range in scores, and calculating the proportion of agreement of test-retest differences with a referent value of +/-1 could provide additional insight into the stability of the questionnaire. It is suggested that the item-by-item proportion of agreement method proposed by Nevill et al. (2001) should be used to supplement existing methods and could be especially helpful in identifying rogue items in the initial stages of psychometric questionnaire validation.     

Stability of psychometric questionnaires

In 1999, Wilson and Batterham proposed a new approach to assessing the test-retest stability of psychometric questionnaires. They recommended assessing the proportion of agreement - that is, the proportion of participants that record the same response to an item - using a test-retest design. They went on to use a bootstrapping technique to estimate the uncertainty of the proportion of agreement. The aims of this short communication are (1) to demonstrate that the sampling distribution of the proportion of agreement is well known (the binomial distribution), making the technique of 'bootstrapping' redundant, and (2) to suggest a much simpler, more sensitive method of assessing the stability of a psychometric questionnaire, based on the test-retest differences (within-individuals) for each item. Adopting methods similar to Wilson and Batterham, 97 sport students completed the Social Physique Anxiety Scale on two occasions. Test-retest differences were calculated for each item. Our results show that the proportion of agreement ignores the nature of disagreement. Items 4 and 11 showed similar agreement (44.3% and 43.3% respectively), but 89 of the participants (91.8%) differed by just +/-1 point when responding to item 4, indicating a relatively stable item. In contrast, only 78 of the participants (80.4%) recorded a difference within +/- 1 point when responding to item 11, suggesting quite contrasting stability for the two items. We recommend that, when assessing the stability of self-report questionnaires using a 5-point scale, most participants (90%) should record test-retest differences within a reference value of +/- 1.     

Functional literacy of Young Guyanese Adults

Functional literacy is interpreted as the ability of the individual to apply skills in reading, writing, calculation and basic problem-solving in those activities in which literacy is required for effective functioning in his/her own group and community. The paper describes the rationale, development and administration of the test used for measuring levels (high, moderate, low) of achievement in functional literacy in three domains (document, prose and quantitative). An assumption of the study was that a high level of functional literacy was required for the individual to function effectively in his/her own group and community. The context of the study is Guyana the most underdeveloped and impoverished country in the English-speaking Caribbean. The subjects are out of school youth in Guyana aged 14–25.Amongst the main findings are: only approximately 11% of the young people show a high level of functional literacy; females tend to have a higher level of functional literacy than males: and most of those at the low level never went beyond primary and low status secondary schools and usually end up unemployed or in semi- or unskilled jobs.Attention is drawn to the difficulty of attracting funding for literacy programmes from international aid agencies, given the inflated adult literacy rate which is reported for Guyana in international statistics. While they credit Guyana with an adult literacy rate of 97.5%, the study suggests that a more realistic figure is in the 70s. The importance of adult and continuing education is underscored in view of the need to help those who are out of school to meet the ever-changing demands of society for improved skills in literacy and numeracy.     

The Roles of Intuition and Informants’ Expertise in Children's Epistemic Trust

This study examined how children's intuitions and informants’ expertise influence children's trust in informants’ claims. Three‐ to 8‐year‐olds (N = 192) watched videos in which experts (animal/biology experts or artifact/physics experts) made either intuitively plausible or counterintuitive claims about obscure animals or artifacts. Claims fell either within or beyond experts’ domains of expertise. Children of all ages were more trusting of claims made by informants with relevant, as opposed to irrelevant, expertise. Children also showed greater acceptance of intuitive rather than counterintuitive claims, a differentiation that increased with age as they developed firmer intuitions about what can ordinarily happen. In summary, children's trust in testimony depends on whether informants have the relevant expertise as well as on children's own developing intuitions.