Application of Generalizability Theory in Estimating the Reliability of a Motor Performance Test

Abstract

This paper discusses the use of generalizability theory, a flexible intraclass approach, for estimating the reliability of a motor performance task. The generalizability technique provides variance component estimates for each factor of an analysis of variance design. These variance estimates can be utilized to define a number of reliability coefficients which are dependent upon the logical or theoretical rationale for assigning variance component estimates as true score or error score variance. The second section of this paper presents a study illustrating the use of generalizability theory in estimating the reliability of a motor performance task. The purpose of this study was to determine the reliability of first-ball scores for beginning college bowlers. The facets of the generalizability study were identified as sex of the bowler, trials, and days. Three reliability coefficients, R ₁ (.93), R ₂ (.92), and R ₃ (.84), were computed by assigning the different factors to either true score or error score variance. The performance of beginning college bowlers as measured by first-ball scores is reliable. While the information might be useful for the teacher or researcher who is interested in detecting between-subject differences in coeducational bowling classes, the primary purpose of this paper was to demonstrate the use of generalizability theory to estimate reliability coefficients for a set of motor performance test scores.     

Estimating the Stability Reliability of a Score

The internal consistency and factorial validity of the 6-factor Scale of Coaching Performance (SCP; MacLean &; Chelladurai,1995) were examined using the responses of 448 administrators and coaches, randomly selected among NCAA programs. Confirmatory factor analysis, alpha coefficients, composite reliabilities, and variance extracted measures were utilized to test the internal consistency, reliability, and factorial validity of the SCP. The original 6-factor model was also tested against possible alternative models. The SCP had weak factor reliabilities: the public relations behaviors factor had an unacceptable reliability coefficient. The fit statistics (χ²p;= 2641.72; Scaled χ² = 2141.70, p < .001; RMSEA = .081; GFI = .74) indicated unsatisfactory factorial validity of the 6-factor model using the NCAA sample, because of problematic indicators and weak independence of latent variables but none of the alternative models provided better fit to the NCAA data. The SCP may be population-specific to the Canadian athletic system, which differs from the system found in the United States. Further analytic efforts are needed to improve the generalizability of SCP.     

Test–retest Reliability and Construct Validity of an Online and Paper Administered Physical Activity Neighborhood Environment Scale (PANES)

ABSTRACT

The Physical Activity Neighborhood Environment Scale (PANES) has been used internationally; however, PANES properties have not been assessed in all geographical contexts. Our objectives were to assess the reliability and validity of an online and paper version of the PANES in Canadian adults. Reliability was estimated using intraclass correlation coefficients (ICC), percent of overall agreement (p₀) and Cohen’s Kappa coefficient (κ). Lower 95% confidence interval (CI) ICC ranged from 0.10 to 0.70. Lower 95%CI for κ statistics ranged from ?0.20 to 0.64 and p₀ ranged from 80.1% to 95.7%. Cronbach’s alpha coefficients (α) estimated internal consistency of the PANES (α = 0.58 for the paper version and α = 0.55 for the online version). Mean scores for the PANES Built Environment Index (BEI) significantly differed by neighborhood street pattern (p < .05). The PANES administrated via paper or online provides reliable overall agreement and valid estimates of the self-reported neighborhood built environment supportiveness of physical activity.     

Comparison of Two Nonparametric Methods for Estimating the Reliability of Motor Performance Tests

The purpose of this study was to compare the power and sampling distributions of the coefficient of concordance (W) and the weighted average tau (W _a ) for the selected sets of parameters m, n, and α. The values of m and n were selected to represent situations in physical education for which test reliability and judge objectivity might be estimated. The comparison of W and W_a required the generation of data matrices with varying amounts of error (α) for each set of values m and n. The obtained distributions were used to make the necessary comparisons. Several conclusions were drawn: First, the sampling distributions of W and W_a were similar for the sets of parameters investigated in this study. Second, W and W_a were equally powerful for the sets of parameters investigated in this study. And third, the comparisons did not indicate that either statistic is more appropriate for estimating the reliability of motor performance tests. On the basis of the practical considerations of ease of computation and familiarity with the coefficient of concordance (W,) this statistic might be preferred.     

Psychometric Note on the Reliability of the Sport Competition Anxiety Test: Form C

Abstract

Data collected from 736 fourth, fifth, and sixth grade children using Martens SCAT-C yielded moderately high values of split half reliability. Internal consistency coefficients were relatively high (.81 to .85) and similar to the values reported in the Martens test manual when calculated using the appropriate test, eliminating use of the KR-20. The SCAT-C was also found to be generalizable to fourth grade populations even though originally its use was not intended with this age group.     

一种羽毛球灵敏素质测试方案的信效度检验研究

为了验证从运动实践和理论推导设计的羽毛球灵敏素质测试方案的科学性,本研究募集21名大学校队的羽毛球选手进行“六点灵敏测试”和5米折返跑测试验证该灵敏专项测试的同时效度,并对“六点灵敏测试”进行间隔一段时间后的二次测试,检验其重测信度;此外,将测试结果同普通运动者“六点灵敏测试”的测试结果进行比较验证其结构信度。研究结果发现该灵敏素质测试方案具备了较好测试信度和效度。因此,“六点灵敏测试”可以作为羽毛球专项灵敏素质测试和训练的有效方案。     

When faultlines are created: Exploring the conflict triggering process in sport

The purpose of this article is to advance theoretical understanding of the relationship between task conflict and relationship conflict (i.e., triggering process) by discussing the role of social identity as a mediating factor. Specifically, the authors’ proposed theoretical model emphasizes the consequences of intense, long-lasting task conflict by mapping the connection to workgroup identity formation and relationship conflict. The authors apply social identity theory to provide a new lens through which to view the impact of intense, long-lasting task conflict that goes unmanaged in sport organizations. Further, the authors provide several testable propositions regarding identity and subgroup formation that point to major factors influencing the development of relationship conflict within groups. The authors’ propositions bolster current sport management theory by emphasizing how intergroup and intragroup processes are interwoven.     

全民健身日研究

利用文献资料、统计分析和专家访谈等方法,对我国首个"全民健身日"活动开展状况进行全面深入地总结研究,结果表明:我国"全民健身日"的确立是适应人民群众体育健身的需求;是进一步发挥体育的综合功能和社会效应,丰富社会体育文化生活,促进人的全面发展的需要;是促进中国从体育大国向体育强国目标迈进的需要.要提高全民健康素质,全面推进全民健身活动,必须以科学发展观为指导,与以往的全民健身活动紧密结合,建立健全全民健身服务体系.     

The Reliability of a 5km Run Test on a Motorized Treadmill

The purpose of the present study was to determine the reliability of a 5km run test on a motorized treadmill. Over three consecutive weeks, 12 well-trained runners completed three 5km time trials on a treadmill following a standardized warm-up. Runners were partially-blinded to their running speed and distance covered. Total time to complete the run was recorded for analysis of reliability. The highest intraclass correlation coefficient between trials for 5km time trial (TT) time was .99 (90% confidence intervals [CI]; .96–1.00, very high), which occurred between trials 2 and 3. The lowest typical error of measurement (expressed as a coefficient of variation percentage [CV%] and as an absolute value in seconds) also occurred between trials 2 and 3, and was 1.0% and 10.9 seconds, respectively. The testing protocol performed on a motorized treadmill in the current study is reproducible in well-trained runners following a single trial, making it a reliable method for monitoring running performance.     

When the chips are down: effects of attributional feedback on self-efficacy and task performance following initial and repeated failure

In two experiments, we manipulated the controllability and stability of causes of failure and explored the impact of these factors on self-efficacy and performance. In Experiment 1, participants (N=80; mean age 20.0 years, s=1.0) were provided with false negative feedback following performance on a blindfolded dart-throwing task. Consistent with theory and recent research, an induced belief that failure was beyond control and unlikely to change led to lower self-efficacy and poorer performance (all F1,754>5.49, all P<0.05, all η2=0.01). A second experiment (N=80; mean age 22.0 years, s=2.1) demonstrated that following an induced belief that failure was beyond control and unlikely to change, only new perceptions that a repeated failure was within one's control and likely to change resulted in higher self-efficacy and improved performance (all F1,75>4.53, all P<0.05, all η2>0.004). All effects were mediated by self-efficacy: Sobel's (1982) test, all z>1.97 (in absolute magnitude), all P<0.05, all r>0.22 (in absolute magnitude). These findings suggest that in novel circumstances individuals believe in the best for themselves unless possibilities to self-enhance are explicitly precluded, and only reinvest efforts when opportunities for self-enhancement become clearly admissible.     

Reliability,usefulness and construct validity of the Combined Basketball Skill Test (CBST)

This study aimed to assess the reliability, usefulness and construct validity of the newly developed Combined Basketball Skill Test (CBST). Fifteen recreational (age = 22.8 ± 4.2 y, stature = 184.8 ± 6.5 cm, body mass = 81.6 ± 9.6 kg, training experience = 9.8 ± 5.3 y) and fifteen semiprofessional (age = 18.9 ± 3.3 y, stature = 190.5 ± 8.1 cm, body mass = 84.2 ± 11.2 kg, training experience = 11.1 ± 3.5 y) players volunteered to participate in this study. Test–retest reliability and usefulness were examined for recreational players, while construct validity was evaluated comparing the two player groups. The CBST is composed of 12 trials and its outcome measures include: completion time (sum of the 12 trial times); penalty time (sum of the times from the 12 trials); performance time (completion time + penalty time) and total number of errors. Relative reliability analysis showed acceptable ICC values (i.e. ≥0.70) in all the studied variables. Absolute reliability analysis showed a CV < 5% for completion (1.6%) and performance (2.0%) time, while a CV >5% is reported for the remaining variables. The usefulness of the test was considered “Marginal” and “Good” when comparing TE values with SWC₀₂ and SWC₀₅, respectively for all the studied variables. Likely and very likely differences were shown between recreational and semiprofessional players in all investigated variables. Results showed that the CBST is reliable, useful to detect moderate changes and valid to assess basketball skills.     

The Validity and Reliability of the Back Saver Sit-and-Reach Test in Middle School Girls and Boys

The purpose of this article was to highlight important research needs related to physical activity in 3- to 5-year-old children. We identified research needs in 3 major categories: health effects, patterns of physical activity, and interventions and policies. The top research needs include identifying the health effects of physical activity, the effects of physical activity on the development of healthy weight, the effects of physical activity on learning and behavior, and the health implications of sedentary behavior. Research questions concerning patterns of physical activity include determining the prevalence of 3- to 5-year-olds meeting the current physical activity guidelines; the social and environmental factors that influence physical activity in home, preschool, and community settings; and how physical activity tracks into later childhood, adolescence, and adulthood. Research questions about interventions and policies include identifying the most effective strategies to promote physical activity in home, child care, and community settings and to reach diverse populations of young children, identifying effective intervention implementation and dissemination strategies, and determining the effectiveness of national, state, local, and institutional policies for increasing physical activity. In conclusion, research is needed to establish a full understanding of the health implications of physical activity in 3- to 5-year-old children, to better understand the nature of physical activity behavior in this group, and to learn how to promote physical activity in young children.     

球类项目灵敏测试的实践应用:基于信度与效度评价的系统综述

目的：对国内外经检验的球类运动灵敏测试方法的设计结构、信度和效度进行系统综述,为球类运动灵敏素质的评价提供更科学的测试方法。方法：通过数据库对相关关键词进行检索并搜集符合标准的文献,运用PE-Dro评分系统对文献进行质量评估。结果：共纳入33篇文献,质量评价平均得分15分,得分范围13~17分。所有搜集文献共包含59种测试方法,37种为改变方向速度测试（CODT）,22种为反应灵敏测试（RAT）。结果：CODT设计的移动距离相对更长,改变方向的次数相对更多,两类测试方法的变向角度主要采用45°、90°和180° ;CODT的组内相关系数（ICC）范围为0.5~0.99,RAT的ICC为0.33~0.99,3篇文献发现较低的信度且均为RAT ;效度检验主要通过区分运动员水平、比赛位置以及年龄段来实现。结论：球类项目灵敏素质评价主要应用CODT与RAT两类测试方法。由于设计结构的差异,两类方法在信度和效度上各有优劣。CODT具有相对更高的信度,RAT则具有相对更高的效度。认知与决策能力作为高水平球类运动员的关键指标,建议未来在球类项目灵敏素质测量评价中加入刺激源组件     

The Validity and Reliability of the 12-Minute Swim Test in Male Swimmers Ages 13-17

The purpose of this study was to examine the validity and reliability of the Cooper 12-min swim test in high school male swimmers ages 13 to 17. Thirty-three boys performed three 12-min swims and 1 maximal graded treadmill test within a 14-day period. One practice swim was conducted 1 week prior to participation in this study. VO2max was assessed by indirect calorimetry with open-circuit spirometry with the Truemax 2400 metabolic cart (Consentius Technologies, Sandy, UT). Test-retest reliability of the 12-min swim assessed via 1-way analysis of variance indicated moderate reliability (R = .66, 95% confidence interval [CI] = .42-.81), whereas concurrent validity assessed via a Pearson product-moment correlation indicated a moderate relation (r = .47, 95% CI = .15-.70, r2 = .22). Results indicate that the Cooper 12-min swimming test is only moderately reliable after 2 practice swims and does not appear to be a valid field test of aerobic capacity in high school male swimmers ages 13 to 17.     

体育专业术科课程考核影响“效度”、“信度” 的因素分析及对策研究

通过对全国各体育院校、高等师范院校及部分专科学校体育专业术科课程考核的现状进行抽样调查,采用文献资料法、专家访问法、问卷调查法等研究方法,对影响术科课程考核“效度”、“信度”的因素进行分析并提出发展对策,为高校教学考核改革提供一些帮助和依据.