Assessment of interrater and intermethod agreement in the kinesiology literature

The purpose of this article was two-fold (1) provide an overview of the commonly reported and under-reported absolute agreement indices in the kinesiology literature for continuous data; and (2) present examples of these indices for hypothetical data along with recommendations for future use. It is recommended that three types of information be reported as evidence for agreement because no one type is superior to another. Report one graphical display (bivariate plot of raw scores with identity line or Bland-Altman plot), one scaled index (intraclass correlation coefficient with agreement definition or Lin’s Concordance Correlation Coefficient), and at least one unscaled index (root mean squared deviation, total deviation index, coverage probability, or limits of agreement) when there are two methods/raters with no replication. The amount of information recommended exceeds what has commonly been reported in absolute agreement studies published in the kinesiology literature.