The dynamic control ratio at the equilibrium point (DCRe): introducing relative and absolute reliability scores

Analytical methods to assess thigh muscle balance need to provide reliable data to allow meaningful interpretation. However, reproducibility of the dynamic control ratio at the equilibrium point has not been evaluated yet. Therefore, the aim of this study was to compare relative and absolute reliability indices of its angle and moment values with conventional and functional hamstring–quadriceps ratios. Furthermore, effects of familiarisation and angular velocity on reproducibility were analysed. A number of 33 male volunteers participated in 3 identical test sessions. Peak moments (PMs) were determined unilaterally during maximum concentric and eccentric knee flexion (prone) and extension (supine position) at 0.53, 1.57 and 2.62 rad · s^–1. A repeated measure, ANOVA, confirmed systematic bias. Intra-class correlation coefficients and standard errors of measurement indicated relative and absolute reliability. Correlation coefficients were averaged over respective factors and tested for significant differences. All balance scores showed comparable low-to-moderate relative (<0.8–0.9) and good absolute reliability (<10%). Relative reproducibility of dynamic control equilibrium parameters augmented with increasing angular velocity, but not with familiarisation. At 2.62 rad · s^–1, high (moment: 0.906) to moderate (angle: 0.833) relative reliability scores with accordingly high absolute indices (4.9% and 6.4%) became apparent. Thus, the dynamic control equilibrium is an equivalent method for the reliable assessment of thigh muscle balance.     

Toward Cranfield-inspired reusability assessment in interactive information retrieval evaluation

Re-using research resources is essential for advancing knowledge and developing repeatable, empirically solid experiments in scientific fields, including interactive information retrieval (IIR). Despite recent efforts on standardizing research re-use and documentation, how to quantitatively measure the reusability of IIR resources still remains an open challenge. Inspired by the reusability evaluations on Cranfield experiments, our work proactively explores the problem of measuring IIR test collection reusability and makes threefold contributions: (1) constructing a novel usefulness-oriented framework with specific analytical methods for evaluating the reusability of IIR test collections consisting of query sets, document/page sets, and sets of task-document usefulness (tuse); (2) explaining the potential impacts of varying IIR-specific factors (e.g. search tasks, sessions, user characteristics) on test collection reusability; (3) proposing actionable methods for building reusable test collections in IIR and thereby amortizing the true cost of user-oriented evaluations. The Cranfield-inspired reusability assessment framework serves as an initial step towards accurately evaluating the reusability of IIR research resources and measuring the reproducibility of IIR evaluation results. It also demonstrates an innovative approach to integrating the insights from individual heterogeneous user studies with the evaluation techniques developed in standardized ad hoc retrieval experiments, which will facilitate the maturation of IIR fields and eventually benefits both sides of research.     

Evaluating solutions to the problem of false positives

A current challenge for the scientific community is the choice of appropriate policies to reduce the rate of false positives. Existing proposals differ in whether to prioritize tackling omission through transparency requirements, punishing more severe transgressions, or possibly both. We use a formal model to evaluate these possible solutions. We find that a policy that prohibitively increases the cost of ‘misdemeanor’ types of questionable research practices robustly decreases the overall rate of researcher misconduct, because the rate of ‘felonies’, such as fabrication, also decreases. Therefore proposals that aim to prevent lying by omission by enforcing reporting guidelines are likely to be effective in reducing researcher misconduct, but measures such as government audits (purported to counteract pure fraud) can backfire. Moreover, we find that an increase in the rewards of publication need not increase overall misconduct.     

Test–retest reliability of the 30–15 Intermittent Fitness Test: A systematic review

PurposeThis review aimed to synthesize previous findings on the test–retest reliability of the 30–15 Intermittent Fitness Test (IFT).MethodsThe literature searches were performed in 8 databases. Studies that examined the test–retest reliability of the 30–15 IFT and presented the intraclass correlation coefficient (ICC) and/or the coefficient of variation (CV) for maximal velocity and/or peak heart rate were included. The consensus-based standards for the selection of health measurement instruments (COSMIN) checklist was used for the assessment of the methodological quality of the included studies.ResultsSeven studies, with a total of 10 study groups, explored reliability of maximal velocity assessed by the 30–15 IFT. ICCs ranged from 0.80 to 0.99, where 70% of ICCs were ≥0.90. CVs for maximal velocity ranged from 1.5% to 6.0%. Six studies, with a total of 7 study groups, explored reliability of peak heart rate as assessed by the 30–15 IFT. ICCs ranged from 0.90 to 0.97 (i.e., all ICCs were ≥0.90). CVs ranged from 0.6% to 4.8%. All included studies were of excellent methodological quality.ConclusionFrom the results of this systematic review, it can be concluded that the 30–15 IFT has excellent test–retest reliability for both maximal velocity and peak heart rate. The test may, therefore, be used as a reliable measure of fitness in research and sports practice.     

Measurement error in short-term power testing in young people

The aim of this study was to examine the consistency or reproducibility of measuring cycling peak power in children and adults. Twenty-seven pre-pubertal girls and boys and 27 female and male physical education students (age 9.8±0.5 and 24.4±4.3 years, respectively; mean±s) participated in the study. All participants performed five tests over 15 days and underwent a habituation session before the study. Each test included four sprints against four different braking forces. We found that braking forces of 7.5% of body weight in children and 10% of body weight in adults were too high for most of the participants to elicit maximal cycling power. Unlike the children, the physical education students improved their performance between session 1 and session 2 (1025±219 vs 1069±243 W; P<0.001). Therefore, to obtain reproducible measures of cycling peak power, a habituation session including a complete test protocol (i.e. warm-up plus three sprints) is highly recommended. When the protocol included three sprints in children and at least two sprints in adults, measurement of cycling peak power was found to be highly reliable (test-retest coefficient of variation ～3%). Finally, to avoid performance fluctuations, especially over several consecutive evaluations (e.g. longitudinal studies), it is necessary to maintain high motivation in children.     

纳米压痕试验测量不确定度评定

影响纳米压痕试验结果的因素很多,导致分别计算A类和B类不确定度十分困难。该文采取实验室间比对的方法,通过对熔融石英样品进行纳米压痕试验,得出压痕硬度和压痕模量的重复性限r和复现性限R。在r和R的基础上推导出纳米压痕硬度和模量扩展不确定度的近似计算公式,从而大大简化了纳米压痕试验测量不确定度的评定。不确定度评定结果表明纳米压痕试验测量不确定度较大;试验次数超过5次后,其对扩展不确定度的影响不大。     

Test–retest reliability of second lactate turnpoint using two different criteria in competitive cyclists

Abstract

The aim of this study was to determine the relative and absolute reliability of second lactate turnpoint using fixed and individual blood lactate method in competitive cyclists. Twenty-eight male, well-trained cyclists (30.2 ± 10.1 years, 72.0 ± 7.4 kg, 177.3 ± 4.7 cm) were recruited to participate in this study. Cyclists completed two incremental cycling tests to exhaustion over a period of 7 days to determine their peak power output, maximal oxygen uptake, maximal heart rate, maximal blood lactate concentration and two lactate turnpoint criteria. The fixed blood concentration criterion (3.5 mM) and an individual criterion were assessed by a lactate-power curve, considering power output, heart rate and oxygen uptake. The main finding of this study was that both lactate turnpoint criteria showed identical low within-subject variation for power output (2.8% coefficient of variation). High values for test–retest correlations ranging from r = 0.70 to r = 0.94 were found for all variables in both threshold criteria. In conclusion, the individual and fixed method to determine the second lactate turnpoint showed similar high absolute and relative reliability in competitive cyclists.     

Consistency of kinematic and kinetic patterns during a prolonged spell of cricket fast bowling: an exploratory laboratory study

Due to the high incidence of lumbar spine injury in fast bowlers, international cricket organisations advocate limits on workload for bowlers under 19 years of age in training/matches. The purpose of this study was to determine whether significant changes in either fast bowling technique or movement variability could be detected throughout a 10-over bowling spell that exceeded the recommended limit. Twenty-five junior male fast bowlers bowled at competition pace while three-dimensional kinematic and kinetic data were collected for the leading leg, trunk and bowling arm. Separate analyses for the mean and within-participant standard deviation of each variable were performed using repeated measures factorial analyses of variance and computation of effect sizes. No substantial changes were observed in mean values or variability of any kinematic, kinetic or performance variables, which instead revealed a high degree of consistency in kinematic and kinetic patterns. Therefore, the suggestion that exceeding the workload limit per spell causes technique- and loading-related changes associated with lumbar injury risk is not valid and cannot be used to justify the restriction of bowling workload. For injury prevention, the focus instead should be on the long-term effect of repeated spells and on the fast bowling technique itself.     

Assessing child maltreatment prevention via administrative data systems: A case example of reproducibility

Critical issues about scientific reproducibility have been raised about biomedical research, including the reliability of data and analyses within a given study. The case example in this article examined a reproducibility issue pertaining to the use of administrative data systems for evaluation of child maltreatment (CM) prevention, making use of a prevention study conducted over a decade ago that provided a unique opportunity. The place-randomization study, which randomized counties to condition, found that community-wide implementation of a parenting and family support intervention produced positive impact on county-wide rates for substantiated CM cases and out-of-home placements, documented through a state information system. The key consideration is whether and to what extent the administrative record data re-examined retroactively a decade later for the original study’s time period would yield comparable results to those based on data acquired at the time of the study. The results indicated that despite small changes over time, the same data patterns and statistical effects were reproducible for the two archival outcome variables. For substantiated CM, the reproduced analyses reflected higher effect sizes and a clear pattern of reduction as a function of intervention. For out-of-home placements, effect sizes were quite comparable to the original ones, reflecting preventive impact. Overall, this case study illustrated the verifiability of data reproducibility in the context of a population outcome evaluation, which underscores the importance of reliable population-prevalence measurement as an essential part of a comprehensive public health strategy aimed at the prevention of CM.     

Practical perspectives of consumer fairness in recommendation

In recent years, there has been an increasing number of mitigation procedures against consumer unfairness in personalized rankings. However, the experimental protocols adopted so far for evaluating a mitigation procedure were often fundamentally different (e.g., with respect to the fairness definitions, data sets, data splits, and evaluation metrics) and limited to a narrow set of perspectives (e.g., focusing on a single demographic attribute and/or not reporting any analysis on efficiency). This situation makes it challenging for scientists to consciously decide which mitigation procedure better suits their practical setting. In this paper, we investigated the properties a given mitigation procedure against consumer unfairness should be evaluated on, to provide a more holistic view on its effectiveness. We first identified eight technical properties and evaluated the extent to which existing mitigation procedures against consumer unfairness met these properties, qualitatively and quantitatively (when possible), on two public data sets. Then, we outlined the main trends and open issues emerged from our multi-dimensional analysis and provided key practical recommendations for future research. The source code accompanying this paper is available at https://github.com/jackmedda/Perspective-C-Fairness-RecSys.