Establishing Weights for Test Plans for Licensure and Certification Examinations

This study evaluated two methods for establishing weights for test plans for certification examinations. One method required a panel of experts to provide holistic judgments indicating the percentage of test questions to allocate to each content category. Weights were first obtained from individual panel members, discussed by the entire panel, and then finalized by group consensus. The other method derived weights using a statistical model. The model included ratings of task frequency and task criticality provided by a large sample of practitioners as well as information from a panel of experts concerning the linkages between specific tasks and the knowledge and skills required to perform those tasks. The study was replicated for four medical imaging specialties in the field of radiologic technology. The weights for the two methods exhibited moderate to high levels of agreement for sections of the test plans comprised of specific imaging procedures. However, there was much less agreement for those sections of the test plans that addressed more general topics. Possible reasons for the observed pattern of agreement and disagreement are considered.