Establishing Weights for Test Plans for Licensure and Certification Examinations

This study evaluated two methods for establishing weights for test plans for certification examinations. One method required a panel of experts to provide holistic judgments indicating the percentage of test questions to allocate to each content category. Weights were first obtained from individual panel members, discussed by the entire panel, and then finalized by group consensus. The other method derived weights using a statistical model. The model included ratings of task frequency and task criticality provided by a large sample of practitioners as well as information from a panel of experts concerning the linkages between specific tasks and the knowledge and skills required to perform those tasks. The study was replicated for four medical imaging specialties in the field of radiologic technology. The weights for the two methods exhibited moderate to high levels of agreement for sections of the test plans comprised of specific imaging procedures. However, there was much less agreement for those sections of the test plans that addressed more general topics. Possible reasons for the observed pattern of agreement and disagreement are considered.     

Combining Data on Criticality and Frequency in Developing Test Plans for Licensure and Certification Examinations

Job analysis is a critical component in evaluating the validity of many high-stakes testing programs, particularly those used for licensure or certification. The ratings of criticality and frequency of various activities that are derived from such job analyses can be combined in a number of ways. This paper develops a multiplicative model as a natural and effective way to combine ratings o f frequency and criticality in order to obtain estimates of the relative importance of different activities for practice. An example of the model's use is presented. The multiplicative model incorporates adjustments to ensure that the effective weights of frequency and criticality are appropriate.     

Use of Knowledge, Skill, and Ability Statements in Developing Licensure and Certification Examinations

The task inventory approach is commonly used in job analysis for establishing content validity evidence supporting the use and interpretation of licensure and certification examinations. Although the results of a task inventory survey provide job task-related information that can be used as a reliable and valid source for test development, it is often the knowledge, skills, and abilities (KSAs) required for performing the tasks, rather than the job tasks themselves, which are tested by licensure and certification exams. This article presents a framework that addresses the important role of KSAs in developing and validating licensure and certification examinations. This includes the use of KSAs in linking job task survey results to the test content outline, transferring job task weights to test specifications, and eventually applying the results to the development of the test items. The impact of using KSAs in the development of test specifications is illustrated from job analyses for two diverse professions. One method for transferring job task weights from the job analysis to test specifications through KSAs is also presented, along with examples. The two examples demonstrated in this article are taken from nursing certification and real estate licensure programs. However, the methodology for using KSAs to link job tasks and test content is also applicable in the development of teacher credentialing examinations.     

Rejoinder: A Further Discussion of Job Analysis and Use of KSAs in Developing Licensure and Certification Examinations: A Response to LaDuca

In his commentary to our paper on the use of knowledge, skill, and ability statements in developing credentialing examinations ( Wang, Schnipke, & Witt, 2005 ), Dr. LaDuca set forth his concerns while commending our paper for providing helpful insights into the importance of careful delineation of KSAs. We believe that there is little substantive disagreement between our position and Dr. LaDuca's. Based on the counterarguments offered in his commentary, we believe that some misunderstanding occurred regarding our main point. Our response here further clarifies our main point to address Dr. LaDuca's criticisms. Additional explanations are provided to indicate that our paper is intended to address job analysis issues over a broader and diverse array of professions/occupations/fields, and our perspective is not limited to the context of credentialing physicians. We recommend that, for such professions/occupations/fields for which task inventory constitutes an appropriate approach to job analysis, the use of KSAs should be considered to establish a linkage between the credentialing examination and the job tasks.     

Repeating History: Teacher Licensure Testing in Massachusetts

This paper reviews the history of initial teacher licensure testing in the United States from the 1980s and summarizes the issues and problems that have ensued. The author shows how Massachusetts, a state that has only fairly recently started this testing, is experiencing many of the same difficulties. Included are extensive tables, based on the authors research, providing information concerning the specific tests that were used in the past and an update showing what most states participating in this testing are currently using for their teacher licensure purposes.     

考试经济与社会发展——以科举、高考为例

考试使用范围和规模若达到一定程度,就会形成考试产业链,与考试有关的各种产业都被带动,并直接从考试活动中获取经济利益。科举和高考通过各自考试产业的形成发挥了可观的经济功效,进而形成各自社会的考试经济。考试经济兴盛与否,主要取决于考试活动,包括考试制度存在时间的长短及其完善程度,以及考试在所处社会的重要性与影响力之大小。考试经济的兴旺发达积极作用于社会各方面发展的同时,也会带来消极影响。只有对考试经济兴利除弊,才能促进考试和社会的协调发展,才能最大限度地发挥考试的社会功能。     

纽约州教师资格证书考试体系及特征

纽约州教师资格考试已有两个世纪的历史,其考试体系日臻完善。通过对纽约州教师资格考试体系的介绍和分析,可以看出其主要有四个特征：证书种类多,适合不同需要;实行资格证书晋级制度：教师资格考试与教师专业团体联系密切：采用标准参照评价。     

Using Response Time to Detect Item Preknowledge in Computer‐Based Licensure Examinations

This article addresses the issue of how to detect item preknowledge using item response time data in two computer‐based large‐scale licensure examinations. Item preknowledge is indicated by an unexpected short response time and a correct response. Two samples were used for detecting item preknowledge for each examination. The first sample was from the early stage of the operational test and was used for item calibration. The second sample was from the late stage of the operational test, which may feature item preknowledge. The purpose of this research was to explore whether there was evidence of item preknowledge and compromised items in the second sample using the parameters estimated from the first sample. The results showed that for one nonadaptive operational examination, two items (of 111) were potentially exposed, and two candidates (of 1,172) showed some indications of preknowledge on multiple items. For another licensure examination that featured computerized adaptive testing, there was no indication of item preknowledge or compromised items. Implications for detected aberrant examinees and compromised items are discussed in the article.     

Teacher Testing and Certification: An Historical Perspective from Ontario

Certification tests for elementary teachers in Ontario were introduced in 1871 and in 2002. Although the provincial government’s stated goals for the testing programs were similar, the 2002 program was opposed by the initial teacher education programs and the teachers’ associations, but the 1871 program was not. The authors argue that much of this difference can be attributed to the relationship of the tests to other requirements for certification and the process for determining the content of the tests.     

Preparing MSW Students for Social Work Licensure: A Curricular Case Example

Licensing has been a dynamic tension for the social work profession for many years, specifically in social work education. Increasingly, social work programs are using factors related to social work licensing (pass rates, number of test takers, etc.) as an indicator of programmatic success. Yet few, if any, published papers examine curricular innovations designed to prepare students for social work licensing exams. This article describes the initiative that one MSW program implemented to address licensing preparedness. After a brief overview of social work licensing, this article describes the contextual rationale, implementation, and evaluation of the Licensing Preparation Initiative (LPI). Since its inception in spring 2011, LPI data suggest that students are more knowledgeable about the process for applying to sit for the exam and the content areas of the exam, more prepared to take the social work licensing exam, and more confident in their ability to pass the social work licensing exam once they graduate. This article concludes with specific strategies and resources for designing a curricular initiative to address licensing preparedness.     

多维视野中社会考试改革与发展的思考——以上海为例

本文从社会转型发展和教育公平的视野来审视社会考试的改革与发展。为适应经济的转型发展,社会考试须积极主动地与各类职业院校合作,与重点发展产业的行业协会合作,联合开发和创建相关产业的职业资格证书认证考试体系;从教育公平的角度来看,要逐步建立起考试项目的质量评估和监督机制,从管理规范来保障社会考试的公平和安全。最后,社会考试要加强与各类教育机构、行业协会、劳动技能部门和人事部门等横向合作,为构建各类教育纵向衔接、横向沟通的立交桥发挥更大的作用。     

Outlier Detection in High-Stakes Certification Testing

Recent developrnents of person-Jit analysis in computerized adaptive testing (CAT) are discussed. Methods from stutistical process control are presented that have been proposed to classify an item score pattern as fitting or misjitting the underlying item response theory model in CAT. Most person-fit research in CAT is restricted to simulated data. In this study, empirical data from a certification test were used, Alternatives are discussed to generate norms so that bounds can be determined to classify an item score pattern as fitting or misfitting. Using bounds determined from a sample of a high-stakes certification test, the empirical analysis showed that dizerent types of misfit can be distinguished. Further applications using statistical process control methods to detect misfitting item score patterns are discussed.     

考试的社会功能探究

考试尤其是大规模的考试,自诞生之日起就承担着为社会选拔和筛选人才的功能。从社会学的角度看,考试这一选拔和筛选人才的功能实质上是社会分层和社会控制在教育活动中的具体运用。首先,考试的历史发展和考试本身的层次性等皆证明考试是社会分层和选拔的工具。其次,考试的内容和过程等隐含着社会控制。所以,社会分层和社会控制是考试所隐含的社会功能,考试是课程知识合法化的工具。     

提高海船船员适任考试通过率的思考与实践

近年来,全国各院校和培训机构海船船员(操作级)考证通过率不容乐观,该文结合教学实践,从生源质量、学生就业意愿、教学管理、教风学风多方面分析,尝试找出考证合格率低的原因所在,并从学校基本制度、教风学风建设、教学管理等多方面提出解决对策并实践,以提高考证合格率和船员专业素质.     

略论科举制废除后的社会影响

1905年,科举制度废除后,学堂教育迅速发展,但由于科举制度的断然废除导致了社会承受能力的提前到来,引发了相应的社会危机,加上兴办学堂社会动员的失当导致了大量的社会弊病,从而严重地削弱了变革的影响,造成了社会的普遍弱化,最终也加速了清政府的衰亡。     

普通高中学业水平合格性考试试卷设计与命题的思考

在新一轮高考改革的制度设计中,普通高中学业水平考试分为合格性考试(合格考)和等级性考试(等级考)两种。作为基础教育的一个环节和高考制度的一个重要组成部分,合格考的试卷设计与命题工作,要体现立德树人的根本要求,遵循大规模教育考试命题的各项原则和要求,同时也具有其自身的特点。在明确合格考的功能、定位的前提下,开展试卷设计与命题的研究与实践,以充分发挥其在教育、教学、评价中的特殊作用,使高考改革平稳高质有序推进。