THE VALIDITY AND RELIABILITY OF ORAL EXAMINATIONS IN ASSESSING COGNITIVE SKILLS IN MEDICINE1

In order to attempt to assess aspects of clinical competence, not adequately assessed by other means, the Center for the Study of Medical Education, University of Illinois College of Medicine together with the American Board of Orthopaedic Surgery developed oral examinations in formats specifically designed to yield information on high level cognitive functioning. The examinations were administered to 784 candidates for certification in January 1968. Reliability of the oral problem-solving component score pooled from four examiners was approximately .50. Assessment of content, construct, and concurrent validity made by questionnaire and factor analytic studies indicated that the oral tests identified factors not measured by multiple-choice tests and, therefore, significantly improved the relationship between supervisory evaluations and test scores.     

A Model for Training Group Counselors

A teaching model is presented which was used for increasing the skills and interests of practicing school counselors in group counseling. A significant aspect of the model is its inclusion of learnings designed to explore both affective and cognitive aspects of the group process. Although the model described was used in an in-service workshop for counselors, the format might also be useful in designing a university course in group counseling.     

Three Counselor Training Models: A Comparative Study

This study investigated the effects of three prepracticum counselor training approaches on counselor performance: T group, behavioral skills, and discussion control. Pre- and postmeasures consisting of 30-minute video-recorded counseling interviews were administered to all subjects. Counselor performance was evaluated by client completion of the Counseling Evaluation Inventory (CEI) and by three trained judges' ratings of four five-minute interview segments using the Counseling Strategies Checklist (CSC). Data from each instrument were subjected to a 3 × 2 factorial analysis of variance. A significant group × time interaction occurred for the CEI. Significant time effects occurred on four of the seven dependent variables of the CSC. The results of the study have implications for both the length and type of training approach used in effective counselor education practices.     

A FACET-FACTORIAL APPROACH TO THE CONSTRUCTION OF RATING SCALES TO MEASURE COMPLEX BEHAVIORS

The purpose of this study was to examine a technique for the development of performance rating scales to measure achievement in courses whose objectives require complex behaviors not easily measurable with paper and pencil achievement tests. A facet-factorial approach to rating scale construction was employed (i.e. the behavior was conceptualized as multidimensional and items for the scales were selected by employing factor analytical techniques) to construct scales to measure clarinet music performance. The three major results of the study were: 1) a thirty-item rating scale based on a six factor structure of clarinet music performance; 2) high inter-judge reliability estimates for both the total score (above .90) and the scale scores (above .60); and, 3) criterion-related validity coefficients greater than .80. Results of the investigation suggest that the facet-factorial approach can be an effective technique for the construction of rating scales to measure complex behavior such as music performance.