The Influence of Labels and Positions in Rating Scales

On occasion, rating scales are used in which the verbal labels are compressed to saturate one end of the response continuum. In these scales, differences arise between the normal meaning of the label and its scalar position. This paper investigated the influence of the label and the position in defining the meaning of the labeled position. Three forms of labeled rating scale were constructed where the labels FAIR and GOOD were systematically moved. Subjects judged the desirability of premarked responses on the scales. In addition, two forms were used to determine the desirability of the unlabeled scale position and the desirability of the label independent of a rating scale. When label and position differed in meaning, the labeled position was rated as a compromise between the scale value of the label and the position     

DETECTING PARENTAL DECEPTION USING A BEHAVIOR RATING SCALE DURING ASSESSMENT OF ATTENTION‐DEFICIT/HYPERACTIVITY DISORDER: AN EXPERIMENTAL STUDY

It is often assumed that parents completing behavior rating scales during the assessment of attention‐deficit/hyperactivity disorder (ADHD) can deliberately manipulate the outcomes of the assessment. To detect these actions, items designed to detect over‐reporting or under‐reporting of results are sometimes embedded in such rating scales. This study presents the results of an experimental study in which parents (a) read a scenario telling them that their hypothetical son's teacher has suggested their son may have ADHD and (b) considered assigned goals for the assessment. Parents then completed the accompanying Conners 3Parent Short form (Conners 3‐P[S]) in a manner that they believed would achieve their assigned goals. Findings showed that parents are able to engage in deception when completing behavior rating scales. The validity scales embedded in the Conners 3‐P(S), however, demonstrated mixed results for detecting parental deception with the Negative Impression validity scale, accurately detecting attempts to malinger in the majority of cases, whereas the Positive Impression validity scale appears to have little to no diagnostic utility for the detection of defensive responding. Clinicians utilizing behavior rating scales should carefully consider results, and nonresults, obtained from embedded validity scales when interpreting parent responses to behavior rating scales as part of an ADHD assessment.     

Inclusive cultures,policies and practices in Spanish compulsory secondary education schools: teachers’ perceptions in ordinary and specific teaching contexts

This article presents a study that attempts to inquire into the indicators related with inclusive education by taking into account teachers’ responses in the ordinary and specific teaching contexts (experts in special education needs, specialists in therapeutic pedagogy and compensatory education or specific programmes organised to accommodate diversity). We used a Spanish translation of the Index for Inclusion, which has been adapted and divided into three Likert‐type scales about inclusive cultures, organisation in the teaching–learning context and inclusive practices. We did a t‐test to detect any significant differences in teachers’ responses in terms of the ordinary or specific teaching contexts in which they work. No significant differences were seen in the items related to cultures and organisation, but we found significant differences in 12 of the 36 items of the inclusive practices scale. Finally, we discuss the role of the processes for innovation and transformation in secondary schools and provide key factors to build an intercultural inclusive school.     

Validity of the home and community social behavior scales: Comparisons with five behavior‐rating scales

Three separate studies focusing on convergent and discriminant validity evidence for the Home and Community Social Behavior Scales are presented. The HCSBS is a 65‐item social behavior‐rating scale for use by parents and caretakers of children and youth ages 5–18. It is a parent‐rating version of the School Social Behavior Scales. Within these studies, relationships with five behavior‐rating scales were examined: the Social Skills Rating System, Conners Parent Rating Scale–Revised‐Short Form, Child Behavior Checklist, and the child and adolescent versions of the Behavior Assessment System for Children. HCSBS Scale A, Social Competence, evidenced strong positive correlations with measures of social skills and adaptability, strong negative correlations with measures of externalizing behavior problems, and modest negative correlations with measures of internalizing and atypical behavior problems. HCSBS Scale B, Antisocial Behavior, evidenced strong positive correlations with measures of externalizing behavior problems, modest positive correlations with measures of internalizing and atypical behavior problems, and strong negative correlations with measures of social skills and adaptability. These results support the HCSBS as a measure of social competence and antisocial behavior of children and youth. © 2001 John Wiley & Sons, Inc.     

The Personality of Pre-Theological Majors

Two personality scales, each consisting of twenty-two items with Likert type response options, were administered to sixty-one graduate students. The items were keyed by three methods: a.) Likert weighting, b.) unit weighting, dichotomizing between the agree and disagree options, and c.) unit weighting, dichotomizing as close as possible to p = .50. Internal consistency reliability coefficients were computed for both scales, keyed by the three methods. For both scales, keying method c.) resulted in reliability coefficients as high as those obtained with method a.) . Method b.) however, effected a drastic drop in reliability in the scale with many highly skewed item response distributions. It was concluded that multiple-response personality items with highly skewed response distributions can be useful in a unit weighted keying system if they are dichotomized close to p = .50 rather than between the agree and disagree responses.     

The arts,social inclusion and social class: The case of dance

This article places the results of an empirical research study on the relationship between the social class factor and young people's perceptions of dance within the context of recent British government initiatives promoting social and educational inclusion through the arts. Four Likert‐type dance attitude scales that were developed from pupil opinion expressed during prior discussions were included in a survey questionnaire and responses sought from a nation‐wide representative sample of 1298 11–16‐year‐old boys and girls. Analyses of the data showed that attitudes to aspects of dance varied with social class and that there was an interaction between gender and social class in just one instance. A major implication of the research is that dance and the arts should be more widely available in schools so that all children and young people can have access to aesthetic experiences that have the potential to improve quality of life. It is concluded that the current limited provision within the National Curriculum may be contributing to the social and educational exclusion of some pupils.     

Improving the quality of student ratings of instruction: A look at two strategies

The effects of rating scale format (behaviorally anchored vs. Likert) and rater training on leniency and halo in student ratings of instruction were investigated. The subjects (N=269) were students enrolled in required courses at a graduate theological seminary in the Southwest United States. A repeated measures design controlling for teacher and course was used. Findings indicated: (a) training was effective in reducing leniency and halo in ratings from both instruments; (b) trained raters exhibited less leniency on two rating dimensions when using behaviorally anchored rating scales (BARS's) than when using the Likert scale; and (c) trained raters exhibited less halo when using the Likert than when using the BARS. The findings demonstrate the importance of focusing efforts to improve quality of ratings on the students rather than on the format of the instrument.Presented at the Twenty-Eighth Annual Forum of the Association for Institutional Research, Phoenix, Ariz., May 1988.     

Gender Differences in Pupil Attitudes to the National Curriculum Foundation Subjects of English,Mathematics, Science and Technology in Key Stage 3 in South Wales

The attitudes of pupils in South Wales to mathematics, English, science and technology were tested using a Likert‐type attitude scale. Pupils were selected from National Curriculum Key Stage 3, specifically Year 9 (age 13‐14 years). Schools were selected by their position in the 1992 National League tables produced by the Welsh Office, the schools being placed into one of four bands. The number of schools involved was 34 and the number of pupils 4263. This represents 15.3% of the total population of school children of age 13‐14 years in the region (Welsh Office, 1993). The results show gender differences in attitudes to English, mathematics, science and technology and some between school differences. The results are subjected to a factor analysis and are discussed in relation to groups of statements reflecting key aspects of each subject scale.     

Same constructs,different results: Examining the consistency of two behavior‐rating scales with referred preschoolers

Broad‐band or multidimensional behavior‐rating scales are common tools for evaluating children. Two popular behavior‐rating scales, the Behavior Assessment System for Children, Second Edition (BASC‐2; Reynolds & Kamphaus, 2004) and the Child Behavior Checklist (CBCL; Achenbach & Rescorla, 2000), have undergone downward extensions so that the preschool versions are available for children as young as 18 months. Limited research, however, has been conducted on the preschool versions of the BASC‐2 and CBCL. This study examined the consistency of results from the two preschool versions when completed by parents of clinically referred preschoolers. Comparisons of similarly named scales found significant correlations. Mean scores for several of the constructs were significantly different, however, and often resulted in inconsistent classification decisions. © 2010 Wiley Periodicals, Inc.     

Training Staff to Measure the Engagement of Children with Disabilities in Inclusive Childcare Centres

Practitioner use of the revised Individual Child Engagement Record – Revised (ICER‐R) for observing children with disabilities in inclusive childcare is examined. Training in the use of the ICER‐R, which includes both a momentary time sampling observation system and rating scales, was provided across two training phases with five to seven participants. Four of the five participants who completed both phases of the training achieved greater than 80% inter‐observer agreement on the observation measure within the allocated time, and were able to maintain this level of agreement over subsequent observation sessions. There was consistency in ratings across observers and positive, statistically significant correlations were found for data gathered using momentary time sampling and data from rating scales. The practical value of the ICER‐R was supported by feedback provided through evaluation questionnaires completed by participants. Training childcare practitioners to systematically observe children with disabilities in inclusive childcare has the potential to benefit both children and staff.     

Observations and ratings of preschool children's social behavior: Issues of representativeness and validity

Data were gathered from videotaped recordings of two preschool children engaged in unstructured free play over 12 days each. Observers coded behavior from the videotapes and completed a behavior rating scale for each child after every two observation sessions. Teachers also completed two behavior rating scales per child. Results indicated that at least three 30‐min observation sessions were required to reliably represent a child's overall behavior. Moderate correlations were obtained when observations were compared with teachers' and observers' own ratings, indicating the behavior rating scale did an adequate job of reflecting actual observed behavior. The implications of these results for researchers and practitioners are discussed. © 2005 Wiley Periodicals, Inc. Psychol Schs 42: 13–26, 2005.     

Likert等级量表调查结果有效性的影响因素探析

Likert等级量表是社会科学研究领域广泛使用的一种调查工具。从量表的构制、调查的开展、数据的录入等三个方面入手,探析影响Likert等级量表调查结果有效性的各种因素,同时提出解决这些问题的一些建议和对策。     

Stability of the adjustment scales for children and adolescents

Investigation of the short‐term (90‐day) stability of the Adjustment Scales for Children and Adolescents is reported for 124 randomly selected children in grades ranging from kindergarten to grade 12. Significant test‐retest stability coefficients were obtained and mean differences across the retest interval did not exceed .8 raw score points. The Solitary Aggressive‐Impulsive, Diffident, and Lethargic/Hypoactive syndromes and the global Underactivity scale showed significant raw score and T score changes across the retest interval, but the effect strengths were small. Syndromic Profile Classifications and Discriminant Classifications were also significantly consistent across the retest interval. Results were similar to those obtained in other stability studies of teacher report behavior rating scales. © 2001 John Wiley & Sons, Inc.     

Perceptions as Measures of Training Transfer

This study investigated the extent to which trainees' perceptions of the applicability of what they learned, gathered immediately after training, can predict their perceptions of actual application of the training six months after they return to the job. It also investigated whether support to apply what trainees learned during training after they returned to the job affected their six‐month perceptions. The study found that in all cases, the six‐month perceptions were statistically significantly lower than the immediate after‐course perceptions. However, using a three‐point Likert scale, with “good” being the highest rating, the actual practical judgments about the training program retained the same rating (good) for the program that received the most job environment support. For the training program that received less support, the decisions made from the initial data and the follow‐up data were different. The initial ratings were “good,” and the follow‐up ratings were “fair.” This study suggests that the immediate after‐course perceptions of training are a reasonable estimate of the six‐month perceptions when there is substantial reinforcement from the home job environment.     

Student evaluation of teaching: the use of best–worst scaling

An important purpose of student evaluation of teaching is to inform an educator’s reflection about the strengths and weaknesses of their teaching approaches. Quantitative instruments are one way of obtaining student responses. They have traditionally taken the form of surveys in which students provide their responses to various statements using item-by-item agree/disagree ratings. Previous research has identified shortcomings of such rating scales, including response bias and the associated lack of discrimination amongst the items evaluated. In this paper, best–worst scaling is proposed as a novel method for quantitative teaching evaluation. The way in which best–worst scaling can be used in this context is illustrated in three different applications. Two applications demonstrate how it can be used for evaluations in a small-size classroom environment. The third application is a broader evaluation of university courses on a larger scale. In comparison with conventional rating scales, the best–worst scaling approach enables better highlighting of the differences between evaluation items. In doing so, it can provide enhanced guidance to educators in their reflection about their teaching. Moreover, implementation and analysis of a best–worst scaling evaluation is relatively straightforward, which establishes it a feasible method for teaching practitioners and researchers.     

Skills for the 21st Century Supervisor: What Factory Personnel Think

The study reported here is an attempt to fill a void in research regarding the opinions of people who actually work on the shop floor. In addition to providing historical and modern perspectives on the supervisory profession, its primary contribution is a prioritized set of 30 supervisory skills, synthesized from extant literature in the area of human performance. Factory personnel believe these skills are what are needed for leading and improving employee performance in the complex manufacturing environment of the 21st century. Using Likert‐type rating scales to serve as the data collection instrument, subject groups (sorted by employees, supervisors, and managers) representing manufacturing firms in the Waterloo/Cedar Falls, Iowa, metropolitan areas were asked to rate the importance of each of the 30 skills. Three data analyses were conducted: a Pareto analysis to determine which categories and skills were most important, a comparative analysis to measure how the groups differed in their ratings, and a one‐way analysis of variance F‐test to determine significant differences between the mean ratings of the groups. Where significant differences were discovered, apost hoc test was also conducted to assess pairwise differences. This study was successful in identifying a Taxonomy of Supervisory Skills for improving employee performance. Although the subject groups rated all 30 skills relatively high, six specific skills were found to be most important. In the rating of nine particular skills, significant pairwise differences were discovered.     

Evidence‐Based Survey Design: The Use of a Midpoint on the Likert Scale

Likert‐type scales are often used in survey instruments, and practitioners and researchers need to clearly understand the appropriate use of a midpoint in these scales. The authors of this article explore research studies from various disciplines to indicate that there are circumstances when a midpoint should be included and others where it should not. They provide tables, summarizing the benefits and problems in each case as well as evidence‐based strategies to employ.     

Inter-parent agreement on four preschool behavior rating scales: Effects of parent and child gender

This study investigated inter-parent agreement on four preschool behavior rating scales. Sixty-four cohabiting biological parents rated their four and five year-old children's positive and negative behaviors. Results indicated low to moderate inter-parent correlations across all four preschool behavior rating scales for the total sample. Although mean score differences were not significantly different by parent or child gender, absolute differences between pairs of mothers' and fathers' ratings were large and consistent across instruments, as well as by gender of child and gender of parent. © 1996 John Wiley & Sons, Inc.     

Applying Multigroup Confirmatory Factor Models for Continuous Outcomes to Likert Scale Data Complicates Meaningful Group Comparisons

Treating Likert rating scale data as continuous outcomes in confirmatory factor analysis violates the assumption of multivariate normality. Given certain requirements pertaining to the number of categories, skewness, size of the factor loadings, and so forth, it seems nevertheless possible to recover true parameter values if the data stem from a single homogeneous population. It is shown that, in a multigroup context, an analysis of Likert data under the assumption of multivariate normality may distort the factor structure differently across groups. In that case, investigations of measurement invariance (MI), which are necessary for meaningful group comparisons, are problematic. Analyzing subscale scores computed from Likert items does not seem to solve the problem.