The Parent Threat Inventory: development, reliability, and validity

OBJECTIVE: The goal was to develop a retrospective inventory of parental threatening behavior to facilitate a better understanding of such behavior's role in the etiology of psychological distress. METHOD: Inventory items were developed based on theory and 135 students' responses to a question eliciting examples of threatening parental behavior. Following item development, two additional student samples (n = 200 and n = 603) completed batteries of self-report measures. Responses were used to eliminate unstable or redundant items from the inventory and to examine the inventory's psychometric properties. RESULTS: Factor analysis of the inventory revealed three factors, accounting for 66.2% of variance; this factor structure is compatible with theory, and consistent across maternal behavior scores, paternal behavior scores, and combined maternal and paternal scores. Cronbach's coefficient alphas indicated acceptable internal consistency; Pearson correlation coefficients indicated acceptable 4-week test-retest reliability. Moderate intercorrelations with two retrospective measures of childhood experiences suggested construct validity. Regression analyses demonstrated the ability of the inventory to predict both anxious and depressive symptomatology and lifetime symptoms of anxiety and depressive disorder. Normative data on combined parent scores, maternal scores, and paternal scores are also presented. CONCLUSIONS: Initial psychometric testing of the Parent Threat Inventory (PTI) suggests it is a reliable and valid tool for investigating the developmental antecedents of adult psychological distress. Further research should focus on addressing two limitations: (1) lack of normative and psychometric data on men and women suffering from clinical disorders, and (2) lack of validation by parental reporting.     

A FACET-FACTORIAL APPROACH TO THE CONSTRUCTION OF RATING SCALES TO MEASURE COMPLEX BEHAVIORS

The purpose of this study was to examine a technique for the development of performance rating scales to measure achievement in courses whose objectives require complex behaviors not easily measurable with paper and pencil achievement tests. A facet-factorial approach to rating scale construction was employed (i.e. the behavior was conceptualized as multidimensional and items for the scales were selected by employing factor analytical techniques) to construct scales to measure clarinet music performance. The three major results of the study were: 1) a thirty-item rating scale based on a six factor structure of clarinet music performance; 2) high inter-judge reliability estimates for both the total score (above .90) and the scale scores (above .60); and, 3) criterion-related validity coefficients greater than .80. Results of the investigation suggest that the facet-factorial approach can be an effective technique for the construction of rating scales to measure complex behavior such as music performance.     

美国大学生满意度调查方法评介

美国全国大学生满意度调查始于1994年,至今已进行了12次.调查所采用的《大学生满意度量表》有5种版本,分别面向不同类型的高等教育机构.量表由70多个项目构成,且类化成12个指标.每个项目和指标均有重要性、满意度和绩差3种得分.通过调查结果,不仅可以让院校了解其强优项和挑战项,调整校生关系政策,提高满意度,促进学生实现学术成就,还可以让政府和社会掌握高等教育机构的办学状态,对高等教育机构实行有效问责.     

The development and validation of the Attitudinal Learning Inventory (ALI): a measure of attitudinal learning and instruction

In this paper, we present the development and validation of a new measure of attitudinal learning—the Attitudinal Learning Inventory (ALI). While specific scales are available for measuring attitudes, they largely focus on established attitudes, not the impact of instruction on those attitudes. We developed the inventory with two explicit objectives: (1) to measure a broad range of attitude constructs representing a holistic view of attitudinal learning and instruction; and (2) to facilitate the measurement of attitudinal learning that can be useful for educational researchers beyond traditional metrics. The ALI was developed and validated across two samples of a total of 1009 participants with diverse demographics. The ALI comprises 15 scale items and exhibited good psychometric properties and conformed to the theoretical four-dimensional structure of attitudinal learning: cognitive, affective, behavioral, and social. The ALI was also shown to correlate with behavioral metrics of class engagement. Future uses of the new measure are discussed. Participants were taken from entirely online populations, and while demographically diverse, implementation of the scale with face-to-face instruction, in varied settings, and across different groups of learners is needed to provide additional evidence of its intended generalizability and consider possible biases.     

A Short Test of Personality: The S-ident Form

Abstract

On the premise that affective measure" are needed by those who would accurately predict the classroom behavior of teachers, an Inventory was developed to assess a teacher's attitude toward one aspect of the teaching process, namely. Instructional objectives stated In terms of student behavior. The inventory consisted of 20 instructional objectives, some behavioral and some nonbehavioral, to be rated according to a five-point scheme. A modest but significant correlation of .25 was obtained between 50 subjects' scores on the inventory and their observed classroom use of behavioral objectives. A construct validity study involving an examination of the pre-and post-instructor factor structure on Inventory scores of 100 prospective teachers also suggested that the instrument possesses a degree of validity.     

Administrative and structural changes in student evaluations of teaching and their effects on overall instructor scores

Using nine years of student evaluation of teaching (SET) data from a large US research university, we examine whether changes to the SET instrument have a substantial impact on overall instructor scores. Our study exploits four distinct natural experiments that arose when the SET instrument was changed. To maximise power, we compare the same course/instructor before and after each of the four changes occurred. We find that switching from in-class, paper course evaluations to online evaluations generates an average change of ?0.14 points on a five-point scale, or 0.25 standard deviations (SDs) in the overall instructor ratings. Changing labelling of the scale and the wording of the overall instructor question generates another decrease in the average rating: ?0.15 of a point (0.27 SDs). In contrast, extending the evaluation period to include the final examination and offering an incentive (early grade release) for completing the evaluations do not have a statistically significant effect on the overall instructor rating. The cumulative impact of these individual changes is ?0.29 points (0.52 SDs). This large decrease shows that SET scores are not comparable over time when instruments change. Therefore, administrators should measure and account for such changes when using historical benchmarks for evaluative purposes (e.g. appointments and compensation).     

Relationships between age and raw score increases on the Kaufman-assessment battery for children

Raw scores on the 16 K-ABC subtests and the total raw scores on the sequential and simultaneous processing scales and the achievement scale were correlated with age in months for two separate samples, each subdivided by race and sex: the K-ABC standardization sample (N = 2000) and an additional group of blacks and whites tested during the development of the K-ABC sociocultural norms (N = 615). Within each sample, the highest and lowest correlations from all race/sex groups were contrasted across all K-ABC subtests and scales. All correlations between age and raw scores were statistically significant (p ⩽ .05). No significant differences occurred in the magnitude of these relationships as a function of race/sex grouping, supporting the construct validity of the K-ABC as a developmental measure of children's aptitude and achievement for blacks, whites, Hispanics, males, and females.     

Measuring styles of learning and motivation

The theoretical basis, the development and the use of an inventory designed to measure styles of learning and motivation are described. On the basis of previous research, items within the inventory were located on 20 sub-scales. The ten motivation sub-scales were derived from a theory of school motivation which was based on extensive interview and inventory work carried out in Hungary with children, their parents and teachers. The ten scales describing styles of learning were derived from concepts describing learning approaches and styles which emerged from interviews and experimental research with students in British and Swedish higher education. Versions were prepared in both English and Hungarian and given to samples of 614 British and 579 Hungarian pupils aged between 13 and 17 years. A remarkably similar factor structure was obtained in each sample. An analysis of mean scores indicated distinctive differences both between boys and girls, and between British and Hungarian pupils. Cluster analysis was used to investigate the possible existence of styles which combined both approaches to learning and aspects of school motivation. The potentiality of the inventory for use in conjunction with study skills courses is discussed.     

Measuring Spanish orthographic development in private,public and subsidised schools in Chile

The current study has a twofold purpose: first, to determine the reliability of a tool for assessing orthographic development in Spanish; second, to assess differences in students’ performance on the measure across multiple types of primary schools in a large city in Chile. A Spanish developmental spelling inventory that contained words of increasing orthographic difficulty was administered twice to students in first through fifth grades in three types of schools: public, private and partially subsidised. These school types in large part represent varying levels of student socio-economic status because of costs associated with attending them. Results demonstrate that (a) the Spanish spelling inventory is able to reliably measure the development of orthographic knowledge in Spanish, and (b) there is a relationship between school type and student achievement. The authors describe individual cases to highlight the qualitative nature of differential performance for students in greater- or lesser-resourced schools.     

Validity and Reliability of Turkish Version of the Balanced Index of Psychological Mindedness

The purpose of current study was to explore psychometric properties of the Turkish version of the Balanced Index of Psychological Mindedness (BIPM). Studies regarding psychological mindedness (PM) are still limited in Turkey; and there are few valid instruments to measure PM. The Turkish version of the BIPM and the 20-Item Toronto Alexithymia Scale (TAS) was administered to 654 voluntary undergraduate students. Confirmatory factor analysis indicated a good fit for the model including two subscales (interest and ability dimensions of PM) with a total of 14 items (x²/df?=?3.11, p?<?.001, RMSEA?=?0.060, S-RMR?=?0.059, GFI?=?0.95, AGFI?=?.0.93, CFI?=?0.94). McDonald’s Omega (ω) for scores on the overall scale was .85, .73 for scores on the Interest Subscale and .76 for scores on the Insight Subscale. Test–retest reliability coefficients were found as .61 for the scores on the BIPM total score, .55 for scores on the Interest subscale and .56 for scores on the Insight subscale (p?<?.01). As expected, the BIPM total scale score were negatively correlated with TAS total score (r?=?-.59).

     

The classroom practices inventory: An observation instrument based on NAEYC's guidelines for developmentally appropriate practices for 4- and 5-year-old children

The controversy over what is an appropriate early childhood curriculum has created a need for research instruments designed to measure classroom practices. This article reports on the development of a new observational measure based on the Guidelines for Developmentally Appropriate Practices of the National Association for the Education of Young Children (NAEYC). The Classroom Practices Inventory (CPI) is a 26-item rating scale tapping the curricular emphasis and emotional climate of programs for 4- and 5-year-old children. The scale demonstrated a high degree of internal consistency. Over half the measure's variance was accounted for by a factor tapping encouragement of curiosity, creativity, and provision of concrete materials. In a study of 10 preschool programs, CPI scores correlated significantly with teachers' and parents' educational attitudes. Modest relationships were found between the CPI scores of children's preschools and measures of academic skills, creativity, and anxiety. The CPI appears to be a promising measure for critically examining the concept of developmentally appropriate practices in early childhood education.     

Measures of Children's Knowledge and Regulation of Cognition

Two studies were conducted to investigate measures of children's metacognition. Experiment 1 presented two versions of a self-report inventory, the Jr. MAI, appropriate for assessing metacognition in children in grades 3–9. Factor analyses are interpreted that illustrate how the items measure components of metacognition. Experiment 2 further addressed properties of the two versions and compared the instrument to other inventories, teacher ratings of children's metacognition, and student achievement scores. Findings indicated high correlations between the Jr. MAI and an existing metacognitive problem-solving inventory (Fortunato, Hecht, Tittle, & Alvarez, 1991). Moderate correlations between the Jr. MAI and other self-report instruments of metacognition while reading (Jacobs & Paris, 1987; Schmitt, 1990) and low correlations between the Jr. MAI and teacher ratings of metacognition and overall achievement scores were also found. Gender and grade level differences in metacognition are presented and discussed. The instruments are appended.     

Intercultural sensitivity among university students: measurement of the construct and its relationship with international mobility programmes / Sensibilidad intercultural de los estudiantes universitarios: medición del constructo y su relación con los programas de movilidad internacional

The purpose of this study was to validate the IDI (Intercultural Development Inventory) and to assess the impact of international mobility programmes on the intercultural sensitivity of university students. For this, a quasi-experimental study was carried out using a longitudinal design of repeated pre-test and post-test measures, with an experimental group of students who carried out international mobility programmes and a control group of students who did not. The sample consisted of 3,047 university students. The inventory yielded a five-factor structure (negation/defence, minimization, reversion, acceptance/adaptation and encapsulated marginality) with alpha coefficients ranging from .82 to .91, revealing adequate reliability and validity. The results indicate the effectiveness of the IDI to measure intercultural sensitivity. Although the post-test scores are higher than the pre-test scores for both groups, there are significant differences in favour of the experimental group (p = .000) in all factors except the negation/defence stage. The conclusions focus on the implications of this study for curriculum development and policy at an educational level.     

A new science and engineering career interest survey for middle school students

This study describes the development and validation of a science and engineering (S/E) career interest survey (CIS). This 56 question survey was developed to measure the overall S/E career interests of 7th through 9th grade students. In the CIS, a S/E career is characterized as one which requires the completion of at least a four-year college program with a major in science, science education, or engineering. The CIS is divided into four major parts. In Part I (30 questions), students are expected to select from occupational activities, while in Part II (20 questions) they are to select from various occupations. Part III (5 questions) and Part IV together make up the CIS internal verification scale. The CIS test-retest reliability coefficients for one week and eight months were calculated as 0.96 (n = 57, grades 7–9) and 0.78 (n = 1937, grade 8), respectively. The KR-21 estimate for the CIS was calculated as 0.92. Criterion-related validity coefficients were calculated in two ways: (a) CIS scores were correlated with the Kuder GIS science subscale (r = 0.75, n = 45, grades 7–9), and (b) CIS scores were correlated with a CIS internal verification scale (r = 0.59, n = 127, grades 7–9). Evidence to support the construct validity of the CIS was collected by two methods: (a) for 7–9 grade students (n = 45), the CIS score was found to correlate 0.75 with the scientific subscale and ?0.42 with the artistic sub-scale, of the Kuder GIS. (b) the second method compared the scores of known groups. Test results for students in grades 7-9 (n = 127; n = 1937) showed a statistically significant difference between the scores of boys and girls on S/E career interest. The readability of the CIS was seventh grade level.     

Assessment of performance in practical science and pupil attributes

Performance assessment in the UK science General Certificate of Secondary Education (GCSE) currently relies on pupil reports of their investigations. These are widely criticized. Written tests of procedural understanding could be used as an alternative, but what exactly do they measure? This paper describes small‐scale research in which there was an analysis of assessments of pupils' GCSE scores of substantive ideas, their coursework performance assessment and a novel written evidence test. Results from these different assessments were compared with each other and with baseline data on CAT scores and pupils' attributes. Significant predictors of performance on each of these assessments were determined. The data reported shows that a choice could be made between practical coursework that links to ‘behaviour’ and written evidence tests which link, albeit less strongly, with ‘quickness’. There would be differential effects on pupils.     

Development of an instrument measuring student teachers’ perceived stressors about the practicum

The Stressors about Practicum Inventory, a self-report measure of perceived stressors about the practicum, was designed to provide those responsible for the training of primary school teachers with an informative, inexpensive and psychometrically sound tool. The present study describes the development and validation of the 94-item inventory in a sample of 138 Greek primary student teachers. It also investigates its initial factor structure and reliability. The scale was administered both before and after a four-week period of practicum. Content validity was examined using a panel of experts and student teachers, whereas convergent validity was assessed with the use of a measure of neuroticism. Factor analyses revealed four sources of perceived stress. No gender differences were found. However, the experience of the practicum significantly reduced student teachers’ perceived stress. All perceived stressors were moderately correlated with neuroticism, but after the practicum the size of the correlations was reduced.     

The design and validation of an instrument to measure teachers’ professional development at work

After graduation, it is essential for teachers to continue working on their professional development since they need to be prepared for the requirements of tomorrow's knowledge- and technology-based society, a more varied student and pupil population creating new responsibilities, and higher social expectations from schools and the society as a whole. Although attention for teachers’ professional development at work is increasing, the availability of valid survey instruments that measure their participation in professional development at work is still limited. Based on a literature review, such an instrument was designed. This 21-item self-report measure, using a 4-point rating scale, makes an inventory of teachers’ participation in activities contributing to professional development. The purpose of the instrument is to make teachers, school leaders and other stakeholders aware of whether, and to what degree, teachers engage in learning at work. This is important because ultimately the quality of education depends on it. A survey was administered in 9 Dutch primary schools and 15 Dutch secondary schools. Exploratory and Confirmatory Factor Analysis confirmed a six-factor structure, and augments earlier findings in the field. The reliabilities of the six subscales were sufficient to good.     

Differential Improvement in Student Understanding of Mathematical Principles Following Formative Assessment Intervention

ABSTRACT

The authors describe results from a study of a middle school mathematics formative assessment strategy. They employed a randomized, controlled design to address the following question: Does using our strategy improve student performance on assessments of key mathematical ideas relative to a comparison group? Eighty-five teachers and 4,091 students were included. Students took a pretest and a transfer measure at the end of the year. Treatment students completed formative assessments. Treatment teachers had exposure to professional development and instructional resources. Results indicated students with higher pretest scores benefited more from the treatment compared to students with lower pretest scores. In addition treatment students significantly outperformed control students on distributive property items. This effect was larger as pretest scores increased. Results, limitations, and future directions are discussed.     

A Scale for the Evaluation of Structure Needs and Perceptions in Adolescence

An inventory was constructed which purports to measure the extent to which adolescents want structure and also perceive themselves as having such structure. “Structure”, as here defined, refers to guidance, advice, information, clarity, or direction offered to the adolescent by an adult authority figure. The scale was administered to approximately 1, 700 adolescents at junior and senior high school level on two separate testing occasions. Item analysis reduced the total number of items from 77 to 47 and improved the reliability considerably. Low correlations between the Wants Structure (WS) and Has Structure (HS) sub-scales suggest that the two are not measuring the same thing. Support given a number of hypotheses based on expected relationships between the structure inventory and other tests suggests that the former is measuring what it claims to measure, but further validation is desirable. The inventory seems to have some promise for research in an area which has been largely overlooked.