What Are They Thinking? The Development and Use of an Instrument That Identifies Common Science Misconceptions

This article describes the rationale for, and development of, an online instrument that helps identify commonly held science misconceptions. Science Beliefs is a 47-item instrument that targets topics in chemistry, physics, biology, earth science, and astronomy. It utilizes a true or false, along with a written-explanation, format. The true or false responses provide a cursory view of the extent to which specific beliefs are prevalent, while the accompanying explanations reveal underlying reasons for those beliefs. The stages of instrument development, reliability and validity information, along with the original sources of the items are discussed. The developed instrument has the potential to help science educators understand some specific barriers to deepening understanding across a range of science topics.     

The Derivation of Norms

This study focused on the development of an instrument for the measurement of the quality of a student’s perceived relationship with his peers in an elementary school classroom. Jackson’s model of person-group relationships was employed for describing a student’s group membership. An initial 36-item instrument was administered to 181 fifth and sixth grade students. A factor analysis performed on these data resulted in a new 22-item instrument (My Classmates). This instrument was then administered to 621 fifth and sixth grade students. The quality of a student’s group membership was found to be related to his or her achievement level in reading, level of school-related anxiety, attitudes toward school and self-concept as a learner.     

Development of an instrument to measure perceived cognitive,affective, and psychomotor learning in traditional and virtual classroom higher education settings

The purpose of this study was to develop and validate a self-report instrument that can be used to measure learning in the cognitive, affective, and psychomotor domains. The study underwent three phases, each with its own data collection and analysis. Phase I featured the development, testing, and factor analysis of an 80-item instrument that addressed cognitive, affective, and psychomotor learning that was administered to a sample of 142 online and face-to-face learners. Based on the results, the instrument was reduced to 21 items for Phase II and tested with a new sample of 171 online and face-to-face students. The results of confirmatory factor analysis suggested a better data fit with an even smaller 9-item instrument, which was then administered to a new sample of 221 online and face-to-face students in Phase III. The results of this final phase are presented along with the resulting CAP Perceived Learning Scale, a 9-item self-report measure of perceived cognitive, affective, and psychomotor learning. Implications and usage of the CAP Perceived Learning Scale for research and practice are also discussed.     

Construction and Validation of a Chinese Value of Learning Scale (CVLS)

The Chinese Value of Learning Scale (CVLS) is a 5-item instrument for assessing the value of learning in Chinese culture. The psychometric properties of CVLS were investigated in a convenient sample of 583 Chinese soon-to-be-aged adults in Hong Kong. CVLS possesses high internal consistency and exhibited satisfactory short-term test-retest reliability. CVLS also exhibited high face validity and mild convergent validity with other 48-item openness-to-experience scale (OTES) from Revised NEO Personality Inventory, and is significantly related to the past and actual learning experience. In view of its brevity and easy administration, CVLS offers a useful instrument to further investigate the value of learning in Chinese culture and its possible relationship with adults' learning behaviour.     

The Implementation of School Inclusion Practices for Students with Special Needs in Israel: Teachers’ Perceptions

The research examined how school inclusion practices for students with special needs in Israel are perceived by teachers. Participating in the study were 121 elementary and secondary teachers in 10 schools around the country. The instrument was the 50-item ‘Implementation of inclusion practices in schools for students with special needs’ questionnaire, developed by the author together with seminar students. The findings indicate that inclusion practice as a whole was perceived to be implemented in the schools to a moderate degree. When testing the five components of inclusion separately, moral obligation was perceived to be implemented to a significantly greater degree and the knowledge component to a significantly lesser degree than the others. Inclusion practice was found to be implemented more in schools with medium-high socio-economic-level students than in those with low socio-economic-level students, and to a greater degree in the Jewish sector than in the Arab.     

理工科学生英语语用能力发展的纵向干预研究

本项研究以58名理工科学生为实验对象,考察了不同的教学方法对言语行为语用能力发展的效用。这些受试接受了持续12周、每周半小时的关于13种言语行为的语用知识的明示和暗示教学。前测和后测结果的t值检验表明,在明示教学的四个测试项目上受试有着显著进步,暗示教学则未带来任何显著变化。在大多数课堂教学中未涉及的言语行为测试项目上,受试或表现退步,或困惑不解。本项研究亦表明语用能力测试问卷的信度和效度还有待提高。     

Measuring Elementary Student’s Mathematics Motivation: A Validity Study

The psychometric properties of a 10-item math motivation scale were empirically validated with an independent sample consisting of 182 elementary-school students. Analysis of the model dimensionality supported a one-factor structure fit. Item parameter estimates from a Classical Test Theory framework revealed that most items were highly discriminating, and the survey is informative for students of low to average math motivation. Differential item functioning (DIF) analyses found two items exhibiting gender bias. Overall, the instrument was a psychometrically valid instrument for measuring math motivation at the elementary level.     

The Development of Measures of Individual Differences in Self-Regulatory Control and Their Relationship to Academic Performance

Two studies are reported describing the development and validation of the Strategic Flexibility Questionnaire (SFQ): a self-report instrument aimed at eliciting students’ beliefs about the need for, and conditional nature of, self-regulatory control over learning. In Study 1, 281 first-year university education students completed a 40-item pilot questionnaire. Factor analysis of responses revealed a 21-item instrument indicating three types of control beliefs: adaptive executive control, inflexible executive control, and irresolute executive control. In Study 2, the predictive validity of these conceptions was tested against the academic performance of 105 third-year university education students. Results indicated that students reporting adaptive executive control beliefs were more successful academically, while those students reporting inflexible or irresolute control beliefs were significantly less successful academically.     

Validation of the attitude toward women in science scale for early adolescents

The purpose of this study was to measure attitude of adolescents toward women in science careers. To do so, researchers in the Career Oriented Modules to Explore Topics in Science Project validated a 27-item instrument, the Women in Science Scale (WiSS). The scale had high reliability whether measured by estimates of internal consistency or test-retest reliability. The validity was established using the known groups and correlates procedures. The test clearly distinguished between known groups and resulted in the predicted pattern of correlations with measures of other constructs. The WiSS could prove to be a valuable instrument for basic research or curriculum evaluation where adolescent attitude toward women in science careers is a variable of interest.     

Development and Validation of the Supervisee Disclosure in Supervision Scale

The authors developed and initially validated the Supervisee Disclosure in Supervision Scale (SDSS). Exploratory and confirmatory factor analyses with 2 independent samples revealed that the SDSS is a 17-item instrument with 2 subscales showing promising psychometric properties. The authors discuss the findings with implications for supervision training, practices, and research.     

Self-appraisal of Curiosity

Abstract

A 41-item self-rating inventory was developed for the purpose of determining whether such an instrument could be used to identify groups of children differing in curiosity as effectively as could combined teacher and peer rating. An estimated reliability of .91 had been obtained for the instrument by using the split-half method. The instrument was administered to groups of children differing in curiosity level who had been selected from 577 fifth-grade pupils attending suburban middle class schools. The groups were controlled for intelligence and had been subdivided by sex. In all cases, the differences between the means of high-curiosity children and the means of low-curiosity children were significant at the .05 level or better.     

What Do You Know about Alternative Energy? Development and Use of a Diagnostic Instrument for Upper Secondary School Science

The need for renewable and non-fossil fuels is now recognised by nations throughout the world. Consequently, an understanding of alternative energy is needed both in schools and in everyday life-long learning situations. This study developed a two-tier instrument to diagnose students' understanding and alternative conceptions about alternative energy in terms of: sources of alternative energy, greenhouse gas emission, as well as advantages, and disadvantages. Results obtained with Years 10 and 11 students (n?=?491) using the 12-item two-tier instrument (α?=?0.61) showed that students' understanding of alternative energy was low (M?=?7.03; SD?=?3.90). The 23 alternative conceptions about alternative energy sources that could be identified from the instrument are reported. The implications for teaching and learning about alternative energy and suggestions for further development and improvement of the instrument are presented.     

Development and validation of the meaning of work inventory among French workers

The purpose of this study was to validate a psychometric instrument among French workers for assessing the meaning of work. Following an empirical framework, a two-step procedure consisted of exploring and then validating the scale among distinctive samples. The consequent Meaning of Work Inventory is a 15-item scale based on a four-factor model, and assesses the importance, understanding, direction, and purpose of work. The results indicate that this instrument is reliable for assessing the meaning of work in the French context. Implications for career counseling practices are discussed.     

Measuring approaches to learning in preschoolers: validating the structure of an instrument for teachers and parents

This study examined a 13-item instrument measuring approaches to learning (AtL) as a component of school readiness in the context of early childhood socio-emotional development. Few instruments, limited to preschool teacher ratings, measure AtL among kindergarteners with short easy-to-use questionnaires. We investigated psychometric properties of the instrument designed to provide practical measures of AtL behaviours identified in the Arizona Early Learning Standards with teacher (n?=?205) and guardian (n?=?1025) samples. We found a one-factor structure via exploratory factor analysis and confirmatory factor analysis (CFA). The multi-group CFA for combined teacher and guardian models indicated a good fit, which demonstrated the structure validity of the AtL instrument. This finding, combined with evidence of reliability of the instrument, supported the educational utility of the AtL as a new tool for measuring school readiness among kindergarteners in Arizona.     

Development of a scale on learners' views on blended learning and its implementation process

The purpose of this study was to extend the evaluation of learners' views on blended learning and its implementation process by developing and validating an objective assessment instrument. Cronbach' alpha, item analysis and item discrimination indices, principal component analysis, varimax rotation, and discriminant validity were used to measure the reliability and validity of the scale. A 50-item refined version of the scale was found to be highly reliable and of reasonable length.Further refinement was made based on principal component analysis. This indicated two major components, which facilitate the elicitation of learners' views on blended learning and its implementation process in relation to the complexity levels of the learning process.The scale is recommended to identify individual's views on blended learning and its implementation process and thus, to contribute to the improvement of a blended learning environment for learners.     

Development of the teaching in urban schools scale

This study presents the development of a 91-item instrument to measure what pre-service teachers know about effective teaching in urban schools. The instrument, Teaching in Urban Schools Scale, was piloted and then refined and conducted with 248 graduate students in a teacher education program. Validity of the scale was investigated and revealed evidence of discriminant and content validity and a reliability coefficient of .91. Results also indicated that certification and number of program credits completed bore no significant influence on respondents’ knowledge base. However, participants who expressed interest in teaching in urban schools scored significantly higher on the instrument.     

Needs assessment in STEM disciplines: reliability,validity and factor structure of the Student Support Needs Scale (SSNS)

Retention is a major problem in most colleges and universities. High dropout rates, especially in the STEM disciplines (science, technology, engineering and mathematics), have proved intractable despite the offering of supplemental instruction. A broad model of support systems that includes psychological factors is needed to address retention in STEM fields. The purpose of our study was to develop an instrument to identify the support needs of college students. We adapted the theoretical model of the performance pyramid to create a 48-item measure called the Student Support Needs Scale. We examined the psychometric properties of our scale by subjecting the measure to a principal component analysis, which resulted in a robust 36-item, six-factor solution. Finally, we established the reliability and validity of the resulting instrument. Once student needs have been assessed using our scale, interventions may be tailored to the needs of a minority group, discipline, geographic area and/or institution. This instrument could help university programmes to make informed decisions about resource allocation based on students’ needs.     

MEASURING FACULTY REACTION TO HOMOGENEOUS GROUPING

A 6-item instrument to estimate teacher reaction to homogeneous grouping is described. With slight changes in wording, its format can be adapted to evaluate opinion toward other educational practices.     

Promising Areas for Psychometric Research

Many credentialing agencies today are either administering their examinations by computer or are likely to be doing so in the coming years. Unfortunately, although several promising computer-based test designs are available, little is known about how well they function in examination settings. The goal of this study was to compare fixed-length examinations (both operational forms and newly constructed forms) with several variations of multistage test designs for making pass-fail decisions. Results were produced for 3 passing scores. Four operational 60-item examinations were compared to (a) 3 new 60-item forms, (b) 60-item 3-stage tests, and (c) 40-item 2-stage tests; all were constructed using automated test assembly software. The study was carried out using computer simulation techniques that were set to mimic common examination practices. All 60-item tests, regardless of design or passing score, produced accurate ability estimates and acceptable and similar levels of decision consistency and decision accuracy. One interesting finding was that the 40-item test results were poorer than the 60-item test results, as expected, but were in the range of acceptability. This raises the practical policy question of whether content-valid 40-item tests with lower item exposure levels and/or savings in item development costs are an acceptable trade-off for a small loss in decision accuracy and consistency.     

Different administrative directions and student ratings of instruction: Cognitive versus affective effects

A manipulation of the instructions students received prior to completing the 7-item Endeavor Instructional Rating card differentially affected their ratings on two types of items. Specifically, when students were led to believe their ratings would have a strong impact on the instructor's career, they tended to be more lenient on items measuring rapport (i.e., the affective domain); this same effect was not observed for items measuring pedagogical skill (i.e., the cognitive domain). The different items on our instructional rating instrument appear to be measuring different things. One implication of this observation is that the inconsistent findings reported in past research on student ratings of instruction may be due to the differential mix of items from one instrument to another. When instructors are compared on ratings given them by students, unbiased interpretation requires that the multidimensional nature of teaching (and of the rating instrument) be considered.