The Academic Rational Beliefs Scale: Development,Validation, and Implications for College Counselors

This study reports on a new instrument, the Academic Rational Beliefs Scale, designed to measure college students' academic beliefs along a rational‐irrational continuum. The new instrument is potentially useful when working with students experiencing academic difficulties. Information about test construction, reliability, validity, and generalizability are presented and implications, benefits, and cautions associated with use of the instrument as a college counseling tool are discussed.     

Design,validation, and use of an evaluation instrument for monitoring systemic reform

Over the past decade, state and national policymakers have promoted systemic reform as a way to achieve high‐quality science education for all students. However, few instruments are available to measure changes in key dimensions relevant to systemic reform such as teaching practices, student attitudes, or home and peer support. Furthermore, Rasch methods of analysis are needed to permit valid comparison of different cohorts of students during different years of a reform effort. This article describes the design, development, validation, and use of an instrument that measures student attitudes and several environment dimensions (standards‐based teaching, home support, and peer support) using a three‐step process that incorporated expert opinion, factor analysis, and item response theory. The instrument was validated with over 8,000 science and mathematics students, taught by more than 1,000 teachers in over 200 schools as part of a comprehensive assessment of the effectiveness of Ohio's systemic reform initiative. When the new four‐factor, 20‐item questionnaire was used to explore the relative influence of the class, home, and peer environment on student achievement and attitudes, findings were remarkably consistent across 3 years and different units and methods of analysis. All three environments accounted for unique variance in student attitudes, but only the environment of the class accounted for unique variance in student achievement. However, the class environment (standards‐based teaching practices) was the strongest independent predictor of both achievement and attitude, and appreciable amounts of the total variance in attitudes were common to the three environments. © 2001 John Wiley & Sons, Inc. J Res Sci Teach 38: 646–662, 2001     

A VALID SELF‐APPRAISAL INSTRUMENT FOR COMMUNITY COLLEGE ADMINISTRATORS

This study resulted from a concern that a valid and reliable self‐assessment instrument was needed for community college administrators. The instrument constructed included thirty performance dimensions. Administrators from three community colleges assisted in the establishment of the validity and reliability of the instrument. The process for developing a self‐assessment instrument for performance appraisal was validated. Implications for practice include a method for administrator self‐improvement of performance and the development of administrator training programs.     

AN INSTRUMENT TO MEASURE SELF‐RESPONSIBILITY FOR WELLNESS IN OLDER ADULTS

Literature relating to the well‐being of older adults was reviewed to identify indicators relevant to the construct of self‐responsibility for wellness. The wellness model proposed by Travis (1981) has produced a variety of concepts which can be useful in improving the quality of life for older adults. The purpose of this study was to develop an instrument which would assess an individual's self‐responsibility for wellness. A 47‐item instrument developed for this purpose was evaluated by experts in gerontology and psychology. After revision and reevaluation it was field‐tested on a sample of 180 older adults (60 years of age and over). In order to take preliminary steps in establishing the validity and reliability of this instrument, the data were evaluated and an item analysis conducted to identify poor items. Cronbach's coefficient alpha was also computed (α = .90). A test‐retest correlation coefficient was computed, and an analysis of variance was performed to test for the relationship between self‐responsibility for wellness and demographic variables obtained during the field test.

The field testing of the instrument served as an educational needs assessment study. Evidence has been provided that there is a significant need for education programs which can provide training in the wellness skills as assessed by the instrument.     

System competence modelling: Theoretical foundation and empirical validation of a model involving natural,social and human‐environment systems

Constituting a metacognitive strategy, system competence or systems thinking can only assume its assigned key function as a basic concept for the school subject of geography in Germany after a theoretical and empirical foundation has been established. A measurement instrument is required which is suitable both for supporting students and for the evaluation of methodical‐didactic measures. Such a tool is theoretically anchored in an empirically validated geography‐didactic and cognition‐psychological competence model, providing a differentiated representation of both the internal structure of a competency and the proficiency levels. The starting point of this foundation was the development of a normative‐theoretically derived model of geographic system competence. Its empirical validation was performed in different phases aimed at operationalising the competence model by means of test problems. In order to analyse the factor structure of the theoretical model, various item response models were estimated. The item levels of difficulty expected in the competence model were related to the empirical levels of difficulty and predicted by means of ordinary least squares regression to verify the model for proficiency levels. The two‐dimensional competence model – with the two dimensions ‘system organisation and behaviour’ and ‘system‐adequate intention to act’ – exhibits a better fit in reference to the model fit criteria than the one‐dimensional and three‐dimensional models. The correlations between the expected and empirical item difficulties are positive. Items that should be more difficult according to the competence model are actually shown to be more difficult. These findings suggest the reliability and validity of this new measurement instrument for diagnosing and promoting geographical system competence. It has to be implemented in practice as the next step.     

Using behavioral and academic indicators in the classroom to screen for at‐risk status

The present study validated a brief at‐risk screening instrument designed for easy use by teachers in the elementary school. School performance measures were collected for students in first to sixth grade one year following initial teacher ratings using the Screening For At‐Risk Status screening instrument. Findings indicated that the instrument is best seen as measuring a single at‐risk construct with items drawn from three domains: academic skills, social confidence, and social cooperation. Correlations between at‐risk scores and school performance measures taken one year later demonstrated predictive validity. The screening instrument correctly identified at‐risk students with 88% accuracy and not‐at‐risk students with 74% accuracy. There were 12% false negatives. Use of the instrument provides teachers with a quick, easy screening of students who may develop difficulties in the future. For schools, the screening can be used as the first step in a supportive response system to assist at‐risk students from developing serious school difficulties and possibly failure in the longer term. © 2005 Wiley Periodicals, Inc. Psychol Schs 42: 151–158, 2005.     

Development and validation of an instrument to assess secondary school students’ perceptions of assessment tasks

Research aimed at developing and validating an instrument to assess secondary school students’ perceptions of assessment tasks was conducted. Following a review of literature, a five‐scale instrument of 40 items was trialled with a sample of 658 science students in 11 English secondary schools. Based on internal consistency reliability data and exploratory factor analysis, refinement decisions resulted in a five‐scale instrument called the Perceptions of assessment tasks inventory (PATI). The scales of the PATI are Congruence with planned learning, Authenticity, Student consultation, Transparency and Diversity.     

An Exploratory Investigation of the Counseling Competencies Scale: A Measure of Counseling Skills,Dispositions, and Behaviors

The authors examined the psychometric properties of the Counseling Competencies Scale (CCS; University of Central Florida Counselor Education Faculty, 2009 ), an instrument designed to assess trainee competencies as measured in their counseling skills, dispositions, and behaviors. There was strong internal consistency for the 4‐factor model for midterm data (.927) and the 5‐factor model for final data (.933). Interrater reliability for the total CCS score was .570, and criterion‐related validity (correlation between the total score on the final CCS and semester grade) yielded a moderate correlation (r= .407, p < .01). Thus, the results provide initial support for using the CCS to assess counseling students’ professional competencies.     

Evaluating Secondary Students’ Scientific Reasoning in Genetics Using a Two‐Tier Diagnostic Instrument

While genetics has remained as one key topic in school science, it continues to be conceptually and linguistically difficult for students with the concomitant debates as to what should be taught in the age of biotechnology. This article documents the development and implementation of a two‐tier multiple‐choice instrument for diagnosing grades 10 and 12 students’ understanding of genetics in terms of reasoning. The pretest and posttest forms of the diagnostic instrument were used alongside other methods in evaluating students’ understanding of genetics in a case‐based qualitative study on teaching and learning with multiple representations in three Western Australian secondary schools. Previous studies have shown that a two‐tier diagnostic instrument is useful in probing students’ understanding or misunderstanding of scientific concepts and ideas. The diagnostic instrument in this study was designed and then progressively refined, improved, and implemented to evaluate student understanding of genetics in three case schools. The final version of the instrument had Cronbach’s alpha reliability of 0.75 and 0.64, respectively, for its pretest and the posttest forms when it was administered to a group of grade 12 students (n = 17). This two‐tier diagnostic instrument complemented other qualitative data collection methods in this research in generating a more holistic picture of student conceptual learning of genetics in terms of scientific reasoning. Implications of the findings of this study using the diagnostic instrument are discussed.     

Supporting school improvement: the development of a scale for assessing pupils' emotional and behavioural development

This paper reports the construction of a teacher rating instrument designed to monitor the social and emotional development of school age children and young people (4–17 years). The instrument was developed by reviewing previously implemented checklists to build an extensive list of behavioural and emotional criteria and through the use of focus groups to establish the views of key stakeholders. The criteria were categorized according to three areas: conduct, emotion and learning. The initial instrument of 21 items was piloted and amended accordingly. The final trial of the instrument was carried out on a sample of 7285 pupils from a cross‐section of UK schools. A principal component factor analysis confirmed the division of the scale into three factors. The instrument was supported by trends showing that the distributions were different for different types of schools and between males and females. The final version of the instrument was amended to include 15 items, five in each category (conduct, emotion, learning), all expressed positively on a six‐point scale. The scale is a useful tool for providing a basis for a strategic discourse between staff in planning approaches to the emotional and behavioural development of students in school.     

Education and Learning for the Elderly: Why,How, What

Seventy‐seven children of divorce and their selected grandparents completed questionnaires that included a demographic section, the Family Adaptability and Cohesion Evaluation Scale‐II (FACES II; Olson, Bell, &; Portner, 1982) to assess their relationships, and an instrument designed to identify shared activities and desires in these areas. Partners had similar present and future perceptions of relationship and activity factors. All participants desired future increases in both areas. Grandchildren and grandparents connected emotional bonding with grandparents’ listening, keeping them safe, and gift giving; grandparents also thought a good relationship with the custodial parent was important. Most healthy and least healthy subgroups were identified. Membership in the healthiest group was related to geographic proximity, feminine gender for both partners, and being married and employed full‐time for grandparents. Grandparents who were related by blood to custodial parents, had a good relationship with them, and negotiated their role were more likely to have a healthier relationship. Helping professionals are encouraged to explore these variables as they help family members negotiate the grandparent‐grandchild relationship, use skill training to foster development in areas amenable to change, and use supportive tactics in areas of liability.     

The Student Actions Coding Sheet (SACS): An instrument for illuminating the shifts toward student‐centered science classrooms

This study describes the development of an instrument to investigate the extent to which student‐centered actions are occurring in science classrooms. The instrument was developed through the following five stages: (1) student action identification, (2) use of both national and international content experts to establish content validity, (3) refinement of the item pool based on reviewer comments, (4) pilot testing of the instrument, and (5) statistical reliability and item analysis leading to additional refinement and finalization of the instrument. In the field test, the instrument consisted of 26 items separated into four categories originally derived from student‐centered instruction literature and used by the authors to sort student actions in previous research. The SACS was administered across 22 Grade 6–8 classrooms by 22 groups of observers, with a total of 67 SACS ratings completed. The finalized instrument was found to be internally consistent, with acceptable estimates from inter‐rater intraclass correlation reliability coefficients at the p < 0.01 level. After the final stage of development, the SACS instrument consisted of 24 items separated into three categories, which aligned with the factor analysis clustering of the items. Additionally, concurrent validity of the SACS was established with the Reformed Teaching Observation Protocol. Based on the analyses completed, the SACS appears to be a useful instrument for inclusion in comprehensive assessment packages for illuminating the extent to which student‐centered actions are occurring in science classrooms.     

Factors related to the underachievement of university students in Turkey

The study aims to elucidate the characteristics of gifted underachievers at the university level and the reasons for their underachievement. The sample consisted of students from Bogazici University in Istanbul. The School Attitude Assessment Survey‐Revised (SAAS‐R) instrument was administered to 91 underachievers and a comparison group of 74 university students. The instrument employed five subscales: academic self‐perceptions, attitudes toward teachers, attitudes toward school, goal valuation, and motivation/self‐regulation. Findings indicated that the comparison students and the underachieving students differed on all five subscales of the SAAS‐R. In addition, the instrument correctly classified almost 80% of the students in the present sample as either underachievers or comparison students. Among the five subscales, the motivation/self‐regulation subscale was most predictive of underachievement status.     

Teacher self‐regulation: examining a multidimensional construct

This study aimed to develop and validate an instrument to assess the multidimensional nature of teacher self‐regulation. A nine‐factor structure was proposed: goal setting, intrinsic interest, performance goal orientation, mastery goal orientation, self‐instruction, emotional control, self‐evaluation, self‐reaction, and help‐seeking. Through a series of confirmatory factor analyses with different samples, this nine‐factor structure was supported. Reliability estimates were also satisfactory. Further validation evidence was provided through canonical correlation analysis between teacher self‐regulation and teachers’ sense of efficacy. Results indicated that the Teacher Self‐Regulation Scale can be utilised as a valid and reliable instrument to assess teachers’ self‐regulatory strategies.     

A preliminary validation of Attention, Relevance, Confidence and Satisfaction model-based Instructional Material Motivational Survey in a computer-based tutorial setting

This paper describes a preliminary validation study of the Instructional Material Motivational Survey (IMMS) derived from the Attention, Relevance, Confidence and Satisfaction motivational design model. Previous studies related to the IMMS, however, suggest its practical application for motivational evaluation in various instructional settings without the support of empirical data. Moreover, there is a lack of discussion regarding the validity of the instrument. Therefore, this study empirically examined the IMMS as a motivational evaluation instrument. A computer‐based tutorial setting was selected owing to its wide application in teaching large entry‐level college courses. Data collected from 875 subjects were subjected to exploratory and confirmatory factor analyses, and measurement modelling LISREL. Findings suggested that 16 original items should be excluded from the IMMS; the retained 20 items were found to fall into different constructs, indicating that instructional features of the tutorial may influence the validity of the survey items. The implication of the study supports the situational feature of the IMMS. Therefore, a preevaluation adjustment on the IMMS items is recommended to identify suitable items before the full motivational evaluation. Future research should focus on the further validation of the IMMS based on this preliminary evidence.     

Assessing student exposure to and use of computer technologies through an examination of course syllabi

A syllabus analysis instrument was developed to assist program evaluators, administrators and faculty in the identification of skills that students use as they complete their college coursework. While this instrument can be tailored for use with a variety of learning domains, we used it to assess students' use of and exposure to computer technology skills. The reliability and validity of the instrument was examined through an analysis of 88 syllabi from courses within the teacher education program and the core curriculum at a private Midwest US university. Results indicate that the instrument has good inter‐rater reliability and ratings by and interviews with faculty and students provide evidence of construct validity. The use and limitations of the instrument in educational program evaluation are discussed.     

Development of chemistry attitudes and experiences questionnaire (CAEQ)

In this article we describe the development of the Chemistry Attitudes and Experiences Questionnaire (CAEQ) that measures first‐year university chemistry students' attitude toward chemistry, chemistry self‐efficacy, and learning experiences. The instrument was developed as part of a larger study and sought to fulfill a need for an instrument to investigate factors that influence student enrollment choice. We set out to design the instrument in a manner that would maximize construct validity. The CAEQ was piloted with a cohort of science and technology students (n = 129) at the end of their first year. Based on statistical analysis the instrument was modified and subsequently administered on two occasions at two tertiary institutions (n = 669). Statistical data along with additional data gathered from interviews suggest that the CAEQ possesses good construct validity and will prove a useful tool for tertiary level educators who wish to gain an understanding of factors that influence student choice of chemistry enrolment. © 2003 Wiley Periodicals, Inc. J Res Sci Teach 40: 649–668, 2003     

Undergraduates' attitudes and beliefs about subject matter and pedagogy measured periodically in a reform‐based mathematics and science teacher preparation program

This study describes the design and use of a valid and reliable instrument to measure teacher candidates' attitudes and beliefs about mathematics and science and the teaching of those subjects. The instrument, Attitudes and Beliefs about the Nature of and the Teaching of Mathematics and Science, was developed for the Maryland Collaborative for Teacher Preparation (MCTP), a statewide, standards‐based project in the National Science Foundation's Collaborative in Excellence in Teaching Preparation (CETP) Program. We report on two applications of the instrument: (a) a contrast between MCTP teacher candidates' and non‐MCTP teacher candidates' attitudes and beliefs about mathematics and science as they initially encountered reform‐based instruction in their undergraduate courses, and (b) a landscaping of how the MCTP teacher candidates' attitudes toward and beliefs about mathematics and science evolved over a 2.5‐year period. In support of current reform in science and mathematics teacher education, we determined that over an extended period the MCTP teacher candidates' attitudes and beliefs moved substantively and significantly in the direction intended. However, we also found that the non‐MCTP teacher candidates in the same reform‐based courses did not mirror this improvement in their attitudes and beliefs about mathematics and science or the teaching of those subjects. © 2002 Wiley Periodicals, Inc. J Res Sci Teach 39: 713–737, 2002     

Physical education students’ attitudes and self‐efficacy towards the participation of children with special needs in regular classes

The purpose of the present study was to investigate the relationship between personal variables of physical education students and their attitudes towards participation of children with disabilities and self‐efficacy (SE) in teaching students with disabilities in regular classes. A total 153 PE majors (95 females and 58 males) participated in the study. A 15‐item attitude instrument and a 15‐item SE instrument concerning dilemmas during educational tasks were administered as a part of the didactic assignments. Factor analysis revealed one challenge and two threat factors in the attitude instrument. The statistical analysis revealed significant effects on attitudes to gender (females higher than males) and years in college (advanced students higher than novices). Significant effects on SE were found in the coursework, previous experience and years in college variables. SE was inversely related to both threat factors of the attitude instrument (r?=???0.42 and ??0.43 respectively).     

Locus of Control,Self‐Efficacy,and Motivation in Different Schools: Is moderation the key to success?

This study used a novel multidimensional locus of control instrument (I‐SEE) to investigate the relationship between locus of control, motivation, and academic achievement in three different types of school. The strengths of the I‐SEE are that it incorporates the construct of self‐efficacy and that it is embedded in a model of personality and action based on field‐theoretical conceptions. Further, it includes the role of the environment and personality in determining action. The results support a multidimensional conceptualisation of locus of control and the utility of the I‐SEE. There were statistically significant differences between schools for motivation and achievement and also a mediating effect between locus of control and school type, suggesting that interactional models are required in investigations of motivation and achievement. Furthermore, moderate levels of locus of control and self‐efficacy appear to be more adaptive than either extremely high or low levels.