PSYCHOMETRIC ANALYSIS OF A 5E LEARNING CYCLE LESSON PLAN ASSESSMENT INSTRUMENT

The purpose of this paper is to describe the procedures and the analysis of an instrument designed to measure preservice teachers’ ability to develop appropriate 5E learning cycle lesson plans. The 5E inquiry lesson plan (ILP) rubric is comprised of 12 items with a scoring range of zero to four points per item. Content validity was determined through the expertise of a panel of five science educators. Sixty six preservice teachers enrolled in elementary science methods at three universities prepared lesson plans, which were scored by their instructors using the ILP rubric. Using a Pearson two-tailed correlation, inter-rater reliability was established at a value of 0.83. An exploratory factor analysis provided evidence of construct validity, with three factors. The factors included (1) explore, (2) engage/explain/elaborate, and (3) evaluate. In addition, a secondary analysis revealed the means and standard deviations of the students' performance on each of the phases of the 5E that include: engage, explore, explain, elaborate, and evaluate. The engage item held the highest mean rating, and the evaluation items had the lowest mean ratings. Examination of the instrument's structure in light of the 5E phases is discussed and provides directions for future revisions and research.     

Development of Instruments to Assess Teacher and Student Perceptions of Inquiry Experiences in Science Classrooms

This study describes the development of two instruments to investigate the extent to which students are engaged in scientific inquiry. As a result of the instrument development process employed, each finalized instrument consisted of 20-items separated into five categories. Both instruments were found to be internally consistent, with high reliability estimates. Factor analysis showed two factors for each instrument that, while not clustering the items into the five categories, did show item clustering that is consistent with research literature about students’ engagement in inquiry experiences. Based on the analyses completed, the instruments appear to be useful instruments for use in comprehensive assessment packages for assessing the extent to which students are experiencing inquiry in science classrooms.     

Technology Use in Science Instruction (TUSI): Aligning the Integration of Technology in Science Instruction in Ways Supportive of Science Education Reform

This study describes the development of an instrument to investigate the extent to which technology is integrated in science instruction in ways aligned to science reform outlined in standards documents. The instrument was developed by: (a) creating items consistent with the five dimensions identified in science education literature, (b) establishing content validity with both national and international content experts, (c) refining the item pool based on content expert feedback, (d) piloting testing of the instrument, (e) checking statistical reliability and item analysis, and (f) subsequently refining and finalization of the instrument. The TUSI was administered in a field test across eleven classrooms by three observers, with a total of 33 TUSI ratings completed. The finalized instrument was found to have acceptable inter-rater intraclass correlation reliability estimates. After the final stage of development, the TUSI instrument consisted of 26-items separated into the original five categories, which aligned with the exploratory factor analysis clustering of the items. Additionally, concurrent validity of the TUSI was established with the Reformed Teaching Observation Protocol. Finally, a subsequent set of 17 different classrooms were observed during the spring of 2011, and for the 9 classrooms where technology integration was observed, an overall Cronbach alpha reliability coefficient of 0.913 was found. Based on the analyses completed, the TUSI appears to be a useful instrument for measuring how technology is integrated into science classrooms and is seen as one mechanism for measuring the intersection of technological, pedagogical, and content knowledge in science classrooms.     

The Student Actions Coding Sheet (SACS): An instrument for illuminating the shifts toward student‐centered science classrooms

This study describes the development of an instrument to investigate the extent to which student‐centered actions are occurring in science classrooms. The instrument was developed through the following five stages: (1) student action identification, (2) use of both national and international content experts to establish content validity, (3) refinement of the item pool based on reviewer comments, (4) pilot testing of the instrument, and (5) statistical reliability and item analysis leading to additional refinement and finalization of the instrument. In the field test, the instrument consisted of 26 items separated into four categories originally derived from student‐centered instruction literature and used by the authors to sort student actions in previous research. The SACS was administered across 22 Grade 6–8 classrooms by 22 groups of observers, with a total of 67 SACS ratings completed. The finalized instrument was found to be internally consistent, with acceptable estimates from inter‐rater intraclass correlation reliability coefficients at the p < 0.01 level. After the final stage of development, the SACS instrument consisted of 24 items separated into three categories, which aligned with the factor analysis clustering of the items. Additionally, concurrent validity of the SACS was established with the Reformed Teaching Observation Protocol. Based on the analyses completed, the SACS appears to be a useful instrument for inclusion in comprehensive assessment packages for illuminating the extent to which student‐centered actions are occurring in science classrooms.     

Elementary School Students’ Science Talk Ability in Inquiry-Oriented Settings in Taiwan: Test Development,Verification, and Performance Benchmarks

The purpose of this study was to develop a computer-based measure of elementary students’ science talk and to report students’ benchmarks. The development procedure had three steps: defining the framework of the test, collecting and identifying key reference sets of science talk, and developing and verifying the science talk instrument. The Science Talk Test (STT), consisting of semi-structured items, was developed for a framework of exemplars from inquiry classrooms representing three inquiry components (i.e. identifying questions, designing methods and presenting evidence, and drawing evidence-based conclusions) for two types of science talk (i.e. factually oriented and evaluative discourses). Grades 4 and 6 students (N = 281) were selected from three levels of urbanization representative of Taiwan to verify the STT and to establish performance benchmarks. The STT’s reliability, validity, and item parameters were found to be reasonable. Students performed better in the factually oriented talk than in the evaluative talk. Designing methods and presenting evidence was more difficult than the other two types of inquiry talk. Students’ science talk ability was predictable with prior scientific knowledge and language ability measures. Practical use of the STT, explicit instruction, and future research are recommended.     

农村教师未来时间洞察力问卷的编制

目的：编制农村教师未来时间洞察力问卷。方法该研究以187名农村教师为被试,通过项目分析和探索性因素分析,最终确定包含21个项目的问卷。结果：问卷包括远目标探知、未来效能、行为承诺、未来意象、目标完成5个因子,它们可解释总变异量的59．718％;总问卷的内部一致性信度、重测信度和分半信度分别为0．772、0．784和0．638;问卷中各因素之间具有较显著的相关（r＝-0．289-0．699,P＜0．05）。各因素与总分具有很高的相（r＝0．251-0．765,P＜0．05）。结论：编制了农村教师未来时间洞察力问卷,包含5个因子,问卷具有较好信度效度,适合作为农村教师未来时间洞察力的测量工具。     

Investigating the Impact of NGSS-Aligned Professional Development on PreK-3 Teachers’ Science Content Knowledge and Pedagogy

This pilot study investigates the impact of a 2-week professional development Summer Institute on PK-3 teachers’ knowledge and practices. This Summer Institute is a component of [program], a large-scale early-childhood science project that aims to transform PK-3 science teaching. The mixed-methods study examined concept maps, lesson plans, and classroom observations to measure possible changes in PK-3 teachers’ science content knowledge and classroom practice from 11 teachers who attended the 2014 Summer Institute. Analysis of the concept maps demonstrated statistically significant growth in teachers’ science content knowledge. Analysis of teachers’ lesson plans demonstrated that the teachers could design high quality science inquiry lessons aligned to the Next Generation Science Standards following the professional development. Finally, examination of teachers’ pre- and post-Summer Institute videotaped inquiry lessons showed evidence that teachers were incorporating new inquiry practices into their teaching, especially regarding classroom discourse. Our results suggest that an immersive inquiry experience is effective at beginning a shift towards reform-aligned science and engineering instruction but that early elementary educators require additional support for full mastery.     

Measuring Knowledge Integration Learning of Energy Topics: A two-year longitudinal study

Although researchers call for inquiry learning in science, science assessments rarely capture the impact of inquiry instruction. This paper reports on the development and validation of assessments designed to measure middle-school students’ progress in gaining integrated understanding of energy while studying an inquiry-oriented curriculum. The assessment development was guided by the knowledge integration framework. Over 2 years of implementation, more than 4,000 students from 4 schools participated in the study, including a cross-sectional and a longitudinal cohort. Results from item response modeling analyses revealed that: (a) the assessments demonstrated satisfactory psychometric properties in terms of reliability and validity; (b) both the cross-sectional and longitudinal cohorts made progress on integrating their understanding energy concepts; and (c) among many factors (e.g. gender, grade, school, and home language) associated with students’ science performance, unit implementation was the strongest predictor.     

Preservice science teachers’ ecological value orientation: A comparative study between Indonesia and Korea

Abstract

The current study aimed to examine differences in ecological values between Indonesian and Korean preservice science teachers. Fifteen items from revised-New Ecological Paradigm based on value orientations were used as the research instrument. Two hundred seventy-three preservice science teachers participated in the study. Rasch analyses of dimensionality, item fit, and differential item functioning were used to explore the validity of the instrument. The independent-sample T-test and Pearson’s correlation test were utilized to compare data from the two countries. Preservice science teachers from the two different countries showed significant differences in only one of the three value orientations, which is egoistic value. Relationships among cultural, educational, and economic factors, as well as environmental values in these countries are discussed to fully explore the findings.     

Improving Science Teachers’ Views about Scientific Inquiry

Science & Education - The present study specifically focuses on science teachers’ views about scientific inquiry and their use of scientific inquiry in their lesson plans, which were...     

Development and validation of Kindergarten Environment Rating Scale

The authors present the development and the validation of the four‐dimensional, twenty‐five‐item, five‐point Kindergarten Environment Rating Scale (KERS). The Cohen’s Kappa of the items indicates acceptable reliability for the instrument. The content validity and confirmatory factor analysis indicates that the data obtained using the KERS could differentiate the kindergarten environmental quality in reality, as they proposed to do. As previous literature pointed out, the quality of kindergartens in this study varied greatly across different types and different areas. How to create and utilise the physical environment and how to interact with and provide appropriate support for young children are still common difficulties for kindergartens.     

Evaluating Instrument Quality in Science Education: Rasch‐based analyses of a Nature of Science test

Given the central importance of the Nature of Science (NOS) and Scientific Inquiry (SI) in national and international science standards and science learning, empirical support for the theoretical delineation of these constructs is of considerable significance. Furthermore, tests of the effects of varying magnitudes of NOS knowledge on domain‐specific science understanding and belief require the application of instruments validated in accordance with AERA, APA, and NCME assessment standards. Our study explores three interrelated aspects of a recently developed NOS instrument: (1) validity and reliability; (2) instrument dimensionality; and (3) item scales, properties, and qualities within the context of Classical Test Theory and Item Response Theory (Rasch modeling). A construct analysis revealed that the instrument did not match published operationalizations of NOS concepts. Rasch analysis of the original instrument—as well as a reduced item set—indicated that a two‐dimensional Rasch model fit significantly better than a one‐dimensional model in both cases. Thus, our study revealed that NOS and SI are supported as two separate dimensions, corroborating theoretical distinctions in the literature. To identify items with unacceptable fit values, item quality analyses were used. A Wright Map revealed that few items sufficiently distinguished high performers in the sample and excessive numbers of items were present at the low end of the performance scale. Overall, our study outlines an approach for how Rasch modeling may be used to evaluate and improve Likert‐type instruments in science education.     

师范生择业效能感问卷的初步编制

为了编制师范生择业效能感问卷,并检验所编制问卷的信度和效度,采用探索性因素分析初步探究理论结构,用验证性因素分析验证理论结构的合理性和正确性。验证结果为:探索性因素分析确定该问卷包括5个因子,全问卷的内部一致性系数为0.96,各分问卷的信度均在0.85以上,各分问卷全问卷得分之间的相关系数在0.859~0.918之间。从而得出结论:编制的师范生择业效能感问卷共包括5个因子,量表具有较好的信效度。     

STUDENT INTEREST IN TECHNOLOGY AND SCIENCE (SITS) SURVEY: DEVELOPMENT,VALIDATION, AND USE OF A NEW INSTRUMENT

This study presents the systematic development, validation, and use of a new instrument for measuring student interest in science and technology. The Student Interest in Technology and Science (SITS) survey is composed of 5 sub-sections assessing the following dimensions: interest in learning science, using technology to learn science, science careers, technology careers, and attitudes toward biotechnology. Our development process included review of existing instrumentation, pilot testing, and expert panel review. The resulting instrument was administered before and after implementation of a biotechnology intervention which used a computer-based game to engage learners in the use of biotechnology to address a societal issue. We employed item response theory (IRT) to explore instrument validity and precision. Results of the psychometric analyses suggest that the SITS survey has a well-defined structure and meets IRT assumptions. Difficulty and discrimination parameters as well as reliability analyses indicate that SITS items provide useful measures of student interest. Finally, we use the SITS to explore the extent to which the intervention used in this study supports changes in student interest and association between students’ interest and related content knowledge. Implications for the future use of this instrument are discussed.     

Enactment of Scientific Inquiry: Observation of Two Cases at Different Grade Levels in China Mainland

Enactment of scientific inquiry in classroom has attracted a great attention of science educators around the world. In this study, we examined two competent teachers’ (one Grade 9 chemistry teacher and one Grade 4 science teacher) enactment of scientific inquiry in selected teaching units to reveal the characteristics of enacted inquiry at different grade levels by analyzing lesson sequence videos. The coding schemes for enacted inquiry consist of ontological properties and instructional practices. Pre-topic and post-topic teacher interviews and the two teachers’ responses to a questionnaire were adopted to identify the factors influencing teacher’s enactment. The results indicate that the two case teachers’ enactment involved a range of inquiry activities. The enacted inquiry at fourth-grade level covered all the inquiry elements, tending to engage students in the whole procedure of inquiry. The ninth-grade chemistry class placed emphasis on the elements “making plans” to solve problems in authentic context. Important factors influencing the enactment include teacher’s understanding about scientific inquiry, textbooks, assessment, students and resource. Implications for inquiry enactment and instruction improvement have been provided.     

A survey to examine teachers’ perceptions of design dispositions,lesson design practices,and their relationships with technological pedagogical content knowledge (TPACK)

This study investigates 201 Singaporean teachers’ perceptions of their technological pedagogical content knowledge (TPACK), lesson design practices, and design dispositions through a survey instrument. Investigation of these constructs reveal important variables influencing teachers’ perceptions of TPACK which have not yet been explored. The confirmatory factor analysis and reliability analysis confirm the validity and reliability of the instrument. The structural equation model shows that the teachers’ perceptions of design dispositions (orientations towards design) and lesson design practices (approaches used for lesson design) have direct relationships with the teachers’ perceptions of TPACK. The results of this study show that to enhance teachers’ TPACK perceptions, teacher educators need to help teachers develop lesson design practices that support ideation and iteration. They also need to develop teachers’ design dispositions that are amenable to exploring and resolving conflicting lesson design ideas. Going beyond TPACK, understandings of teachers’ lesson design practices and design dispositions are important for teacher educators to better design professional development for integration of information and communications technology.     

Validity and Reliability of a Shortened,Revised Version of the Constructivist Learning Environment Survey (CLES)

The purpose of the study was to investigate the use of an existing instrument, the Constructivist Learning Environment Survey (CLES)(Taylor, Dawson & Fraser, 1995; Taylor, Fraser & Fisher, 1993, 1997), for providing insights into the classroom learning environments of beginning science teachers. In the first year of the study, the CLES was used with 290 upper elementary, middle, and high school science teachers and preservice teachers. As part of a larger study of the classroom environments and teaching practices of beginning science teachers, data also were gathered through classroom observations of and interviews with some of the participating teachers. Exploratory factor analysis and internal consistency reliability analysis, as well as examination of each item and of participants' questions and comments about them, led to a shortened, revised version of the CLES, named the CLES 2(20). The five original scales were retained, but the number of items in each scale was reduced from six to four. The single negatively worded item was eliminated. Some of the original items were rephrased. The revised CLES was then used in the second, third and fourth years of the study. Examples of feedback based on CLES data is provided to researchers to assist them in writing teacher profiles.     

小学教师课例研究能力评价指标体系构建研究

教师的课例研究能力对课例研究开展的成效具有决定性作用,因此科学地评价教师的课例研究能力具有重要意义。在借鉴参考教师教育研究能力、行动研究能力结构模型的基础上,立足于课例研究的五个基本环节,构建了由确定研究主题的能力、合作设计教案的能力、课堂观察的能力、课后研讨的能力、撰写课例研究报告的能力5个一级指标,发现问题的能力、收集整理资料的能力等35个二级指标构成的指标体系。通过对小学教师课例研究能力评价指标体系的信效度检验以及对教育专家、一线教师、学校管理者、教研员的认同度调查,初步验证了构建指标体系的合理性。     

The development of a recycling awareness scale for prospective science teachers

The purpose of this study was to develop a scale for measuring prospective science teachers’ awareness of waste recycling. The study was conducted with the participation of 382 prospective teachers attending a university located in northern Turkey. The five-point Likert type scale that was developed contained 82 items relating to prospective science teachers’ awareness of waste recycling. The factor analysis conducted showed that five items had factor loadings below 0.30, and five were cross loading items. Factor analysis was repeated after removing these items. A further 24 items were removed from the list at the end of the factor analysis, and the remaining 48 items were grouped under ten dimensions. The reliability coefficient for the factors extracted varied between 0.717 and 0.805, and the internal reliability coefficient for the whole scale was 0.905.     

The Development of an Instrument to Measure the Project Competences of College Students in Online Project-Based Learning

This study sought to develop a self-report instrument to be used in the assessment of the project competences of college students engaged in online project-based learning. Three scales of the KIPSSE instrument developed for this study, namely, the knowledge integration, project skills, and self-efficacy scales, were based on related theories and the analysis results of three project advisor interviews. Those items of knowledge integration and project skill scales focused on the integration of different disciplines and technological skills separately. Two samples of data were collected from information technology-related courses taught with an online project-based learning strategy over different semesters at a college in southern Taiwan. The validity and reliability of the KIPSSE instrument were confirmed through item analysis and confirmatory factor analysis using structural equation modeling of two samples of students’ online response sets separately. The Cronbach’s alpha reliability coefficient for the entire instrument was 0.931; for each scale, the alpha ranged from 0.832 to 0.907. There was also a significant correlation (r = 0.55, p < 0.01) between the KIPSSE instrument results and the students’ product evaluation scores. The findings of this study confirmed the validity and reliability of the KIPSSE instrument. The confirmation process and related implications are also discussed.