The Language Factor in Elementary Mathematics Assessments: Computational Skills and Applied Problem Solving in a Multidimensional IRT Framework

The results of an exploratory study into measurement of elementary mathematics ability are presented. The focus is on the abilities involved in solving standard computation problems on the one hand and problems presented in a realistic context on the other. The objectives were to assess to what extent these abilities are shared or distinct, and the extent to which students' language level plays a differential role in these abilities. Data from a sample of over 2,000 students from first, second, and third grade in the Netherlands were analyzed in a multidimensional item response theory (IRT) framework. The latent correlation between the two ability dimensions (computational skills and applied mathematics problem solving) ranged from .81 in grade 1 to .87 in grade 3, indicating that the ability dimensions are highly correlated but still distinct. Moreover, students' language level had differential effects on the two mathematical abilities: Effects were larger on applied problem solving than on computational skills. The implications of these findings for measurement practices in the field of elementary mathematics are discussed.     

Instructional Climate in Dutch Secondary Education

Research, aimed at constructing an instrument for measuring instructional climate, was conducted in 121 Dutch schools for senior general secondary education. The instrument comprised scales for students, teachers, heads of department and principals. All scales proved to be sufficiently reliable and valid. The results show that the following variables are significantly related to students' results in mathematics: students' enjoyment of mathematics; students' attitude towards high grades; students' enjoyment of learning; appreciation of how teachers prepare students for tests; students' appreciation of the teacher as a person; students' appreciation of teaching abilities; an orderly instructional climate in the classroom; frequent monitoring of student results; emphasizing achievement by teachers; and the principals' emphasis on a task oriented climate.     

A systematic review of bilingual education teachers’ competences

Ongoing globalisation processes call forth a need to foster students' intercultural competences and language skills. Simultaneously, teachers face an increasing diversity of students' first languages and cultures within classrooms, demanding responsive and integrative practices from them which address all students. Bilingual education is one possible format in which these challenges can be met; however, it requires a high level of teacher professionalism. This systematic literature review harvests international research on bilingual education teachers published between 1995 and 2020. It compares frameworks and research on bilingual education teachers’ required competences for secondary education and generates a professional competence model for bilingual education teachers. This systematic review of 79 individual reports identifies a total of 16 converging competences, many focusing on language proficiency and pedagogical/psychological knowledge. Furthermore, it yields important implications for bilingual teacher training such as the fostering of (academic) language proficiency, cooperation skills and knowledge of bilingual education research.     

The implications of content versus item validity on science tests

The use of content validity as the primary assurance of the measurement accuracy for science assessment examinations is questioned. An alternative accuracy measure, item validity, is proposed. Item validity is based on research using qualitative comparisons between (a) student answers to objective items on the examination, (b) clinical interviews with examinees designed to ascertain their knowledge and understanding of the objective examination items, and (c) student answers to essay examination items prepared as an equivalent to the objective examination items. Calculations of item validity are used to show that selected objective items from the science assessment examination overestimated the actual student understanding of science content. Overestimation occurs when a student correctly answers an examination item, but for a reason other than that needed for an understanding of the content in question. There was little evidence that students incorrectly answered the items studied for the wrong reason, resulting in underestimation of the students' knowledge. The equivalent essay items were found to limit the amount of mismeasurement of the students' knowledge. Specific examples are cited and general suggestions are made on how to improve the measurement accuracy of objective examinations.     

The impact of the pre-instructional cognitive profile on learning gain and final exam of physics courses: a case study

The case study described in this paper investigates the relationship among some pre-instructional knowledge, the learning gain and the final physics performance of computing engineering students in the introductory physics course. The results of the entrance engineering test (EET) have been used as a measurement of reading comprehension, logic and mathematics skills and basic physics knowledge of a sample of 47 Computing Engineering freshmen at the University of Palermo (Italy). These data give a significant picture of the initial knowledge status of a student choosing engineering studies. The students' physics learning gain has been calculated using a standardized tool in mechanics: the force concept inventory (FCI). The analysis shows that mathematical and physical background contribute to achieve a good final preparation in physics courses of engineering faculties; however the students' learning gain in physics is independent of students' initial level of mathematics skills and physics knowledge. Initial logic skills and reading comprehension abilities are not significant factors for the learning physics gain and the performance on physics courses.     

Understanding Mental Health Scale: Development,Validation, and Implications for College Counselors

The development and validation of the 29‐item Understanding Mental Health Scale (UMHS) is described. This new questionnaire measures college students' understanding of mental health issues. The authors also identified significant demographic differences in students' understanding of mental health issues. A principal factor analysis revealed a 2‐factor solution. The dimensions were labeled Risk‐Factor Awareness and External Support Awareness. Findings suggest the UMHS demonstrates adequate reliability and validity. Implications for college counseling are discussed.     

An empirical exploration of metacognitive assessment activities in a third-year civil engineering hydraulics course

Threshold concepts are transformative, integrative, and provocative; understanding these difficult concepts allows students to be capable of solving advanced problems. This investigation and evaluation of a metacognitive curricular approach explore variation in students' and teachers' discernment of structural complexity of concepts and its potential for enhancing students' learning and conceptual understanding of threshold concepts. Three trials of a metacognitive assessment activity administered to two cohorts of a civil engineering course (n?=?276 and n?=?264) were investigated. Students were presented with several answers (varying in structural complexity) to a question about a threshold concept and asked to mark each response. Quantitative analyses compared students' and teachers' marking schemes within and across trials, and qualitative analyses explored students' written reflections following the activity. Students' justifications for their marking schemes, their reflections on the activity's usefulness, and the convergence of students' and teachers' marking schemes suggest that the activity supported deep forms of student learning.     

Eliciting Mathematics Interest: New Directions for Context Personalization and Example Choice

Building on common assumptions in theories of interest and mathematics education, this experimental study examined the effect of context personalization based on individual preferences, group personalization, and example choice with preselected popular examples on middle school students' situational interest and performance in mathematics. Participants (N = 713) learned a principle in probability calculus in one of four instructional conditions. Individual interest and perceived competence were examined as moderators on triggered and maintained situational interest, perceived value, task effort, and performance. Results showed that example choice triggered situational interest in the learning activity and that context personalization influenced perceived value and effort, contingent on students' perceived competence and individual interest in mathematics. We discuss results in relation to previous findings on interest-based interventions and the theoretical and practical implications.     

在专业课程框架下培养高职生的关键能力研究——基于国际商务专业的统计分析

关键能力是个人职业生涯中除岗位专业能力之外的基本能力,它适用于各种职业,适应岗位的不断变换,是伴随其终生的可持续发展能力。具备关键能力成为众多企业对人才的迫切需求,因此,高职院校越来越重视对学生关键能力的培养。调研显示,国际商务专业在专业课程中的外语学习、信息处理以及与人交流能力培养等方面效果都较好,教学方法多样有效,而解决问题、与人合作能力则需要进行更多的探索。     

Production d'exemples et de contre-exemples en analyse: étude de cas en première d'université

The history of mathematics shows that counter-examples have played a substantial role in the development of functional concepts. We undertook a long-term study about the construction of examples and counter-examples in a college level calculus course. The subjects were three pairs of first year students majoring in mathematics/physics at the University of Fes (Morocco). In addition to the instructional role of constructing examples and counter-examples, our research addressed the students' thought processes. For this propose, their interactions, as well as all their verbal and written productions were collected and transcribed. Each sentence was then classified according to the four qualitative variables: Activity, Expression, Content and Correctness. A factor analysis put in evidence two types of strategies that we interpreted as global and local ones. In addition, a qualitative analysis allowed us to distinguish between `winning' and `losing' strategies.     

Disposition assessment in teacher education: A framework and instrument for assessing technology disposition

Abstract

This study proposes and verifies the concept of a technology disposition in teacher education, which is comprised of predisposition and competence. It examines the reliability and validity of a measure of the technology disposition of teacher education students, the Technology Disposition Scale for Teacher Education Students (TDS‐T). The TDS‐T is comprised of two subscales: technology predisposition and technology competence. In this study, the TDS‐T was completed by 656 teacher education students. Results showed the proposed disposition model of predisposition and competence had significant relationships with gender and class year. Male students had significantly higher overall technology disposition scores than did female students, but these differences were due to the male students' strong self‐concept, especially self‐confidence. Students technology competence level was significantly higher for seniors than for sophomores, and the overall technology disposition scores significantly increased from students' junior to senior years. The TDS‐T demonstrated content validity through factor analysis and convergent and discriminate validity through item analysis. The value for Cronbach's alpha was .93, indicating highly satisfactory reliability.     

Assessment of Chemistry Laboratory Classroom Environments

The Science Laboratory Environment Inventory (SLEI) is a recently developed classroom environment instrument for assessing students' or teachers' perceptions of their science laboratory classroom environment. This paper describes its development and reports on the validation and application of its modified form, the Chemistry Laboratory Environment Inventory (CLEI), with a Singapore secondary school sample. The sample consisted of 1 592 final-year secondary school (i.e. tenth grade) chemistry students from 56 intact classes from 28 randomly selected co-educational government secondary schools in Singapore. Various item and factor analyses supported the reliability and validity of the instrument for assessing students' perceptions of their chemistry laboratory environment specifically in Singapore.     

Qualitative methods to assess intercultural competence in higher education research: A systematic review with practical implications

A number of methods have been recorded in the literature with regard to assessing intercultural competence, but extant reviews tend to focus on self-report instruments in which students' responses can be easily quantified. To address the gap, this paper provides an overview of qualitative methods used to assess students' intercultural competence in higher education research. Based on 34 studies, 25 qualitative assessment methods have been identified and captured in a newly-developed typology. Results show that (1) the majority of qualitative methods to assess intercultural competence were writing-based and ‘expressive’ in nature; (2) the assessment results were mainly analysed in a ‘thematic’ or ‘scalar’ manner; and (3) very little assessment took place among STEM majors. These results are further discussed and problematized to provide implications for developing and assessing intercultural competence in higher education research and curriculum.     

Student evaluation of teaching quality in higher education: development of an instrument based on 10 Likert‐scales

Students' evaluation of teaching skills has been an important yet controversial tool in the improvement of teaching quality during the last few decades. When searching for an apt student questionnaire to measure instructional skills, it appeared that most existing questionnaires the authors were able to collect are based on a single‐item type of evaluation. Additionally, most of these instruments lack a theoretical foundation and hardly any instrument was tested with modern tests for reliability and validity. The authors managed to create a 31‐item instrument which comprises 10 Likert scales and is based on both the educational theory and empirical data. In this article, they present the different steps in constructing the instrument and discuss its reliability and validity. The results of this study underline the value of the use of a scaling technique in students' evaluation of teacher performance.     

The ecologies of civic competence: students' perceptions from one Singapore school

A growing body of research suggests that traditional assessments of democratic participation overlook students' present realities, and fail to capture the knowledge, skills and dispositions necessary to resolve public issues in the twenty-first century. Addressing these concerns, we employed an interpretive perspective in examining students' perceptions of civic competence in one Singaporean school. Analyses of qualitative data reveal students' perceived lack of civic competence to effect systemic change within their school environment and in the larger political arena. This perceived lack of civic competence can be attributed largely to rigid and hierarchical classroom, school and political structures, and the pragmatic focus of Singapore society. We discuss findings in light of civic education research, policy and practice.     

Teachers' judgments of students' early literacy skills measured by the Early Literacy Skills Assessment: Comparisons of teachers with and without assessment administration experience

Teacher judgments about students' academic abilities are important for several reasons, including their day‐to‐day instructional decision making. Not surprisingly, previous studies have investigated the accuracy of teachers' judgments about their students' reading abilities. Previous research, however, has not investigated teachers' judgments about students' early literacy skills, nor has previous research systematically examined how teachers' training and use of an objective assessment instrument impact their judgments of students' performance on that instrument. This exploratory study offers the first investigation of teachers' judgment accuracy of pre‐kindergarten students' early literacy skills, and compares the judgment accuracy of teachers with and without opportunities to administer the Early Literacy Skills Assessment (ELSA). Findings suggest that teachers with opportunities to administer the ELSA had a significantly higher percentage of accurate judgments across half of the ELSA subtests, but their judgment accuracy was still no better than 50%–60% on all but one ELSA subtest. Implications for school psychologist practitioners and researchers are presented. © 2010 Wiley Periodicals, Inc.     

The Academic Rational Beliefs Scale: Development,Validation, and Implications for College Counselors

This study reports on a new instrument, the Academic Rational Beliefs Scale, designed to measure college students' academic beliefs along a rational‐irrational continuum. The new instrument is potentially useful when working with students experiencing academic difficulties. Information about test construction, reliability, validity, and generalizability are presented and implications, benefits, and cautions associated with use of the instrument as a college counseling tool are discussed.     

Assessing reflective thinking in solving design problems: The development of a questionnaire

Reflection is a critical factor in solving design problems. Using good methods to observe designers' reflection is essential to inform the design of the learning environments that support the development of design problem‐solving skills. In this study, we have developed and validated a novel self‐reporting questionnaire as an efficient instrument to explore reflection, called Assessing Reflective Thinking in Solving Design Problems (ARTiD). This questionnaire has been developed based on the three‐dimensional model for reflective thinking: the timing, the objects and the levels of reflection. A total of 294 undergraduate and graduate students in the fields of engineering, computer science and instructional design were recruited to participate in one of four iterative, formative tests through which reliability and validity analyses were performed to revise and confirm the questionnaire. Examples of participants' reflection patterns were demonstrated to show the practical value of this questionnaire. In conclusion, the final version of ARTiD was presented as a valid instrument to explore students' reflection for research and educational practice. In addition, we found that ARTiD can be used as an effective learning tool to guide student designers' reflections toward better learning and performance.     

数学建模竞赛模式对教改的启示

数学理论知识是很多课程的基础,在实际工作领域中也有广泛的应用,而理工科大学生在抽象数学学习中常存在学习容易倦怠的情况。数学建模竞赛有助于提高学生的学习积极性和效率,提升教师的教学和科研水平,应在教学中进行更大范围的推广。文章在具体分析数学建模竞赛对师生教与学的促进作用的基础上,探讨了推广数学建模竞赛的方式。     

Differences in Competence and Beliefs About Autism Among Teacher Education Students

Students with autism often are placed in general education classrooms, indicating general education teachers should be prepared to meet these students' needs. This survey study of teacher education students examined differences in educational training, beliefs about autism, and competence working with students with autism, according to college level and major area of study. Results showed that teacher education students held accurate beliefs about most aspects of autism, but they demonstrated misconceptions about special, gifted abilities. Secondary teacher education students were less likely to demonstrate accurate beliefs about the social emotional features of autism over the course of their training. General education teachers reported a lack of competence working with students with autism and believed additional training was needed. Findings suggested ongoing preservice training is needed to foster accurate beliefs about autism and competence working with students with autism.