Testing Students with Special Needs: A Model for Understanding the Interaction Between Assessment and Student Characteristics in a Universally Designed Environment

This article presents a model of assessment development integrating student characteristics with the conceptualization, design, and implementation of standardized achievement tests. The model extends the assessment triangle proposed by the National Research Council ( Pellegrino, Chudowsky, & Glaser, 2001 ) to consider the needs of students with disabilities and English learners on two dimensions: cognitive interaction and observation interaction. Specific steps in the test development cycle for including students with special needs are proposed following the guidelines provided by Downing (2006) . Because this model of test development considers the range of student needs before test development commences, student characteristics are supported by applying the principles of universal design and appropriately aligning accommodations to address student needs. Specific guidelines for test development are presented.     

Meta-assessment in a project-based systems engineering course

Project-based learning (PBL) facilitates significant learning, but it poses a major assessment challenge for assessing individual content knowledge. We developed and implemented an assessment approach and tool for a mandatory undergraduate systems engineering PBL-based course. We call this type of assessment student-oriented meta-assessment. Research participants included 131 undergraduate engineering students who carried out team projects requiring conceptual modelling of complex systems. Next, individual students assessed their peer team projects. Finally, the course staff assessed students individually, based on the content knowledge that was reflected in students’ comments to their peers, by using the new meta-assessment tool. The research findings validated the meta-assessment tool which can serve for assessing various PBL courses. Our contribution is twofold: a new approach for assessing project-based undergraduate engineering courses, and classification of meta-assessment methods into three types, where a student-oriented meta-assessment as one of the three meta-assessment methods is our newly introduced and validated method.     

Using Tree Diagrams as an Assessment Tool in Statistics Education

This study examines the potential of the tree diagram, a type of graphic organizer, as an assessment tool to measure students' knowledge structures in statistics education. Students' knowledge structures in statistics have not been sufficiently assessed in statistics, despite their importance. This article first presents the rationale and method for using tree diagrams as assessment tools in statistics education, followed by an empirical study examining the technical quality of tree diagram assessment. Thirty-seven university students enrolled in an introductory statistics course, and four statistics experts participated in the study. The results provide evidence of reliability and validity for the proposed function of using tree diagrams to measure knowledge structures in statistics education. As reliability evidence, the interrater reliability of tree diagram scoring is high (Pearson's r = .96); alpha coefficient of the 21 linked vertex pairs in tree diagrams is .89. As validity evidence, experts performed better than novices on the tree diagram assessment; students' performance with the tree diagram is significantly correlated with their performance on statistics achievement tests (Pearson's correlation coefficient r = .62); tree diagrams are sensitive to the discrepancies in students' knowledge structures, and most students considered tree diagrams helpful to their organization of statistics knowledge.     

Effectiveness of a digital literacy programme for primary education students / Efectividad de un programa de alfabetización digital para estudiantes de educación primaria

This study carried out an assessment of the effectiveness of a digital literacy programme for primary education students. A quasi-experimental design with repeated measures of assessment was used. The sample consisted of 364 students (206 male and 158 female) from the sixth year of primary education. The students were assessed in their own schools and the sample was divided into two groups: experimental group (N = 190) and control group (N = 174). The assessment was carried out in three sessions: pre-intervention, post-intervention and six-month follow-up. The programme was composed of three sessions aimed at improving digital literacy, as a way of developing a safe and responsible use of the internet. The results obtained after the intervention showed statistically significant differences in the comparison between the two groups. Specifically, the experimental group reached a significantly higher degree of digital literacy (conceptual, procedural and attitudinal) than the control group after passing through the programme.     

Scientific Processes in PISA Tests Observed for Science Teachers

A research study, mainly based on the notion of ‘scientific literacy’ from the Programme for International Student Assessment (PISA) 2003 assessment framework, was carried out obtaining data from the administration of an open written questionnaire with items covering three central scientific processes—describing, explaining and predicting scientific phenomena; understanding scientific investigation; and interpreting scientific evidence and conclusions—to 30 experienced in‐service secondary school science teachers. The purpose was to analyse their views regarding the competences on the mentioned scientific processes assessed by Science PISA tests: which of the competences assessed were the most frequently identified by teachers, which of the competences they considered presenting difficulties for their students, and, finally, which activities they used in their classes to promote similar competences. Our results indicated that teachers had different perceptions of one or other scientific processes considered relevant for scientific literacy in the PISA framework. Their awareness of the expected students’ difficulties did not necessarily match the competences intended to be assessed by either PISA or what they thought to be assessed. Moreover, their views differed depending not only on the type of scientific process but also on the underlying subject. Concern about the students’ need of reading fluently with understanding and of paying special attention during the test time was also observed.     

The relationship between students’ perceptions of portfolio assessment practice and their approaches to learning

This study focuses on students’ learning approaches in the context of a competency‐based program on Applied Sciences, with portfolio assessment as its core mode of assessment. The study examines students’ perceptions of these assessment practices and the relationships to their learning approaches. Additionally, differences in perceptions and learning approaches between first‐year students and second‐year students, who already have one year of experience with the portfolio assessment practice, are investigated. A total of 110 students completed two questionnaires at the end of the academic year: the Revised SPQ (Study process questionnaire) was used to measure their approaches to learning and the AEQ (Assessment experience questionnaire) was used to measure their perceptions of the assessment practice. The results indicate that, when implementing portfolio assessment, feedback is essential. Moreover, it seems not to be the assessment design, but rather how it is implemented, and how students’ learning is stimulated through the assessment, that influences the quality of learning.     

Does formative feedback on search behaviour help students in answering comprehension questions from an available text? / ¿Ayuda la retroalimentación formativa sobre el comportamiento de búsqueda a contestar preguntas de comprensión en lecturas con texto disponible?

This study analyses the efficacy of formative feedback to boost students’ search behaviour when answering comprehension questions in a with-text reading situation, which is a common reading situation in instructional and assessment settings. In these reading situations search strategies play an important role to predict students’ performance. Sixty-five high school students read two texts and answered eight multiple-choice comprehension questions per text using the software Read&Answer, which recorded all the students’ actions. After answering each question, students received either global-search-feedback or specific-search-feedback, which differed in the specificity of their information, or no-feedback. Participants who received any feedback had a second chance to correct their wrong answers. Specific-search-feedback increased students’ search decisions and improved their use of relevant information to repair wrong answers over global-search-feedback. Consequently, specific-search-feedback improved students’ performance when they corrected wrong answers over global-search-feedback. These results have implications for the design and implementation of formative feedback in computer-based systems aimed at improving students’ performance and teaching reading literacy skills.     

Assessing test‐taking strategies of university students: developing a scale and estimating its psychometric indices

Test‐taking strategies are important cognitive skills that strongly affect students’ performance in tests. Using appropriate test‐taking strategies improves students’ achievement and grades, improves students’ attitudes toward tests and reduces test anxiety. This results in improving test accuracy and validity. This study aimed at developing a scale to assess students’ test‐taking strategies at university level. The scale developed was passed through several validation procedures that included content, construct and criterion‐related validity. Similarly, scale reliability (internal reliability and stability over time) was assessed through several procedures. Four samples of students (50, 828, 553 and 235) participated by responding to different versions of the scale. The scale developed consists of 31 items distributed into four sub‐scales: Before‐test, Time management, During‐test and After‐test. To the researcher’s knowledge, this is the first comprehensive scale developed to assess test‐taking strategies used by university students.     

Creating first-year assessment support: lecturer perspectives and student access*

While online courses provide opportunities for a range of assessment tools and resources, little is known about how lecturers might best provide assessment cues to support diverse cohorts. This study explored how lecturers communicate messages about assessment expectations to first-year students in two courses at an Australian regional university (Nursing, n = 323; Engineering, n = 153), and how students accessed the assessment resources. A comparative case study methodology was used. Data included interviews with course coordinators to identify the assessment cues provided, course online content related to assessment and data recording student access of assessment resources. Diverse messaging systems were observed and verbal and written cues were abundant and embedded throughout both courses; however, not all students accessed all cues. What remains unclear is how students might bring together these cues in a productive way. Further research must explore the extent to which students can be better supported to improve their learning.     

Measuring the impact of the flipped anatomy classroom: The importance of categorizing an assessment by Bloom's taxonomy

The flipped classroom (FC) model has emerged as an innovative solution to improve student‐centered learning. However, studies measuring student performance of material in the FC relative to the lecture classroom (LC) have shown mixed results. An aim of this study was to determine if the disparity in results of prior research is due to level of cognition (low or high) needed to perform well on the outcome, or course assessment. This study tested the hypothesis that (1) students in a FC would perform better than students in a LC on an assessment requiring higher cognition and (2) there would be no difference in performance for an assessment requiring lower cognition. To test this hypothesis the performance of 28 multiple choice anatomy items that were part of a final examination were compared between two classes of first year medical students at the University of Utah School of Medicine. Items were categorized as requiring knowledge (low cognition), application, or analysis (high cognition). Thirty hours of anatomy content was delivered in LC format to 101 students in 2013 and in FC format to 104 students in 2014. Mann Whitney tests indicated FC students performed better than LC students on analysis items, U = 4243.00, P = 0.030, r = 0.19, but there were no differences in performance between FC and LC students for knowledge, U = 5002.00, P = 0.720 or application, U = 4990.00, P = 0.700, items. The FC may benefit retention when students are expected to analyze material. Anat Sci Educ 10: 170–175. © 2016 American Association of Anatomists.     

Individual differences in negative group work experiences in collaborative student learning

This paper reports the development of the Negative Group Work Experiences questionnaire (NGWE), an assessment tool for measuring negative experiences of group work. Study 1 involved two samples of undergraduate psychology students (second‐year sample n = 425; first‐year sample n = 443), who completed research modules incorporating substantial elements of assessed group work. Participants completed a 39‐item inventory designed to measure their negative experiences of group work as part of their general module evaluation. Exploratory and confirmatory factor analysis of the second‐year sample indicated four interpretable and reliable factors: lack of group commitment, group fractionation, task disorganisation, and storming. Confirmatory factor analysis of the first‐year data set confirmed and cross‐validated the factor structure of the second‐year sample. Study 2 (n > 254) confirmed the criterion validity of the NGWE. The NGWE is proposed as a useful tool for evaluating group processes, especially in large group teaching which involves collaborative group work.     

Service learning as a tool to enhance future teachers’ media competence / Aprendizaje-Servicio como estrategia favorecedora del desarrollo de la competencia mediática en futuros docentes

Abstract

This paper presents a study of service learning (SL) as a methodology to facilitate the development of media competence in future teachers. The SL methodology fulfils a training and service need for two different populations: in our case, a preschool/primary school and students from the Bachelor’s in Education degrees at the Universidad de Jaén. The study involved 193 university students who carried out 63 projects involving new technologies. The goal of these projects was to produce materials to be used on smartboards, advertising materials, picture-reading materials or multimedia resources, among others. These materials were designed and created in the university classes as a part of the practical sessions of a course and were then given away to be used at a preschool/primary school. At the end of the project, the university students filled out a questionnaire that required them to compare SL to traditional activities in terms of how well each type of practice helps them to achieve the 20 competences of the course. The results show that students recognize the value of the SL methodology more than traditional practice, which suggests the relevance of this methodology for university students.     

A quantitative and qualitative inquiry into the attitudes toward science of nonscience college students

The attitudes toward science of nonscience college students were investigated using quantitative and qualitative forms of inquiry. Quantitative methods were used to determine (a) how attitudes toward science of nonscience college students compare with attitudes of science majors, and (b) whether attitudes toward science change with instruction. Qualitative assessment was used to investigate attitude development as it relates to science. The subjects were 102 nonscience students and 81 science students. Six attitudinal variables were investigated using the Attitudes Toward Science Inventory (ATSI) as the quantitative instrument. Hotelling's T² showed a significant difference (p = 0.0001) in attitudes between the two groups. T tests revealed significant differences between the two groups for all six variables. A significant difference (p = 0.0001) was found between pretest and posttest results for the nonscience students. T tests showed significant differences between the two sets of scores for all six variables, indicating a favorable change in attitudes. An interview questionnaire was used to investigate factors contributing to attitude development. The interview results suggested that attitudes toward science are formed by interactions of both school and nonschool variables.     

Optimizing quality of dental carving by preclinical dental students through anatomy theory reinforcement

Knowledge of dental anatomy is of great importance in the practice of dentistry, especially in oral rehabilitation, because without this knowledge, professional practice is not possible. Dental carving plays a major role in training dental students as it develops their manual dexterity. This randomized controlled trial aimed to evaluate the influence of didactic‐theoretical reinforcement on the theoretical and practical knowledge of dental anatomy of preclinical students by examining the quality of the anatomical restorations performed by these students before and after a didactic‐theoretical reinforcement. For the evaluation of theoretical knowledge, a questionnaire with closed questions about dental anatomy was used. To evaluate the effect of didactic reinforcement on dental carvings, two groups of 15 preclinical students were assessed. Experimental group (G1) received a three‐hour theoretical tutoring on dental anatomy, while the control group (G2) did not. The dental carving scores obtained by the two different groups were compared using Student's t‐test. Cohen's d was used to estimate the effect sizes between groups. The frequency of correct answers given for each theoretical knowledge question was compared in each group using Fisher's exact test. T‐test was also used to compare the means of the two groups' final scores of theoretical evaluations. To compare these final scores obtained in both carving and theoretical tests, a principal component analysis was performed with different items assessed in each test to obtain factor loading scores and a final weighted score, where factor loadings were considered for each item. Weighted scores were compared using t‐test. Also, scores obtained during the head and neck course were assessed and compared using t‐test. Spearman's correlation test was used to assess the correlation between scores obtained prior to the anatomy course and scores obtained in the dental carving exercise. The theoretical evaluation revealed no significant difference between the grades (mean ± SD) of G1 (85.1 ± 6.6%) and G2 (86.2 ± 9.1%) with the grades of a baseline test that was previously obtained when students submitted to the study (P = 0.725). Regarding the tooth carving assessment, the dental carving quality by students of G1 has significantly improved, except for tooth #23 (P = 0.096). Theoretical reinforcement of dental anatomy seems to improve the students' carving performance but does not enhance their knowledge about dental anatomy. Anat Sci Educ 11: 377–384. © 2017 American Association of Anatomists.     

El impacto de la cultura en el autoconcepto. Un estudio con mestizos de distintos entornos educativos de Chiapas

Resumen

El siguiente estudio explora el autoconcepto en 410 estudiantes mestizos de la Universidad Intercultural de Chiapas (144), la Universidad Autónoma de Chiapas (124) y la Universidad de los Altos de Chiapas (142). Nuestro objetivo era comparar los resultados de la tarea de autoconcepto personal y social (TAPS) entre los estudiantes que atendían a una de estas tres universidades. De acuerdo con la perspectiva cultural de la cognición situada, esperábamos encontrar distintos autoconceptos ya que los estudiantes participaban en distintos entornos educativos. Específicamente, esperábamos que los estudiantes que asistían a la Universidad Intercultural de Chiapas puntuarían significativamente superior en categorías sociales en sus autoconceptos que los estudiantes que asistían a la Universidad privada de los Altos de Chiapas y la Universidad Autónoma de Chiapas. Los resultados apoyan dicha hipótesis. Se realizó un análisis de la varianza (ANOVA) mostrando diferencias estadísticamente significativas en los grupos estudiados. Se sugiere que los procesos proximales son los mecanismos a través de los cuales la cultura influye en los individuos.     

Assessing dyslexia in higher education: the York adult assessment battery‐revised

Although there are a number of standardised measures to assess dyslexia in children, there are comparatively fewer instruments suitable for the assessment of dyslexia in adults. Given the growing number of students entering UK higher education institutions, there is a need to develop reliable tools for assessing the additional needs of those with dyslexia and related difficulties. This study reports data from a revised version of the York Adult Assessment: An Assessment Battery for Dyslexia Screening in Higher Education. The current York Adult Assessment‐Revised (YAA‐R) is an assessment battery consisting of tests of reading, spelling, writing and phonological skills. Data from a normative sample of 106 adults without dyslexia and a validation sample of 20 adults with dyslexia illustrate significant group differences on the tests comprising the YAA‐R. Additionally, the YAA‐R has good discriminatory power yielding 80% sensitivity and 97% specificity. Taken together, the YAA‐R is a suitable test battery for the assessment and identification of dyslexia in university students.     

Development of human resources in and for bilingual intercultural education in Latin America

Worked as consultant for the Deutsche Gesellschaft für Technische Zusammenarbeit on the Bilingual Education Programme of Puno (Peru). Professor of Andean Linguistics at the Universidad Nacional del Altiplano. Co-author (with Ingrid Jung) of Las lenguas en la educación bilingüe: el caso de Puno,and compiler of Pesquisas en linguística andinaand Pueblos indios, estados y educación.