Evaluating standards-based assessment rubrics in a postgraduate public health subject

Standards-based assessment rubrics are recognised as an important tool for ensuring clear criteria in higher education. Most of the available evidence on rubrics comes from studies with face-to-face undergraduate students. Our study evaluated the introduction of rubrics into a postgraduate subject that included both online and face-to-face students. We conducted formative evaluation with former students to develop the rubrics, as well as surveying current students for process and impact evaluation, supplemented with a focus group. Process results showed that the rubrics were used widely and were generally considered to be of at least some use. Clarity of the rubrics was the main issue affecting perceived usefulness, particularly for the assessment tasks that had undergone comparatively less formative evaluation. Impact evaluation showed mixed results, with students that reported always using rubrics having higher marks on average compared to students who did not always use the rubrics. However, an expected improvement in marks across the whole student cohort compared to previous years did not eventuate, although this could be explained by a number of factors independent of the rubrics. Our results provide some evidence for the use of rubrics in postgraduate and online settings but highlight the need for rigorous formative evaluation.     

A plan for the co-construction and collaborative use of rubrics for student learning

Abstract

The use of assessment rubrics in the higher education sector is now widespread in a number of disciplines. Typically, these rubrics are constructed by teachers who also tend to be the main users of the rubrics throughout the grading process. In recent years, questions have been raised about this teacher-directed approach and some educators have begun to explore an alternate approach to rubric construction; that is, engaging students in collaboration with their teachers to co-construct assessment rubrics. This paper outlines the processes employed in a project that investigated the co-construction of rubrics within six different contexts. The project aimed to engage students in collaboration with their teachers to co-construct rubrics which are co-owned by teacher and student. A mixed methods approach was utilized to explore the effectiveness of the strategy. Questionnaires, interviews and focus groups were utilized to gather data from both the teacher-participants and student-participants regarding their experiences of being involved in the study. Findings are presented from the perspectives of both students and teachers, relating their views of rubrics and the activity of rubric co-construction. The paper concludes with recommendations for practical approaches to rubric co-construction and future research directions.     

Unpaving the Road to Hell: Disrupting Good Intentions and Bad Science About Islam and the Middle East

Teachers across all subject areas engage students, in some way, in the study of otherness—other societies, other cultures, other practices. Often teachers and teacher educators attend to teaching about others with strong desires toward social justice as they seek to make a difference and do good. However, with insufficient tools to interrogate their practices and beliefs to think critically what good actually entails, they can unwittingly pave the road to hell. When good intentions are additionally coupled with the bad science of incomplete knowledge, misinformation, and weak arguments, the road can get treacherous. In this article, we examine the road to hell as it winds through teaching about a specific other, Muslims. We examine how good intentions and bad science about Muslims and Islam have worked to cement stereotypes, promote intolerance, shut down learning, and in doing so thwart education for social justice. Peering closely, we examine commonly voiced student conceptions of Muslims/Islam/The East and highlight the good intentions and bad science behind many of the popular discourses that students advance. We then offer strategies for building a different path, by problematizing good intentions and repairing bad science.     

Demystifying the rubric: a five-step pedagogy to improve student understanding and utilisation of marking criteria

Marking-criteria rubrics are commonly used to judge the quality of student work, but few students receive instruction to effectively use and apply rubrics. This study investigates an intervention designed to encourage effective utilisation of rubrics. The study, completed over two years (2011: n?=?189 respondents; 2012: n?=?464 respondents) in a large first-year course, explored how five instructional activities, used formatively and sequentially impacted student learning. This intervention comprised: (1) deconstruction of the rubric and standardising the marking method; (2) examples and exemplars; (3) peer review; (4) self-review; and (5) a reflective diary. Results showed an increase in student confidence with marking criteria and assessment (89% in 2011) and a statistically significant improvement in marks. Most students (77% in 2011 and 90% in 2012) rated each intervention as useful. Student feedback indicated that substantial changes occurred between initial drafts and final submission, which resulted in improvements in student performance scores.     

Exploring variation in nurse educators’ perceptions of the inclusive curriculum

This paper reports findings from a study into how nurse educators view the notion of an inclusive curriculum within their discipline. UK nurse education is professionally accredited, with substantial levels of work-based learning. Therefore, this analysis should be useful to practitioners on other professional courses. The study was based on a phenomenographic analysis of data collected from interviews with 15 nurse educators in one university. Analysis exposed wide variation in respondents’ perceptions and experiences of developing and delivering an inclusive curriculum. Much discussion focused on teaching students with disabilities and tutors expressed concern over implications regarding fitness for practice and public safety. However, there was recognition that diversity was a feature of the contemporary educational environment and that nursing courses had much to gain from inclusive practices. Key differences identified related to the extent to which change was required to current processes and practices. The findings indicated a concern over support for educators to manage these issues and suggested that in the absence of more robust direction, the student experience is shaped by the attitudes of individual tutors. In light of this, the author suggests that a coordinated and consistent response from the higher education establishment, professional bodies, practitioners and policy-makers is necessary to fully establish the notion of inclusive curricula in any professionally accredited course.     

Rubrics as a way of providing transparency in assessment

This paper reports on a study where rubrics have been used to convey assessment expectations to students (n?=?176) in three different assessment situations in professional education. These situations are: (1) the development of a survey instrument, which was part of a course in statistics and epidemiology; (2) an inspection of a house, which was part of a course about the functions of buildings for real estate brokers and (3) a workshop in communication with patients, which was part of a course in the evaluation of diagnostic procedures and treatments of oral infections in dental education. In all situations, students’ perceptions and uses of the rubrics were investigated. Findings suggest that it is indeed possible to convey expectations to students through the use of rubrics, in the sense that students not only appreciate the efforts to make assessment criteria transparent, but may also use the criteria in order to support and self-assess their performance. Important features of the rubrics, which were found to facilitate students’ understanding and use of the criteria in these situations, are presented and discussed.     

Investigating the Alignment of Bhutanese Mathematics Teachers’ Planned Approaches Within the Context of a Reformed Curriculum

Reform of mathematics education has been in focus in many countries including those in major economic transition. This paper reports a segment of a study which was conducted in Bhutan, where a reformed elementary mathematics curriculum has been recently introduced. The reformed curriculum is based on social constructivism and its design has been influenced by the USA’s National Council of Teachers of Mathematics (NCTM) standards. This paper reports upon the planned teaching of a sample of elementary (primary) teachers in their implementation of the reformed curriculum. The data for this part of the investigation were aggregated from a qualitative survey of randomly selected elementary school teachers (N = 72) from 40 schools across the country. A set of 3 analytical rubrics, based on intentions embedded in the curriculum, was developed to analyse the alignment of teachers’ planned sample learning activities with the intentions of the curriculum. Our conclusions are that while the curriculum’s intentions might be meaningfully stated in the curriculum documents, implementation remains problematic because there appears to be a mis-alignment between curriculum intentions and classroom practices. For instance, teachers by not adopting evidence-based approaches recommended in the curriculum document and are continuing to deny students opportunities to achieve deep understanding in achieving national priorities. This paper also argues that the proposed analytical rubrics may be of value for Bhutanese mathematics educators and, with minor modification, educators in other contexts, as they critique planned and actual practices.     

The Urban Bridge is a symbol

Response to Intervention (RtI) is a general education initiative that takes place prior to evaluation for placement in special education. Essentially, the first two tiers of RtI require general education teachers to use research-based instruction with all students and then to evaluate the effectiveness of that instruction. Given that the use of RtI as an assessment tool begins in the general education classroom, the following question remains: Are general educators aware of the development and implementation of RtI as a process for assessing and, eventually, possibly placing students into special education? The purpose of this study was to review the literature to answer the following two questions:

1. To what extent is RtI evident in the professional literature of educator groups (teachers, administrators, and teacher educators) who will be responsible for implementing these school practices?

2. What is the nature of the published literature (concept papers, assessment practices, instructional practices, and research) on RtI?

     

Modeling Resilience in an Alternative Education Context

Black and Latino students and the educators who serve as their advocates must be resilient in a highly contentious and racialized public education system. Modeling resilience in the public education system relies on individual assets and the extent to which one’s school possesses the resources and support needed to cultivate such assets. This study used interviews and site observations to explore the degree to which educators and students model resilience in an alternative education program. Findings suggest educators engage a sense of agency when they advocate for inclusion of Black and Latino students. In addition, an alternative education context can provide resources and cultivate strong relationships, a sense of self-efficacy and future orientation among educators and students. These findings highlight the assets educators and advocates need and the programs required to promote resilience in the public education system.     

You are never alone: understanding the educational potential of an ‘urban solo’ in promoting place-responsiveness

Abstract

Solos have a long history in outdoor learning (education) for their use in ‘wilderness’ settings. They have been theorised in experiential education literature and through phenomenological concepts where dwelling and solitude provide unstructured time for individual reflection. Place-Based Education provides opportunities for educators to develop place-responsive practices in areas local to where people live, work and study. This paper reports on an exploratory investigation into a Masters-level course. Students were required to undertake an ‘urban solo’ in a familiar place and consider how ontological disruption might be nurtured in city-based locations. Six students participated in semi-structured interviews which were then analysed thematically using a constructivist grounded theory approach. Findings revealed that their solo experiences had had a profound and unexpected effect on all students. These changes seemed to occur as the result of an ontological shift in the students from being passive receptors of stimuli to engaging more actively with their surroundings. We concluded that, because of its simplicity and close proximity to school grounds, the urban solo is one way for teachers to overcome the barriers they consistently report to outdoor learning. We recommend that future studies draw further on urban theorists to develop more city-based, place-responsive practices.     

Contrasting disclosure practices and experiences of school support for Australian adolescents with cancer,cystic fibrosis and anorexia nervosa: parent perspectives

Schools in Australia are required to promote equitable access to education and provide support services to students with chronic health conditions (CHCs). This qualitative study was conducted to explore stakeholders’ experiences and perceptions regarding school-based support. In-depth interviews were conducted with 38 parents of upper secondary school students with cystic fibrosis, anorexia nervosa or cancer (which were illustrative of different types of severe congenital, mental health or somatic CHCs). This enabled comparison of findings to determine whether there were any patterns in perceptions and experiences that were tied to the different CHCs. Data were analysed using a modified grounded theory approach to identify themes. Even though analysis indicated that all cohorts of parents viewed informed educators as best placed to provide tailored school-based support, major cohort differences were reported in parental experiences of school-based support as well as parental diagnosis disclosure behaviour. This latter finding was associated with cohort-related perceptions of risk to their adolescent child. Barriers to school-based support were perceived as varying by condition, the advocacy experience of parents and the nature of health service supports. Taking these findings into account, modification of current policies and practices within education and health sectors may lead to more equitable educational outcomes.     

Knowledge typologies for professional learning: educators’ (re)generation of knowledge when learning open educational practice

Open education resources (OER) and accompanying open education practices (OEP), are changing the education landscape. For educators to take full advantage of the opportunities OER offer they must engage in learning activities to facilitate the extension and adaption of their practice. This paper forms part of a larger study exploring how adult educators learn from and through their engagement with OER in the contexts of their work. Following a quantitative investigation of the learning behaviours of 521 educators around OER use, follow up interviews were conducted with 30 participants. The interviews explore in greater detail the ways knowledge is being (re)generated and used by the educators as they learn new practices with and through OER. Six broad knowledge types were identified as supporting the expansion of practice. The data suggest educators not only require multiple types of knowledge, but also must be able to move fluidly among these different types of knowledge.     

Quality and Qualms in the Marking of University Assignments by Sessional Staff: An Exploratory Study

The higher education sector is increasingly reliant upon casual (‘sessional’) staff for teaching and marking purposes. While this practice has been little examined in the past, over the last few years increasing attention has been paid to the quality of marking, mainly because students and academic staff alike are becoming increasingly likely to question examples of poor practice. Hence, many universities in Australia are now attempting to introduce stricter procedures relating to marking. Despite current concerns, there is little published research on marking practices in Australian universities. This paper adds to the body of knowledge by reporting on two pieces of empirical research into the use of casual markers. A project at Charles Sturt University comprised focus groups of, respectively, students, lecturers and markers, and a survey of distance education students. Research at the University of South Australia focused on pedagogical issues relating to marking, comparing the approaches of permanent lecturing staff with those of sessional markers. The results of these projects provide a useful insight into areas of current concern to university staff and management.     

The moral aspects of teacher educators' practices

The growing political, social and scientific attention that is being devoted to the moral aspects of teaching has implications for teacher education. This paper reports on a study of the actual moral education practices of 54 teacher educators within one institution. We encouraged these teacher educators to make their values explicit and to explain how they put them into practice. Nine teacher educators were studied in detail. These teacher educators were then stimulated to reflect on their values by completing charts to analyse the moral aspects of their practices. In addition, one of their lessons was videotaped and discussed. An important conclusion of this study is that whilst the responsibility for preparing student teachers for moral education rests with individual teacher educators, this process is largely implicit and unplanned. This is due in part to the lack of a language for expressing the moral dimension in teaching. Both teacher educators and students emphasise the importance of the role that attitudes play in the expression of values by teacher educators.     

‘I’m just not that comfortable with technology’: student perceptions of and preferences for Web 2.0 technologies in reflective journals

Encouraging reflective practice and developing reflective practitioners is a goal of many disciplines in higher education. A variety of pedagogical techniques have been used to promote critical reflection including portfolios, narratives and reflective journals. Over the past decade, the use of Web 2.0 technologies with students has been increasingly adopted in higher education settings and many educators have integrated these technologies into reflective assignments. These educators assume that students, who are members of the Net Generation, are technologically savvy and have the ability to integrate the use of Web 2.0 technologies into learning. However, while there have been studies examining the outputs of reflective assignments using Web 2.0 technologies such as blogs, e-portfolios and wikis, there has been little research examining whether or not students actually use technology for these types of assignment if given the choice. Therefore, the purpose of this study was to explore if technology was appropriated or rejected by students for a reflective journaling assignment. Results are based on a content analysis of 42 student journaling assignments and interviews with eight students. Findings suggest that (1) students are not as technologically competent as assumed; (2) students chose to use basic/fundamental technologies (e.g. word processing) because they viewed it as the easiest way to complete the reflective journaling assignment; (3) student perceptions of what makes an assignment ‘good’ influenced their choice to use Web 2.0 technologies; and (4) overarching student perceptions of higher education and learning impacted their appropriation of technology. Implications are discussed and recommendations for both research and practice are made.     

分数功能的异化与回归

在现实的学校教育中,分数经常被误用和滥用,并对学生的发展造成伤害。然而,学校教育又不可能从根本上消灭分数。其实,分数本身无所谓好与坏,关键在于如何合理、有效地使用分数。合理使用分数的基本标准在于:能帮助教师诊断和改进教学;能激励和强化学生的学习;切实体现和反映学校教育的总体目标;评价的手段、方法应与评价的内容、对象相适应。依此标准,在对学生的学习进行评分时,教师应突出和强化对学生学习过程的评分;充分考虑学生的努力与进步程度;采用灵活、弹性的评分;注重自我评分;面向学生的全面发展评分;采用多元化的评分方法与手段等。     

Self‐Assessment in Art Education through a Visual Rubric

A tool for self assessment in secondary art education was developed and tested. The tool includes rubrics for assessing production and reception activities in art education and consists of visual and text rubrics. The criteria in the rubrics are based on the Common European Framework of Reference for Visual Literacy which was developed by The European Network of Visual Literacy (ENViL). The way teachers and students use the rubrics, whether they consider them helpful and to what extent students’ self‐assessments are in line with teacher assessments was studied. It was concluded that teachers work with the rubrics intensively and both students and teachers appreciate its visual form. However, it was found that the agreement between teachers and students about the students’ scores was moderate and needed to improve. The results show that it is untrue that students, or boys in particular, overestimate their own performance in art education. The current study contributes to the development of feasible and valid assessment criteria and instruments in secondary art education.     

An interrogation of the role of critical thinking in English language pedagogy in Chile

The present study aims to critically interrogate the role of Critical Thinking (CT) in English language education in Chile through the analysis of teacher educators’ and postgraduate students’ perceptions and understandings of CT in relation to their academic trajectories through university. Five postgraduate students and five teacher educators from three different Masters of Teaching English as a Foreign Language (MTEFL) programs agreed to take part in the research. Informed by a qualitative paradigm, semi-structured interviews were conducted aimed mainly at (1) unpacking teacher educators’ and postgraduate students’ understandings of CT; (2) investigating postgraduates’ views of the place of critical thinking in their English teacher education training; (3) examining the ways in which teacher educators deal with CT in their pedagogies; and (4) identifying students’ preparedness to approach reading and writing critically, as viewed by both groups of participants. Analysis of responses revealed, on the one hand, students’ recognition of CT skills as necessary to succeed in academic life and, on the other, great concerns for the limited importance accorded to teaching these skills in teacher education courses. Teacher educators’ responses generally showed an increasing interest in trying to incorporate the teaching of CT skills in their pedagogies, but admitted to an overall lack of consistency in the implementation process. The paper concludes with critical questions about the perceived pedagogical mismatches between teacher educators’ and postgraduate students’ perceptions of the role of CT in their academic journeys, and about the role of Chilean teacher education programs in addressing these issues.     

An Investigation of Counselor Educators' Attitudes Toward Evidence‐Based Practices

Evidence‐based practices (EBPs) receive little attention in counselor education curricula despite the ethical obligation for counselor educators to teach EBPs to counseling students. This study investigates counselor educators’ attitudes toward EBPs along with perceived barriers to the inclusion of EBPs in counselor education curricula.     

Design of Online Professional Development in Science Content and Pedagogy: A Pilot Study in Florida

The Exploring Florida Science project goals were: (1) increasing content knowledge of secondary science teachers, specifically in topics that are important to the future of the state, and (2) providing secondary science students with digital media for use in project-based learning. A team of instructional designers, educators, scientists, web designers and teacher educators designed an online professional development environment. Needs analysis included identification of the high incidence topics on the state science achievement test and topics of socio-scientific importance in the state. Development followed tenets of design-based research, and was guided by E-Learning for Educators standards and evaluated for content, pedagogy and usability using rubrics based on established guidelines. This paper details design, development, and evaluation frameworks, and summarizes pilot testing outcomes.