Exploring the potential for and promise of incorporating distributive and procedural justices into post-secondary assessment of student learning

This paper explores the potential of incorporating constructs of distributive justice and procedural justice into summative assessment of student learning in higher education. I systematically compare the process used by managers to evaluate employee performance in organizations – performance appraisal (PA) – with processes used by professors to assess student learning in academe. These procedures are remarkably similar in terms of objectives to facilitate and measure performance/learning, criteria for defining desired performance/learning, the procedures used by raters to evaluate performance/learning, and the procedure used by employees and students to assess fairness of their PA/grades. I then summarize empirical research regarding distributive justice (DJ) and procedural justice (PJ) in organizations. Finally, I posit how positive attitudes and behaviors associated with DJ and PJ in organizations might transfer to benefit students, professors, administrators, and universities in academe.     

Grading leniency, grade discrepancy, and student ratings of instruction

The purpose of this study was to examine how grading leniency and grade discrepancy (the difference between expected grades and deserved grades) were associated with various dimensions of student ratings of instruction. A sample of 754 undergraduate college students completed a student ratings of instruction instrument and provided responses to a number of other questions on topics such as course difficulty and workload. A series of multilevel regression analyses were conducted and results showed that an instructor's grading leniency, as perceived by students, was positively associated with student ratings on 11 of 12 dimensions of instruction examined. This finding suggests that more lenient instructors tend to receive higher student ratings. The second finding shows that grade discrepancy was negatively associated with most dimensions of instruction. This supports the self-serving bias hypothesis under attribution theory (Gigliotti & Buchtel, 1990) in that students tended to punish instructors with lower ratings when expected grades were lower than students believed they deserved, yet little evidence of a pattern of rewards existed in student ratings when students expected grades higher than they deserved.     

Developing an online justice studies degree program: A case study

Online technology allows higher education to reach additional students. Because of its convenience many universities will offer a variety of criminology and criminal justice courses online. This paper presents a case study of the transfer of a graduate justice studies degree program from traditional instruction to online delivery. The transformation must involve the coordination of several university departments. Online course instruction includes development, instruction, and assessment stages that are distinct from traditional courses. The lessons learned from this process are discussed.     

Formatives Assessment im Leseunterricht der Grundschule – Implementation und Wirksamkeit eines modularen,materialgestützten Konzepts

Definitions of formative assessment include assessment, feedback and differentiated instruction as key components. We investigated the effects of prepared teaching materials designed to support teachers using learning progress assessment (LPA) to give feedback and adapt differentiated instruction. We also examined to what extend this modular approach can be implemented in regular reading lessons in third grade. In a three-group design all teachers (N?=?44, N?=?945 students) employed a computer-based LPA tool, while teachers in two conditions additionally received prepared feedback material (FB) or feedback and reading instruction material (FB+FM), to support the implementation of the different components of formative assessment. Over the course of one schoolyear, we assessed the implementation outcomes using questionnaires as well as students’ reading achievement and further student outcomes. While acceptability is high, teacher ratings of feasibility are low. In comparison to the LPA group, the additional support in form of prepared materials had no effects on student outcomes. Results are discussed regarding the question of how teachers can optimally be supported in using formative assessment.     

The development of science activities via on-line peer assessment: the role of scientific epistemological views

This study implemented an online peer assessment learning module to help 36 college students with the major of pre-school education to develop science activities for future instruction. Each student was asked to submit a science activity project for pre-school children, and then experienced three rounds of peer assessment. The effects of the online peer assessment module on student learning were examined, and the role of Scientific Epistemological Views (SEVs) in the learning process was carefully investigated. This study found that student peers displayed valid scoring that was consistent with an expert’s marks. Through the online peer assessment, the students could enhance the design of science activities for future instruction; for instance, the science activities became more creative, science-embedded, feasible and more suitable for the developmental stage of pre-school children. More importantly, students with more sophisticated (constructivist-oriented) SEVs tended to progress significantly more for designing science activities with more fun, higher creativity and greater relevancy to scientific knowledge, implying that learners with constructivist-oriented SEVs might benefit more from the online peer assessment learning process. These students also tended to offer more feedback to their peers, and much of the peer feedback provided by these students was categorized as guiding or helping peers to carefully appraise and plan their science activity projects. This study finally suggested that an appropriate understanding regarding the constructivist epistemology may be a prerequisite for utilizing peer assessment learning activities in science education.     

External and internal assessment in the final examination in secondary schools in Slovenia

This article examines the introduction of the final examination as an external national system of assessment of students in the transition from secondary school to university education. Since 1991 Slovenia has followed foreign models, the British one in particular; however, it has also considered its own traditional system of education. Students take the final examination (the matura) in three compulsory and two elective subjects; of which sociology can be one. A case study considering this subject demonstrates how external assessment can be fine tuned using internally supervised and assessed project work, in which seminar essays are produced. This article conducts a qualitative and quantitative analysis of the essays and structured questions forming the written examination; the same procedure is applied in the case of seminar essays. Also included are the opinions and experiences of students regarding their preparations for the final examination and their work in the seminars; the students’ feedback is decidedly positive. So far, the external and the internal results have shown little correlation, a fact that points to certain problems in the final examination assessment system. The positive results in sociology, considered in this contribution, suggest that it would be worthwhile applying similar assessment strategies in other subjects. It is, however, impossible to provide a straightforward answer to the question of a positive impact of the final examination on the desired improvement in the quality of the education process.     

School-Based Tests of the Triarchic Theory of Intelligence: Three Settings,Three Samples,Three Syllabi

Three studies were conducted at the middle and high school levels to assess the effectiveness of triarchically based instruction and assessment—which emphasizes analytical, creative, and practical thinking and learning skills as well as memory-oriented skills—to conventional instruction and assessment. Interventions emphasized reading in the context of instruction in language arts, math, physical sciences, social sciences, history, foreign languages, and the arts. There were 809 fifth-grade students in Study 1, 62 middle school students in Study 2, and 432 high school students in Study 3. Almost all students were from relatively lower socioeconomic status (SES) backgrounds, and they were highly diverse in terms of ethnic background. In all studies, triarchic instruction was more effective than conventional instruction in improving student reading achievement.     

Student knowledgeability,student sex,class size,and class level: Their interactions and influences on student ratings of instruction

Student knowledgeability, class size, and class level were found to significantly influence students' ratings of instruction. In general, the more knowledgeable the student in an area, the higher his ratings of courses and instructors in that area. Also, large courses and advanced courses were most highly rated by students. The effect of student sex on student ratings of instruction varied as a function of the particular aspect of instruction being evaluated. Significant interactions among the four main effects were also found across the judgmental dimensions students utilized in evaluating instruction, as assessed by factor analysis. Student knowledgeability and class size were found to be the main predictors of student ratings on these dimensions.     

Student Grades and Average Ratings of Instructional Quality: The Need for Adjustment

Abstract

The authors compared the average grades given in 165 behavioral and social science courses with the average ratings given by students to the instructors who taught the courses. Significant positive correlations were found between the average ratings for instructional quality and the average grades received by students. The courses in which the average grades were the highest were also those in which students gave teachers the highest ratings. Among possible reasons for the correlations are that better teachers attracted better students or that quality teachers provided more effective instruction, resulting in more student learning and, thus, higher average grades. Another explanation is that most college students tend to bias their ratings of instructional quality in favor of teachers who grade leniently (I. Neath, 1996). If correct, the latter reasoning begins to explain why the widespread use of student evaluations in the United States in recent decades has been accompanied by increases in the average grades that university students received. To prevent grade inflation, and particularly to avoid rewarding and promoting instructors who use increasingly lax grading standards, administrators should adjust student ratings of instructional quality for the average grades given for a course. In general, only courses near the extremely high and low ends in terms of students' average grades were significantly affected by the statistical adjustment.     

Reviewing criminal justice baccalaureate curricula: The importance of student input

While criminal justice undergraduate enrollments continue to burgeon, rarely do criminal justice departments conduct comprehensive surveys of their majors as part of the baccalaureate curriculum review process. This paper describes one such survey completed by 238 criminal justice majors at Wayne State University in Detroit. Major survey findings are presented along with their implications for student and faculty diversity, curriculum design, teaching and grading methods, student advising, and department administration.     

After-school tutoring in the context of no Child Left Behind: Effectiveness of two programs in the Pittsburgh Public Schools

The No Child Left Behind (NCLB) legislation has created pressure for districts to improve their students’ proficiency levels on state tests. Districts that fail to meet their academic targets for 3 years must use their Title I funds to pay for supplemental education services (SES) that provide tutoring or other academic instruction. Many districts, including the Pittsburgh Public Schools (PPS), have also adopted additional tutoring programs designed to help students reach proficiency goals. This paper examines student participation and achievement in two PPS tutoring programs—the NCLB-mandated SES program and a state-developed tutoring program. We examine the characteristics of students participating in each program, the effects of participation on student achievement, and the program features that are associated with improved achievement.     

Ratings of university teacher instruction: how much do student and course characteristics really matter?

Several student and course characteristics were examined in relation to student ratings of instruction. Students at a major Canadian university completed the Universal Student Ratings of Instruction instrument at the end of every course over a three‐year period, providing 371,131 student ratings. Analyses of between‐group differences indicate that students who attend class often and expect high grades provide high ratings of their instructors (p < .001). In addition, lab‐type courses receive higher ratings than lectures or tutorials, and courses in the social sciences receive higher ratings than courses in the natural sciences (p < .001). Regression analyses indicated, however, that student and course characteristics explain little variance in student ratings of their instructors (<7%). It is concluded that student ratings are more related to teaching instruction and behavior of the instructor than to these variables.     

Correlations,trends and potential biases among publicly accessible web-based student evaluations of teaching: a large-scale study of RateMyProfessors.com data

Student evaluations of teaching are widely adopted across academic institutions, but there are many underlying trends and biases that can influence their interpretation. Publicly accessible web-based student evaluations of teaching are of particular relevance, due to their widespread use by students in the course selection process and the quantity of data available for analysis. In this study, data from the most popular of these websites, RateMyProfessors.com, is analysed for correlations between measures of instruction quality, easiness, physical attractiveness, discipline and gender. This study of 7,882,980 RateMyProfessors ratings (from 190,006 US professors with at least 20 student ratings) provides further insight into student perceptions of academic instruction and possible variables in student evaluations. Positive correlations were observed between ratings of instruction quality and easiness, as well as between instruction quality and attractiveness. On average, professors in science and engineering disciplines have lower ratings than in the humanities and arts. When looking at RateMyProfessors as a whole, the effect of a professor’s gender on rating criteria is small but statistically significant. When analysing the data as a function of discipline, however, the effects of gender are significantly more pronounced, albeit more complex. The potential implications are discussed.     

Faculty Perspectives on Course and Teacher Evaluations

Student ratings of instruction have been the subject of numerous studies with much of the research focusing on the validity and reliability of the ratings themselves. Comparatively little empirical investigation has been devoted to the perceptions of the individuals who are the subjects of the ratings, that is, the faculty. The current study explored faculty perspectives on the usefulness of student ratings for formative and summative purposes, and the actual use of student ratings for summative purposes. Contrary to what might have been deduced from the anecdotal literature, the results of this study do not portray a great deal of resistance to student ratings in general or to their use for formative and summative evaluation. It was also found that student ratings are actually being used for the latter purpose. The usefulness of the student feedback was viewed differentially by the faculty, with feedback on their interaction with students seen as most useful, followed by feedback on their grading practices, global ratings of instructor and course, and finally structural issues of the course.     

Perceived autonomy support,intrinsic motivation,and student ratings of instruction

Motivation theory suggests that autonomy supportiveness in instruction often leads to many positive outcomes in the classroom, such as higher levels of intrinsic motivation and engagement. The purpose of this study was to determine whether perceived autonomy support and course-related intrinsic motivation in college classrooms positively predict student ratings of instruction. Data were collected from 47 undergraduate education courses and 914 students. Consistent with expectations, the results indicated that both intrinsic motivation and autonomy support were positively associated with multiple dimensions of student ratings of instruction. Results also showed that intrinsic motivation moderated the association between autonomy support and instructional ratings—the higher intrinsic motivation, the less predictive autonomy support, and the lower intrinsic motivation, the more predictive autonomy support. These results suggest that incorporating classroom activities that engender autonomy support may lead to improved student perceptions of classroom instruction and may also enhance both student motivation and learning.     

专题研讨式教学在研究生思想政治理论课中的运用

专题研讨式教学适应了研究生思想政治理论课的特点,适应了教学对象———研究生的特征,促进了任课教师的发展。专题研讨式教学的实施包括专题内容的设置、专题的讲授与研讨,教学的考核、教学的评价与总结四个方面。实施专题研讨式教学应注意三点,即坚持意识形态的导向性,提升任课教师的素养,与其他教学方式相结合。     

EIGHTH GRADE MATHEMATICS TEACHERS’ FORMATIVE ASSESSMENT PRACTICES IN SES-DIFFERENT CLASSROOMS: A TAIWAN STUDY

In this qualitative case study, the researchers explored 3 8th grade teachers?? formative assessment practices as it occurred during their mathematics instruction at socioeconomic status (SES)-different schools in Taiwan. Instructionally embedded formative assessment we refer to as discourse-based assessment practice (DAP). In Taiwan, there is a large mathematics achievement gap among high SES and low SES and minority students in 8th grade. Addressing this issue at the level of classroom practice, we attempt to document teachers?? DAP using a parsimonious, but practical framework for analyzing classroom discourse in 23 lessons. For the purpose of this study, 3 lessons for each teacher are represented. The extended sequence (Mehan, 1979) was the primary unit of analysis, to which we applied our framework for illuminating and differentiating the teachers?? press for student understanding through questions and feedback. The analysis of the teacher??s instruction at the high SES indicated that it was more formative than the others, pressing for mathematical understanding. The 2 teachers at the low SES schools pressed for rote understanding, though there were qualitative differences between them. The findings raise more questions and implications for research and education policy. In Taiwan where there is scant research on classroom practices, DAP as an instructional improvement may be a starting point for educational improvement for all its students.     

Occupational Therapy Students as Peer Assessors in Viva Examinations

This study was undertaken in order to evaluate the use of students as peer assessors, in collaboration with academic tutors, in the assessment of second-year viva examinations as part of a problem based learning occupational therapy curriculum. Data were collected from three consecutive cohorts of second-year students (N = 93), and an assessment was made of the reliability of the academic tutor marking, and the reliability of peer marking against the tutor marks. Results demonstrated that overall ratings of the viva examination performances given by the panel of assessors (two peer assessors and one academic tutor), were significantly correlated. On some occasions, such as the assessment of a borderline student, the ratings given were not as closely correlated. Some modifications of the examination process are suggested in order to optimise the reliability of the outcomes, but the study results lend support for the practice of peer assessment.     

Understanding and addressing the achievement gap through individualized instruction and formative assessment

This article synthesizes findings regarding the development of competence and learned helplessness and factors influencing persistence and intrinsic motivation, suggests the process through which small differences in early achievement are magnified by the current structure of schools, and reviews evidence suggesting that the characteristics of a specific type of individualized instruction and assessment system may be especially suited to remediate these differences. Age‐graded schools and group tests label students as ‘below’ and ‘above’ average, inadvertently demoralizing below‐average students, depressing effort and achievement, and perpetuating the gap in achievement between poor students and their more affluent peers. Analysis of the research literature suggests that the psychological experience of school for both high and low achieving students may be altered through a structure where instruction is individualized, students are challenged at their own levels, and each student receives objective assessment information confirming that he or she is successfully advancing to higher levels.     

Show and tell: Student and instructor perceptions of screencast assessment

This study addresses students’ and instructors’ perceptions of screencast assessment and compares this feedback method to traditional paper or digital text-based comments. Screencast assessment allows for asynchronous audio and visual commenting on student papers using screencast software. A pilot study using surveys of 39 students enrolled in lower-level composition classes and surveys of five composition instructors have indicated that screencast assessment promotes detailed and effective feedback on student writing. This feedback method reveals instructor's thought process while grading, and this transparency fosters student–teacher communication about writing. Screencast assessment changes the detail and types of comments, fostering an increased use of macro-level comments over micro-level concerns. The method does not necessarily save time grading but allows for feedback in the same amount of time. Because of the suggestion that students and instructors value screencast assessment, this study warrants a larger scale project in order to investigate the student preference for this feedback and whether the method leads to improvement in student performance.