The Accuracy of Aggregate Student Growth Percentiles as Indicators of Educator Performance

Mean or median student growth percentiles (MGPs) are a popular measure of educator performance, but they lack rigorous evaluation. This study investigates the error in MGP due to test score measurement error (ME). Using analytic derivations, we find that errors in the commonly used MGP are correlated with average prior latent achievement: Teachers with low prior achieving students have MGPs that underestimate true teacher performance and vice versa for teachers with high achieving students. We evaluate alternative MGP estimators, showing that aggregates of SGP that correct for ME only contain errors independent of prior achievement. The alternatives are thus more fair because they are not biased by prior mean achievement and have smaller overall variance and larger marginal reliability than the Standard MGP approach. In addition, the mean estimators always outperform their median counterparts.     

Examining the Reliability of Student Growth Percentiles Using Multidimensional IRT

Student growth percentiles (SGPs, Betebenner, 2009) are used to locate a student's current score in a conditional distribution based on the student's past scores. Currently, following Betebenner (2009), quantile regression (QR) is most often used operationally to estimate the SGPs. Alternatively, multidimensional item response theory (MIRT) may also be used to estimate SGPs, as proposed by Lockwood and Castellano (2015). A benefit of using MIRT to estimate SGPs is that techniques and methods already developed for MIRT may readily be applied to the specific context of SGP estimation and inference. This research adopts a MIRT framework to explore the reliability of SGPs. More specifically, we propose a straightforward method for estimating SGP reliability. In addition, we use this measure to study how SGP reliability is affected by two key factors: the correlation between prior and current latent achievement scores, and the number of prior years included in the SGP analysis. These issues are primarily explored via simulated data. In addition, the QR and MIRT approaches are compared in an empirical application.     

Assessing Learning Outcomes: Student Employees in Student Affairs

Erin Bentrim, Kim Sousa‐Peoples, Garrett Kachellek, and Wendy Powers report the results of assessing the effect of students' work experiences on their personal and academic development.     

Covariate Measurement Error Correction for Student Growth Percentiles Using the SIMEX Method

In this study, we examined the impact of covariate measurement error (ME) on the estimation of quantile regression and student growth percentiles (SGPs), and find that SGPs tend to be overestimated among students with higher prior achievement and underestimated among those with lower prior achievement, a problem we describe as ME endogeneity in this article. We proceeded to assess the effect of covariate ME correction on SGP estimation at two levels—the individual (student) and the aggregate (classroom). Our ME correction approach was limited to the simulation‐extrapolation method known as SIMEX. For both the individual and aggregate SGP, we find SIMEX effective in bias reduction. Further, because SIMEX is especially effective in reducing SGP bias for students with very high or very low prior achievement, it significantly weakens the ME endogeneity. SIMEX is also effective in reducing the MSE of aggregate SGP, provided that the students are sorted to some extent on their latent prior achievement. Our empirical study confirms the pattern of the simulation results: SIMEX mainly affects the mean SGP of classes in the highest and lowest quintiles of the prior score distribution, and significantly lowers the correlation between class SGP and prior achievement.     

Predicting Student Growth in Mathematical Content Knowledge

Abstract

Using national longitudinal data, the authors investigated factors related to student learning or growth in statistics, algebra, and geometry in middle school and high school. The authors used hierarchical linear models to model variation in student rate of growth with factors associated with student characteristics and instructional and environmental factors. In addition, the authors designed the study to identify factors that differentially affect student growth at different levels of secondary school (middle school vs. high school) and for different mathematical content areas (statistics, algebra, geometry). Results indicated substantial growth in all 3 content areas in both middle school and high school. Factors related to student learning were identified and found to differ by level of secondary school and content area.     

The Role of Student Characteristics in Predicting Retention in Online Courses

Given the continued issue of student retention for online classes, past research has suggested several “retention strategies” focused on engaging students as a way to reduce their withdrawal rate from these classes. However, a recent study testing the effects of these strategies on retention in online undergraduate business courses (Leeds et al., Int J Manage Educ 7(1/2), 2013) did not show empirical support for the effectiveness of such strategies. Taking an alternative approach that focuses on individual characteristics of students, this study takes a broader view and examines previous research literature on traditional face-to-face classes to determine how individual characteristics of students may be associated with the likelihood of withdrawal from online classes. Using a sample of undergraduate students (n = 2,314) from a large state university, results from this study identified prior performance in college classes (cumulative GPA) and class standing (senior vs. non-senior) as significant student characteristics related to student retention in online classes for all students. Other factors significantly related to retention rates for students with certain characteristics or within certain majors include previous withdrawal from online courses, gender, and receipt of academic loans.     

The Effects of Reciprocal Imitation on Teacher–Student Relationships and Student Learning Outcomes

Neuroscientific and developmental psychological research in imitation has yielded important insights into building teacher–student relationships and enhancing students' learning. This study investigated the effects of reciprocal imitation on teacher–student relationships and students' learning outcomes in one‐on‐one teacher–student interactions. In a within‐subjects design, participants learned eight English vocabulary words under two conditions: one condition paired with teacher's imitative behaviors and the other with teacher's random behaviors. Students' self‐rating surveys and quiz scores on new words were assessed. When the teacher imitated the students' behaviors in interactions, the students reported significantly higher perceptions of rapport, more confidence in and satisfaction with learning outcomes, and had significantly higher quiz scores. Results had important implications for teachers in using imitation as an effective teaching tool to build teacher–student relationships and enhance students' learning.     

Articulating and Utilizing Communication Student Learning Outcomes in Community Colleges

Academia in the United States stands face-to-face with the constant call for greater levels of assessment and accountability. Academics question whether our students are learning what they’re taught? Are they able to secure employment upon completing their education? Are they prepared to be active, productive citizens in society? Community colleges provide education to those who may typically not have the opportunity to attend an institution of higher education. A foundation of any assessment process is the articulation of the core learning outcomes. It is from this foundation, then, that community college faculty can start assessing the effectiveness of teaching and curricular design. The present investigation details a project that focused on creating discipline-specific student learning outcomes and how these outcomes can be utilized to aligning learning outcomes and mapping to curriculum. Communication studies faculty can utilize the learning outcomes to strengthen and clarify curriculum, demonstrate the acquisition of competencies, and improve departmental offerings.     

Student Self-Reported Learning Outcomes of Field Trips: The pedagogical impact

In this study, we used the classification and regression trees (CART) method to draw relationships between student self-reported learning outcomes in 26 field trips to natural environments and various characteristics of the field trip that include variables associated with preparation and pedagogy. We wished to examine the extent to which the preparation for the field trip, its connection to the school curriculum, and the pedagogies used, affect students’ self-reported outcomes in three domains: cognitive, affective, and behavioral; and the extent the students' socioeconomic group and the guide's affiliation affect students' reported learning outcomes. Given that most of the field trips were guide-centered, the most important variable that affected the three domains of outcomes was the guide's storytelling. Other variables that showed relationships with self-reported outcomes were physical activity and making connections to everyday life—all of which we defined as pedagogical variables. We found no significant differences in student self-reported outcomes with respect to their socioeconomic group and the guide's organizational affiliation.     

Teaching Behavior and Student Learning Outcomes in Dutch Mathematics Classrooms

Abstract

In this article I described the means of identifying teaching behaviors that have cognitive and affective learning effects on students who are taking a course in mathematics. This study was conducted on 50 mathematics teachers who were teaching in the eighth grade. I obtained the data on teaching behaviors through direct systematic observation. Multiple regression was used as the method of analysis. For the cognitive domain, the results showed that effective teaching behaviors are: (a) high-level questions put to a large group of students; (b) probing, followed by a correct student response; (c) teacher waiting after asking a question; (d) successful redirecting; and (e) all forms of positive acknowledgement. Effective teaching behaviors in the affective domain are: (a) all forms of teacher lecture/explanations; (b) probing, followed by correct student response; and (c) all forms of positive acknowledgement. More teaching behaviors have a positive effect on mathematical knowledge than have a positive effect on students’ attitude toward mathematics.     

The Effects of Discipline on Deep Approaches to Student Learning and College Outcomes

“Deep learning” represents student engagement in approaches to learning that emphasize integration, synthesis, and reflection. Because learning is a shared responsibility between students and faculty, it is important to determine whether faculty members emphasize deep approaches to learning and to assess how much students employ these approaches. This study examines the effect of discipline on student use of and faculty members’ emphasis on deep approaches to learning as well as on the relationships between deep approaches to learning and selected educational outcomes. Using data from over 80,000 seniors and 10,000 faculty members we found that deep approaches to learning were more prevalent in Biglan’s soft, pure, and life fields compared to their counterparts. The differences were largest between soft and hard fields. We also found that seniors who engage more frequently in deep learning behaviors report greater educational gains, higher grades, and greater satisfaction with college, and that the strength of these relationships is relatively consistent across disciplinary categories.     

The Impact of Student Learning Outcomes Assessment on Technical and Professional Communication Programs

Because of accreditation, budget, and accountability pressures at the institutional and program levels, technical and professional communication faculty are more than ever involved in assessment-based activities. Using assessment to identify a program's strengths and weaknesses allows faculty to work toward continuous improvement based on their articulation of learning and behavioral goals and outcomes for their graduates. This article describes the processes of program assessment based on pedagogical goals, pointing out options and opportunities that will lead to a meaningful and manageable experience for technical communication faculty, and concludes with a view of how the larger academic body of technical communication programs can benefit from such work. As ATTW members take a careful look at the state of the profession from the academic perspective, we can use assessment to further direct our programs to meet professional expectations and, far more importantly, to help us meet the needs of the well-educated technical communicator.     

The Impact of Student Learning Outcomes Assessment on Technical and Professional Communication Programs

Because of accreditation, budget, and accountability pressures at the institutional and program levels, technical and professional communication faculty are more than ever involved in assessment-based activities. Using assessment to identify a program's strengths and weaknesses allows faculty to work toward continuous improvement based on their articulation of learning and behavioral goals and outcomes for their graduates. This article describes the processes of program assessment based on pedagogical goals, pointing out options and opportunities that will lead to a meaningful and manageable experience for technical communication faculty, and concludes with a view of how the larger academic body of technical communication programs can benefit from such work. As ATTW members take a careful look at the state of the profession from the academic perspective, we can use assessment to further direct our programs to meet professional expectations and, far more importantly, to help us meet the needs of the well-educated technical communicator.     

The Impact of Taiwan's University Multiple-Channel Entrance Policy on Student Learning Outcomes

This research explores the impact of Taiwan's university multiple-channel entrance policy on student learning outcomes, using quantitative research to look for differences in the learning experiences of third-year students who were admitted via different methods (examination and placement, application for admission, recommendation and selection, and gifted aptitude selection) and who were at different types of institutions (public universities, public vocational/technical schools, private universities, and private vocational/technical schools). The source data for this research was a 2005 national survey of third-year university students by the National Taiwan Normal University Center for Educational Research and Evaluation. The data were analyzed using two-way analysis of variance (ANOVA) and found that among students at public universities and vocational/technical schools, those who applied for admission had a better learning experience, while those selected for gifted aptitude had a poorer learning experience. Among students at private schools, those admitted through examination and placement or application for admission had a significantly better learning experience than students admitted through selection. Overall, the learning experience of screened students was generally poor, so it is worth considering a review of this admission method.     

学习产出视角下高等职业教育学生学习评估探析

高职学生学习产出是指学生经过学习后,其职业知识、职业技能、职业行为及职业道德等方面发生的变化。高职学生学习产出具有职业性、实践性和应用性等特点。以学习产出视角反思当前我国高职学生学习评估现状,需要树立系统化的学习评估理念;学校需要从被评者转变为评估主体和主导者;行业企业和学生需要共同参与以学习产出为核心的学习评估标准制定;学习信息的复杂性和多样性需要采用多元收集方法。     

Project‐Based Learning Involving Sensory Panelists Improves Student Learning Outcomes

Project‐based, collaborative learning is an effective teaching method when compared to traditional cognitive learning. The purpose of this study was to assess student learning after the completion of a final meal project that involved a group of sensory panelists. A paper survey was conducted among 73 senior nutrition and dietetics students enrolling in Experimental Food Science, Spring 2013 and 2014. The survey comprised of qualitative and quantitative questions about students’ perceived learning outcomes. The project required students to prepare a 4‐course meal in 90 min, in groups of 4. Eighteen sensory panelists consisting of faculty and graduate students from the Nutrition, Dietetics, and Hospitality Management Department and administrative staff from the college evaluated the foods, asked students several food science‐related questions, and completed a survey about their involvement in the project. Students reported that this project reinforced class material (4.7 ± 0.6), and was enjoyable (4.6 ± 0.6) yet challenging (4.4 ± 0.7); numerous skills were developed, including team‐building (4.7 ± 0.5), food preparation (4.6 ± 0.5), and time management (4.4 ± 0.5). Students perceived that the involvement of sensory panelists was beneficial in terms of providing constructive feedback for improvement and motivating them to prepare high‐quality foods. It also prepared them for serving foods to individuals from different cultural backgrounds. However, some students found the experience stressful. For the sensory panelists, the project helped them to venture out of their areas of expertise. In conclusion, a meal project involving sensory panelists can result in positive learning outcomes.