Self-Reported Learning Gains: A Theory and Test of College Student Survey Response

Recent studies have asserted that self-reported learning gains (SRLG) are valid measures of learning, because gains in specific content areas vary across academic disciplines as theoretically predicted. In contrast, other studies find no relationship between actual and self-reported gains in learning, calling into question the validity of SRLG. I reconcile these two divergent sets of literature by proposing a theory of college student survey response that relies on the belief-sampling model of attitude formation. This theoretical approach demonstrates how students can easily construct answers to SRLG questions that will result in theoretically consistent differences in gains across academic majors, while at the same time lacking the cognitive ability to accurately report their actual learning gains. Four predictions from the theory are tested, using data from the 2006–2009 Wabash National Study. Contrary to previous research, I find little evidence as to the construct and criterion validity of SRLG questions.     

Reported Gains in Student Learning: Do Academic Disciplines Make a Difference?

This study examined differences in students' college experiences and learning outcomes using Biglan's typology of pure and applied disciplines. It was expected that students in applied disciplines would have more positive perceptions of the college environment, be more involved, and report greater gains in learning and intellectual development than students in pure disciplines. An analysis of data from one university identified significant differences in the college experiences and learning outcomes of students in pure and applied disciplines; however, differences in reported learning seemed to be related to disciplinary content, rather than Biglan's pure and applied topology.     

Student Summative Evaluation of Teaching: code of practice

The following code of practice has been developed by the Association of University Staff (AUS) of New Zealand (Inc.) with Dr Cedric Hall, Director, University Teaching Development Centre (UTDC), Victoria University of Wellington. The code deals with the use of student evaluations of teaching for summative decision making (e.g. promotion, tenure, etc.). A draft of the AUS code was distributed to the centres responsible for academic staff development in each university and all committees of AUS for comment. We thank the individuals and groups concerned for their comments and advice. Appropriate changes have been made to the code to take account of the suggestions provided. The AUS code on student summative evaluation of teaching is for the use of AUSNZ members and New Zealand universities. The code is intended to encourage good professional practice in the use of student summative evaluation of teaching and teaching staff. AUS recognises that the contents of the code will need to be reviewed from time to time to take account of changing practices and educational developments within institutions. AUS recognises that student views can be a relevant consideration in overall evaluation of teaching quality, and that they can properly have a place in decision making in relation to tenure and promotion for teaching staff. However, varying practices in ascertaining student views have led (both in New Zealand and internationally) to instances of such decisions being based on invalid and unreliable data from student views. Bad practice in these areas reflects poorly on the institution(s), as well as being unfair to staff; in some instances, it may also be in breach of current legislation. This Code of Practice is based on the belief that any evaluation should provide valid (i.e. relevant and useful) and reliable (i.e. accurate) information on the quality of a person's teaching. AUS believes that teaching quality must be judged in context: the type and purpose of the teaching, the background of the students, the resources available and the voluntary/compulsory nature of the course, should all be considered. Validity and reliability, therefore, relate not only to the measurements that are made of a person's teaching performance, but also to the interpretation of the results.     

Verbal Final Exam in Introductory Biology Yields Gains in Student Content Knowledge and Longitudinal Performance

We studied gains in student learning over eight semesters in which an introductory biology course curriculum was changed to include optional verbal final exams (VFs). Students could opt to demonstrate their mastery of course material via structured oral exams with the professor. In a quantitative assessment of cell biology content knowledge, students who passed the VF outscored their peers on the medical assessment test (MAT), an exam built with 40 Medical College Admissions Test (MCAT) questions (66.4% [n = 160] and 62% [n = 285], respectively; p < 0.001);. The higher-achieving students performed better on MCAT questions in all topic categories tested; the greatest gain occurred on the topic of cellular respiration. Because the VF focused on a conceptually parallel topic, photosynthesis, there may have been authentic knowledge transfer. In longitudinal tracking studies, passing the VF also correlated with higher performance in a range of upper-level science courses, with greatest significance in physiology, biochemistry, and organic chemistry. Participation had a wide range but not equal representation in academic standing, gender, and ethnicity. Yet students nearly unanimously (92%) valued the option. Our findings suggest oral exams at the introductory level may allow instructors to assess and aid students striving to achieve higher-level learning.     

Development of Automated Scoring Algorithms for Complex Performance Assessments: A Comparison of Two Approaches

Performance assessments are typically scored by having experts rate individual performances. The cost associated with using expert raters may represent a serious limitation in many large-scale testing programs. The use of raters may also introduce an additional source of error into the assessment. These limitations have motivated development of automated scoring systems for performance assessments. Preliminary research has shown these systems to have application across a variety of tasks ranging from simple mathematics to architectural problem solving. This study extends research on automated scoring by comparing alternative automated systems for scoring a computer simulation test of physicians'patient management skills; one system uses regression-derived weights for components of the performance, the other uses complex rules to map performances into score levels. The procedures are evaluated by comparing the resulting scores to expert ratings of the same performances.     

Assessment and Evaluation in Social Work Education: Formative and Summative Approaches

Despite a wealth of tacit knowledge in academia regarding effective teaching strategies and a rich theoretical and empirical knowledge base on student learning, social work instructors wishing to identify appropriate ways to measure teaching and learning have little evidence to guide them. This article presents a framework for assessment of student learning and evaluation of instructor teaching that distinguishes between formative methods, which support an ongoing process of improvement, and summative methods, which represent a measure of competence or mastery. While summative methods are often used to meet institutional or programmatic goals, formative methods bridge assessment and evaluation and can result in a more reflective, constructive, and productive experience for both instructors and students.     

Course Format and Student Learning Styles: A Comparison of Political Science Courses

ABSTRACT

This study investigates whether differences in learning styles exist between students in online and face-to-face (FTF) sections of political science courses taught by three instructors. Some studies suggest that student preferences regarding online or FTF formats are influenced by their preferred modes of learning. Independent learners, for example, may prefer online courses since they provide individualistic opportunities to study outside of the traditional classroom. This study uses original survey data to assign students one of six learning styles in order to assess whether independent learners are more common in online courses. Our analysis finds no significant differences in independent learners when comparing the two formats. This finding runs counter to studies that argue that independent learners tend to prefer online courses. In fact, the only learning style where we observe a meaningful difference among online and FTF formats is among dependent learners. Contrary to expectation, students enrolled in online versions demonstrated a greater tendency toward dependent modes of learning. Further survey responses suggest that student lifestyle drove course format selection rather than learning style. These findings have important implications for universities that increasingly turn to online courses to address decreasing enrollments and attempt to remedy the high attrition rates associated with those courses.     

Intercultural Learning,Adaptation, and Personal Growth: A Longitudinal Investigation of International Student Experiences in China

Despite the rapid increase of international students in the Chinese higher education sector, little is known about their experiences in China. This paper reports a longitudinal study investigating experiences of a small group of international students during their undergraduate study in an English-medium medical course at a Chinese university. Data were generated through annual interviews, complemented by two rounds of questionnaire surveys respectively held at the beginning and end of the course. Drawing on self-determination theory, findings reveal the learning process through which the participants, responding to and interacting with new academic and social environments, gradually achieved adaptation and personal growth. Meanwhile, the research presents evidence of the students’ dissatisfaction, arising mainly from three aspects of their university environments: linguistic, pedagogical, and attitudinal. Policy implications are discussed and suggestions for future studies are given.     

科学学习--深层和浅层学习通道的对比

科学学习的本质－有意义学习与机制学习，与学习者建构的“学习通道”有关。学习通道作为学习者学习的方式，会影响学习结果的性质。学习采用深层或浅层通道的学生在五个方面有明显的区别：（1）生成性思考，（2）解释的性质，（3）提出问题，（4）元认知，（5）学习方式。     

Measuring Student Involvement: A Comparison of Classical Test Theory and Item Response Theory in the Construction of Scales from Student Surveys

This study compares the psychometric utility of Classical Test Theory (CTT) and Item Response Theory (IRT) for scale construction with data from higher education student surveys. Using 2008 Your First College Year (YFCY) survey data from the Cooperative Institutional Research Program at the Higher Education Research Institute at UCLA, two scales are built and tested—one measuring social involvement and one measuring academic involvement. Findings indicate that although both CTT and IRT can be used to obtain the same information about the extent to which scale items tap into the latent trait being measured, the two measurement theories provide very different pictures of scale precision. On the whole, IRT provides much richer information about measurement precision as well as a clearer roadmap for scale improvement. The findings support the use of IRT for scale construction and survey development in higher education.     

Using Rasch Measurement Theory to Examine Two Instructional Approaches for Teaching and Learning of French Grammar

ABSTRACT

A new model of reflective practice for student teachers on school placement was implemented into a teacher education program. The model aimed to encourage critical reflection that challenged hegemonic assumptions and power relations. In contrast to this, the analysis of the student teachers’ reflections revealed a desire to fit in and conform. Reflections portrayed student teachers’ allegiance to conventional practices rather than a critique of them. Issues associated with power relations remained uncontested. The authors explore possible reasons for these low levels of critique and examines the normative effect of cultural practices on student teachers’ reflections.     

A Learning Experience in Inquiry-Based Physics with Immersive Virtual Reality: Student Perceptions and an Interaction Effect Between Conceptual Gains and Attitudinal Profiles

Journal of Science Education and Technology - The integration of immersive virtual reality (VR) in authentic science classrooms can result in a totally new learning experience for the students....     

Measuring the Impact of Professional Accreditation on Student Experiences and Learning Outcomes

This research uses a conceptual model to examine the influence of a change in accreditation standards on a representative national sample of 203 engineering programs at 40 institutions. Based on data collected from more than 140 program chairs, 1,200 faculty, 4,300 graduates of 2004, and 5,400 graduates of 1994, the study investigates the differential impact of the change in accreditation standards on programs reviewed in different years during the period of transition. Despite significant variation across most of the measures and groups in 1994 (before the introduction of the new accreditation criteria), the 2004 evidence demonstrates a surprisingly uniform level of student experiences and outcomes. These findings suggest that engineering accreditation is beginning to accomplish its quality assurance goals. This is a revised version of a research paper presented at the meeting of the Association for Institutional Research, May 17, 2006, Chicago, IL.     

The Effects of Discipline on Deep Approaches to Student Learning and College Outcomes

“Deep learning” represents student engagement in approaches to learning that emphasize integration, synthesis, and reflection. Because learning is a shared responsibility between students and faculty, it is important to determine whether faculty members emphasize deep approaches to learning and to assess how much students employ these approaches. This study examines the effect of discipline on student use of and faculty members’ emphasis on deep approaches to learning as well as on the relationships between deep approaches to learning and selected educational outcomes. Using data from over 80,000 seniors and 10,000 faculty members we found that deep approaches to learning were more prevalent in Biglan’s soft, pure, and life fields compared to their counterparts. The differences were largest between soft and hard fields. We also found that seniors who engage more frequently in deep learning behaviors report greater educational gains, higher grades, and greater satisfaction with college, and that the strength of these relationships is relatively consistent across disciplinary categories.     

National Identity in EFL Learning: A Longitudinal Inquiry

This article draws on a longitudinal study of four Chinese students’ English learning experiences during their college years and explores the ways in which EFL learning has influenced their sense of national identity. The study captures the changes they have experienced in constructing identities over a prolonged period in the context of mainland China where strong sense of national identity has been historically promoted and where there is currently unprecedentedly close interaction between local cultures and western cultures. Drawing on interviews and diary studies as the primary data source, this study identifies a three‐stage development in the national identities of the college students that proceeds from initial admiration of English‐speaking cultures, to antagonism towards alien things, before then reaching a stage of conciliation between the national and the global. The findings suggest that China’s deeply‐rooted culture of collectivism and altruism plays a role in shaping English learners’ national identity and that the learners also demonstrate considerable agency in constructing national identities at different stages of English learning.     

Spontaneous Collaborative Learning: A New Dimension in Student Learning Experience?

‐ Many studies have reported positive effects of cooperative learning methods on student achievement. In all such cases, cooperative learning experiences were initiated and structured by teachers, and designed according to special task and reward structures. The present study discusses a different kind of cooperative learning, Spontaneous Collaborative Learning (SCOLL), which is not structured by the teachers, but is student‐initiated. The characteristics of such a learning experience are described, and its effects on the learning strategies adopted by students, the quality of the students’ learning outcomes are examined. Quantitative and qualitative data from 39 Hong Kong tertiary (physiotherapy) students showed that SCOLL, as compared to individual preparation, promoted the engagement of a deep approach incorporating high level cognitive strategies, and in turn learning outcomes reflected in a higher structural complexity of assignments than those approaches and outcomes of students who worked individually.     

Growth Curve Modeling to Studying Change: A Comparison of Approaches Using Longitudinal Dyadic Data With Distinguishable Dyads

Although methodology articles have increasingly emphasized the need to analyze data from two members of a dyad simultaneously, the most popular method in substantive applications is to examine dyad members separately. This might be due to the underappreciation of the extra information simultaneous modeling strategies can provide. Therefore, the goal of this study was to compare multiple growth curve modeling approaches for longitudinal dyadic data (LDD) in both structural equation modeling and multilevel modeling frameworks. Models separately assessing change over time for distinguishable dyad members are compared to simultaneous models fitted to LDD from both dyad members. Furthermore, we compared the simultaneous default versus dependent approaches (whether dyad pairs’ Level 1 [or unique] residuals are allowed to covary and differ in variance). Results indicated that estimates of variance and covariance components led to conflicting results. We recommend the simultaneous dependent approach for inferring differences in change over time within a dyad.