Working with Low Survey Response Rates: The Efficacy of Weighting Adjustments

National data show a continuing decline in the willingness of people to respond to surveys. This trend is troubling given the central role that survey research plays in collecting data for institutional research purposes. This paper examines the effectiveness of a weighting procedure described by Astin and Molm for adjusting survey results to correct for nonresponse bias. Using data from a Cooperative Institutional Research Program (CIRP) follow-up survey, the results indicate that the weighting procedure is highly effective at reducing nonresponse bias in univariate distributions. The effectiveness of the weighting procedure in adjusting correlation and regression analyses is less clear. This may be due in part to the observation that even when individual variables are noticeably biased, their relationships with each other tend not to be.     

Should We Be Concerned About Nonresponse Bias in College Student Surveys? Evidence of Bias from a Validation Study

This study uses college student survey data and corresponding administrative data on campus recreation facility usage, academic performance, physical education class attendance, and co-curricular participation to examine nonresponse bias in college student surveys. Within the context of the Groves (Public Opin Q 70:646–675, 2006) Alternative Cause Model, we found compelling evidence of the presence of nonresponse error observed as student characteristics related to the survey topic that also explain their response propensity. An individual’s survey response propensity has a statistically significant relationship with their actual behavior for 2 of 3 survey topics. In 11 of the 13 survey questions used to measure the survey topic behaviors, we found statistically significant differences between the respondent and nonrespondent behavioral measures. These findings hold important implications for survey researchers and those using student surveys for high-stakes accountability measures because survey summary statistics may not be generalizable to the target population.

     

Comparing Methods for Addressing Missingness in Longitudinal Modeling of Panel Data

Respondent attrition is a common problem in national longitudinal panel surveys. To make full use of the data, weights are provided to account for attrition. Weight adjustments are based on sampling design information and data from the base year; information from subsequent waves is typically not utilized. Alternative methods to address bias from nonresponse are full information maximum likelihood (FIML) or multiple imputation (MI). The effects on bias of growth parameter estimates from using these methods are compared via a simulation study. The results indicate that caution needs to be taken when utilizing panel weights when there is missing data, and to consider methods like FIML and MI, which are not as susceptible to the omission of important auxiliary variables.     

Survey research in higher education

Although textbooks on educational research give only scant attention to survey research methodology, its extensive use in education provides a strong rationale for improving the preparation of educational researchers in effectively applying survey methods. This study reviews methods for dealing with nonresponse bias, the primary problem presented by survey methods. It also provides an updated review of the literature of studies investigating the effectiveness of incentives to increase survey response rates. We hypothesized that a reanalysis of reviewed studies, using a linear trend test, would resolve the inconsistencies found regarding the effectiveness of incentives to increase response rates. The results of the reanalysis and guidelines for educational researchers in systematically selecting and applying incentives to increase response rates are discussed.     

Dealing With Item Nonresponse in Large‐Scale Cognitive Assessments: The Impact of Missing Data Methods on Estimated Explanatory Relationships

Competence data from low‐stakes educational large‐scale assessment studies allow for evaluating relationships between competencies and other variables. The impact of item‐level nonresponse has not been investigated with regard to statistics that determine the size of these relationships (e.g., correlations, regression coefficients). Classical approaches such as ignoring missing values or treating them as incorrect are currently applied in many large‐scale studies, while recent model‐based approaches that can account for nonignorable nonresponse have been developed. Estimates of item and person parameters have been demonstrated to be biased for classical approaches when missing data are missing not at random (MNAR). In our study, we focus on parameter estimates of the structural model (i.e., the true regression coefficient when regressing competence on an explanatory variable), simulating data according to various missing data mechanisms. We found that model‐based approaches and ignoring missing values performed well in retrieving regression coefficients even when we induced missing data that were MNAR. Treating missing values as incorrect responses can lead to substantial bias. We demonstrate the validity of our approach empirically and discuss the relevance of our results.     

项目无回答的成因与降低其水平的途径

本文基于两个全国概率抽样调查的实证数据,采用序列变量逻辑斯蒂回归和稳健回归的分析方法,探讨了在中国社会科学概率抽样调查中影响项目无回答的主要因素,结果发现受访人的受教育水平、城乡、地域属性和年龄是对无回答水平最稳定、最关键的影响因素。在控制了这些人群特征和题目的难度类型之后,测量指标的信度水平对无回答也具有显著的影响。根据分析结果,本文从实践的角度提出了有关降低项目无回答水平的建议。     

Response bias in follow-up studies of college students

The problems of response bias in longitudinal studies of college students are examined. An extensive follow-up questionnaire was sent to 1,253 college seniors who had participated in a similar survey as freshman four years earlier. Careful measure of student responsiveness in relation to various techniques designed to increase the proportion of responders (e.g., postcard, telephone contact) were kept.The less responsive groups were significantly different from their more responsive counterparts on nearly a dozen variables representing a wide variety of content areas, including academic achievement, self-concept, alcohol consumption, social deviance, and major choice preferences. Controlling for sex and socioeconomic status served to reduce, but not eliminate, these biases. Overall, the results indicate that researchers cannot account for follow-up nonresponse bias by making statistical adjustments according to data available at initial testing. The results are discussed in light of identifying the reasons for nonresponse, and attempting to develop categories of nonresponders who may be motivated to cooperate by different types of follow-up techniques.     

The impact of education on political ideology: Evidence from European compulsory education reforms

Previous research documents a correlation between education and political ideology, usually indicating a positive relationship between education and left-wing political views. In this paper, I examine to what extent this association is causal. I merge political ideology data from 25 waves of Eurobarometer surveys with information on 18 educational reforms in 11 European countries. I then instrument for educational attainment with a regression discontinuity design that estimates the increase in education due to compulsory educational reforms. Notably, it appears that omitted variables bias is important here. I find a significant causal effect of education moving individuals to the right when properly addressing the endogeneity whereas there is a significant association between education and left-wing political ideology when treating education as exogenous. I find that on average, among the individuals compelled into additional education from these specific reforms, an additional year of education moves individuals to the right of the political continuum by about 5–6%. However, I also find no evidence of a causal effect on political ideology for a subgroup of countries.     

Learners Emphasize Their Intrinsic Load if Asked About It First: Communicative Aspects of Cognitive Load Measurement

Cognitive load measurement is an important aspect of educational research. Current cognitive load surveys differentiate between intrinsic cognitive load (resulting from the complexity of learning materials) and their extraneous cognitive load (which is increased by a demanding design). In two studies, order effects of cognitive load subscales are demonstrated. Asking learners regarding their intrinsic load first increases their responses concerning this type of load, with little effect on extraneous load ratings. This effect can be replicated even when extraneous load is intentionally induced. This finding has important implications for cognitive load research, as the order of surveys appears to bias cognitive load ratings. As most cognitive load research is conducted to find ways of reducing extraneous load, it may be reasonable to carefully consider whether and when intrinsic load items are included in studies. Generally, the results show that study participants seem to emphasize certain demands, similar to a dialogue.     

Student Survey Response Rates across Institutions: Why Do they Vary?

While many studies have examined nonresponse in student surveys, little research investigates why some schools achieve higher student survey response rates than other schools. Using hierarchical linear modeling, we analyze survey data from 321 institutions that participated in the 2003 National Survey of Student Engagement to understand how characteristics of colleges and universities relate to student survey response rates. We find that the makeup of the student body, as well as institutional characteristics such public/private status and urban location affects response rates, and that the number of computers per undergraduate has a strong positive effect for web survey response rates.     

Assessing Response Rates and Nonresponse Bias in Web and Paper Surveys

Using data collected as part of the second pilot administration of Your First College Year (YFCY), a national survey of first-year college students, this study was designed to examine both response rates and nonresponse bias across four survey administration groups: paper-only, paper with web option, web-only with response incentive, and web-only without response incentive. Findings indicate that response rates vary by mode of administration. Moreover, predictors of response differed by administration group. Results are discussed in light of the recent surge of interest in online survey research.     

Exploring ethical issues associated with using online surveys in educational research

Online surveys are increasingly used in educational research, yet little attention has focused on ethical issues associated with their use in educational settings. Here, we draw on the broader literature to discuss 5 key ethical issues in the context of educational survey research: dual teacher/researcher roles; informed consent; use of incentives; privacy, anonymity, and confidentiality; and data quality. We illustrate methods of addressing these issues with our experiences conducing online surveys in educational contexts. Moving beyond the procedural ethics approach commonly adopted in quantitative educational research, we recommend adopting a situated/process ethics approach to identify and respond to ethical issues that may arise during the conduct, analysis, and reporting of online survey research. The benefits of online surveying in comparison to traditional survey methods are highlighted, including the potential for online surveys to provide ethically defensible methods of conducting research that would not be feasible in offline education research settings.     

中国教育政策的历史考察——兼论教育主权的行使

教育主权的行使是通过教育政策的执行来体现的。通过梳理历朝历代的教育政策概况来探讨当时教育主权的行使,期望从中总结出教育主权行使正确与否的规律,为探讨当前教育主权行使的合理性提供参考依据。     

教育心理学内容体系发展演变及构建探析

随着教育心理学内容的扩充、对其学科性质认识的变化、所持的理论依据不同，教育心理学的内容体系也经历了一个嬗变过程。未来教育心理学内容体系的构建要考虑到教育对象的全人化，教育机构的扩大化和教育内容的新变化，同时要注重教育目标的要求、使用对象的特点且要遵循稳定性和灵活性相结合的原则。     

The Relative Importance of Items of Chemical Information for General Education

This article reports the results of mail surveys of educational researchers who either did or did not employ specially treated control groups (STCGs). Whereas some consistency in actual practice was found, the surveys revealed confusion among researchers over the definition of and relationship between both sources of artifact and placebo, Hawthorne, and attention-placebo controls. The survey data indicated the need for research assessing the effects of various artifact control practices and for a conceptual model of artifact and their controls in educational research.     

数字教育电视节目的传播渠道应用情况调查研究

通过文献研究发现当前对数字教育电视节目的传播渠道及其在不同渠道的应用和创新推广问题研究比较缺乏,本研究就此问题对教育电视节目的专业制作人员开展调研。     

Choices and changes: Eccles’ Expectancy-Value model and upper-secondary school students’ longitudinal reflections about their choice of a STEM education

During the past 30 years, Eccles’ comprehensive social-psychological Expectancy-Value Model of Motivated Behavioural Choices (EV-MBC model) has been proven suitable for studying educational choices related to Science, Technology, Engineering and/or Mathematics (STEM). The reflections of 15 students in their last year in upper-secondary school concerning their choice of tertiary education were examined using quantitative EV-MBC surveys and repeated qualitative interviews. This article presents the analyses of three cases in detail. The analytical focus was whether the factors indicated in the EV-MBC model could be used to detect significant changes in the students’ educational choice processes. An important finding was that the quantitative EV-MBC surveys and the qualitative interviews gave quite different results concerning the students’ considerations about the choice of tertiary education, and that significant changes in the students’ reflections were not captured by the factors of the EV-MBC model. This questions the validity of the EV-MBC surveys. Moreover, the quantitative factors from the EV-MBC model did not sufficiently explain students’ dynamical educational choice processes where students in parallel considered several different potential educational trajectories. We therefore call for further studies of the EV-MBC model's use in describing longitudinal choice processes and especially in investigating significant changes.     

高师院校教育学科课程研究的迷惘与出路

我国高等师范院校的教育学科课程素来不尽人意。虽然其课程实践历史久远,声势浩大,但学科专业与教育专业的"双专业"性质如何体现,遍地开花的群众性课程研究何以改善陈旧的课程结构和传统的课程面貌?课程研究对此身陷迷惘,举步维艰。因此,既要立足教师职业专业化的背景,高扬学科专业与教育专业双专业和谐发展之旗,又要大兴求真务实之风,强化研究规划和过程管理,还要分化教育学科课程,优化课程结构。     

蒋梦麟早期教育思想初探

以主张个性教育、反对封建教育为核心的蒋梦麟早期教育思想,不仅是其教育思想的重要组成部分,而且对当代中国教育体制的创新与发展也颇具借鉴意义。     

确认并讨论研究方法中可能出现的误差

在任何学术研究中第一手资料和第二手资料的研究都是必要的。所有的资料来源于这些研究,这些研究也给学术的调查提供了主要的资源。因此,每一位研究人员都应该认识到自己调查研究中会出现的一些偏差。这篇文章就是针对调研中可能出现的一些偏差进行了一定的研究和解释,并且讨论了怎样可以使其降低至最少并介绍了一些可能办法。