The precision of data obtained in large-scale science assessments: An investigation of bootstrapping and half-sample replication methods

Powerful computational procedures are now available to better determine the accuracy of statistical estimates derived from data that have unknown distributions or do not meet parametric requirements. These techniques are generally called resampling plans and include the recently developed bootstrap. Science educators often face the problem of nonnormal distributions especially when conducting large-scale assessments or evaluating national curriculum projects that require complex sampling plans. Resampling techniques permit the researcher to make inferences without the strong distributional assumptions needed for more traditional parametric approaches. In this study, the bootstrap and a simplified version of a half-sample replication are used to examine the precision of science test scores obtained in a large-scale evaluation of Scope, Sequence, and Coordination, a national science curriculum project. The resampling plans are described in some detail and the results are compared with those obtained from parametric methods. © 1998 John Wiley & Sons, Inc. J Res Sci Teach 35: 697–704, 1998.     

Performance of Bootstrapping Approaches to Model Test Statistics and Parameter Standard Error Estimation in Structural Equation Modeling

Though the common default maximum likelihood estimator used in structural equation modeling is predicated on the assumption of multivariate normality, applied researchers often find themselves with data clearly violating this assumption and without sufficient sample size to utilize distribution-free estimation methods. Fortunately, promising alternatives are being integrated into popular software packages. Bootstrap resampling, which is offered in AMOS (Arbuckle, 1997), is one potential solution for estimating model test statistic p values and parameter standard errors under nonnormal data conditions. This study is an evaluation of the bootstrap method under varied conditions of nonnormality, sample size, model specification, and number of bootstrap samples drawn from the resampling space. Accuracy of the test statistic p values is evaluated in terms of model rejection rates, whereas accuracy of bootstrap standard error estimates takes the form of bias and variability of the standard error estimates themselves.     

M/G/c排队系统中顾客队长排队长的置信区间的Bootstrap估计

研究了M/G/c排队系统中顾客队长、排队长的置信区间的求解方法.利用Bootstrap原理,给出了正态区间法、枢轴量法和百分位区间法等三种计算方法.实际应用表明,三种方法都得到了较好的结果.以置信区间的长度为评价标准,正态区间法和百分位区间法较优.     

Are Robust Standard Errors the Best Approach for Interval Estimation With Nonnormal Data in Structural Equation Modeling?

When the multivariate normality assumption is violated in structural equation modeling, a leading remedy involves estimation via normal theory maximum likelihood with robust corrections to standard errors. We propose that this approach might not be best for forming confidence intervals for quantities with sampling distributions that are slow to approach normality, or for functions of model parameters. We implement and study a robust analog to likelihood-based confidence intervals based on inverting the robust chi-square difference test of Satorra (2000). We compare robust standard errors and the robust likelihood-based approach versus resampling methods in confirmatory factor analysis (Studies 1 & 2) and mediation analysis models (Study 3) for both single parameters and functions of model parameters, and under a variety of nonnormal data generation conditions. The percentile bootstrap emerged as the method with the best calibrated coverage rates and should be preferred if resampling is possible, followed by the robust likelihood-based approach.     

The policy implications of the global flow of tertiary students: a social network analysis

In 2016, approximately 5 million students, about 2% of global tertiary enrolments, studied abroad. As globalisation of education advances, tertiary student mobility is an important channel through which highly skilled immigrants arrive and work in different nation states. Informed by the multidisciplinary internationalisation frameworks, this study applies social network analysis techniques to the UNESCO data, to explore and compare the international tertiary student mobility networks in 1999 and in 2012. Based on the network visualisation and statistical analyses, this research emphasises that an individual country’s economic and political power and geographic location are increasingly significant in determining its position in the network. Compared to the 1999 networks, the developing world has played a more important role in the networks by becoming the new destination for study and sending out more students. Yet it is still the economically leading nations that serve as the critical bridges connecting the less developed countries/regions to the world. Underneath the seemingly balanced development, the developing nations might be in a more disadvantaged and peripheral position in 2012. This study is concluded with a discussion of the brain drain issue and how various nation states confront it in light of the global flow of tertiary students.     

Why educate “little scientists?” Examining the potential of practice‐based scientific literacy

In recent years, a number of curriculum reform projects have championed the notion of having students do science in ways that move beyond hands‐on work with authentic materials and methods, or developing a conceptual grasp of current theories. These reformers have argued that students should come to an understanding of science through doing the discipline and taking a high degree of agency over investigations from start to finish. This stance has occasionally been mocked by its critics as an attempt to create “little scientists”—a mission, it is implied, that is either romantic or without purpose. Here, we make the strong case for a practice‐based scientific literacy, arguing through three related empirical studies that taking the notion of “little scientists” seriously might be more productive in achieving current standards for scientific literacy than continuing to refine ideas and techniques based on the coverage of conceptual content. Study 1 is a classroom case study that illustrates how project‐based instruction can be carried out when teachers develop guidance and support strategies to bootstrap students' participation in forms of inquiry they are still in the process of mastering. Study 2 shows how sustained on‐line work with volunteer scientists appears to influence students' success in formulating credible scientific arguments in written project reports following an authentic genre. Study 3, using data from three suburban high school classes, suggests that involving students in the formulation of research questions and data analysis strategies results in better spontaneous use of empirical data collection and analysis strategies on a transfer task. The study also suggests that failing to involve students in the formulation of research can result in a loss of agency. The implications of these findings for future research and practice are discussed. © 2004 Wiley Periodicals, Inc. J Res Sci Teach 41: 234–266, 2004     

Development of a user-friendly instrument for identifying the learning strategy preferences of adults

Learning strategies offer a means of addressing individual differences. A programmatic line of inquiry at two universities in the United States has led to an extensive body of research related to learning strategies and to the development of two instruments. This study reports on the development of one of these instruments that can be used to quickly identify learning strategy preferences. Although this instrument, Assessing The Learning Strategies of AdultS (ATLAS), appears very simple and is easy to use, a series of multivariate statistical procedures were used to develop and validate it. These procedures are reported here in detail so that practitioners can be aware of its strong research base and use it with confidence. ATLAS can be used either for self-assessment or by an instructor in order to quickly identify how an individual learner approaches a learning task.     

广义自助法置信域及应用

首先模拟研究广义自助法置信区域的性能,选择不同权重、不同分布的数据,利用Matlab软件模拟其覆盖真值的比率,表明它是非渐近最优的,可适用于不同大小的样本.然后把广义自助法置信区域方法应用于检验地方政府GDP造假与虚假年份,获得比较可信的结果.     

Working with students as co-researchers in schools: a matter of inclusion

The research reported in this paper focuses on the process of involving students as co-researchers in schools. The study involved a process of collaboration with two secondary schools in an economically deprived city in the north East of England. Use was made of a framework for developing inclusion that emerged through earlier work. The framework has students' voices at its centre. Through an analysis of the research process, factors that are important when involving students as co-researchers in schools are discussed. The understandings gained through the study have implications for future research. It is argued that such approaches, which place students at the heart of the research process, can be viewed as a matter of inclusion.     

A Bootstrap Experiment on the Statistical Properties of Students' Ratings of Teaching Effectiveness

This paper examines the effects of two background variables in students' ratings of teaching effectiveness (SETs): class size and students' motivation (as surrogated by students' likelihood to respond randomly). Resampling simulation methodology has been employed to test the sensitivity of the SET scale for three hypothetical instructors (excellent, average, and poor). In an ideal scenario without confounding factors, SET statistics unmistakably distinguish the instructors. However, at different class sizes and levels of random responses, SET class averages are significantly biased. Results suggest that evaluations based on SET statistics should look at more than class averages. Resampling methodology (bootstrap simulation) is useful for SET research for scale sensitivity study, research results validation, and actual SET score analyses. Examples will be given on how bootstrap simulation can be applied to real-life SET data comparison.     

汉语作为第二语言学习语法偏误研究综述

本文在对20多年来汉语作为第二语言教学的语法偏误研究成果作出统计和梳理的基础上,对这一领域的研究状况做了一个综述.指出了这一研究领域呈现的特点以及存在的问题,并对今后的发展提出了自己的建议和看法.     

Equating Error and Statistical Bias in Small Sample Linear Equating

A resampling study was conducted to compare the statistical bias and standard errors of nonequivalent-groups linear test equating in small samples of examinees. Sample sizes of 15, 25, 50, and 100 were examined. One thousand samples of each size were drawn with replacement from each of 5 archival data files from teacher subject area tests. For each test, data files from 2 parallel forms were used. Results suggest trivial levels of equating bias even with small samples, but substantial increases in standard errors as sample size decreases. Results were interpreted in terms of applications to testing situations in which small numbers of examinees are available.     

Resampling and Distribution of the Product Methods for Testing Indirect Effects in Complex Models

Recent advances in testing mediation have found that certain resampling methods and tests based on the mathematical distribution of 2 normal random variables substantially outperform the traditional z test. However, these studies have primarily focused only on models with a single mediator and 2 component paths. To address this limitation, a simulation was conducted to evaluate these alternative methods in a more complex path model with multiple mediators and indirect paths with 2 and 3 paths. Methods for testing contrasts of 2 effects were evaluated also. The simulation included 1 exogenous independent variable, 3 mediators and 2 outcomes and varied sample size, number of paths in the mediated effects, test used to evaluate effects, effect sizes for each path, and the value of the contrast. Confidence intervals were used to evaluate the power and Type I error rate of each method, and were examined for coverage and bias. The bias-corrected bootstrap had the least biased confidence intervals, greatest power to detect nonzero effects and contrasts, and the most accurate overall Type I error. All tests had less power to detect 3-path effects and more inaccurate Type I error compared to 2-path effects. Confidence intervals were biased for mediated effects, as found in previous studies. Results for contrasts did not vary greatly by test, although resampling approaches had somewhat greater power and might be preferable because of ease of use and flexibility.     

Relationships between Young Sisters as Revealed in their Overt Responses

The jackknife and bootstrap methods are becoming increasingly popular in research. Although the two approaches have similar goals and use similar strategies, information is lacking with regard to the comparability of their results. In the present study, this issue was systematically investigated for a case of canonical correlation analysis. Bootstrap, jackknife, and Monte Carlo experiments were carried out for 4 sample sizes (n = 200, 100, 50, 20). The jackknife analyses were also varied as regards the number of jackknife observations deleted in each analysis. Some meaningful discrepancies were observed between the bootstrap and jackknife results, especially under small sample-size conditions. Based on the comparisons made with Monte Carlo estimates, the empirical results suggest that the bootstrap technique provides less biased and more consistent results than the jackknife technique does.     

A methodological framework to analyze stakeholder preferences and propose strategic pathways for a sustainable university

Building sustainable universities calls for participative management and collaboration among stakeholders. Combining analytic hierarchy and network processes (AHP/ANP) with statistical analysis, this research proposes a framework that can be used in higher education institutions for integrating stakeholder preferences into strategic decisions. The proposed framework is applied to a private university in Turkey as a case study through a survey of 30 participants, representing key internal stakeholder groups. The present research extends the literature by adding a statistical analysis component involving a diverse sample of stakeholders, while previous applications of AHP/ANP in higher education involve a single or a few decision makers. The survey demonstrates stakeholder priorities with respect to sustainability performance indicators and a set of investment projects as well as how they change under low, medium and high financial constraints. The study finds that, while stakeholders have varying opinions regarding sustainable development, generally their highest priority is teaching, followed closely by research. Further, although stakeholders assign a high priority to environmental initiatives when the concern is service and social responsibility, they do not consider such investments profitable. Lastly, it appears that “high visibility” projects gain priority as the level of financial constraint increases.     

NEETs in Spain: an analysis in a context of economic crisis

This article presents the research results from a study that was conducted on Spanish youths aged 25–29 years who are neither in employment nor education and training (NEET). Their characteristics were analysed as well as the differential aspects in relation to people who are in employment, education or training. A typology of NEETs was also established based on their availability for and attitude towards employment, and the similarities and differences between the diverse subgroups were analysed. A mixed methodology was used: a quantitative perspective, which enabled a statistical and regression analysis using data that were obtained through questionnaires, and a qualitative perspective based on a biographical interview, which made it possible to focus on perceptions and processes. The results enable us to identify the common characteristics of the NEET population as well as the perceptions and determining factors of the three typologies.     

辽宁鞍山地区村落名研究

村落名是地名的重要组成部分,对村落名的描写与研究有其重要的意义。本文从村落名的命名方式、村落名的文化内涵以及村落名的规范问题三个方面对鞍山地区的1 066个村落名进行了研究。鞍山村落名多采用了"通名+专名"的命名方式,文章用统计和分类的方法从通名和专名两个方面对其命名方式进行了探究。鞍山村落名作为文化词汇的一部分,反映了鞍山地区的地理文化、历史文化和社会文化。鞍山村落名存在异体字和重名现象,本文从语言规范的角度对其进行了初步探讨。     

The Role of Institutional Research in a High Profile Study of Undergraduate Research

Armed with a strong toolkit of knowledge and skills, institutional research (IR) professionals often serve as collaborators with campus colleagues who may need assistance with survey design, statistical analysis, program review, and assessment of individual programs or the institution. This paper discusses the role that an IR professional played in a comprehensive and high profile study of undergraduate research. The project was grounded in educational theory and principles of academic assessment. It consumed many long hours, but it also yielded gains in IR visibility, heightened professional relationships with campus colleagues, a second sizable grant, and multiple publications. Implications for the IR practitioner, contributions to the scholarship of assessment, and institutional synergy are discussed.     

Project 12-ways: Measuring outcome of a large in-home service for treatment and prevention of child abuse and neglect

Described here is Project 12-Ways, a large service project employing an ecobehavioral approach to the treatment and prevention of child abuse and neglect. By ecobehavioral it is meant that multifaceted in-home services are provided to clients, and that in-home data are collected on as many variables related to these services as possible. Four levels of research, data collection, and assessment are used to evaluate these services: Included here is a discussion of these levels of assessment. They are: data collected for clinical purposes; data from single-case experiments; research through the use of single-subject design logic applied to several subjects or groups of subjects, or by group statistical research designs; and program evaluation. The particular focus here is the program evaluation data which compare incidents of abuse and neglect during and after treatment between 50 families served by Project 12-Ways and 47 comparison protective service families. The data showed significantly fewer combined abuse and neglect incidents among the families served by Project 12-Ways. Suggestions for additional program evaluation data are provided along with a discussion of the limitations of the current analysis.     

心理资本对大学生创业倾向的影响研究——基于创业态度的中介变量

使用问卷调查法收集了714份有效问卷,通过描述性统计分析、差异性统计分析、相关分析、回归分析和中介效应分析等数理统计方法,探究了心理资本对大学生创业倾向的影响。研究结果显示:心理资本正向预测创业倾向;心理资本正向预测创业态度;心理资本通过创业态度中介变量对创业倾向起间接影响作用。进而,从心理资本积极心理学的视角为提高大学生的创业倾向提出建议,提升创业就业率。