Factors Influencing Intrajudge Consistency During Standard-Setting

What factors influence judges when they set standards? How do judges, test questions, and the standard-setting process interact? How can we improve intrajudge consistency?     

Empirical Performance of Covariates in Education Observational Studies

This article summarizes results from 12 empirical evaluations of observational methods in education contexts. We look at the performance of three common covariate-types in observational studies where the outcome is a standardized reading or math test. They are: pretest measures, local geographic matching, and rich covariate sets with a strong theory of treatment selection. Overall, the review demonstrates that although the pretest often reduces bias in observational studies, it does not always eliminate it. Its performance depends on the pretest's correlation with treatment selection and the outcome, and whether preintervention trends are present. We also find that although local comparisons are prioritized for matching, its performance depends on whether comparable no-treatment cases are available. Otherwise, local comparisons may produce badly biased results. In cases where researchers have a strong theory of selection and rich covariate sets, observational methods perform well, but additional replication studies are needed. Finally, observational methods that rely on demographic covariates without a theory of selection rarely produce unbiased treatment effects. The article concludes by offering education researchers empirically based guidance on covariate selection in observational studies.     

Validity Arguments for High-stakes Testing: In Search of the Evidence

What do we need to do to establish the validity of high-stakes testing programs? How can we move away from a “confirmationist” bias? What questions haven't we been asking that we need to ask?     

The secretary problem

In many spheres of activity, decisions must often be made under uncertain conditions. One such problem relates to selecting a candidate from a known number when: (a) candidates arrive in a sequence; (b) the selection process has to decide on a candidate then and there; (c) the process terminates if a candidate is selected; (d) the process continues if the candidate is not selected. The question is: What is the strategy that maximises the probability of selecting the best candidate? How does one use the ‘scores’ of each of the candidates seen so far to decide if the present candidate must be selected or if the process be continued, in the quest for the best candidate? This is the problem ofoptimal stopping, an example of which is discussed here.     

Domains and Levels of Pedagogical Innovation in Schools Using ICT: Ten Innovative Schools in Israel

The study reported here is based on the results of two international studies: IEA's Second Information Technology in Education Study (SITES) Module 2, focusing on innovative pedagogical practices at the classroom level, and OECD/CERI Case Studies of ICT and Organizational Innovation, focusing on ICT related innovations at the school system level. In the paper we analyze the data collected in ten innovative schools in Israel which have incorporated ICT in unique ways and have succeeded in devising innovative classroom pedagogies and school system changes. The major research questions addressed are: (a) What levels of innovation were observed in the participant schools, and in which domains? (b) How does the level of innovation in the various domains vary among and within schools? (c) What correlation patterns among levels of innovation in the various domains can be identified? The results of ten comprehensive case studies of exemplary schools in Israel were analyzed by mean of the innovation analysis schema developed by us for characterizing ICT-based educational innovations.     

A Comparison of Bias Reduction Methods: Clustering versus Propensity Score Subclassification and Weighting

Propensity score (PS) adjustments have become popular methods used to improve estimates of treatment effects in quasi-experiments. Although researchers continue to develop PS methods, other procedures can also be effective in reducing selection bias. One of these uses clustering to create balanced groups. However, the success of this new method depends on its efficacy compared to that of the existing methods. Therefore, this comparative study used experimental and nonexperimental data to examine bias reduction, case retention, and covariate balance in the clustering method, PS subclassification, and PS weighting. In general, results suggest that the cluster-based methods reduced at least as much bias as the PS methods. Under certain conditions, the PS methods reduced more bias than the cluster-based method, and under other conditions the cluster-based methods were more advantageous. Although all methods were equally effective in retaining cases and balancing covariates, other data-specific conditions may likely favor the use of a cluster-based approach.     

Self-Regulation of Learning within Computer-based Learning Environments: A Critical Analysis

Computer-based learning environments (CBLEs) present important opportunities for fostering learning; however, studies have shown that students have difficulty when learning with these environments. Research has identified that students’ self-regulatory learning (SRL) processes may mediate the hypothesized positive relations between CBLEs and academic performance. In this review, we identified 33 empirical studies of SRL and CBLEs. We address three research questions: (1) How do learner and task characteristics relate to students’ SRL with CBLEs? (2) Can various learning supports or conditions enhance the quality of students’ SRL as they learn with CBLEs? (3) What conceptual, theoretical, and methodological issues exist for this growing area of research? We found evidence that specific SRL processes are more often associated with academic success than others and that SRL skills can be supported. We also identified a number of issues that researchers should aim to address in future investigations, including a more comprehensive measurement of facets of SRL and the quality of SRL processes, the seeming disconnect between SRL processes and learning outcomes, and the distinction between self- and other-regulation. Electronic supplementary material  The online version of this article (doi:) contains supplementary material, which is available to authorized users.     

A developmental perspective on day care

Issues surrounding early day care become clarified when brought within a developmental perspective. Day care is examined in the context of normal developmental issues. A series of questions emerges. How does development commonly unfold in the context of nondisrupted parental care? How is development organized; what are major sequences of development and, especially, how does one phase build upon and set the stage for others? What is required of social partners to promote optimal development? How does this change with age? In brief, what do infants need from the social environment, and how do they typically get it in the context of the family? One then asks questions concerning day care. For example, what kinds of arrangements are and are not generally compatible with infant needs and under what circumstances? A developmental perspective also provides a framework for organizing findings from day care research and even suggests strategies for intervention research. While not fully explicit, Belsky's review (in this issue) provides a valuable service by introducing developmental concepts to the day care discussion. In the present paper this viewpoint is elaborated.     

Financial Aid and College Persistence: Do Student Loans Help or Hurt?

Using data from two freshmen cohorts at a public research university (N = 3730), this study examines the relationship between loan aid and second-year enrollment persistence. Applying a counterfactual analytical framework that relies on propensity score (PS) weighting and matching to address selection bias associated with treatment status, the study estimates that loan aid exerts a significant negative effect on persistence for students from low-income background (i.e., Pell eligible), and those taking up high amounts of loans in order to meet total cost of attendance, including students who exhausted the available amount of subsidized loan aid. However, no significant incremental effect associated with unsubsidized loan aid, net of subsidized loan aid, could be detected. The estimated effect of loan aid on persistence controls for first-year academic experience and takes into account 26 factors related to loan selection and persistence in order to match students with loan aid to a counterfactual case in covariate adjusted regression. Comparison with results from non-matched-sample analysis suggests selection bias may mask the negative effect of loans detected with matched-sample estimation. Validity of covariates determining the loan selection process and criteria for acceptable balance in the matched data are discussed, and implications for future research are addressed.     

Portfolio Assessment: A Theoretical Esthnate of Score Reliability

What components should be included in a portfolio of student work? How can the contents of portfolios be scored? How is reliability affected as the number of entries in the portfolio increases? What will it cost to obtain reliable scores?     

Sociologics: An analytical tool for examining socioscientific discourse

Latour (1987) introduced the framework of sociologics to study the construction, accumulation, and mobilisation of knowledge in the face of controversy by means of unpredictable and heterogeneous networks. The framework centres around five questions: How are causes and effects attributed? What points (ideas) are linked to which other? What are the size and strength of these links? Who are the most legitimate spokespersons? and How are the elements in a network modified during the controversy? Latour calls the method of deriving answers to these five questions “sociologics”. Recognising the usual asymmetry of knowledge production, sociologics is concerned with how some knowledge is rendered more credible and powerful than others. The production of knowledge is considered contentious because knowledge is socially constructed in a world where discourse, politics, knowledge, and power are inextricably related. I argue that the framework of sociologics extends commonly used analytical frameworks in socioscientific research in education as, unlike many previous forms of analysis, sociologics foregrounds the social construction of knowledge (as evidenced in discourse) and highlights the contentious, complex, unpredictable, and dynamic nature of knowledge production prevalent in these issues.     

An Enquiry Into the Role and Importance of Ethics in Scientific Research

In this essay the importance of spirituality (or ethics) in the life of a research scientist is explored. The following four questions are considered: a) Why should the problem be studied? What are the benefits? and For whom? b) How should we approach this problem? c) What if the results of this investigation contradict other theories? What should we do in case of disagreements? and, d) What are the consequences of this study? Who or what is going to be harmed? What are the effects on the environment?

     

Recruiting the Next Generation of Measurement Professionals1

How are we going to find the next generation of measurement professionals? What are the issues pertinent to recruitment and training? If there were any period one would desire to be born in, is it not the age of revolution; when the old and the new stand side by side and admit of being compared; when the energies of all … are searched by fear and by hope; when the historic glories of the old can be compensated by the rich possibilities of the new era? This time, like all times, is a very good one, if we but know what to do with it. (Ralph Waldo Emerson, Phi Beta Kappa Address, 1837.)     

试述数学教学在相关专业实践中的重要意义

如何发挥数学教学在相关专业实践中的重要作用?与之相适应的数学教师应具有怎样的素质?开辟"数学实践"课堂的意义何在?如何引导学生参加数学实践?本文就当前教学改革现状和社会热点回答了上述问题,并通过对以往课堂教学实践的反思,采用实例研究法,设计出具有实践意义的教学引导模型并采用合适的教法,使专业课教学更有活力.     

Covariates and Mixture Modeling: Results of a Simulation Study Exploring the Impact of Misspecified Effects on Class Enumeration

For some time, there have been differing recommendations about how and when to include covariates in the mixture model building process. Some have advocated the inclusion of covariates after enumeration, whereas others recommend including them early on in the modeling process. These conflicting recommendations have led to inconsistent practices and unease in trusting modeling results. In an attempt to resolve this discord, we conducted a Monte Carlo simulation to examine the impact of covariate exclusion and misspecification of covariate effects on the enumeration process. We considered population and analysis models with both direct and indirect paths from the covariates to the latent class indicators. As expected, misspecified covariate effects most commonly led to the overextraction of classes. Findings suggest that the number of classes could be reliably determined using the unconditional latent class model, thus our recommendation is that class enumeration be done prior to the inclusion of covariates.     

Entry points when undergraduate research mentors reflect on their role: a qualitative case study

Abstract

Graduate students and postdoctoral researchers are increasingly taking on mentoring roles in undergraduate research (UR). There is, however, a paucity of research focusing on how they conceptualize their mentoring role. In this qualitative interview study, we identified three entry points that mentors reflect on to define their role: (1) What are the goals of UR? (2) What do the students expect from me? and (3) How should I use my expert knowledge? We discuss how academic developers can use these entry points together with a set of reflective lenses to stimulate critical reflection on the mentoring role and help the mentors to define their role and help the mentors to define their role.     

The Editorial Policy as a Mirror of Petrine Reforms: Textbooks and Their Translators in Early 18th Century Russia

Peter I’s editorial policy appears as a starting point in the birth of secular Russian textbooks. Since the printing production was then organized on a massive scale as a response to the needs of European-like modernization, it should be safely suggested that nearly all books produced during this pioneering period focused teaching objectives. To understand how and why such an impressive cultural blossoming could occur, we must study their history as part of the broad process of the transfer of knowledge that interfered in all the domains of Russian life. As the textbooks were essentially the product of translation, the paper will explore four main points: Who were the translators? What and why did they translate? How did they proceed? What were their main problems and achievements?     

Improving Instructional Practices, Policies, and Student Outcomes for Early Childhood Language and Literacy Through Data-Driven Decision Making

Since the passage of No Child Left Behind, data-driven decision making has become one of the central foci in schools in their attempt to attain and maintain adequate levels of student academic performance. The importance of early childhood education is well established with language and literacy proficiency in the early years being viewed as a leading indicator in children’s educational development. It provides schools with the initial signs of progress towards academic achievement. In this article, a conceptual framework for improving instructional practice and student outcomes in early childhood language and literacy through data-driven decision making was described. Four questions served as the structure around which the conceptual framework was built. These questions include (1) Why do data need to be collected? (2) What kinds of data need to be collected? (3) How are the data collected? (4) How are the data used for making decisions? Responses to these questions serve as tenets for guiding the decision making process.     

Linguistic performance and minority groups

Abstract

Beginning with linguistic performance as currently defined, this article systematically pursues its implications to and through the notion of “successful communication.” The four divisions of the article attempt explorations of the following questions: What does linguistic performance do? What place does it have in a theory of communication? What are the practical implications of such a theory in connection with actual language use? How is all this relevant to problems of what we take to be “linguistically underprivileged” (minority) groups? The article concludes that the concept of linguistic performance, “scientifically” based, is not significant in successful communication, which is an art.