Improving Instructional Practices, Policies, and Student Outcomes for Early Childhood Language and Literacy Through Data-Driven Decision Making

Since the passage of No Child Left Behind, data-driven decision making has become one of the central foci in schools in their attempt to attain and maintain adequate levels of student academic performance. The importance of early childhood education is well established with language and literacy proficiency in the early years being viewed as a leading indicator in children’s educational development. It provides schools with the initial signs of progress towards academic achievement. In this article, a conceptual framework for improving instructional practice and student outcomes in early childhood language and literacy through data-driven decision making was described. Four questions served as the structure around which the conceptual framework was built. These questions include (1) Why do data need to be collected? (2) What kinds of data need to be collected? (3) How are the data collected? (4) How are the data used for making decisions? Responses to these questions serve as tenets for guiding the decision making process.     

Measuring Variability in Proctor Decision Making on High-Stakes Assessments: Improving Test Security in the Digital Age

Remote proctoring, or monitoring test takers through internet-based, video-recording software, has become critical for maintaining test security on high-stakes assessments. The main role of remote proctors is to make judgments about test takers' behaviors and decide whether these behaviors constitute rule violations. Variability in proctor decision making, or the degree to which humans/proctors make different decisions about the same test-taking behaviors, can be problematic for both test takers and test users (e.g., universities). In this paper, we measure variability in proctor decision making over time on a high-stakes English language proficiency test. Our results show that (1) proctors systematically differ in their decision making and (2) these differences are trait-like (i.e., ranging from lenient to strict), but (3) systematic variability in decisions can be reduced. Based on these findings, we recommend that test security providers conduct regular measurements of proctors’ judgments and take actions to reduce variability in proctor decision making.     

Age Differences in Choice Deferrals as Functions of Interattribute Conflict and Decision Domain

The primary purpose of the present study was to examine age differences in choice deferral when young and older adults make high vs. low conflict decisions in two domains (i.e., health and commodity). Sixty young and 60 older adults were presented with four different decision scenarios in which they could either choose an option or use choice deferral (i.e., not choose any of the options for the time being). The high conflict decisions had negative interattribute correlations, while the low conflict decisions had positive interattribute correlations. Older adults were more likely than young adults to choose the deferral option across domains. Participants were more likely to choose the deferral option when making high conflict decisions than when making low conflict decisions. For older adults, an increase in postdecision positive affect was greater for those who deferred choice than for those who did not defer choice in the commodity domain. Implications were discussed in terms of real-life decision making of older adults.     

IDENTIFYING STUDENTS WITH EMOTIONAL DISTURBANCE: SCHOOL PSYCHOLOGISTS’ PRACTICES AND PERCEPTIONS

From its inception as a disability category in the Education for All Handicapped Children Act, serving students under the special education category Emotional Disturbance (ED) has been a challenging task for school psychologists. In particular, the vague and ambiguous federal definition has created an environment in which inconsistent assessment practices and decision making are almost inevitable. In this study, we examined school psychologists’ (N = 214) assessment practices when determining initial eligibility for ED, as well as their perceptions regarding the language contained within the federal ED definition. Findings indicated that although school psychologists recognize the need for a multimethod, multisource assessment model when ED is a classification consideration, in many instances, their actual assessments are missing commonly recommended sources of data for making eligibility decisions, including classroom observations; parent, teacher, and student interviews; and behavior rating scales. The results from the study provide a useful overview of school psychologists’ assessment practices and are couched within the questions and confusion that have surrounded the federal ED definition since its codification into law.     

Bias among Professionals Who Erroneously Declare Students Eligible for Special Services

A computer simulation program was developed for the purpose of studying the processes diagnostic personnel in the schools engage in when making psychoeducational decisions about students. Initially, the program collected demographic data on 159 participants and assessed their knowledge base in assessment. Bogus referral information for one of sixteen “cases” was then provided and subjects were instructed that they were to make eligibility and classification decisions for the child. They were told that scores and other information (e.g., observations) were available to them on a variety of tests from among seven domains. Although the referral data indicated the student might be evidencing academic or behavior problems, all assessment data indicated a normal or average performance. Fiftytwo percent of the subjects (i.e., 83) declared the “average” child eligible for special education services; their classification decisions were further analyzed. Decisions about the 16 different children were found to be more a function of referral information than child performance data. The outcomes are discussed with regard to implications for the practice of psychoeducational assessment.     

English language learners with reading-related LD: linking data from multiple sources to make eligibility determinations

Results are reported for an exploratory study of eligibility decisions made for 21 Spanish-speaking English language learners (ELLs) with learning disabilities (LD) and no secondary disabilities who received special education support in reading. Eligibility determinations by an expert panel resulted in decisions that differed significantly from those of school multidisciplinary teams. The panel agreed that some students appeared to have reading-related LD (n = 5) but also identified students that they believed had disabilities, but not necessarily reading-related LD (n = 6). Another group of students (n = 10) had learning problems that the panel believed could be attributed to factors other than LD or for whom substantive additional data would be required to validate eligibility. Issues associated with referral, assessment, and eligibility determinations for ELLs are discussed, and recommendations for improving practice are offered, with an emphasis on the importance of linking data from multiple sources when deciding whether ELLs qualify for special education.     

Validating English Language Proficiency Assessment Uses for English Learners: Academic Language Proficiency and Content Assessment Performance

States use standards‐based English language proficiency (ELP) assessments to inform relatively high‐stakes decisions for English learner (EL) students. Results from these assessments are one of the primary criteria used to determine EL students’ level of ELP and readiness for reclassification. The results are also used to evaluate the effectiveness of and funding allocation to district or school programs that serve EL students. In an effort to provide empirical validity evidence for such important uses of ELP assessments, this study focused on examining the constructs of ELP assessments as a fundamental validity issue. Particularly, the study examined the types of language proficiency measured in three sample states’ ELP assessments and the relationship between each type of language proficiency and content assessment performance. The results revealed notable variation in the presence of academic and social language in the three ELP assessments. A series of hierarchical linear modeling (HLM) analyses also revealed varied relationships among social language proficiency, academic language proficiency, and content assessment performance. The findings highlight the importance of examining the constructs of ELP assessments for making appropriate interpretations and decisions based on the assessment scores for EL students. Implications for policy and practice are discussed.     

The University of Bath in a National Context

In order to investigate selected aspects of decision‐making in three schools of a South Australian TAFE College, data about modes of decision‐making and extent of involvement in decision‐making and the association of these with biographic variables were obtained from a questionnaire completed by 72 lecturing staff.

The hypotheses tested were:‐‐

1. Teachers will desire modes of decision‐making which permit them to be involved in decision‐making to an extent greater than they currently perceive themselves to be

2. Teachers will desire to be involved in decision‐making to an extent greater than they currently perceive themselves to be.

These hypotheses were supported by the data. Ninety‐four per cent of respondents were found to be in some state of decisional deprivation. Some decisional areas were associated with higher decisional deprivation than others. Whilst different modes of decision‐making were desired for different decisions, the Democratic Centralist mode was most frequently desired for a majority of decisions.

The biographic variable having most association with responses was that of classification, with staff of higher classification being more involved in decision‐making and less deprived than those of lower classification.     

Eligibility and classification decisions in educational settings: Issuing “Passports” in a state of confusion

Eligibility and classification decisions have become an integral part of the American educational system, but the process of making these decisions remains poorly understood. Subjects for this investigation of this decision-making process were 223 educational professionals. Each was given referral and assessment information reflecting average performance for a hypothetical student and asked to make decisions about that student. In their decisions, 51% declared the student eligible for special services; 61% indicated the presence of mental retardation, learning disabilities, and/or emotional disturbance. Several subjects indicated eligibility for services, but did not classify, or indicated ineligibility for services, but classified the student. Learning disabilities was the most popular classification. Student characteristics and the subjects' professional roles and knowledge of assessment/ measurement principles affected decisions made; however, greater training and knowledge did not lead to significantly better decisions. It is apparent that more care must be taken by decision makers in the use and interpretation of assessment information.     

Evaluation practices for college students with LD

To examine current practices in the use of psychoeducational evaluations for service delivery, we surveyed 91 service providers to college students with learning disabilities. The three purposes of the survey were to determine (a) whether service delivery decisions are based on information from psychoeducational evaluations, (b) which sections of the psychoeducational report are most useful in making service delivery decisions, and (c) the respondents' satisfaction with the tests and measurements for service delivery. The findings supported the common belief that data from psychoeducational evaluations serve as the primary basis for both eligibility and specific accommodation determinations. Respondents reported that all sections of the psychoeducational evaluation written report were useful, with the least useful section being test scores and the most useful being the summary of cognitive strengths and weaknesses. However, the section used most often for service delivery decisions was the professional's recommendations.     

Adequacy of selected services to international students in an urban technical college

The purpose of the study was to evaluate the adequacy of selected academic and personal services to international students at New York City Technical College of the City University of New York. A population (137) of international students enrolled for one semester was surveyed via a self-developed questionnaire. Analysis of data was by mean weighted scores, chi squares at .05 level of significance, and percentage variances. The findings indicated that academic needs of international students were not satisfied even to minimum levels, and two areas of personal needs attained minimum satisfactory levels. Significant difference in opinions of respondents pertaining to satisfying academic needs was seen between or among the groups in the variables gender, national origin, and native language. In satisfying personal needs significant difference in opinions of respondents was seen among the groups in the variables proficiency in reading in English, proficiency in speaking in English, and proficiency in reading and speaking in one's native language.     

Conceptualizing parents' sign language use in bilingual early intervention

This research study investigates how parents and interveners conceptualize parents' early sign language use. Three groups of respondents involved in the same bilingual early intervention program were being interviewed: hearing parents (n = 12), hearing teachers (n = 6), and Deaf consultants (n = 6). The search and retrieve program 'The Ethnograph' was used for data analysis. The results demonstrate that neither a framework of linguistic proficiency nor one of communicative competence captures the complexity of the issues involved for parents and interveners in conceptualizing parents' early sign language use. The interpretative frameworks of each group emphasize different aspects of the task facing parents: balancing the quest for Sign Language proficiency with emotional and practical considerations in the family, acknowledging the overriding importance of communication regardless of what one may label the language used, and confirming the visual quality and child appropriateness of the signing. Results are related to the stocks of knowledge on which respondents draw and the implications for Deaf and hearing interveners assessing parents' progress.     

In and out of home care decisions: The influence of confirmation bias in developing decision supportive reasoning

The aims of this study were to identify the themes Social Workers regard as important in supporting decisions to remove children from, or return them to, the care of their parents. To further elicit underlying hypotheses that are discernible in interpretation of evidence. A case study, comprising a two-part vignette with a questionnaire, recorded demographic information, child welfare attitudes and risk assessments, using scales derived from standardised instruments, was completed by 202 Social Workers in Northern Ireland. There were two manipulated variables, mother's attitude to removal and child's attitude to reunification 2 years later. In this paper we use data derived from respondents’ qualitative comments explaining their reasoning for in and out of home care decisions. Some 60.9% of respondent's chose the parental care option at part one, with 94% choosing to have the child remain in foster care at part two. The manipulated variables were found to have no significant statistical effect. However, three underlying hypotheses were found to underpin decisions; (a) child rescue, (b) kinship defence and (c) a hedged position on calculation of risk subject to further assessment. Reasoning strategies utilised by social workers to support their decision making suggest that they tend to selectively interpret information either positively or negatively to support pre-existing underlying hypotheses. This finding is in keeping with the literature on ‘confirmation bias.’ The research further draws attention to the need to incorporate open questions in quantitative studies, to help guard against surface reading of data, which often does not ‘speak for itself.’     

Deafness, language skills, and rehearsal: a model for the development of a memory strategy

We review a series of studies that evaluate the role of developing language skills as a prerequisite to spontaneous rehearsal use in deaf populations. We propose a theoretical model that summarizes the contribution of language skills by highlighting the interrelations among age, language proficiency, and automatized language skills. In an initial test of the model, an index of deaf children's language experience was found to be an almost complete mediator of strategy use. Increasing language proficiency, therefore, was implicated as a critical variable in predicting children's spontaneous rehearsal use, minimizing any direct contribution from age per se. In a more direct assessment of the link between general language proficiency and rehearsal use, pragmatic language skills (as measured by the Language Proficiency Profile: LPP-I) were found to be a nearly complete mediator of rehearsal use, with the remaining contributions of age and language experience being nonsignificant. Language proficiency, therefore, was identified as a strong and necessary prerequisite for rehearsal to be used spontaneously as a memory strategy. The additional hypothesis that the automatization of these general language skills, measured by a rapid automatized naming task (RAN), contributes to spontaneous rehearsal was also evaluated. Automatized language emerged as a partial mediator of the language proficiency → rehearsal use relation but additional language-processing variables were implicated. We discuss these findings as they relate to issues around the nature of language proficiency and the identification of those automatized language skills involved in rehearsal.     

The relationship of child abuse indicators to the assessment of perceived risk and to the court''s decision to separate

The most serious decision taken in child abuse cases is to formally separate the child and the family. There has been concern that the criteria for making these decisions should be made explicit. The consequences of the separation for the child, family, and society is one set of reasons for this concern. Another is that with the increasing number of cases being reported, there is a need for reliable, consistent, and efficient decisions across cases. This paper operationally defines a set of indicators for the separation decision. One hundred fifty-two cases of suspected child abuse from the Royal Children's Hospital were coded in terms of the indicators and the outcome of the case. The outcome was the court's decision to separate the child. The study investigated whether and in what way these indicators were related to the separation outcome. In addition the perceived risk to the child was operationally defined, and the relationship of the indicators and the court's decision to perceived risk was studied. Statistical analysis of the data showed that the indicators most associated with the separation outcome were severity of abuse, parenting, and the family social system. Two important implications for practice are that information on explicit indicators should be recorded for cases, and that they can provide a base for training of new child protection workers.     

Australian Postgraduate Students' Perceptions,Preferences and Mobility

Postgraduate students' choice of university, and related mobility issues, have been of interest to Australian university researchers, supervisors and administrators for some time. However, with the release of the Commonwealth Government's 1999 Green Paper on research and research training, which proposes greater portability of research student funding, a sense of urgency regarding these issues has developed. This paper reports on students' choice of a particular university for postgraduate research and on what information they made those decisions. A survey was conducted in 1997 of 938 applicants for Postgraduate Research Scholarships to Flinders University, Macquarie University and the Universities of Western Australia, Adelaide and Melbourne. The survey sought to identify respondents' choices and possible movement to undertake research awards. Questions dealt with a wide range of issues with a particular focus on sources used to access information on higher degrees, preference(s) for where they wished to study, factors leading to their preferred choice, and influences on the decision to move or not move to another institution. Approximately 50% of respondents were younger than 25 years and only 27% were living with their parents. An Honour's degree was the most common prior qualification for students (73%), but only 61% were enrolling directly from their previous degree. Thirty-six per cent reported that they regularly read their main local paper for information related to postgraduate research opportunities and 39% of respondents did not regularly access any source of media for postgraduate information. Forty-two per cent of students did not explore any opportunities at other universities prior to making their decision about enrolment. Almost all students who were planning to enrol at the same university were remaining in the same department. Commensurate with this, 73% reported that their Honour's supervisor or academic staff in their original university were their major source of information on study destinations. These results provide baseline data, which can be used to approach the issue of postgraduate mobility more strategically.     

Beyond English language proficiency scores: understanding the academic performance of international undergraduate students during the first year of study

ABSTRACT

Past research into the relationship between English proficiency test (EPT) scores and score profiles, such as the IELTS and the TOEFL, has shown that there is not always a clear relationship between those scores and students’ subsequent academic achievement. Information about students’ academic self-concept (ASC) may provide additional information that helps predict future academic success. Research has consistently shown a positive relationship between students’ ASC and subsequent academic achievement and educational attainment in both school and higher education settings. The purpose of the current study was to examine the relationship between the academic performance of international students and their language proficiency and academic self-concept as well as other characteristics related to academic success. The study focused on first year international students in undergraduate business programs at an English-medium university in Canada. The following information was collected about the student participants: grades in degree program courses, annual GPA, and EPT scores (including subscores). In addition, students completed an academic self-concept scale. To obtain additional information about success in first-year business courses, instructors in two required courses were interviewed about the academic and language requirements in their courses and the profile of successful students. Correlations between the students’ course grades, GPA, EPT scores, and ASC score were calculated. The instructor interviews were analyzed using a content analysis procedure. The findings from all data sources were triangulated and show that language ability, ASC, and other factors impact academic success during the first year in a business program. The implications of these findings are discussed.     

Why do faculty members leave a university?

This paper uses the results of a survey of tenured faculty members who resigned from the University of Minnesota to analyze the factors that influenced their decisions to leave. The results indicate that the probability of accepting an outside offer is positively related to the expected salary gain, but that other general factors are also important determinants of faculty decision making in this area. However, even with the survey results, we know very little in any systematic sense about specific factors that influence individual faculty decisions, particularly about differences in these factors for those faculty who leave as opposed to those who remain. To analyze faculty retention more completely, we need additional information about the personal circumstances of faculty members who are and are not retention cases and more extensive information on why the retention cases made their eventual choices about staying at or leaving the institution.     

Psychoeducational decision making as a function of the amount of information reviewed

Psychoeducational assessment is commonplace in America's public school systems. Important decisions for and about school children are made on the basis of assessment results; test scores and information derived from the administration of tests clearly contribute to these decisions. Current “best practice” is guided by the concept of multidisciplinary team decision making; parents, teachers, psychologists, and other professionals review assessment data when participating as team members. In this research, decision makers were allowed to review information from a variety of sources prior to making decisions about a case study child. Individuals who reviewed large amounts of information made decisions similar to those of individuals who reviewed little information. Approximately 49% of the participants made accurate decisions when judged against an a priori criterion. The results are discussed with regard to psychoeducational assessment practice.     

The contribution of higher education to the development of objective and subjective judgment in day‐to‐day decision‐making

In order to make effective decisions based on probability, one must be able to distinguish between the objective‐factual aspects of the decision and the subjective aspects. This study examines the hypothesis that academic studies contribute to the development of the ability to make such distinctions in day‐to‐day decision‐making. Some 940 subjects, including some holding Bachelor's degrees and others with Master's degrees in the Exact Sciences, the Social Sciences, and the Humanities, responded to a questionnaire on decision‐making situations which examined the degree of their awareness of the objective and subjective elements of decisions. It was found that the respondents made the distinction in the situations with which they were presented. A rise in academic level from the Bachelor's degree to the Master's Degree led to increased awareness of the objective aspect of decisions among scholars in the Exact Sciences and the Humanities. Holders of Master's degrees in all of the fields of specialty showed a greater awareness of the subjective aspect than did holders of Bachelor's degrees. The application of these insights in the field of education is discussed.