Psychoeducational decision making as a function of the amount of information reviewed

Psychoeducational assessment is commonplace in America's public school systems. Important decisions for and about school children are made on the basis of assessment results; test scores and information derived from the administration of tests clearly contribute to these decisions. Current “best practice” is guided by the concept of multidisciplinary team decision making; parents, teachers, psychologists, and other professionals review assessment data when participating as team members. In this research, decision makers were allowed to review information from a variety of sources prior to making decisions about a case study child. Individuals who reviewed large amounts of information made decisions similar to those of individuals who reviewed little information. Approximately 49% of the participants made accurate decisions when judged against an a priori criterion. The results are discussed with regard to psychoeducational assessment practice.     

Eligibility and classification decisions in educational settings: Issuing “Passports” in a state of confusion

Eligibility and classification decisions have become an integral part of the American educational system, but the process of making these decisions remains poorly understood. Subjects for this investigation of this decision-making process were 223 educational professionals. Each was given referral and assessment information reflecting average performance for a hypothetical student and asked to make decisions about that student. In their decisions, 51% declared the student eligible for special services; 61% indicated the presence of mental retardation, learning disabilities, and/or emotional disturbance. Several subjects indicated eligibility for services, but did not classify, or indicated ineligibility for services, but classified the student. Learning disabilities was the most popular classification. Student characteristics and the subjects' professional roles and knowledge of assessment/ measurement principles affected decisions made; however, greater training and knowledge did not lead to significantly better decisions. It is apparent that more care must be taken by decision makers in the use and interpretation of assessment information.     

浅论高校公共突发事件应急决策的逻辑生成路径

在高校公共突发事件应急决策过程中,由于决策环境的非确定性、约束条件的非常态化和决策过程的非程序性等方面的影响,高校公共突发事件的应急决策复杂而多变,这给决策者带来极大的挑战与风险。为了最大程度地保证应急决策方案的良好效果,提高应急决策的科学性,决策主体需要从正确的逻辑起点、科学的逻辑方法、有效的逻辑策略和综合的逻辑过程等逻辑生成角度构建明确的应急决策目标指向、合理的应急决策评估机制、动态的应急决策调整方案和积极的应急决策情境交互思维,掌握高校公共突发事件的演化规律并对其进行科学认知,形成科学判断,提高应急决策的生态理性。     

Bias among Professionals Who Erroneously Declare Students Eligible for Special Services

A computer simulation program was developed for the purpose of studying the processes diagnostic personnel in the schools engage in when making psychoeducational decisions about students. Initially, the program collected demographic data on 159 participants and assessed their knowledge base in assessment. Bogus referral information for one of sixteen “cases” was then provided and subjects were instructed that they were to make eligibility and classification decisions for the child. They were told that scores and other information (e.g., observations) were available to them on a variety of tests from among seven domains. Although the referral data indicated the student might be evidencing academic or behavior problems, all assessment data indicated a normal or average performance. Fiftytwo percent of the subjects (i.e., 83) declared the “average” child eligible for special education services; their classification decisions were further analyzed. Decisions about the 16 different children were found to be more a function of referral information than child performance data. The outcomes are discussed with regard to implications for the practice of psychoeducational assessment.     

Understanding and Improving Teachers' Classroom Assessment Decision Making: Implications for Theory and Practice

Teacher assessment decision making is described as a process that may be important in developing a measurement theory for classroom assessment. Teacher beliefs and values about what and how to assess often conflict with the pressures of external demands, especially recent high-stakes large-scale testing. With this sense of tension, teachers make assessment decisions that will engage and motivate students and enhance learning. Implications for a measurement theory of classroom assessment, as well as teacher preservice and inservice, are explored. The article suggests that measurement specialists need to adapt technical testing principles to be more relevant to the nature of classroom assessment decision making.     

Accuracy of teachers’ tracking decisions: short- and long-term effects of accountability

Bias in teachers’ judgment formation and decision making has long been acknowledged. More specifically, studies have repeatedly demonstrated discrepancies between teacher ratings of minority and majority students with similar academic profiles. Studies have also demonstrated that increasing accountability reduced bias. Little is known, however, about the effect of accountability and bias on the accuracy of decisions. This study investigated the short- and long-term effect of accountability priming on the accuracy of transition decisions. It considered both the extent to which teacher decision accuracy differed for minority and majority students with similar academic profiles (accuracy bias) and differences in levels of confidence for accurate versus erroneous decisions (metacognitive judgment bias). In a longitudinal experimental design, we presented 38 primary school teachers with 9 student vignettes at 3 points in time (baseline, post priming, and 6-month follow-up), varying students’ ethnic background, and asked them to make a school tracking decision for each student. We measured decision accuracy as well as teachers’ level of confidence for each decision. Accuracy and confidence levels were combined to provide two indices of metacognitive judgment accuracy. Results confirmed the hypothesis that accuracy of decisions would improve as a result of increased level of accountability. More specifically, we found that teachers made more accurate decisions after priming, whereby ethnic background differences disappeared. In addition, teachers’ metacognitions varied, whereby after priming decision accuracy was better matched with teachers’ confidence levels. Although accuracy levels were still higher at follow-up than at pre-test, the ethnic bias recurred. This study shows that increased levels of accountability are associated with not only increased decision accuracy but also reduced metacognitive judgment bias, especially in regard to minority students. It also demonstrates accountability may be an effective way of reducing systematic errors in decision making. Findings are discussed in terms of theory and current changes in educational practice.     

Decision making in determining eligibility of culturally and linguistically diverse learners: Reasons given by assessment personnel

Assessment personnel from the southern borderlands area of Texas participated in analyzing one of four different hypothetical cases. Respondents were expected to defer making eligibility decisions due to language proficiency, environment, culture, or lack of data. When data were lacking, the investigators proposed that respondents would recognize the need for additional information and defer making a decision. Differences in eligibility decisions were found when the respondents analyzed cases in which a significant discrepancy existed and in cases in which language proficiency information was provided. Experience in assessment was significant in the determination that additional information was needed to reach a decision, F(2, 87) = 3.99, p < .05. A majority of the respondents (83%) made an eligibility decision using insufficient data.     

判断评价与课程标准一致性的若干问题

Webb一致性程序是判断评价与课程标准一致性的重工具,它有4个标准:类别一致性、知识深度一致性、知识范围一致性和知识分布平衡性,分别对应着多级可接受水平。一致性研究的挑战在于判断"什么样的一致性才是最好的",这一判断过程虽然可以基于众多实证研究,但也包含某种程度的主观性。     

A new look at resits: are they simply a second chance?

The article challenges common practice in relation to resits of ignoring the main assessment results and making high‐stakes decisions on the basis of resit results alone. Implications for institutions, students and educators are discussed. The proposal is that not all resits should be treated equally and that, in some circumstances, more robust decisions can be made by combining resit results with the results from the main assessment.     

Issues Related to Judging the Alignment of Curriculum Standards and Assessments

Abstract

A process for judging the alignment between curriculum standards and assessments developed by the author is presented. This process produces information on the relationship of standards and assessments on four alignment criteria: Categorical Concurrence, Depth of Knowledge Consistency, Range of Knowledge Correspondence, and Balance of Representation. Five issues are identified—but not resolved—that have arisen from conducting alignment studies. All of these issues relate to making a decision about what alignment is good enough. Pragmatic decisions have been made to specify acceptable levels for each of the alignment criteria. The assumptions are described. The issues discussed arise from a change in the underlying assumptions and from considering variations in the purpose for an assessment. The existence of such issues reinforces that alignment judgments have an element of subjectivity.     

The Impact of Varying the Number of Measurement Invariance Constraints on the Assessment of Between-Group Differences of Latent Means

The objective was to offer guidelines for applied researchers on how to weigh the consequences of errors made in evaluating measurement invariance (MI) on the assessment of factor mean differences. We conducted a simulation study to supplement the MI literature by focusing on choosing among analysis models with different number of between-group constraints imposed on loadings and intercepts of indicators. Data were generated with varying proportions, patterns, and magnitudes of differences in loadings and intercepts as well as factor mean differences and sample size. Based on the findings, we concluded that researchers who conduct MI analyses should recognize that relaxing as well as imposing constraints can affect Type I error rate, power, and bias of estimates in factor mean differences. In addition, fit indexes can be misleading in making decisions about constraints of loadings and intercepts. We offer suggestions for making MI decisions under uncertainty when assessing factor mean differences.     

Conversations With Environmental Educators: A Conversation With Four Classroom Teachers

Abstract

With the ultimate goal of providing safe, high-quality experiential educational opportunities, decision making on the part of the outdoor instructor has become a critical component in successful programming. Within the outdoor pursuits setting, decisions can be categorized by specific situations and by the person or group affected by the decisions. In addition, decisions can be classified according to the frequency and severity of the consequences of a wrong choice. Correct decision making can be hindered by a variety of situations, such as stress and adversity, which are often present in the outdoor setting. Despite these problems, a number of techniques, such as consensus decision making, can aid the outdoor instructor in making correct decisions.     

Assessing Learning Quality: reconciling institutional,staff and educational demands

Two major frameworks for educational decision making, including decisions on assessment, can be distinguished: quantitative, whichis adequate for construing some kinds of learning; and qualitative, which should be the appropriate framework for enabling decisions flowing from most tertiary educational aims. However, for various reasons, institutions implicitly encourage a quantitative framework for assessment‐related decision making, particularly evident in the recent emphasis on accountability and performance indicators. This is unfortunate because, through the backwash effect, quantitative modes of assessment encourage surface approaches to learning, which typically lead to low cognitive‐level outcomes that are not compatible with stated course objectives. It is argued that an institution and its educational practices comprise a system in equilibrium, and that if educational goals are to be realised, the whole system needs to be compatible with those aims.     

Evaluation practices for college students with LD

To examine current practices in the use of psychoeducational evaluations for service delivery, we surveyed 91 service providers to college students with learning disabilities. The three purposes of the survey were to determine (a) whether service delivery decisions are based on information from psychoeducational evaluations, (b) which sections of the psychoeducational report are most useful in making service delivery decisions, and (c) the respondents' satisfaction with the tests and measurements for service delivery. The findings supported the common belief that data from psychoeducational evaluations serve as the primary basis for both eligibility and specific accommodation determinations. Respondents reported that all sections of the psychoeducational evaluation written report were useful, with the least useful section being test scores and the most useful being the summary of cognitive strengths and weaknesses. However, the section used most often for service delivery decisions was the professional's recommendations.     

Factors influencing the decision to administer psychoeducational tests

Ninety-nine NASP members participated in a study designed to investigate bias in the early stages of the referral process (i.e., in the decision to administer psychological tests). Each school psychologist received one of eight case studies, which described a child referred for academic learning problems. The case studies included typical referral information and varied student race (Black, White), socioeconomic status (higher, lower), and group achievement test scores (average, below average). The decision to administer individual psychoeducational tests was not influenced by the student's race or socioeconomic status. School psychologists were influenced by the group achievement test data. Students who showed lower achievement test results were more likely to be recommended for testing than were those who showed average performance levels. Thus, these school psychologists were not biased by knowledge of a child's race or socioeconomic status, but were influenced by instructionally relevant data (i.e., achievement test scores). In addition, when objective test data indicated average achievement levels, the psychologists did not generally recommend subsequent individual psychoeducational testing. The findings suggested that, under certain conditions, testing may not automatically follow receipt of a referral.     

Toward a Critical Deliberative Strategy for Addressing Ideology in Educational Policy Processes

Is it possible and desirable to transcend ideological perspectives within educational policy research and decision making? If so, what would it entail and how should we proceed? At heart, the relation among research, ideology, and policy may be characterized as a complex and interconnected web. The common view of this web is that good social and educational policy decisions are generally made based on research evidence and largely independent of ideological positions. However, it is rather more likely that research and political ideology interact throughout the research process, from the choice of research question, to the organization that provides funding for the research, to the way that findings are interpreted and used. The purpose of this article is to propose a promising strategy that can be utilized by policymakers in the process of making educational policy decisions.     

“The verdict is in: A study of jury decision making factors,moment of personal decision,and jury deliberations—from the jurors' point of view”

The purpose of this study was to gain insight into jury decision waking factors, jury decision timing, and the jury decision making process by talking to actual trial jurors. Three criminal jury trials were observed, and seventeen of the jurors were interviewed after each trial's completion. The data indicate that jurors were influenced by the evidence, witnesses, lawyers, and defendant in the trials. In two trials, jurors made their decisions early; one trial produced later decisions. The jurors' reports also demonstrate that jurors considered evidence and discussed the key issues during deliberation.     

An Examination of Factors Influencing Students Selection of Business Majors Using TRA Framework

Making decisions regarding the selection of a business major is both very important and challenging for students. An understanding of this decision‐making process can be valuable for students, parents, and university programs. The current study applies the Theory of Reasoned Action (TRA) consumer decision‐making model to examine factors that influence college students’ intentions to choose a business major. A total of 670 undergraduate students enrolled at a large midwestern university participated in the study. Social image, job availability, and aptitude were found to be significant factors that impact students’ decisions to select a business major. The results also reveal that family, high‐school counselors, and professors have a major influence on students’ decisions. Furthermore, some unique differences were found related to gender and decided/undecided status of students. The implications of these results for promoting different majors and future research are discussed.     

A model for making decisions about ethical dilemmas in student assessment

In this mixed-methods study we investigated the development of a generalized ethics decision-making model that can be applied in considering ethical dilemmas related to student assessment. For the study, we developed five scenarios that describe ethical dilemmas associated with student assessment. Survey participants (i.e., educators) completed an online survey to express their decision-making process when faced with ethical dilemmas relating to student assessment. Based on the literature and the educators’ written responses to the scenarios, elements to consider in an ethics decision-making model related to student assessment include the following: (1) the critical incident giving rise to the ethical dilemma; (2) identification of the conflict elements; (3) decisions about the ethicality of the elements; (4) justification of the decisions; (5) implications; and (6) alternative suggestions. This model offers guidance to educators in considering the dimensions of an ethical dilemma in assessment prior to making a decision.     

Socioeconomic bias in special education placement decisions

This study analyzed the planning and placement decisions of the Educational Placement and Planning Committees for 973 students in the State of Michigan. An attempt was made to determine if evidence suggesting a socioeconomic bias in placement existed. Data included students from five categories: learning disabled (N = 323), educable mentally impaired (N = 163), emotionally impaired (N = 83), otherwise impaired (N = 70), and nonimpaired (N = 334). No statistically significant differences that would indicate an SES bias were found. However, other factors (sex, age, intelligence, location of district) may contribute substantially to special education placement. Significant agreement was found between school psychologists and teacher consultants with respect to placement decisions. Results are discussed in terms of the apparent absence of any socioeconomic bias in team decision making.