Linking assessment to undergraduate student capabilities through portfolio examination

Portfolios are an established method of assessment, although concerns do exist around their validity for capabilities such as reflection and self‐direction. This article describes an e‐portfolio which closely aligns learning and reflection to graduate capabilities, incorporating features that address concerns about portfolios. Students are required to complete assessments linked to graduate capabilities. In Year 3, a portfolio review occurs (205–248 students per year), focusing on students' grades and feedback from assessments and a reflective essay is submitted. In the essay, students reflect on their progress, identify areas of weakness and detail plans for improvement. Progress in each capability is summatively graded against specific criteria and feedback is provided. Students progressively accumulate evidence of learning linked to the graduate capabilities. The provision of sufficient structure prevents evasion of areas of weakness. Importantly, the equal weighting given to all graduate capabilities emphasises that competence in all areas is required. The requirement for a degree of self‐direction and reflection in all assessments promotes regular review of progress. This e‐portfolio explicitly links graduate outcomes with assessment in order to drive learning. Further research is required to evaluate acceptability to students, as well as the efficacy of portfolios in developing reflective practice and self‐directed learning.     

Career guidance,career assessment,and consultancy

Summary Although career guidance activities can positively influence students' career development, the effectiveness of such activities is often impaired because they attempt to provide the same services to all students at the same time. Programs typically do not consider either students' differential developmental patterns or the fact that many students have particular needs due to learning, physical, or emotional handicaps. Therefore, a primary focus of refining programs must be the individualization of services to meet a wide variety of student needs.Referral to career assessment or to professionals trained in career assessment and consultation is a valuable option open to school counselors who have neither the time nor the specialized training to conduct such assessments themselves. Career assessment centers offer comprehensive assessment services, including both more traditional assessments as well as innovative activities. Using a consultation paradigm, the school counselor and career assessment officer can better provide comprehensive, individualized assessment and counseling tailored to the specific needs of students, regardless of their handicaps and their different levels of developmental readiness     

Science assessments for all: Integrating science simulations into balanced state science assessment systems

This article reports on the collaboration of six states to study how simulation‐based science assessments can become transformative components of multi‐level, balanced state science assessment systems. The project studied the psychometric quality, feasibility, and utility of simulation‐based science assessments designed to serve formative purposes during a unit and to provide summative evidence of end‐of‐unit proficiencies. The frameworks of evidence‐centered assessment design and model‐based learning shaped the specifications for the assessments. The simulations provided the three most common forms of accommodations in state testing programs: audio recording of text, screen magnification, and support for extended time. The SimScientists program at WestEd developed simulation‐based, curriculum‐embedded, and unit benchmark assessments for two middle school topics, Ecosystems and Force & Motion. These were field‐tested in three states. Data included student characteristics, responses to the assessments, cognitive labs, classroom observations, and teacher surveys and interviews. UCLA CRESST conducted an evaluation of the implementation. Feasibility and utility were examined in classroom observations, teacher surveys and interviews, and by the six‐state Design Panel. Technical quality data included AAAS reviews of the items' alignment with standards and quality of the science, cognitive labs, and assessment data. Student data were analyzed using multidimensional Item Response Theory (IRT) methods. IRT analyses demonstrated the high psychometric quality (reliability and validity) of the assessments and their discrimination between content knowledge and inquiry practices. Students performed better on the interactive, simulation‐based assessments than on the static, conventional items in the posttest. Importantly, gaps between performance of the general population and English language learners and students with disabilities were considerably smaller on the simulation‐based assessments than on the posttests. The Design Panel participated in development of two models for integrating science simulations into a balanced state science assessment system. © 2012 Wiley Periodicals, Inc. J Res Sci Teach 49: 363–393, 2012     

Constructive guidance and feedback for learning: the usefulness of exemplars,marking sheets and different types of feedback in a first year law subject

Criteria and standards‐based assessment models are increasingly being adopted by universities as effective practice. However the promise of these models of assessment may not be realised unless teachers can find ways of making criteria and standards understandable to students. Exemplars or examples of previous students’ work of high and low quality can make criteria and standards concrete. Recent research has focussed on the use of exemplars to help students understand criteria and standards, and less emphasis has been given to exemplars simply as guides for students. This mixed methods study explores students’ perceptions of the usefulness of exemplars and different types of feedback for guiding them in completing assessments. A combination of engaging in marking and discussing exemplars, and receiving individualised and standards‐based feedback provides the most helpful guidance for students’ effective learning.     

Needs Assessments: An Integrated Assignment in Civic Service

An undergraduate social work program developed a service‐learning experience in partnership with a local United Way organization to complete a community needs assessment project. The experience integrated the curricula of a social work research methods course and a generalist‐macro practice course with the principles and actions of experiential and service‐ learning, evidence based practice, and civic responsibility. This integrated curricula project yielded practical, “real world” outcomes for the local community and learning outcomes for the students. Needs assessments conducted by social students resulted in particular groups of community citizens receiving targeted services from agencies receiving funding from United Way. Evidence obtained from the students who participated in this experience indicates that two important learning outcomes were achieved; a heightened awareness of the local community's resource system and a sense of connection to the local community, along with the enhancement of students ‘self‐esteem and increased self‐confidence in their abilities.     

Contextualizing instruction: Leveraging students' prior knowledge and experiences to foster understanding of middle school science

Contextualizing science instruction involves utilizing students' prior knowledge and everyday experiences as a catalyst for understanding challenging science concepts. This study of two middle school science classrooms examined how students utilized the contextualizing aspects of project‐based instruction and its relationship to their science learning. Observations of focus students' participation during instruction were described in terms of a contextualizing score for their use of the project features to support their learning. Pre/posttests were administered and students' final artifacts were collected and evaluated. The results of these assessments were compared with students' contextualizing scores, demonstrating a strong positive correlation between them. These findings provide evidence to support claims of contextualizing instruction as a means to facilitate student learning, and point toward future consideration of this instructional method in broader research studies and the design of science learning environments. © 2007 Wiley Periodicals, Inc. J Res Sci Teach 45: 79–100, 2008     

Articulating the validity evidence for a science alternate assessment

Students with the most significant cognitive disabilities (SCD) are the 1% of the total student population who have a disability or multiple disabilities that significantly impact intellectual functioning and adaptive behaviors and who require individualized instruction and substantial supports. Historically, these students have received little instruction in science and the science assessments they have participated in have not included age‐appropriate science content. Guided by a theory of action for a new assessment system, an eight‐state consortium developed multidimensional alternate content standards and alternate assessments in science for students in three grade bands (3–5, 6–8, 9–12) that are linked to the Next Generation Science Standards (NGSS Lead States, 2013 ) and A Framework for K‐12 Science Education (Framework; National Research Council, 2012 ). The great variability within the population of students with SCD necessitates variability in the assessment content, which creates inherent challenges in establishing technical quality. To address this issue, a primary feature of this assessment system is the use of hypothetical cognitive models to provide a structure for variability in assessed content. System features and subsequent validity studies were guided by a theory of action that explains how the proposed claims about score interpretation and use depend on specific assumptions about the assessment, as well as precursors to the assessment. This paper describes evidence for the main claim that test scores represent what students know and can do. We present validity evidence for the assumptions about the assessment and its precursors, related to this main claim. The assessment was administered to over 21,000 students in eight states in 2015–2016. We present selected evidence from system components, procedural evidence, and validity studies. We evaluate the validity argument and demonstrate how it supports the claim about score interpretation and use.     

TRAINING FOR EXPERTISE VERSUS TRAINING FOR TRUST: ISSUES IN RURAL GERONTOLOGY

Traditional gerontological instruction uses an expertise‐based model. Such an approach can be counterproductive for gerontologists serving the elderly in rural areas. To meet the needs of both the rural elderly who need services and the professionals who wish to provide those services, educators in gerontology need to instruct students interested in a rural gerontology practice on the issues that face a professional who is entering a rural area: the diversity of rural areas; the influx of urbanites and what it means to locals; the slow acceptance of the need for human services in rural areas; and, most important, the fact that in rural areas, authority is based on trust, not on expertise. Rather than encouraging the development of expertise and professional authority, gerontological instruction for students interested in serving the rural elderly should emphasize the need for the professional to “stay low” in the client‐clinician relationship, learn the local language and culture, and accept the lack of separation between professional and personal roles in rural areas. These practices will help the professional who is new to the rural area gain the trust of the elderly residents.     

Assessment as learning: Enhancing discourse,understanding, and achievement in innovative science curricula

An assessment‐oriented design‐based research model was applied to existing inquiry‐oriented multimedia programs in astronomy, biology, and ecology. Building on emerging situative theories of assessment, the model extends prevailing views of formative assessment for learning by embedding “discursive” formative assessment more directly into the curriculum. Three twenty‐hour curricula were designed and aligned to content standards, and three levels of assessments were developed and used to assess and enhance learning for each curriculum. These assessments included three or four informal “activity‐oriented” quizzes and discursive formative feedback rubrics supporting collective discourse, a “curriculum‐oriented” examination of individual conceptual understanding, and a “standards‐oriented” test measuring aggregated achievement of targeted standards. After two design‐research cycles, worthwhile scientific argumentation and statistically significant gains were attained for two of the three packages on the exam and test. Achievement gains were comparable to or larger than those of students in comparison classrooms. Many existing innovations could be enhanced and evaluated in this fashion; designing these strategies directly into innovations could have an even greater impact on discourse, understanding, and achievement. © 2012 Wiley Periodicals, Inc. J Res Sci Teach 49: 1240–1270, 2012     

Accuracy of students' self‐assessment and their beliefs about its utility

The purpose of the two studies presented here was to evaluate the accuracy of students' self‐assessment ability, to examine whether this ability improves over time and to investigate whether self‐assessment is more accurate if students believe that it contributes to improving learning. To that end, the accuracy of the self‐assessments of 3588 first‐year students enrolled in a post‐secondary institution was studied throughout a semester during which each student made approximately 80 self‐assessments about his or her own learning process. These self‐assessments were then compared with multiple judgements by peers and tutors. The overall correlations between the scores of self‐, peer and tutor assessments suggest weak to moderate accuracy of student self‐assessment ability. The findings also reveal an ability effect; students judged as more academically competent were able to self‐assess with higher accuracy than their less competent peers. Comparing the accuracy of student self‐assessment averaged over four consecutive periods indicates that the accuracy does not improve over time. In a second study, a questionnaire aimed at eliciting student's beliefs about the effects of self‐assessment on their learning was administered to 936 first‐year students. Based on their responses, sub‐groups of students were identified: those who either believed in the usefulness of self‐assessment or did not. Results suggest that there is no significant association between student beliefs about the utility of self‐assessment and the accuracy of their self‐assessments.     

Autism spectrum disorder assessment and evaluation research in 10 school psychology journals from 2007 to 2017

School psychologists are increasingly engaged in service provisions for students eligible for special education services under the eligibility category of autism, including conducting school‐based assessments and evaluations. Evaluations occur for a variety of reasons such as special education eligibility decision‐making, treatment and intervention planning, and progress monitoring. Publications in school psychology journals emphasizing autism spectrum disorder (ASD) assessment and evaluation are vital to quality training and practitioner utilization of quality practices. In the current study, researchers conducted a systematic review of publicaftions in 10 school psychology journals from 2007 to 2017 to assess the current state of ASD assessment and evaluation research in the field of school psychology. Implications for researchers, trainers, and practitioners are discussed.     

Cultural‐competence considerations for contemporary school‐based threat assessment

Threat‐assessment procedures are advanced for their utility in reducing racial disparities in punitive and exclusionary school discipline outcomes. Generally unexamined, however, is bias in who gets referred for school‐based threat assessment and under what circumstances. Cultural‐competence considerations hold promise for addressing sources of bias in the evaluation of threats made in the school setting. Using a quantitative approach, this systematic literature review examines the degree to which contemporary cultural‐competence considerations are embedded in 24 school‐based threat‐assessment articles published between 2007 and 2017. Results indicate generally poor coverage of considerations for cultural competence both within and across threat‐assessment articles. An analysis of change in cultural‐competence considerations by year of publication suggests that more recently published threat‐assessment literature has generally not integrated concurrent advancements in concepts of cultural competence. Preliminary guidance for incorporating contemporary cultural‐competence considerations into school‐based threat‐assessment procedures are provided.     

A metasynthesis of the complementarity of culturally responsive and inquiry‐based science education in K‐12 settings: Implications for advancing equitable science teaching and learning

Employing metasynthesis as a method, this study examined 52 empirical articles on culturally relevant and responsive science education in K‐12 settings to determine the nature and scope of complementarity between culturally responsive and inquiry‐based science practices (i.e., science and engineering practices identified in the National Research Council's Framework for K‐12 Science Education). The findings from this study indicate several areas of complementarity. Most often, the inquiry‐based practices Obtaining, Evaluating, and Communicating Information, Constructing Explanations and Designing Solutions, and Developing and Using Models were used to advance culturally responsive instruction and assessment. The use and development of models, in particular, allowed students to explore scientific concepts through families’ funds of knowledge and explain content from Western science and Indigenous Knowledge perspectives. Moreover, students frequently Analyzed and Interpreted Data when interrogating science content in sociopolitical consciousness‐raising experiences, such as identifying pollution and asthma incidences in an urban area according to neighborhood location. Specific inquiry‐based practices were underutilized when advancing culturally responsive science instruction, though. For example, Using Mathematics and Computational Thinking and Engaging in Argument from Evidence were infrequently encountered. However, culturally responsive engineering‐related practices were most often connected with these, and thus, represent potential areas for future complementarity, particularly as the United States embraces the Next Generation Science Standards. In considering innovative directions for advancing equitable science education, several possibilities are discussed in light of the findings of this study.© 2017 Wiley Periodicals, Inc. J Res Sci Teach 54:1143–1173, 2017     

National survey of school psychologists regarding use of dynamic assessment and other nontraditional assessment techniques

A national survey investigating the use of dynamic assessment and other nontraditional assessment techniques among school psychologists (N = 226) was conducted. Results of the survey indicated that 42% of respondents were at least “somewhat familiar” with dynamic assessment. However, of those familiar with dynamic assessment, only 39% reported using the techniques once a year or more. The most frequently endorsed reasons for not using dynamic assessment (if familiar with it) were lack of knowledge and time restraints. Learning disabled students were the population of students most often evaluated using dynamic assessment and the dynamic assessment was most often used to determine processing strengths and weaknesses. The majority of those familiar with dynamic assessment became so through independent reading. Only 10% reported learning about dynamic assessment through course work. In response to questions regarding assessment techniques most often used with minority students the majority of respondents reported using traditional assessment tools including the WISC‐III (Wechsler Intelligence Scale for Children–Third Edition), BINET IV (Stanford‐Binet Intelligence Scale–Fourth Edition), or KABC (Kaufman Assessment Battery for Children). Overall, the results of the survey suggest that although the population is becoming increasingly more diverse and changes in PL94‐142 (Public Law 94‐142) demand functional assessments, school psychologists continue to rely heavily upon traditional assessment techniques to address referral concerns of all students. This may in large part be due to weaknesses in graduate training programs. © 1999 John Wiley & Sons, Inc.     

The delicate dance: the assessment implications of awarding students vocational qualifications within university degrees

This paper explores the implications for assessment of embedding qualifications from the vocational education and training (VET) sector within university qualifications. As VET qualifications are now all competency based, assessment in the two sectors is quite different, since universities have generally eschewed competency‐based training and assessment. A general discussion of the issues is followed by the results of a small‐scale research study carried out in the VET discipline area itself, with participants drawn from the AVTEC list of VET teacher‐educators. Telephone interviews were conducted with nine Australian university academics involved with such qualifications; and data were extracted from a broader study of students who had undertaken the Certificate IV in Assessment and Workplace Training embedded within degrees and graduate diplomas in VET at one of these universities. It is clear that there are many assessment challenges involved with the practice of embedding, which have not yet been systematically addressed.     

Integrating Scenario‐Based and Component Reading Skill Measures to Understand the Reading Behavior of Struggling Readers

In recent years, researchers, educators, and policy makers have called for a new generation of reading comprehension assessments (e.g., Partnership for 21st Century Skills, 2008). Advocates of this movement argue for a deeper type of reading assessment, one that captures students’ ability to not only understand single texts in isolation but also to engage in purposeful, multisource integration of sources. While this shift in how we define and measure reading comprehension is laudable, assessments must also measure the fundamental reading skills that may impede higher‐level comprehension processes. This article presents data from two assessments that were designed to work in tandem to provide a more complete picture of reading comprehension. Middle school students were given a component skills battery which measured core reading skills such as word recognition, decoding, vocabulary, and morphology, as well a second assessment designed to measure reading comprehension. Reading comprehension was measured using a scenario‐based assessment approach, which required students to read a range of sources to fulfill a particular reading goal. The results indicate that students, including struggling readers, were able to read, understand, and problem solve in complex learning environments, but students’ ability to do so was often tempered by their basic reading skills. We argue that including a measure of component skills alongside a measure of higher‐level comprehension is beneficial in interpreting student performance. Accordingly, we present the results on the scenario‐based measure as a function of reading component skills and argue for the value of using this approach for struggling readers.