A Strategy for Evaluating District Developed Assessments for State Accountability

As part of Nebraska's assessment and accountability system, districts' local assessment systems are evaluated for their psychometric quality. This article provides an overview of a two-stage evaluation strategy, discusses how it was applied in Nebraska, and presents results from the first three years of the evaluation process. Benefits of the method include an emphasis on formative evaluation and promotion of improved assessment quality at the local level. A limitation of the model is the inability to make refined comparisons of student performance across districts on the assessments. Results from the first three years suggest that greater specificity in the review criteria and additional reviewer calibration activities are needed.     

From the Editor

Educational Assessment, Evaluation and Accountability -     

Breaking in: Gaining entry to prisons and other hard-to-access criminal justice organizations

There exists a large body of literature on doing research with hard-to-access populations such as the mentally ill, gang members, prison inmates, correctional officers, and wardens. There is little discussion, however, on getting access to criminal justice organizations in the first place. This article offers several practical suggestions for “breaking in” to prisons and other criminal justice organizations. This article emphasizes getting access to agency-collected data.     

Outside the Comfort Zone of the Classroom

What comes to mind when the word learning is mentioned? Is it a classroom full of students and a teacher dictating notes as the students struggle emphatically to write down every word that is said? Is it a picture of a student in the library with a stack of books piled on a desk, trying to read them all? These examples of learning may fit the traditional definition but this paper expands learning to include service. This paper describes a non‐traditional style of learning that takes the student “out of the comfort zone of the classroom” into the community and explains how this experience has an impact on the students’ education as well as their lives. The authors of this paper hope to influence its readers to re‐think the definition of learning by understanding the importance of service.     

COMMUNITY COLLEGE FACULTY RECRUITMENT: EFFECTS OF JOB MOBILITY,RECRUITER SIMILARITY-DISSIMILARITY,AND APPLICANT GENDER

The study reported in this article was a factorial experiment that addressed the usefulness of the Rynes and Barber (1990) applicant attraction model and the Byrne (1971) similarity-attraction hypothesis as theoretical frameworks for conducting empirical research about community college faculty recruitment. Study participants (N = 136) were business professionals completing the masters of business administration degree who position announcements of community college business faculty vacancies. The independent variables were job mobility (relocation required vs. relocation not required), recruiter background (business vs. education), and applicant gender. The dependent variable was a composite rating for applicant reaction to a job described in a faculty position announcement. Both male and female applicants rated jobs most favorably when the job did not require relocation F(1,128) = 5.16, p.05 and the recruiter conveying the job message had a professional background similar to that of the applicant F(1,128) = 11.97, p.001. These findings have implications for theory, educational recruitment practice, and future research.     

Assessment matters: Making the familiar strange: How a culture audit can boost your advising impact

Chad Ahren, Helen-Grace Ryan, and Amanda Suniti Niskodé-Dossett introduce an assessment tool to help advisors better understand their student organizations.     

Metamemory Development: Understanding the Role of Similarity in False Memories

Research on the development of metamemory has focused primarily on children's understanding of the variables that influence how likely a person is to remember something. But metamemory also involves an understanding of why people occasionally misremember things. In this study, 5- and 6-year-olds ( N  =   38) were asked to decide whether another child's mistakes in a memory game were due to false memories or guesses. Some of the fictitious child's mistakes were similar to material he had seen earlier and some were not. Six-year-olds, but not 5-year-olds, consistently attributed more similar than dissimilar mistakes to false memories. Understanding the link between similarity and false memories improves significantly between 5 and 6 years of age.     

Teacher Implementation and the Impact of Game-Based Science Curriculum Materials

Research-based digital games hold great potential to be effective tools in supporting next-generation science learning. However, as with all instructional materials, teachers significantly influence their implementation and contribute to their effectiveness. To more fully understand the contributions and challenges of teacher implementation of digital games, we studied the replacement of existing high school biology genetics lessons over a 3- to 6-week period with Geniverse, an immersive, game-like learning environment designed to be used in classrooms. The Geniverse materials infuse virtual experimentation in genetics with a narrative of a quest to heal a genetic disease; incorporate the topics of meiosis and protein synthesis with inheritance; and include the science practices of explanation and argumentation. The research design involved a quasi-experiment with 48 high school teachers and about 2000 students, student science content knowledge and argumentation outcome measures, and analysis using hierarchical linear modeling. Results indicate that when Geniverse was implemented as the designers intended, student learning of genetics content was significantly greater than in the comparison, business-as-usual group. However, a wide range of levels of Geniverse implementation resulted in no significant difference between the groups as a whole. Students’ abilities to engage in scientific explanation and argumentation were greater in the Geniverse group, but these differences were not statistically significant. Observation, survey, and interview data indicate a range of barriers to implementation and teacher instructional decisions that may have influenced student outcomes. Implications for the role of the teacher in the implementation of game-based instructional materials are discussed.