A validation study of the Teacher Collaboration Assessment Survey

Although teacher collaboration is a school improvement imperative, it persists as an under-empiricized construct that has proven difficult to establish and assess with certainty. In this article, the authors present a validation study of the Teacher Collaboration Assessment Survey (TCAS). The TCAS operationalizes and measures 4 key domains of teacher collaboration: dialogue, decision making, action, and evaluation, and has been used to examine the quality of teacher teaming in district-wide comprehensive school reform efforts in the Northeastern and Mid-Atlantic regions of the United States. Five sources of validity evidence recommended by Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1999 American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. [Google Scholar]) are explicated, which establish a strong argument in support of the instruments' validity. The authors discuss how educational leaders and researchers can use the TCAS for leveraging teacher collaboration for instructional innovation and student achievement, and to systematically examine teacher teaming and its relationship to other educational outcomes.     

The Challenges of Meeting the Standards: A Perspective from the Test Publishing Community

Some of the challenges that test publishers face in constructing educational assessments that meet high technical quality as prescribed in the Standards for Educational and Psychological Testing ( AERA, APA, NCME, 1999 ) are examined. Federal educational initiatives are used to illustrate demands on technical quality that challenge the efforts of test publishers to adhere to the Standards . Possible pitfalls to avoid when updating the Standards are also discussed.     

Code of Fair Testing Practices in Education (Revised)

A note from the Working Group of the Joint Committee on Testing Practices:
The Code of Fair Testing Practices in Education (Code) prepared by the Joint Committee on Testing Practices (JCTP) has just been revised for the first time since its initial introduction in 1988.
The revision of the Code was inspired primarily by the revision of the Standards for Educational and Psychological Testing (AERA, APA, NCME, 1999) and recent changes in testing practices in a variety of professions.
The revised Code (JCTP, 2004) has been endorsed by all seven member associations of the Joint Committee, which includes representatives of the National Council on Measurement in Education, as well as the American Counseling Association, American Educational Research Association, American Psychological Association, American Speech-Language-Hearing Association, National Association of School Psychologists, and National Association of Test Directors. The mission of the JCTP is to provide "a means by which professional organizations and test publishers can work together to improve the use of tests in assessment and appraisal."
The revised Code is likely to continue having a major impact on the development and use of testing and assessment in education. It is presented in this issue of Educational Measurement: Issues and Practice and may be reproduced in whole or in part without fees or permission, provided that any reproduction includes appropriate citation. Additional information about the Code may be obtained from: Joint Committee on Testing Practices, Science Directorate, American Psychological Association, 750 First Street, NE, Washington, DC 20002-4242.     

Pilot Testing a New Short Screen for the Assessment of Older Women's PTSD Symptomatology

It is difficult for busy health care providers to perform routine screening for older women's posttraumatic stress symptomatology. This difficulty is due, at least partially, to a paucity of instruments specifically tested on such a population. To address this issue, in this preliminary study we tested an abbreviated screen from the set of 20 items comprising the Distressing Event Questionnaire (DEQ; Kubany, Leisen, Kaplan, & Kelly, 2000a Kubany , S. E. , Leisen , M. B. , Kaplan , A. S. , & Kelly , M. P. ( 2000a ). Validation of a brief measure of PTSD: The Distressing Event Questionnaire . Psychological Assessment , 12 ( 2 ), 197 – 209 .[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]) on a convenience sample of 94 ethnically diverse older women (age 52–105). This new five-item derivation, named Brief Posttraumatic Stress Screening Scale (BPSSS), assesses posttraumatic stress based on the Diagnostic and Statistical Manual of Mental Disorders (DSM–IV; American Psychiatric Association, 1994 American Psychiatric Association . ( 1994 ). Diagnostic and statistical manual of mental disorders , (4th ed.) . Washington , DC : American Psychiatric Press . [Google Scholar]). Its conciseness reduces the likelihood that older women would become fatigued during assessment, making it ideal for use in busy health care settings. In view of the hypothesized cohesiveness of the tool's five items, we expected the screen to be reliable. Because the BPSSS has only five items, a single factor was hypothesized to account for a large proportion of the variance in its items. We also expected that scores on the screen would correlate (to a certain extent) with those on measures of depression and perceived stress of a nontraumatic and nonmedical nature. A standardized alpha of .86 demonstrated high internal consistency of the BPSSS, and the exploratory factor analysis showed that one factor accounted for 58% of the five items' variance. Moreover, the correlations of BPSSS scores with scores on nontraumatic stress and depression were moderate yet significant (r = .37, p < .01 and r = .50, p < .01, respectively).     

Modifying Achievement Test Items: A Theory-Guided and Data-Based Approach for Better Measurement of What Students With Disabilities Know

Federal regulations allow up to 2% of the student population of a state to achieve proficiency for adequate yearly progress by taking an alternate assessment based on modified academic achievement standards (AA-MAS). Such tests are likely to be easier, but as long as a test is considered a valid measure of grade level content, it is allowable as an AA-MAS (U.S. Department of Education, 2007b U.S. Department of Education. 2007b. Standards and assessments peer review guidance, Washington, DC: Author.  [Google Scholar]). In this article, we examine procedures for developing, modifying, and evaluating items and tests using an evolving modification paradigm, as well as a classic reliability and validity framework. Theoretical influences, such as principles of universal design, cognitive load theory, and item development research, are discussed. The Test Accessibility and Modification Inventory, a tool that provides systematic and comprehensive guidance to help educators modify grade-level tests, is introduced. Cognitive lab methods and experimental field tests are then described, along with examples and key findings from each, relevant to AA-MASs. The article concludes with a discussion of precautions, lessons learned, and questions generated about the methods used to improve both access and test score validity for the students who are eligible for this new alternate assessment.     

History,Pending

Tesconi-Warren collaborations began 50 years ago at the University of Illinois-Chicago, then abbreviated fondly as “Chicago Circle,” perhaps the only American institution of higher learning named for a traffic-control installation. We offered Foundations courses in teacher preparation programs of the College of Education, occasionally as a team. Our wives and children became friends. Van Cleve Morris was our dean and president of the American Educational Studies Association (AESA) around the time Tesconi brought Warren into the young association early in the 1970s. We jointly designed a PhD specialization in education policy studies and secured grants to support multicultural curriculum development and recruitment of American Indian students from Chicago's Native community. Later, we went separate ways. Our work became more administrative, Tesconi as dean of education and social services at the University of Vermont and American University and Warren in a similar capacity at Indiana University.

?This article relies extensively on personal memories, refreshed and fact-checked through AESA archives. Readers may find it fantastical and on some points mistaken. But the want of a history of AESA makes searching its past for coherence a stagger through patchy recollections. Believing is seeing.

?Although our association experiences have extended across 5 decades, the ones we emphasize here occurred in the 1970s and 1980s. All had rippling effects. Specifically, we cite Pride and Promise, an AESA-sponsored publication in 1984; enactment of standards for assessing Foundations courses and sequences in professional education programs; and the AESA campaign to establish what was originally called the Council of Learned Societies in Education, a bonding of Foundations-related associations subsequently renamed the Council for Social Foundations of Education (American Educational Studies Association, 2017 American Educational Studies Association. (2017, May 6). Weekly Roundup. Retrieved from http://www.educationalstudies.org/executivecouncilstatementofconcern [Google Scholar], sec. 1978; Tozer, Gallegos, Henry, Greiner, & Price, 2011 Tozer, S., Gallegos, B. P., Henry, A., Greiner, M. B., & Price, P. G. (Eds.). (2011). Handbook of research on the social foundations of education. New York, NY: Routledge. [Google Scholar]).

An AESA narrative should be constructed as multiple stories with shifting centers of gravity, a design permitting transdisciplinary understanding of how diversity can grow as one without compromising unique traits and histories. Sturdy purposes, reinterpreted and reaffirmed, injected the powerful secret ingredients. The association has always been hydra-headed, but united nonetheless. Membership analyses would confirm whether the guesses have merit. AESA has changed without coherently pinpointing when, how, and why. It devised long-range plans but did not follow them religiously. The flexibility may well have proved to be an asset.     

Researching female professors: the difficulties of representation,positionality and power in feminist research

This article draws on findings from my doctoral research to exemplify some of the difficulties I encountered when interviewing 20 female professors¹1. In the UK context, the title of ‘professor’ is reserved for full professors only – that is, those with either an established or personal chair. and subsequently writing about their life histories. In this article, I discuss how I addressed the issues of representing and positioning my participants, and I reflect on the power dynamics present in the research process. The article contends that by drawing on Bourdieu and Wacquant's [An Invitation to Reflexive Sociology. Chicago, IL: University of Chicago Press] understanding of reflexivity and feminist interpretations of their work including McNay [“Gender, Habitus and the Field: Pierre Bourdieu and the Limits of Reflexivity.” Theory Culture Society 16: 95] and Adkins [“Reflexivity Freedom or Habit of Gender?” Theory Culture Society 20: 21], valuable research can be produced, despite the difficulties. A related aim of this paper is to reflect back after four years in an academic post on my position as a novice researcher, and some of the associated issues facing doctoral students, particularly the problems with researching upwards [Walford, G. 2011 Walford, G. 2011. “Researching the Powerful.” British Educational Research Association on-line resource. Accessed March 17, 2014. www.bera.ac.uk/system/files/Researching%20the%20Powerful.pdf [Google Scholar]. “Researching the Powerful. British Educational Research Association On-line Resource.” Accessed March 17, 2014. www.bera.ac.uk/system/files/Researching%20the%20Powerful.pdfWalford 2011].     

Stories of Smartness and Whiteness in School Pictures and Yearbooks

This article explores the way that discourses of smartness and whiteness are produced and reproduced in schooling. Using an approach grounded in narrative research, I explore the convergences and contradictions between my own educational autobiography and the representations of schooling found in my school pictures and yearbooks. In my analysis, I argue that white supremacy played an important role in the construction of my own story of smartness throughout my primary and secondary schooling experiences. I also argue that yearbooks form powerful “artifacts of smartness” (Hatt, 2011 Hatt, B. (2011). Smartness as a cultural practice in schools. American Educational Research Journal, 49(3), 438–460. doi: 10.3102/0002831211415661[Crossref] , [Google Scholar], p. 448) that can be used to interpret and interrogate personal experiences as well as larger societal discourses of smartness and whiteness in schooling.     

Charter Schools: Implications for Teachers and Administrators

Young adolescents’ low scores on the National Assessment of Educational Progress (NAEP) force the question of whether these students will be ready for college in four years. Our efforts to build a college-going culture emphasize strengthening students’ writing skills by using preservice teachers to lead writing marathons for at-risk middle school students on university visits. Structured interviews, surveys, and written reflections reveal that what students write about changes with age, their motivation to write varies, and their college aspirations and perceptions become more positive after completing several annual writing marathons on a college campus. The writing marathon structure makes the college visit truly meaningful to students, and it provides a compelling incentive to write by supporting characteristics of middle school students, as defined by the National Middle School Association (National Middle School Association [NMSA] 2003 National Middle School Association (NMSA). 2003. This we believe. Successful schools for young adolescents, Westerville, OH: National Middle School Association.  [Google Scholar]). The marathon model focuses on four key elements: setting, timing, small groups, and writers’ level of commitment.     

Educating ‘the new Norwegian we’: an examination of national and cosmopolitan education policy discourses in the context of extremism and Islamophobia

Across Europe there are growing concerns about Islamophobia and far-right and anti-democratic movements. Until Anders Behring Breivik’s July 2011 attacks in which 77 died, Norway’s vulnerability was not perceived as great as that of other jurisdictions. Breivik declared his abhorrence of multiculturalism but also drew the world’s attention to intolerance and xenophobia in Europe, increasingly directed towards Muslims. In response, Prime Minister Stoltenberg spoke widely of ‘the new Norwegian we’, cautioning against exclusionary discourses and underlining that minorities are an equal part of Norwegian society. This paper examines education policy in the context both of extreme right political activity and national and international debates about nationalism and cosmopolitanism, considering ways in which policy supports an inclusive notion of nationhood, and ways in which it promotes an exclusive model of national identity. Drawing on framing questions from the International Association for the Evaluation of Educational Achievement’s 1995 International Association for the Evaluation of Educational Achievement (1995). National case studies final guidelines. Amsterdam: IEA. [Google Scholar] civic education study, relating to national identity and social cohesion, it examines how programmes address social cohesion, human rights, cultural diversity, and conceptualise minorities. It assesses the effectiveness of cross-curricular approaches to contribute to justice and equality and challenge racism in a multicultural society, proposing an inclusive model of education for cosmopolitan citizenship, which incorporates diversity.     

Revisiting “The Master's Tools”: Challenging Common Sense in Cross-Cultural Teacher Education

Although school climate has been thought to be especially important for racial minority and poor students (Booker, 2006 Booker, K. C. 2006. School belonging and the African American adolescent: What do we know and where should we go?. The High School Journal, 89(4): 1–7. [Crossref] , [Google Scholar]; Haynes, Emmons, &; Ben-Avie, 1997 Haynes, N. M., Emmons, C. and Ben-Avie, M. 1997. School climate as a factor in student adjustment and achievement. Journal of Educational and Psychological Consultation, 8(3): 321–329. [Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]), little research has explored the significance of racial climate for these students. Furthermore, research in the area has tended to treat race, socioeconomic class, and gender separately, ignoring the ways in which they interact. Using quantitative survey data from 842 African American and white middle school students, this study examined the associations of race, class, and gender with school racial climate perceptions. Results indicated students’ perceptions of racial climate differed by race, class, and gender. African American, poor, and female students perceived the racial climate in more negative terms than their white, non-poor, and male counterparts, respectively. Results also indicated joint associations between race and class and climate perceptions. Non-poor, African American students perceived a more negative racial climate than did non-poor Whites. There was limited support for a race and gender interaction. African American females tended to perceive less racial fairness in school than African American males. We discuss the conceptual and methodological tradeoffs of examining students’ school racial climate perceptions from a perspective that considers race, class, and gender jointly.     

Early College Awareness as an Enrollment Management Strategy

The Institute of Medicine and The Educated Citizen and Public Health Initiative suggest that an understanding of public health is a core component of an educated public and is necessary to develop social responsibility (Association of American Colleges and Universities [AAC&;U], 2011 Association of American Colleges and Universities (AAC&;U) . ( 2011 ). The educated citizen and public health . Retrieved from http://www.aacu.org/public_health/  [Google Scholar]; National Research Council, 2003 National Research Council . ( 2003 ). Who will keep the public healthy? Educating public health professionals for the 21 st century (Consensus report). Washington, DC: The National Academies Press.  [Google Scholar]). To respond to this call, the University of New Hampshire collaborated with faculty from the New Hampshire Community College System to introduce the public health field through a course called Global Public Health Issues. This article will discuss the development, implementation, and evaluation of a public health course between a two-year and four-year college. It will also discuss the potential for such a partnership and public health course to encourage life-long learning and a commitment to social responsibility, allow for new course/major/minor development in public health, enable faculty to expand their expertise, broaden the attractiveness of two-year colleges, and introduce a vocation-based student population to the value of public health and the public health workforce.     

A Critical Challenge: The Engagement and Assessment of Contingent,Part-Time Adjunct Faculty Professors in United States Community Colleges

In 2011, according to a National Center for Education Statistics report, part-time instructional staff in all higher education institutions exceeded full-time faculty members for the first time, accounting for 50% of all instructional staff (National Center for Education Statistics [NCES], 2012 National Center for Education Statistics . ( 2012 ). IPEDS, Digest of education statistics, Winter 2011–12, human resources component, fall staff section: Table 286 [data file]. Retrieved from http://nces.ed.gov/programs/digest/d12/tables/dt12_286.asp  [Google Scholar]). The same report indicates part-time faculty in community colleges exceeds 70% of instructional staff. Perhaps more alarming are the numbers of contingent instructional staff—faculty without long-term employment commitments. According to this measure, nearly 70% of faculty members in all areas of higher education have little-to-no job stability (American Association of University Professors [AAUP], 2013 American Association of University Professors (AAUP) . ( 2013 ). Background facts on contingent faculty. Retrieved from http://www.aaup.org/issues/contingency/background-facts  [Google Scholar]; Schuster & Finklestein, 2006 Schuster , J. H. , & Finklestein , M. J. ( 2006 ). The American faculty: The restructuring of academic work and careers . Baltimore , MD : The Johns Hopkins University Press . [Google Scholar]). However, limited research exists on the working experiences of this major subpopulation of United States professors.

The purpose of this qualitative study was to explore the experiences of part-time contingent community college faculty regarding the assessment processes their institutions implemented. Through interviews, researchers gathered data identifying what 20 part-time contingent faculty professors reported regarding teaching conditions and institutional assessment procedures. Participant interviews revealed two major themes centered on a lack of institutional engagement and meaningful assessment policies or procedures.     

Retaking the Test

Data-driven teaching and learning is common sense in education today, and it is common sense that these data should come from standardized tests. Critiques of standardization either make no constructive suggestions for what to use in place of the tests or they call for better, more scientifically rigorous, reliable, and authentic forms of testing. This article aims to critique the underlying logic of testing within neoliberal education by a return to the question of the ontology of testing as such. We do this by distinguishing the practices and technologies used in current testing from the fundamentally educational and philosophical aspect of human experience that critical theorist Avital Ronell (2005 Ronell, A. (2005). The test drive. Urbana, IL: University of Illinois Press. [Google Scholar]) calls the “test drive.” Posing the test drive against the current testing regime will allow us to reclaim health, personality, and taste from the high-stakes notion of testing that prevails today.     

Here,There, and Back Again with the Social Foundations of Education: Tracing Conocimientos In/Toward Coalitional Journeys

This article narrates a parallel personal and professional journey of scholarly engagement with the field of the social foundations of education and its home in the American Educational Studies Association. It draws on Gloria Anzaldúa's (2000 Anzaldúa, G. E. (2000). Interviews/Entrevistas (A. Keating, ed.). New York, NY: Routledge. [Google Scholar]) notion of conocimiento as multileveled and connective knowings, and María Lugones's (2008 Lugones, M. (2008). From within germinative stasis: Creating active subjectivity,resistant agency. In A. Keating (Ed.), Entremundos/Among worlds: New perspectives on Gloria Anzaldúa (pp. 85–99). New York, NY: Palgrave Macmillan. [Google Scholar]) idea of coalitional journeys to describe the histories and possibilities for the field.     

Recommendations for Preparing and Scoring Constructed-Response Items: What the Experts Say

We determined the recommendations for preparing and scoring constructed-response (CR) test items in 25 sources (textbooks and chapters) on educational and psychological measurement. The project was similar to Haladyna's (2004) Haladyna, T. M. 2004. Developing and validating multiple-choice test items , 3rd, Mahwah, NJ: Erlbaum. [Crossref] , [Google Scholar] analysis for multiple-choice items. We identified 12 recommendations for preparing CR items given by multiple sources, with 4 of these given by at least half of the sources; and 13 recommendations for scoring CR items given by multiple sources, with 5 given by at least half of the sources. Many recommendations received minority support or were unique to individual sources. Research is needed both on the effect of the recommendations for measurement properties and the extent to which the recommendations are adopted in practice.     

Argumentation Surrounding Argument-Based Validation: A Systematic Review of Validation Methodology in Peer-Reviewed Articles

Since it was formalized by Kane, the argument-based approach to validation has been promoted as the preferred method for validating interpretations and uses of test scores. Because validation is discussed in terms of arguments, and arguments are both interactive and social, the present review systematically examines the scholarly arguments which appear in 83 papers on argument-based validation methods published in peer-reviewed journals. Findings suggest that scholars generally agree on the nature and importance of argument-based validation but disagree on whether validation should be structured or unstructured, formal or informal. Implications are discussed, including promotion of the Standards for Educational and Psychological Testing (AERA, APA, and NCME) as a foundation for consensus in the field.     

Multistage Computerized Adaptive Testing With Uniform Item Exposure

This article describes a computerized adaptive test (CAT) based on the uniform item exposure multi-form structure (uMFS). The uMFS is a specialization of the multi-form structure (MFS) idea described by Armstrong, Jones, Berliner, and Pashley (1998 Armstrong, R. D., Jones, D. H., Berliner, N. and Pashley, P. June 1998. Computerized adaptive tests with multiple form structures, June, Champaign-Urbana, IL: Paper presented at the annual meeting of the Psychometric Society.  [Google Scholar]). In an MFS CAT, the examinee first responds to a small fixed block of items. The items comprising that block may be unrelated to each other, or they may comprise a testlet (Wainer and Kiely, 1987 Wainer, H. and Kiely, G. L. 1987. Item clusters and computerized adaptive testing: A case for testlets. Journal of Educational Measurement, 24: 185–201. [Crossref], [Web of Science ®] , [Google Scholar]) After the first block of items has been administered, adaptation takes place in the choice of the next block to be administered and subsequent blocks. The uMFS design integrates item exposure control, as well as content balancing and other test development needs, into the design of the CAT, instead of placing those activities in the online implementation. We show that it is possible to implement item exposure control, in a very thorough way, in the psychometric specifications of the item blocks.     

Interrogating Discursive Data: How News Media Narratives Assemble Truths About the Teaching Profession

This project provides a framework for interrogating the language often used in media reporting on education, and making those often hidden ideological underpinnings more visible. Explored through this analysis are the language, metaphors, logic, and rhetorical devices used by various news media reporting on educational concerns, and in particular the concerns revolving around teacher quality and teacher preparation, which allow the construction and circulation of specific kinds of knowledge and assert certain kinds of truths. This analysis is guided by the premise that education writ large, which includes the profession of teaching as well as the field of teacher preparation, is charged with helping students, our youngest citizens, develop an understanding of the practices and problems of people-power, self-rule, and shared governance in our democratic society. If education primarily involves preparing students for democratic citizenship, as a long history of scholarship has established (Barber, 2001 Barber, B. (2001). An aristocracy of everyone. In S. J. Goodlad (Ed.), The last best hope: A democracy reader (pp. 11–22). San Francisco, CA: Jossey-Bass. [Google Scholar]; Bean &; Apple, 1995 Beane, J. A., &; Apple, M. W. (1995). Democratic schools. Alexandria, VA: Association for Supervision and Curriculum Development. [Google Scholar]; Beard, 1937 Beard, C. (1937). The unique function of education in American democracy. Washington, DC: National Education Association. [Google Scholar]; Dewey, 1916 Dewey, J. (1916). Democracy and education. New York, NY: Free Press. [Google Scholar]; Bode, 1937 Bode, B. H. (1937). Democracy as a way of life. New York, NY: Macmillan. [Google Scholar]; Giroux, 1989 Giroux, H. (1989). Schooling for democracy: Critical pedagogy in the modern age. London, England: Routledge. [Google Scholar]; Gore, 1993 Gore, J. (1993). The struggle for pedagogies: Critical and feminist discourses as regimes of truth. New York, NY: Routledge. [Google Scholar]; Gutman, 1999 Gutmann, A. (1999). Democratic education. Princeton, NJ: Princeton University Press.[Crossref] , [Google Scholar]; McLaren, 2001 McLaren, P., &; Farahmandpur, R. (2001). Educational policy and the socialist imagination: Revolutionary citizenship as a pedagogy of resistance. Educational Policy, 15(3), 343–378. https://doi.org/10.1177/0895904801015003002.[Crossref], [Web of Science ®] , [Google Scholar]; Nussbaum, 2010 Nussbaum, M. (2010). Not for Profit: Why Democracy Needs the Humanities. Princeton, NJ: Princeton University Press. [Google Scholar]), educating citizens to critically examine, analyze, and understand social worlds is essential to democracy.