Reliability,validity, and all that jazz

Summary

In this article my purpose has not been to indicate what kinds of things can and can't be assessed appropriately with tests. Rather, I have tried to illuminate how the key ideas of reliability and validity are used by test developers and what this means in practice — not least in terms of the decisions that are made about individual students on the basis of their test results. As I have stressed throughout this article, these limitations are not the fault of test developers. However inconvenient these limitations are for proponents of school testing, they are inherent in the nature of tests of academic achievement, and are as real as rocks. All users of the results of educational tests must understand what a limited technology this is.     

Serpents and Sheep: The Harriman Expedition,Alaska, and the Metaphoric Reconstruction of American Wilderness

This article examines one mechanism for the radical change in public perception of wilderness. The published papers of the Harriman Expedition, a scientific expedition at the end of the nineteenth century, are used to illustrate a process that I call "metaphoric reconstruction." I argue that conceptual metaphors are one of the tools through which texts and rhetorical contexts can be mutually transformative.     

Serpents and Sheep: The Harriman Expedition, Alaska, and the Metaphoric Reconstruction of American Wilderness

This article examines one mechanism for the radical change in public perception of wilderness. The published papers of the Harriman Expedition, a scientific expedition at the end of the nineteenth century, are used to illustrate a process that I call “metaphoric reconstruction.” I argue that conceptual metaphors are one of the tools through which texts and rhetorical contexts can be mutually transformative.     

A Way Forward: Nurturing the Imagination at the Intersection of Race,Class, Gender,and Age

Those who are oppressed often find themselves internalizing voices that limit their ability. This article focuses on a population that falls on the non-hegemonic side of the intersection of race, class, gender, and age: Black girls from poor and working-class backgrounds. From my work with youth, I have noticed that internalizing these limiting voices lead to a sense of personal hopelessness. I suggest that Christian educators combat personal hopelessness by nurturing the theological imagination of their youth.     

Aesthetic Learning,Creative Writing and English Teaching

My article argues that the concept of ‘aesthetic learning’ can be helpful for English teachers on two levels. First, it can be a useful identity for English teachers and students to adopt, based upon my own experiences as a secondary English teacher, creative writer and PhD student. Second, I argue that ‘aesthetic learning’ is an effective and productive way of analysing some of the learning processes that happen in the English teacher’s classroom. In order to arrive at these conclusions, I examine my own creative writing, teaching and learning processes from which I extrapolate the notion that we are all ‘aesthetic learners’ in the sense that we learn to appreciate the qualities of the worlds we inhabit, whether these are actual or virtual. Throughout, my own writing, learning and teaching are used to illustrate my argument. In particular, the article seeks to re-position my own teaching in secondary schools within the context of ‘aesthetic learning’.     

Wittgenstein,Social Views and Intransitive Learning

Wittgenstein often refers to matters of learning, and there have been efforts to extract a social conception of learning from his writings. In the first half of this article, I look at three such efforts, those of Meredith Williams, Christopher Winch, and David Bakhurst, and I say why I think these efforts fail. As I go on to argue, though, there is a fairly trivial sense in which learning is a social rather than a psychological phenomenon: ordinarily, there are public criteria for whether someone has learned something. Yet, in the second half of the article, I point to an exception to this general rule. Taking a cue from Wittgenstein, I call this ‘intransitive learning’, as it refers to learning experiences where we cannot say what we have learned or where there simply isn't anything in particular that we have learned. This is a use that is not easily accommodated by received definitions of learning. It also represents a genuinely psychological use of the word ‘learn’. In contrast to ordinary cases of learning, claims about intransitive learning function like expressions and are marked by first‐person authority.     

One-to-One Encounters: Facilitators,Participants, and Friendship

In this article, I explore the claim that one-to-one encounters between community music facilitators and music participants can be described as friendships. By exploring the relational structure through the call and the welcome, I make some general comments on friendship before finally tackling the question lying at the heart of this article: How might we understand the notion of being a friend within an educative relationship?     

On identity, diversity, and educational change

In this article, I discuss two areas of Andy Hargreaves’ scholarship, teachers’ identity and biography and the three periods of educational change spanning the 1960s to 1990s, that have influenced my work as a teacher educator and researcher. I describe research projects, including self-studies, in which I have examined the influence of teachers’ identities and biographies on their beliefs and practices of responding to student diversity. I also explore how the topic of teachers’ identity and biography are integrated into the courses I teach. Additionally, I describe how I have related the three periods of educational change—a period of optimism and innovation, a period of complexity and contradiction, and a period of marketization and standardization, along with the monocultural restoration—to corresponding policies and practices of responding to student diversity in the US and Ontario, Canada. Finally, I discuss a current project in which Andy and I are exploring core issues related to educational leadership and diversity.     

The tangled roots of real estate markets,the state,and public education

In this article, I use a case study of education in nineteenth-century Oregon to place the twentieth-century phenomenon of ‘shopping for schools’ within a broader historical context. In doing so, I make three points. First, documenting previously unacknowledged links between the spread of public education and efforts to market land, I suggest that education, real estate markets, and the state have deeper and more tangled roots than scholars have imagined. Second, I contend that wrestling with these tangled roots suggests the need for scholars to reexamine the role of private markets in shaping the organisation of educational opportunity across places and time. Third, I argue that histories like Oregon’s demonstrate that ‘state’ and ‘market’ are neither stable nor mutually exclusive categories.     

The Interrelationship of Beliefs,Context, and Learning: The Case of a Teacher Reacting to Language Policy

In this article, I trace the complex ways that literacy and language policy are translated into classroom practice through the examination of a single telling case.1 I demonstrate that the role teachers play in the process can best be understood by considering a variety of factors which have been advanced in policy research to explain variations in policy implementation. These include the nature of the local school context, the beliefs and experience of the teacher, and ways in which the teacher might learn from the new policy context. I underscore these claims with empirical data related to a teacher's role in the enactment of Proposition 227-the California initiative designed to end bilingual education. The article presents select findings from a 11/2-year-long ethnographic study of a California district that allowed individual schools to develop their own Proposition 227 implementation plans. Through detailed examinations of the classroom literacy practice of a former bilingual teacher, this article illustrates how the individual qualities of a teacher played a significant role in the enactment of literacy practice.     

Changing Subjectivities,New Learning

In this article I argue that today's social transformations are of such depth and significance that they demand a fundamental rethinking of the nature of pedagogy. I address 3 closely interwoven facets of change: the nature of subjectivity, where as citizens, workers, and cultural beings, we are more and more required to be users and players rather than spectators or delegates; the growing intensity and significance of lifeworld diversity in which the local is more multifarious and the global more proximate; and the changing means of production of meaning where the semantic web is increasingly ubiquitous and its meanings multimodal. The response of a New Learning must be multifaceted and holistic. More than reactive, the New Learning must be creative, itself an agent of change rather than merely reflective of change. The transformations wrought by the New Learning must be systemic, rhetorical, and evident in the everyday practices of teaching and learning. In the article I highlight just one of a number of possible pathways, and that is the shift from a psychological/cognitivist view of learning to an epistemological one.     

Teacher Empowerment,Consultation, and the Creation of New Programs in Schools

In this article, I discuss implications of empowerment theory, as developed by community psychologists, for consultants involved in the development of new programs in schools. I consider the nature of empowerment, including the distinction between psychological and political empowerment. Factors in the planning process and the organizational environment that influence empowerment are specified, including clear and meaningful roles, self-efficacy, self-determination, impact, political and socio-emotional support, access to strategic information and resources, inspiring and shared leadership, and a strengths-based work culture. I present data from a study of teachers showing how these factors contribute to greater openness to change, and I summarize two other studies that highlight dilemmas encountered in pursuing empowerment. I conclude with action guidelines for consultants who wish to increase teacher support for new programs.     

Playing with computers,playing with ideas

In this article I react to the thoughts of Hennessey, addressing mainly the student, and Esquivel, addressing mainly the teacher. These authors present complementary, interesting, and consistent pictures of creativity education. Here I comment on a few issues that are less clear and discuss implications of these authors' work for computers in education. I conclude that teachers need to play with ideas about how they might use computers to build a culture that promotes deep, meaningful, and creative learning.Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the National Science Foundation.     

The Unity of Teaching,Learning, and Doing (November 2, 1927)

The unity of teaching, learning, and doing is our school's instructional policy. Our school's foundation stands upon this slogan, and there is no task more important than making this slogan crystal clear. But, strangely enough, I never gave a speech about this theme in my school, and my comrades never expressed serious questions about the slogan. Everyone seemingly believed this to be the daily diet at Hsiao-chuang School, and there was no point in wasting breath discussing it. But recently I ran across two things which made me feel that among our comrades there were some who really did not understand the meaning of our instructional policy. One occurred when I saw one instructor's notes. His activities were divided into three phases, called the teaching phase, the learning phase, and the doing phase. This is the separation of teaching, learning, and doing, not the unity of teaching, learning, and doing. The second occurred when I read an article by a colleague on Hsiaochuang School in Notes on Rural Education [Hsiang-chiao tzuhsün]. In it he says, "The extracurricular activities at Hsiaochuang School are the teaching, learning, and doing of agriculture." But, there is no term "extracurricular activities" in the dictionary of the school that unifies teaching, learning, and doing. "Extracurricular activities" is a declaration that life is divorced from curriculum and that teaching, learning, and doing are divorced from one another. This spring Mr. Shen Hung started an education unit for movie performers. On the enrollment announcement it said they used the "teaching," 'learning," and "doing" methods. When I saw the announcement, I felt that Mr. Hung had not completely understood the unity of teaching, learning, and doing. If he had really understood, he certainly would have written the "teaching, learning, and doing" method and certainly would not have written the "teaching," 'learning," and "doing" methods. His misunderstanding is of the same sort as the two misunderstandings mentioned above. Having experienced these two reminders in succession, I felt that unless I talked this matter over thoroughly with everyone beginning with the fundamentals, there might be a really great misunderstanding. If there is misunderstanding in thought, contradictions will inevitably arise when applying the thought. Therefore, it is imperative to discuss this matter at least once. When I came back to the country and saw that teachers here just concerned themselves with teaching and students just concerned themselves with receiving teaching, I knew for sure that there was need for reform. This situation was worse in the universities. The instructors were called professors [literally, transmitters of teaching], and everyone thought that to be called "professor" was an honor. Their methods were called pedagogy [literally, teaching transmission methods]. It was as if they were using knowledge as a rescue operation. It was at that time that I proposed replacing "teaching transmission" with "teaching and learning." At the Nanking Upper-Level Normal School discussion meeting on educational affairs, I stood at the podium for two hours but could not get it passed, and, as a result, I did not get the title Chairman of Educational Curriculum. But, in 1919, in response to an article in Shih Pao [a Shanghai newspaper] by Chiang Monlin, a leader of the new educational tide, I wrote an article on the unity of teaching and learning, advocating that teaching methods ought to be based on learning methods. At this time, the head of education in Soochow approved the use of the teaching-learning method. After the May Fourth Movement, our colleagues at Nanking Upper-Level Normal School became incomparably stronger, so I changed all of the "teaching methods" in the curriculum to "teaching-learning methods." This was the beginning of putting the unity of teaching and learning into practice. Later, when the new education system was promulgated, I went one step further and proposed that as things are done, so they are learned, and as they are learned, so they are taught. Therefore, teaching methods should be based on learning methods and learning methods on doing methods.     

Authority,Inquiry, and Education: A Response to Dewey's Critics

In this article I seek to investigate and to rebut charges that Dewey had either too authoritative a conception or use of philosophical and educational inquiry, or not enough of an authoritative use. I look specifically at two critics, one in the discipline of education and one in the discipline of history. These two are, respectively, Clarence Karier and J. P. Diggins. In the final analysis, I conclude that Dewey's talk of authority generally, and his talk of inquiry specifically, functions according to the needs and problems of the community that it take place in, and cannot, therefore, be hooked to some larger, metaphysical end or aim beyond this.     

Environmental Thought,the Language of Sustainability,and Digital Watches

The field of education, and derivatively environmental education, is filled with inherently difficult concepts, yet we have not, I believe, given sufficient attention to understanding these ideas that ground our work. For example, the term 'education for sustainability,' has gained rapid acceptance yet little critical attention has been given to the term. Before launching into a critique, I do acknowledge the importance of 'sustainability' and the usefulness of this concept in environmental thinking. Many ecological processes are not sustained-not kept going continuously. Species are going extinct at an alarming rate and whole ecosystems are at risk. So, it is important to talk about sustainability. However, I argue that this alone is not sufficient. In this article I examine limitations associated with the language of sustainability and their implications for environmental thinking. Using the idea of digital watches as a metaphor, I explore what sustainability is not. Contemporary examples from advertising, curriculum development, and 'wildlife management' will be used to illustrate the central theme.     

Beginning Reading and Developmentally Appropriate Practice (DAP): Past,Present, and Future

In this article, I address issues and trends, past, present, and future, in beginning reading instruction. I also trace the development of a related issue in the broader context of early childhood education: developmentally appropriate practice. Historical changes in reading education for young learners, as well as changes concerning guidelines for developmentally appropriate practice, are addressed. I conclude by suggesting that the fields of reading education and early childhood education share more similarities than differences. With collaborative initiatives, these two fields can strengthen their positive impact on the education of young learners as beginning readers.     

Reply to Yen,Burket, and Fitzpatrick

In their response to my article, "Item Response Theory, Vertical Scaling, and Something's Awry in the State of Test Mark," Yen, Burket, and Fitzpatrick (this issue) question the validity of my field observations. I present evidence that validates those observations. They claim that my simulation was unrealistic. I present evidence (convincing, I believe) that they are simply misinformed. They argue that Thurstone scaling has several weaknesses. I present information that should enable them to understand the procedure better and that reveals that the supposed weaknesses do not, in fact, exist. They say they are very "up front" about not being able to measure students at the extremes accurately but claim the vast majority of students are assessed well, thus implying that my use of data for students at the 2nd and 98th percentiles led to conclusions that would not be found if other segments of the score distribution were examined. I duplicated the analyses at the 15th and 85th percentile points and demonstrated that they were wrong. Yen et al. seem to be convinced that the variance of performance decreases (they use the term "homogenization") as learning progresses. Using their published data for 7 on-grade tests administered at the beginning and end of each school year, when the same on-grade test form was used-thus eliminating any confounding introduced by scaling—I show that in 67 of 77 instances the variance increased. This should serve as convincing evidence to the most doubtful person that the variance of performance increases as learning progresses. Given that there is a serious problem, as clearly illustrated in Figure 2, I suggest some avenues that research could take to address it.     

State Education Governance and Policy: Dynamic Challenges,Diverse Approaches,and New Frontiers

State governments are crucial actors in the nation's system of education governance. This issue of the Peabody Journal of Education underscores the wide-ranging roles that state governments play in the oversight, development, and implementation of elementary and secondary education policy in the United States. In this article, I consider these individual analyses by reflecting on the broader themes and patterns that they suggest. I argue that at the dawn of the 21st century, the 50 U.S. states face dynamic challenges that are testing their institutions of governance. States are also simultaneously crossing new frontiers that will influence future policy and opportunities for the nation's students.     

Sanitizing the Struggle: Barack Obama,Selma, and Civil Rights Memory

In this article, I examine Barack Obama’s use of collective memory during his 2007 campaign speech in Selma, Alabama. Specifically, I argue that Obama appeals to collective memory to challenge the Black community’s relationship to the past. He reframed the memory of the Civil Rights Movement in a manner favorable to his political aspirations. While this frame was politically expedient, it weakened Black political agency. It robbed the Black community of crucial rhetorical resources used to challenge structural inequalities in society. This article offers insight into the complex relationship between metaphor and collective memory. Specifically, I interrogate the role of metaphor as a crucial site where public memory is formed, disrupted, and rearticulated. Furthermore, I consider the implications of his use of memory for the future of the Black freedom struggle.