Assessing a State-Mandated Institutional Accountability Program: The Perceptions of Selected Community College Leaders

ABSTRACT

In the spring of 2002 we conducted a structure-focused case study at 4 North Carolina community colleges to understand how selected senior campus leaders assessed a new legislatively-mandated institutional-accountability program. Using confidential interviews and document analysis we collected, analyzed, and interpreted data that revealed clear differences in how leaders regarded the accountability program. These distinct, campus-specific perspectives are characterized as “bureaucratic meddling,” “benign intrusion,” “an opportunity to demonstrate accountability,” and “the divided leaders.” Despite these distinct perspectives, however, we also found 2 overarching themes that illuminated common reactions to the accountability program. First, leaders at 3 of the 4 institutions reported that performance ratings under the accountability program were instrumental in prompting changes in instructional programs or staffing. Second, faculty leaders at 3 institutions exhibited an apathy or unawareness of state indicators, even though some state funding was linked to measures regarding student performance.     

Student Response Technology: Empirically grounded or just a gimmick?

Student Response Technology (SRT) involves the use of hand‐held remotes by students during classroom lectures to electronically respond to questions. This study surveyed 350 students enrolled in one of 13 lower‐division university science classes taught by five different instructors who used SRT. The survey probed students’ perceptions of SRT in terms of enhancing student learning, and investigated which features of SRT students felt had the greatest/least impact on student learning. The majority of students reported that the SRT increased their content understanding, class participation, alertness, and interactions with fellow students, helped with examination preparation, provided important and immediate instructor feedback, and made class more enjoyable. Students in this study scored more positively than peer groups on survey questions related to student engagement in academic and intellectual experiences, suggesting that SRT helps to promote student engagement. Important instructor actions identified during this study that augment the enhancing effects of SRT on student learning, even in large lecture settings, include designing clear, substantive questions, reviewing correct and incorrect answers with students, and making pedagogical adjustments based on class responses.     

A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models

Disengaged item responses pose a threat to the validity of the results provided by large-scale assessments. Several procedures for identifying disengaged responses on the basis of observed response times have been suggested, and item response theory (IRT) models for response engagement have been proposed. We outline that response time-based procedures for classifying response engagement and IRT models for response engagement are based on common ideas, and we propose the distinction between independent and dependent latent class IRT models. In all IRT models considered, response engagement is represented by an item-level latent class variable, but the models assume that response times either reflect or predict engagement. We summarize existing IRT models that belong to each group and extend them to increase their flexibility. Furthermore, we propose a flexible multilevel mixture IRT framework in which all IRT models can be estimated by means of marginal maximum likelihood. The framework is based on the widespread Mplus software, thereby making the procedure accessible to a broad audience. The procedures are illustrated on the basis of publicly available large-scale data. Our results show that the different IRT models for response engagement provided slightly different adjustments of item parameters of individuals’ proficiency estimates relative to a conventional IRT model.     

Le diagnostic du potentiel d’apprentissage par LPAD: Une etude de la fidélité

The present study was intended to test some of the psychometric properties of Feuerstein’s Learning Potential Assessment Device (LPAD), in particular the reliability of the Set Variations II, considered to assess modificability or learning capacity. Thirty 10-year-old children were studied: an experimental group (N=20) submitted to two training sessions with Set Variations II and a control group (N=10). Both groups were administered three times the Standard Progressive Matrices (SPM). The experimenter’s interventions during training were semi-standardized, based on an examination guide, and the helps given were classified in four types. Analyses dealt with the number and type of helps provided as well as with errors during training. Correlations between these indices for the two sessions of training were rather low, indicating weak stability, in particular with respect to the helps provided. Correlations were also computed between the gains in SPM for the three administrations, indicating low stability in amount of progression. Further, it was also noted that the experimental group did not progress more than the control group. Conclusions are drawn with respect to reliability and validity of the Set Variations Test, as well as, on a more theoretical level, with respect to the concept of cognitive modifiability.     

Tier 3 specialized writing instruction for students with dyslexia

Two instructional experiments used randomized, controlled designs to evaluate the effectiveness of writing instruction for students with carefully diagnosed dyslexia, which is both an oral reading and writing disorder, characterized by impaired word decoding, reading, and spelling. In Study 1 (4th to 6th grade sample and 7th to 9th grade sample), students were randomly assigned to orthographic or morphological spelling treatment but all students were taught strategies for planning, writing, and reviewing/revising narrative and expository texts. Both treatments resulted in improvement on three measures of spelling and one measure of composition. Morphological treatment resulted in better improvement in spelling pseudowords, whereas orthographic treatment resulted in better improvement in rate of reading pseudowords. In Study 2 (4th to 6th grade sample), students were randomly assigned to explicit language treatment (phonological working memory + phonological-orthographic spelling + science report writing training) or nonverbal problem solving treatment (virtual reality-based computer simulation, drawing maps, and building clay models). Both treatments used science content material and resulted in significant improvement in spelling and reading pseudowords (accuracy and rate). The surprising finding was that hands-on, engaging science problem solving led to more improvement in phonological working memory than did specialized phonological instruction. Only when spelling instruction emphasized orthography or morphology (Study 1) did real word spelling improve. Preparation of this chapter was supported by Grant Nos. HD25858 and P50 33812 from the National Institute of Child Health and Human Development (NICHD). In Study 1 teaching assistants included Sylvia Abbott, Marci Anderson-Youngstrom, Belle Chennault, Kate Eschen, Noelia Garcia, Sandra Hiramatsu, Lynn Junell, Julia Kim, Stephanie King, Shin-Ju (Cindy) Lin, Rebecca Metzger, Heather Murphy, Jennifer Norton, Sue Palewicz, Rebecca Pirie, and Suzanna West. Also on the assessment team were Sylvia Abbott, Belle Chenault, Kate Eschen, and Rebecca Pirie. Certified teachers Diana Hoffer and Christina Johnson implemented the composition lessons written by the first author. In Study 2, teaching assistants included Nicole Alston-Abel, Leah Altemeier, Wade Altom, Marci Anderson-Youngstrom, Annie Boyd, Nicole Garcia, Laura Gould, Marta Holsinger, Yen-Ling Lee, Shin-Ju (Cindy) Lin, Laurie McCarthy, Heather Murphy, Hema O’Shea, Michelle Proux, Kristine Tardiff, and Kari Terjeson. Janine Jones supervised the assessment team. Margaret McShane and Doug Stock served as administrative assistants.     

Pre-exercise antioxidant enzyme activities determine the antioxidant enzyme erythrocyte response to exercise

Free radical production increases during exercise and oxidative damage occurs in several tissues. We examined the effects of three different exercise tests on the pattern of change of erythrocyte enzyme antioxidant activities. The tests were a short maximal exercise test, a submaximal prolonged exercise test and a cycling stage during competition. The participants were amateur and professional cyclists with different training statuses and different basal erythrocyte antioxidant enzyme activities. The maximal test produced no changes in the erythrocyte antioxidant enzyme activities of amateur sportsmen. The submaximal test, performed at 80% of maximal oxygen uptake, decreased erythrocyte catalase (12%), glutathione peroxidase determined with H2O2 (14%) and glutathione reductase (16%); superoxide dismutase activity increased by about 25%. The cycling stage performed by professional cyclists increased erythrocyte catalase (29%) and glutathione reductase (10%) activities. The in vivo changes in glutathione reductase activity were confirmed by in vitro measurements: hydrogen peroxide decreased and the presence of catalase increased the activity of this enzyme. In conclusion, we suggest that the different erythrocyte antioxidant enzyme responses to diverse exercise tests can be explained by the effects of hydrogen peroxide and the superoxide anion on the antioxidant enzyme activities in erythrocytes.     

THE DEVELOPMENT OF PERSONALITY,ABILITIES AND SOCIAL RELATIONS IN A SPECIAL CLASS

Two years have passed since our first report on the project with primary school children at Törökszentmiklós. New results indicate that participation in the project has accelerated the development of abilities, has led to mainly favourable personality changes and has promoted the formation of advantageous social structures in the classroom. In the future, in addition to increasing support for the children's personal wellbeing (for instance, provision of more free time, promotion of communication skills), the work of four terms will be completed in three.