A Study in Correlation of Allied English Abilities

ABSTRACT

The authors explored teachers' and principals’ perceptions of the feedback report from the National Tests in Trinidad and Tobago and the extent to which they used the report in making curricular decisions to impact student learning. The sample comprised 133 primary school teachers (79 from low-performing and 54 from high-performing schools) and 10 principals. Results of the quantitative and qualitative data indicated that while many teachers were uncomfortable with interpreting the data presented in the report, teachers in higher performing schools were more inclined through department-wide collaboration to use the report to make pedagogical and curricular decisions. The major conclusion drawn was the need for teacher training in the use and interpretation of assessment data. Other issues emerging from the data and a possible subject for further research included the branding of schools as good schools and bad schools based on the school performance on the tests.     

A utility model for teaching load decisions in academic departments

The authors model the class size and teaching load decisions of academic departments in terms of a departmental utility function. Utility is postulated to be asymmetric around class size and teaching load norms, and variables for curricular structure, disciplinary domain, and institutional type are taken into account. Maximization of the utility function produces decision rules for the number of sections to be offered for each course, and hence the faculty's overall teaching load. A nonlinear estimator is developed for the decision rules' parameters and applied to data from four liberal arts colleges and two research universities. Results are consistent with theories about faculty discretionary time and with expectations about the effects of curricular structure on class size. The paper concludes with a discussion about the effects of enrollment uncertainty on faculty load decisions.     

Curricular Differences and Unidimensionality of Achievement Test Data: An Exploratory Analysis

The purpose of this study was to investigate whether a linear factor analytic method commonly used to investigate violation of the item response theory (IRT) unidimensionality assumption is sensitive to measurable curricular differences within a school district and to examine the possibility of differential item performance for groups of students receiving different instruction. For grades 3 and 6 in reading and mathematics, personnel from two midwestern school systems that regularly administer standardized achievement tests identified the formal textbook series used and provided ratings of test-instructional match for each school building (classroom). For both districts, the factor analysis results suggested no differences in percentages of variance for large first factors and relatively small second factors across ratings or series groups. The IRT analyses indicated little, if any, differential item performance for curricular subgroups. Thus, the impact of factors that might be related to curricular differences was judged to be minor.     

Does Test Item Performance Increase with Test-to-Standards Alignment?

Variation in test performance among examinees from different regions or national jurisdictions is often partially attributed to differences in the degree of content correspondence between local school or training program curricula, and the test of interest. This posited relationship between test-curriculum correspondence, or “alignment,” and test performance is usually inferred from highly distal evidence, rather than directly examined. Utilizing mathematics standards content analysis data and achievement test item data from ten U.S. states, we examine the relationship between topic-specific alignment and test item performance. When a particular item’s content type is emphasized by the standards, we find evidence of a positive relationship between the alignment measure and proportion-correct test item difficulty, although this effect is not consistent across samples. Implications of the results for curricular achievement test development and score interpretation are discussed.     

Simultaneous decisions at study: time allocation, ordering, and spacing

Learners of all ages face complex decisions about how to study effectively. Here we investigated three such decisions made in concert—time allocation, ordering, and spacing. First, college students were presented with, and made judgments of learning about, 16 word-synonym pairs. Then, when presented with all 16 pairs, they created their own study schedule by choosing when and how long to study each item. The results indicated that (a) the most study time was allocated to difficult items, (b) relatively easy items tended to be studied first, and (c) participants spaced their study at a rate significantly greater than chance. The spacing data, which are of particular interest, differ from previous findings that have suggested that people, including adults, believe massing is more effective than spacing.     

Stratified and Maximum Information Item Selection Procedures in Computer Adaptive Testing

In this study we evaluated and compared three item selection procedures: the maximum Fisher information procedure (F), the a-stratified multistage computer adaptive testing (CAT) (STR), and a refined stratification procedure that allows more items to be selected from the high a strata and fewer items from the low a strata (USTR), along with completely random item selection (RAN). The comparisons were with respect to error variances, reliability of ability estimates and item usage through CATs simulated under nine test conditions of various practical constraints and item selection space. The results showed that F had an apparent precision advantage over STR and USTR under unconstrained item selection, but with very poor item usage. USTR reduced error variances for STR under various conditions, with small compromises in item usage. Compared to F, USTR enhanced item usage while achieving comparable precision in ability estimates; it achieved a precision level similar to F with improved item usage when items were selected under exposure control and with limited item selection space. The results provide implications for choosing an appropriate item selection procedure in applied settings.     

Sensitivity of Cross-State Assessment Item Difficulty to Differences in State Curricular Content Standards

It has long been argued that U.S. states’ differential performance on nationwide assessments may reflect differences in students’ opportunity to learn the tested content that is primarily due to variation in curricular content standards, rather than in instructional quality or educational investment. To quantify the effect of differences in states’ intended curricular goals on test item performance in the mid-to-late 2000s, we use fractional logit regression of state-specific mathematics item difficulty values on a measure of content emphasis in state elementary school mathematics curricular standards documents. Finding weak but positive associations between content emphasis in state standards and proportion-correct item difficulty, we conclude that variations in states’ intended curriculum content, alone, appear to have had limited influence on cross-state mathematics test item performance during the time frame examined. Implications for cross-state assessment are discussed.     

Multidimensional CAT Item Selection Methods for Domain Scores and Composite Scores With Item Exposure Control and Content Constraints

The intent of this research was to find an item selection procedure in the multidimensional computer adaptive testing (CAT) framework that yielded higher precision for both the domain and composite abilities, had a higher usage of the item pool, and controlled the exposure rate. Five multidimensional CAT item selection procedures (minimum angle; volume; minimum error variance of the linear combination; minimum error variance of the composite score with optimized weight; and Kullback‐Leibler information) were studied and compared with two methods for item exposure control (the Sympson‐Hetter procedure and the fixed‐rate procedure, the latter simply refers to putting a limit on the item exposure rate) using simulated data. The maximum priority index method was used for the content constraints. Results showed that the Sympson‐Hetter procedure yielded better precision than the fixed‐rate procedure but had much lower item pool usage and took more time. The five item selection procedures performed similarly under Sympson‐Hetter. For the fixed‐rate procedure, there was a trade‐off between the precision of the ability estimates and the item pool usage: the five procedures had different patterns. It was found that (1) Kullback‐Leibler had better precision but lower item pool usage; (2) minimum angle and volume had balanced precision and item pool usage; and (3) the two methods minimizing the error variance had the best item pool usage and comparable overall score recovery but less precision for certain domains. The priority index for content constraints and item exposure was implemented successfully.     

Rater Agreement in Test‐to‐Curriculum Alignment Reviews

During the development of large‐scale curricular achievement tests, recruited panels of independent subject‐matter experts use systematic judgmental methods—often collectively labeled “alignment” methods—to rate the correspondence between a given test's items and the objective statements in a particular curricular standards document. High disagreement among the expert panelists may indicate problems with training, feedback, or other steps of the alignment procedure. Existing procedural recommendations for alignment reviews have been derived largely from single‐panel research studies; support for their use during operational large‐scale test development may be limited. Synthesizing data from more than 1,000 alignment reviews of state achievement tests, this study identifies features of test–standards alignment review procedures that impact agreement about test item content. The researchers then use their meta‐regression results to propose some practical suggestions for alignment review implementation.     

Item Selection in Computerized Adaptive Testing: Should More Discriminating Items be Used First?

During computerized adaptive testing (CAT), items are selected continuously according to the test-taker's estimated ability. The traditional method of attaining the highest efficiency in ability estimation is to select items of maximum Fisher information at the currently estimated ability. Test security has become a problem because high-discrimination items are more likely to be selected and become overexposed. So, there seems to be a tradeoff between high efficiency in ability estimations and balanced usage of items. This series of four studies with simulated data addressed the dilemma by focusing on the notion of whether more or less discriminating items should be used first in CAT. The first study demonstrated that the common maximum information method with Sympson and Hetter (1985) control resulted in the use of more discriminating items first. The remaining studies showed that using items in the reverse order (i.e., less discriminating items first), as described in Chang and Ying's (1999) stratified method had potential advantages: (a) a more balanced item usage and (b) a relatively stable resultant item pool structure with easy and inexpensive management. This stratified method may have ability-estimation efficiency better than or close to that of other methods, particularly for operational item pools when retired items cannot be totally replenished with similar highly discriminating items. It is argued that the judicious selection of items, as in the stratified method, is a more active control of item exposure, which can successfully even out the usage of all items.     

The learning potential of online student-constructed tests with citing peer-generated questions

In view of contribution-based pedagogy and observational learning theory, students’ perceived uses, preferences, usage, and selection considerations with regard to citing peers’ work were examined in an online learning environment targeting student-constructed tests. Data were collected from 84 fifth-grade students who participated in online student-constructed tests with and without citing in an 11-week study. Quantitative and qualitative data in response to an end-of-session questionnaire and actual online citing behaviour were analyzed. Several major findings were obtained. First, significantly more participants supported and preferred “citing” over “no citing” for online student-constructed tests. Second, data with regard to perceived uses, preferences, and reported usage all supported the potential of citing for providing an observational learning space. Third, citing allowed the participants to attend to areas pinpointed by their peers but initially ignored by them, thus making social construction of knowledge possible. Fourth, the quality and the author of the item are the two determining factors affecting citing decisions. Fifth, a statistically significant positive correlation between students’ academic achievement and their generated questions cited by peers was confirmed. Finally, actual online citing behaviour varied greatly among participants, with the majority using the citing function during online test-construction to various extents.     

Planning and Doing in Professional Teaching Practice. A Study with Early Childhood Education Teachers Working with ICT (3–6 years)

Planning is one of the professional tasks teachers have to carry out before their direct action in the classrooms. This planning is closely interrelated to the way teachers teach. The question about how and why teachers reach their decisions in their pre-class planning is a classical one in the research into curricular design and development. The aim of this paper will therefore be to establish whether there is a relationship between curricular planning and curricular practices, studying how nine early childhood education teachers using an ICT resource plan their actions and execute them. For the research, we obtained video recordings of classroom practices and interviewed the teachers just before they went into class. By applying qualitative data analysis, we have been able to identify the elements taken into consideration when the teachers make decisions in lesson, their conceptions about ICT, and the types of activity that are held in all the classes. The results confirm the conclusions reached by previous studies on the relationships between planning and doing, in the sense that the activities, understood to be teaching strategies, are the ones that link the design of what is to be done and direct action. Likewise, the results ratify prior research on the role of materials and resources as aspects that teachers can rely on for support in the management and presentation of classroom tasks and content. The introduction of ICT does not modify the teachers’ curricular planning and development.     

Scaling,Linking, and Reporting in a Periodic Assessment System

A new entry in the testing lexicon is through‐course summative assessment, a system consisting of components administered periodically during the academic year. As defined in the Race to the Top program, these assessments are intended to yield a yearly summative score for accountability purposes. They must provide for both individual and group proficiency estimates and allow for the measurement of growth. They must accommodate students who vary in their patterns of curricular exposure. Because they are meant to provide actionable information to teachers they must be instructionally sensitive, so item‐operating characteristics can be expected to change relative to one another as a function of patterns of curricular exposure. This paper discusses methodology one can draw upon to tackle this ambitious collection of inferences. We consider a modeling framework that consists of an item response theory component and a population component, as in the National Assessment of Educational Progress, and show how performance and growth could be expressed in terms of expected performance on a market basket of tasks. We discuss conditions under which modeling simplifications might be possible and discuss studies that would be needed to fit models, estimate parameters, and evaluate data requirements.     

Effects of Curricular Differences on Achievement Test Data at Item and Objective Levels

The objectives of this study were to examine the impact of different curricula on standardized achievement test scores at item and objective levels and to determine if different curricula generate different patterns of item factor loadings. School buildings from a middle-sized district were rated regarding the degree to which their curricula matched the content of the standardized test, and the actual textbook series used within each building (classroom) was determined. Covariate analyses of objective scores and plots and correlations of item p values indicated very small, nonsignificant differential effects across ratings and textbook series. Factor patterns indicated no curricular effects on large first factors. These findings parallel the results of a previous study conducted at the subtest level. We conclude that educators need not be unduly concerned about the impact of specific and generally small differences in curricular offerings within a district on standardized test scores or inferences to a broad content domain.     

Elementary school teachers’ uses of mathematics curricular resources

With the adoption of new content standards, teachers are often left without adequate curriculum resources. This study examined how educators used their curricular resources to teach new mathematics standards in the USA. Analyses of open-ended survey responses from 257 teachers and teacher–leaders in Grades 3 through Grade 5 indicated that every educator reported supplementing their districts’ or schools’ primary curricular resources with other materials. These supplements primarily included resources found for free on websites and resources that claimed to be aligned to the new standards, but varied in terms of alignment to national standards for effective mathematics curriculum. Implications for this study include further research on how teachers make decisions regarding curriculum resources as well as increasing teachers’ access to quality curriculum materials that can support students’ mathematical learning.     

Curriculum integration = course disintegration: What does this mean for anatomy?

Many basic scientists including anatomists are currently involved in decisions related to revisions of the undergraduate medical curriculum. Integration is a common theme in many of these decisions. As described by Harden, integration can occur along a multistep continuum from independent, discipline‐based courses to a completely interdisciplinary curriculum. For anatomy, each derivative of curricular integration can be shown to involve progressive disruptions of the temporal and topographical relationship between organ systems in a body region, of the temporal relationship with other courses in a harmonized curriculum, and of the relationships between components of organ systems when integration is implemented in thematic curricula. Drawing from our experience teaching in various types of integrated medical curricula, we encourage readers to proceed cautiously with their curricular decisions because each one can have gains and losses that may impact learning in the new format. Anat Sci Educ. © 2013 American Association of Anatomists.     

An update on the status of anatomical sciences education in United States medical schools

Curricular changes continue at United States medical schools and directors of gross anatomy, microscopic anatomy, neuroscience/neuroanatomy, and embryology courses continue to adjust and modify their offerings. Developing and supplying data related to current trends in anatomical sciences education is important if informed decisions are going to be made in a time of curricular and course revision. Thus, a survey was sent to course directors during the 2012–2013 academic years to gather information on total course hours, lecture and laboratory hours, the type of laboratory experiences, testing and competency evaluation, and the type of curricular approach used at their institution. The data gathered were compared to information obtained from previous surveys and conclusions reached were that only small or no change was observed in total course, lecture and laboratory hours in all four courses; more gross anatomy courses were part of an integrated curriculum since the previous survey; virtual microscopy with and without microscopes was the primary laboratory activity in microscopic anatomy courses; and neuroscience/neuroanatomy and embryology courses were unchanged. Anat Sci Educ 7: 321–325. © 2014 American Association of Anatomists.     

数学课程弹性化的初步研究

数学课程弹性化是当今世界各国数学课程改革的新趋势。日本、韩国和英国弹性化数学课程都有各自的具体表现方式。数学课程弹性化的丰富内涵应当从数学课程发展维度、数学课程项目维度、数学课程对象维度进行认识。构建我国弹性化的数学课程应当坚持3条原则：分析各国数学课程弹性化的新近发展，吸取先进的课程理念；研究我国数学课程弹性化发展历程，把握我国数学课程弹性化发展的主要趋势；结合数学教育现状，全方位构建我国弹性化的数学教学课程体系。     

Extra-Curricular Activity and the Transition from Higher Education to Work: A Survey of Graduates in the United Kingdom

This paper focuses on the effects of extra‐curricular activity on graduates' transition from higher education to the labour market. The study is based on a survey of 119 graduates conducted in 2004 in the UK. The data gathered cover a large range of social and leisure activities that the graduates carried on while students at their universities. Several aspects of their transitional process from student to worker are also covered. Data were analysed by means of linear and logistic regression models. Results show that extra‐curricular activity has a significant influence on the transition process. First, extra‐curricular experience gives access to better occupational status but lengthens the period of unemployment preceding the first job. Second, as compared with the most frequently observed extra‐curricular behaviour, two profiles could be distinguished: the one better performing than average, and the other worse performing. Results suggest extra‐curricular strategies to better enable graduates' effective transition to work.     

Sensitivity of Item Difficulties to Curricular Validity

The issue of the match of what is taught to what is tested recently has received increased emphasis. Many recent studies have examined the extent of the mismatch between a local set of objectives/instruction and the content of a nationally standardized test. Phillips and Mehrens (1985, 1987; see also Mehrens & Phillips, 1986) have conducted a series of studies that show that, within a district, different textbook series and informal curricula generally have no significant impact on test total, objective, or item scores. The present study explored further the curricular validity differences across textbooks and the relationship between item statistics and measures of curricular validity. In particular, this study sought to determine whether item p-values appear to be related to measures of curricular validity based on three mathematics textbook series used at a given grade and in a previous grade. The results of this study indicated that the textbooks differed somewhat in content coverage when using a 180-cell matrix classification. However, these differences were not great, especially when the textbooks in both grades 5 and 6 were considered. Further, all three series covered almost all of the 53 cells in the matrix covered by the Stanford Achievement Test, and the differences that did exist in textbook content coverage had no observable relationship to differences in item p-values. In addition, the mean difficulty level of the Stanford Achievement Test items classified by cell were similar for students using the different textbooks, despite the differences among textbooks in location, presentation, and organization of content.