Use of Knowledge, Skill, and Ability Statements in Developing Licensure and Certification Examinations

The task inventory approach is commonly used in job analysis for establishing content validity evidence supporting the use and interpretation of licensure and certification examinations. Although the results of a task inventory survey provide job task-related information that can be used as a reliable and valid source for test development, it is often the knowledge, skills, and abilities (KSAs) required for performing the tasks, rather than the job tasks themselves, which are tested by licensure and certification exams. This article presents a framework that addresses the important role of KSAs in developing and validating licensure and certification examinations. This includes the use of KSAs in linking job task survey results to the test content outline, transferring job task weights to test specifications, and eventually applying the results to the development of the test items. The impact of using KSAs in the development of test specifications is illustrated from job analyses for two diverse professions. One method for transferring job task weights from the job analysis to test specifications through KSAs is also presented, along with examples. The two examples demonstrated in this article are taken from nursing certification and real estate licensure programs. However, the methodology for using KSAs to link job tasks and test content is also applicable in the development of teacher credentialing examinations.     

Self-Regulation and Task Engagement as Predictors of Emergent Language and Literacy Skills

Research Findings: A growing emphasis in the literature on children’s self-regulation signals the need for increased understanding of the ways in which young children become active players in the acquisition of knowledge. In particular, self-regulation may be linked to subsequent academic achievement through greater engagement with the learning tasks and activities made available in the preschool classroom. This study tested preschoolers’ (N = 603) observed task engagement in the classroom as mediating the relations between directly assessed self-regulation and changes in their language and literacy outcomes during the preschool year. Findings indicate that self-regulation is directly related to observed task engagement as well as changes in a host of language and literacy skills. Engagement with tasks and activities in the classroom also partially mediates the association between self-regulation and changes in expressive vocabulary. Mediation through task engagement was not found for receptive language or early literacy skills. Practice or Policy: Findings suggest that the development and evaluation of clearly articulated preschool curricula designed to promote academic achievement by fostering self-regulation is an important direction for future research.     

The performance of college students on piaget-type tasks dealing with distance,time, and speed

Research was done to probe and extend Piaget's theory of the conception of speed. Specifically tested was the hypothesis that there is no hierarchical relationship in performance on the following Piaget-type tasks: conservation of distance, asymmetric series of speeds, one-to-many (circular) speeds, symmetric speeds, time, and proportional reasoning. The research also tested the gender-related performance on these six tasks. One hundred freshman and sophomore college students were shown demonstrations of equipment individually for each of the six tasks. A set of open-ended questions based on the demonstrations was administered to each subject in a 45-minute interview. Subjects were scored as “pass” on a given task if they manipulated objects in certain ways and gave specific types of explanations. A scalogram analysis of the data yielded a z-score of — 3.7 which indicated that subjects passed the task in a certain sequence. The tasks were found to form a unidimensional scale and to be increasingly difficult in the order listed above. A chi-square test for two independent samples showed a significant difference (alpha = 0.05) in performance between males and females on all speed tasks. No significant differences in performance between males and females were found for the distance and time tasks.     

“I Don't Have Time to Dig Back Through This”: The Role of Semantic Search in Supporting Physician Information Seeking in an Electronic Health Record

The purpose of the study was twofold: to understand how usability affected physicians' performance as they used an electronic health record (EHR) and to ascertain whether use of a semantic search feature would better support physician performance during an information‐seeking task. Participants (n = 10) were asked to complete two search tasks to find pertinent patient information. In the first task, participants located the information as they normally would (through browsing the EHR). In the second task, participants employed a semantic search tool. Upon task completion, participants were interviewed to further understand their perceptions and information‐seeking behavior in an EHR. Statistically significant results confirmed that participants were able to more efficiently navigate through an EHR in terms of time and number of clicks when using the semantic search feature. Moreover, participants were more confident in the accuracy of their answers when using semantic search compared with the browsing method. Implications for practice are discussed.     

Understanding students’ conceptions of task assignments

Building on research emphasizing the importance of task in reading performance, the present study examines students’ conceptions or definitions of five common academic task assignments (i.e., argument, essay, opinion, summary and research tasks). Results showed that while students have robust schema for some task assignments (e.g., argument, research report), citing a large variety of task elements in defining each of these assignment, other task assignments are only superficially conceptualized by learners (e.g., opinion, essay). While prior work has demonstrated the important role of task assignment in students’ text processing and performance, this is among the first studies to associate emergent differences not only with the tasks assigned but also with students’ task conceptions. Given the emphasis placed on argument tasks in supporting deeper level processing, we were especially interested in students’ conceptions of argument tasks vis-à-vis summary tasks. Students’ conceptions of these and other task assignments were compared using Cochran’s Q, a non-parametric alternative to the repeated measures ANOVA. Finally, linear regression found students’ conceptions of an argument task to be associated with performance on an argument writing task assignment. For instance, students considering argument construction to require citation use was associated with their inclusion of citations in their written responses. This study is among the first to examine students’ own conceptions of common task assignments and to link task conceptions with performance. Implications for instruction are discussed.     

Seizing the Opportunity to Create Uncertainty in Learning Mathematics

The paper is a reflective account of the design and implementation of mathematical tasks that evoke uncertainty for the learner. Three types of uncertainty associated with mathematical tasks are discussed and illustrated: competing claims, unknown path or questionable conclusion, and non-readily verifiable outcomes. One task is presented in depth, pointing to the dynamic nature of task design, and the added value stimulated by the uncertainty component entailed in the task in terms of mathematical and pedagogical musing.     

The Relations Between Children’s Reading Comprehension, Working Memory, Language Skills and Components of Reading Decoding in a Normal Sample

The primary aim of the current study was to identify the strongest independent predictors of reading comprehension using word reading, language and memory variables in a normal sample of 180 children in grades 3–5, with a range of word reading skills. It was hypothesized that orthographic processing, receptive vocabulary and verbal working memory would all make independent contributions to reading comprehension. The contributions of reading speed, receptive grammatical skills, exposure to print, visuospatial working memory and verbal learning and retrieval (a measure of longer-term retention) were also investigated. Working memory tasks that required the processing and storage of numerical and spatial material were used. One of the numerical working memory tasks was based on the number span task developed by Yuill, Oakhill, and Parkin British Journal of Psychology, 1989, 80, 351–361. A visuospatial equivalent of that task was developed from the forward Corsi block task [Corsi, Abstracts International, 1973, 34, 891]. The results revealed that, after controlling for age and general intellectual ability, the word reading and the language variables had a much stronger relation with reading comprehension than the memory variables. The strongest independent predictor of reading comprehension was orthographic processing since it captured variance in both word reading, language skills and verbal working memory. The forward Corsi task and performance on a measure of verbal learning and retrieval each made small independent contributions to reading comprehension but the contribution of verbal working memory was not significant. It was concluded that tasks measuring the interplay between short-term and long-term memory, in which new information is combined with information already stored in long-term memory, may better predict reading comprehension measured with the text available than working memory tasks which only have a short-term memory component.     

Helping Primary Student Teachers Understand Pupils' Learning: Exploring the student‐mentor interaction

The school-based task is firmly established in teacher education in the UK. In the study reported, primary student teachers carried out tasks exploring children's learning in science and were told to discuss the outcomes with their classteacher-mentors. The tasks and their university-based training followed a "constructivist" approach to learning science. The mentors' role in debriefing school-based tasks has been assumed to be one of "reflective practitioner" able to unpack issues of pedagogy with the student teacher. The research reported here shows that discussions often lacked depth or reflected a superficial or different understanding of constructivism by mentors that conflicted with the student teachers' aspirations. The implications of the research for future practice in teacher education and specifically for the training of mentors in science are discussed.     

建构数学学习发展性评价体系的实践与思考

发展性评价体系中一个重要的工作就是开发多样化的评价工具和方法。在课题研究中我们重点开展了如下七个方面的评价工具的开发与实践:日常观察,等级和分项测试,成长记录袋,表现性任务,通过浓缩孩子成长特点的评价,二次评价,通过家长、学校合作,取得了较好的效果。     

Lack of consistent individual differences in rats on tasks that require response inhibition

Hungry rats performed four different tasks which all required responding to be withheld for food reinforcement to be obtained. The tasks were the following: an omission schedule, a go/no-go procedure, a differential reinforcement of low rate schedule, and a choice task in which the subject could respond for a small immediate reward or a large delayed reward. Each rat experienced each task twice. For each task, the rank ordering of the subjects’ performance on the test and retest was highly consistent, indicating the reliability of the task for measuring a particular “ability.” However, there were no consistencies in the performance of individual rats between tasks. This suggests that claims regarding consistent individual differences in brain-lesioned nonhumans across a range of such tasks are not supported in intact subjects.     

Correlates of formal reasoning: Content and problem effects

Piaget's structural theory of formal thought suggests that a general construct of formal reasoning exists. The content of the task and type of problem employed are often ignored in Piagetian based studies but are important for generalizing findings to other studies and to educational problems. The study reported here examines content and problem effects of formal thought in 13-year-old adolescents. Specifically, three controlling variables tasks with different content and two question type tests (analysis and controlling questions) were administered to 120 seventh graders. Ability measures and personality dimensions associated with formal reasoning in the literature are used to clarify what formal reasoning is and how content and problem type are involved in formal reasoning. Significant main effects were found for problem and content effects (p = 0.001), and a significant interaction was found between the two (p = 0.001). Across the three tasks general ability, field dependency, and locus of control were consistently and significantly related to the controlling questions. However, only measures of field dependency were related to the analysis questions. Combinations of ability and personality factors were found to be uniquely related to each task, within each question type. Results have implications for a theory of formal thought and the teaching of the controlling variables strategy.     

An NCME Instructional Module on Developing and Administering Practice Analysis Questionnaires

The purpose of a credentialing examination is to assure the public that individuals who work in an occupation or profession have met certain standards. To be consistent with this purpose, credentialing examinations must be job related, and this requirement is typically met by developing test plans based on an empirical job or practice analysis. The purpose of this module is to describe procedures for developing practice analysis surveys, with emphasis on task inventory questionnaires. Editorial guidelines for writing task statements are presented, followed by a discussion of issues related to the development of scales for rating tasks and job responsibilities. The module also offers guidelines for designing and formatting both mail-out and Internet-based questionnaires. It concludes with a brief overview of the types of data analyses useful for practice analysis questionnaires.     

Selecting learning tasks: Effects of adaptation and shared control on learning efficiency and task involvement

Complex skill acquisition by performing authentic learning tasks is constrained by limited working memory capacity [Baddeley, A. D. (1992). Working memory. Science, 255, 556–559]. To prevent cognitive overload, task difficulty and support of each newly selected learning task can be adapted to the learner’s competence level and perceived task load, either by some external agent, the learner herself, or both. Health sciences students (N = 55) participated in a study using a 2 × 2 factorial design with the factors adaptation (present or absent) and control over task-selection (program control or shared control). As hypothesized, adaptation led to more efficient learning; that is, higher learning outcomes combined with less effort invested in performing the learning tasks. Shared control over task-selection led to higher task involvement, that is, higher learning outcomes combined with more effort directly invested in learning. Adaptation also produced greater task involvement.     

Analyzing Mathematical Tasks: A Catalyst for Change?

In this study we investigate a strategy for engaging high school mathematics teachers in an initial examination of their teaching in a way that is non-threatening and at the same time effectively supports the development of teachers’ pedagogical content knowledge [Shulman (1986). Educational Researcher, 15(2), 4–14]. Based on the work undertaken by the QUASAR project with middle school mathematics teachers, we engaged a group of seven high school mathematics teachers in learning about the Levels of Cognitive Demand, a set of criteria that can be used to examine mathematical tasks critically. Using qualitative methods of data collection and analysis, we sought to understand how focusing the teachers on critically examining mathematical tasks influenced their thinking about the nature of mathematical tasks as well as their choice of tasks to use in their classrooms. Our research indicates that the teachers showed growth in the ways that they consider tasks, and that some of the teachers changed their patterns of task choice. Further, this study provides a new research instrument for measuring teachers’ growth in pedagogical content knowledge. An earlier version of this paper was presented at the American Educational Research Association Annual Meeting, New Orleans, LA, April 2002.     

The negative impact of goal-oriented instructions

The phrasing of task instructions can facilitate or hinder the learning process. In this study, three groups of participants (N = 526) performed a foreign vocabulary memorization task, with modified instructions for each group. The instructions were either learning oriented, encouraging participants to improve their abilities; outcome oriented, prompting participants to achieve a positive evaluation of their performance; or neutral, with no goal orientation, for the control group. Participants’ performance in the task was measured along with several factors pertinent to the learning process. Results showed that learning-oriented instructions led to lower performance levels, while outcome-oriented instructions reduced participants’ language risk-taking, both of which negatively impact learning. The control group had the best overall results, indicating that it is better to refrain from using goal-oriented instructions in learning tasks.     

The differential effects of task complexity on domain-specific and peer assessment skills

In this study the relationship between domain-specific skills and peer assessment skills as a function of task complexity is investigated. We hypothesised that peer assessment skills were superposed on domain-specific skills and will therefore suffer more when higher cognitive load is induced by increased task complexity. In a mixed factorial design with the between-subjects factor task complexity (simple, n?=?51; complex, n?=?59) and within-subjects factor task type (domain-specific, peer assessment), secondary school students studied four integrated study tasks, requiring them to learn a domain-specific skill (i.e. identifying the six steps of scientific research) and to learn how to assess a fictitious peer performing the same skill. Additionally, the students performed two domain-specific test tasks and two peer assessment test tasks. The interaction effect found on test performance supports our hypothesis. Implications for the teaching and learning of peer assessment skills are discussed.     

Comparing Class- and Task-Level Measures of Achievement Goals

Models of achievement goals suggest that different tasks and contexts influence the goals students adopt at a given time. However, many studies of achievement goals rely on measures assessed at the class level, analyze results with a variable-centered approach, and employ self-report questionnaires, which may reduce understanding of the contextual factors that arise as students interact with tasks. We compared a behavioral, task-level measure of middle school students' achievement goals to a self-reported, class-level measure and analyzed both measures' relations to task performance and quarterly grades using both variable-centered and person-centered approaches. Task-level goals predicted task performance but not quarterly grades, while class-level goals predicted grades but not task performance. The two measures of achievement goals were not related. This work demonstrates the importance of measuring goals at multiple levels and suggests opportunities for changing students' behaviors and achievement through motivation at the task level.     

Associations and indirect effects between LEGO® construction and mathematics performance

There is a known association between LEGO® construction ability and mathematics achievement, yet the mechanisms which drive this association are largely unknown. This study investigated the spatial mechanisms underlying this association, and whether this differs for concrete versus digital construction. Between January 2020 and July 2021, children aged 7–9 years (N = 358, 189 female, ethnicity not recorded) completed spatial and mathematics tasks, and either a concrete or digital Lego construction task. Mediation analyses examining direct and indirect pathways (through spatial skills) between Lego construction ability and mathematics explained 8.4% to 26.6% of variance in mathematics scores. Exploratory moderated mediation analyses revealed that only the indirect path through mental rotation differed between Lego conditions. Findings are discussed in relation to theories of spatial-numerical associations and the potential of Lego training for mathematics improvement.