What Do HPT Consultants Do for Performance Analysis?

This study was conducted to contribute to the field of Human Performance Technology (HPT) through the validation of the performance analysis process of the International Society for Performance Improvement (ISPI) HPT model, the most representative and frequently utilized process model in the HPT field. The study was conducted using content analysis as the research methodology to investigate thirty HPT business cases. The findings in this research indicate that the detailed processes and components as depicted in the performance analysis process in the ISPI HPT model were not fully present in the HPT business cases. Actual processes used in the business cases to identify performance gaps differed in part from the model. In addition, the procedure of analysis in the model is depicted as a logical sequence and in the cases is an actual sequence. In addition, the refined performance analysis process is proposed based on the research findings.     

What Do Tests Test?

Howard Gardner, the John H. and Elisabeth A. Hobbs Professor in Cognition and Education at the Harvard Graduate School of Education and the "father" of multiple intelligences theory, weighs in on the issue of high-stakes testing in this commentary: "We must proceed cautiously before we place students' minds and hearts at risk with tests of dubious quality whose meaning can be over-interpreted and whose consequences can be devastating," writes Gardner.     

What Kind of English Class Do You Like?

I hope I’ll have more chances to speak English in class,because we can learn English well only by using it.However,we have no such a chance at home.I hope the teacher will arrange all kinds of activities in class,such as dialogues,short plays,reading contests and so on.I don’t think it good for the teacher to talk all the time while the students take notes only.I think it necessary to learn some grammar.But we shouldn’t study grammatical rules in     

When Do Students Benefit From Performance Feedback? A Test of Feedback Intervention Theory in Speaking Improvement

This study reviews the important role of feedback in instructional communication Feedback Intervention Theory (FIT). FIT proposes that feedback efficacy is related to three primary constructs: message cues (locus of attention), personality traits, and the nature of the task to be learned. In an experimental design, the effect of grading, along with assessment of personality differences in feedback attributional sensitivity and perceptions of feedback utility are examined for their impact on feedback provided to student speakers. Changes in observed behavior for both eye contact and length of introduction are found. In addition, it is noted that traits and message cues function independently in predicting changes in behavior. Suggestions for future research are provided.     

Do They Really Care? A Critical Race Theory Analysis of Culturally Relevant Caring for Black Male Students

The Urban Review - One segment of the student population that is regularly and systematically ignored is Black male students labeled “overage, under-credited” (OA/UC) based on their age...     

What Do Learning Scientists Do? A Survey of the ISLS Membership

This study responds to a question that people working in the field of learning sciences get asked regularly: What do learning scientists do? Earlier attempts to answer this question came from a need to define a new field of educational research. Now that the International Society of the Learning Sciences (ISLS) has grown into a robust and productive society, it is time to gain a more nuanced understanding of learning sciences research and practices—including where it takes place, for whom, and in what form—as defined by members of the learning sciences community. Here we report on the responses of 253 ISLS members to a survey conducted in 2014. We discuss implications of the findings in terms of the type of impact learning scientists have. We also discuss how these results might be used to advise prospective students and to create a vision for our future.     

Test Score or Student Progress? A Value-Added Evaluation of School Effectiveness in Urban China

Outcome-oriented evaluation of school effectiveness is often based on student test scores in certain critical examinations. This study provides another method of evaluation—value-added—which is based on student achievement progress. This paper introduces the method of estimating the value-added score of schools in multi-level models. Based on longitudinal student achievement data, two measures of school effectiveness in one local education authority in China are compared. It is found that the between-school difference in both test-score and value-added is large comparable with that of Western countries. The results of the two measures of school effectiveness are highly different. The value-added measures lack consistency across different subject areas within schools while the test score measures are highly correlated between subjects. Teachers show their preference for value-added measures over test-score measures of education quality. It is suggested that value-added measures of school effectiveness should be used as a complement to rather than a substitute for test-score measures. The shortcomings of value-added approach are also discussed.     

“Wait,What Do You Mean by College?” A Critical Race Analysis of Latina/o Students and Their Pathways to Community College

ABSTRACT

As a group, Latina/o students are more likely to experience a substandard K–12 education complete with underresourced schools, high teacher turnover, and fewer college-preparatory courses. It is this same inferior education that denies many Latina/o high school students the opportunity to engage in college-choice—leading to their disproportionate enrollment in community colleges over 4-year colleges or universities. In California alone, approximately 75% of Latina/o students in higher education can be found in the community college sector—making this an important pathway for many Latina/o students. This qualitative study incorporated a Critical Race Theory (CRT) in Education framework to focus on the racialized K–12 experiences of four Latina/o graduate students who started their postsecondary career at a community college. This study was undertaken to better understand what led Latina/o students to enroll in community colleges after high school. Exploring the pathways of Latina/o students from high school to community college is imperative to community college practitioners (i.e., faculty, staff, and administrators) when considering best practices for their large Latina/o student body, as is found in California. The initial findings suggest that racism in K–12 in the forms of tracking, limited college information, and low expectations from academic personnel had a direct impact on the postsecondary experiences and opportunities available to Latina/o students. Lastly, the findings challenge prevailing portrayals where Latina/o students passively accept their marginalized position in education by highlighting their voice, resiliency, and agency in the face of systematic racism, as evidenced by their successes in academia.     

What Do We Mean When We Say “Pass Rate”?

Pass rates are key assessment statistics which are calculated for nearly all high-stakes examinations. In this article, we define the terminal, first attempt, total attempts, and repeat attempts pass rates, and discuss the uses of each statistic. We also explain why in many situations one should expect the terminal pass rate to be the highest, first attempt pass rate to be the second highest, total attempts pass rate to be the third highest, and repeat attempts pass rate to be the lowest when repeat attempts are allowed. Analyses of data from 14 credentialing programs showed that the expected relationship held for 13 out of 14 of the programs. Additional analyses of pass rates for educational programs in radiography in one state showed that the general relationship held at the state level, but only held for 6 out of 34 educational programs. It is suggested that credentialing programs need to clearly state their pass rate definitions and carefully consider how repeat examinees may influence pass rate statistics. It is also suggested that credentialing programs need to think carefully about the meaning and uses of different pass rate statistics when choosing which pass rates to report to stakeholders.     

What Do We Teach When We Teach the Learning Sciences? A Document Analysis of 75 Graduate Programs

The learning sciences, as an academic community investigating human learning, emerged more than 30 years ago. Since then, graduate learning sciences programs have been established worldwide. Little is currently known, however, about their disciplinary backgrounds and the topics and research methods they address. In this document analysis of the websites of 75 international graduate learning sciences programs, we examine central concepts and research methods across institutions, compare the programs, and assess the homogeneity of different subgroups. Results reveal that the concepts addressed most frequently were real-world learning in formal and informal contexts, designing learning environments, cognition and metacognition, and using technology to support learning. Among research methods, design-based research (DBR), discourse and dialog analyses, and basic statistics stand out. Results show substantial differences between programs, yet programs focusing on DBR show the greatest similarity regarding the other concepts and methods they teach. Interpreting the similarity of the graduate programs using a community of practice perspective, there is a set of relatively coherent programs at the core of the learning sciences, pointing to the emergence of a discipline, and a variety of multidisciplinary and more heterogeneous programs “orbiting” the core in the periphery, shaping and innovating the field.     

Getting Round to Clarity: What Do We Mean by Circle Time?

The use of circle time is becoming increasingly widespread in this country, but what exactly it involves and should achieve is subject to a wide range of interpretations. Such a situation can make the identification of what should count as good or appropriate practice very hard to undertake. This paper describes three examples of the development of circle time in other countries. These models are used as a means of critically reflecting on practice here. Some key points raised are the importance of the relationship between theory and practice and between specific processes and their outcomes. Lack of evaluative research and adequate training are also identified as significant issues.     

Competence or Performance? A Bernsteinian Analysis of Basic Competency Assessment in Hong Kong

This article analyses an assessment initiative known as the Basic Competency Assessment (BCA) in Hong Kong using Bernstein’s performance and competence models of pedagogic practice. On the one hand, BCA is aligned with a competence model through its official projection as a low-stakes, student-focussed and formative assessment. However, an examination of the public perception of and context for BCA reveals its high-stakes, disciplines-centred and summative nature; it is argued that BCA reflects a performance model with strong classification, strong framing and visible pedagogies. The example of Hong Kong illustrates the usefulness of Bernstein’s theory and extends the existing research on the interactions between local culture and pedagogic practices.     

Does Work Harm Academic Performance of Students? Evidence Using Propensity Score Matching

In this article we analyze the effects of student work on academic performance for college students. In order to reduce the endogeneity bias due to selection into treatment, we use propensity score matching technique. This approach allows us to estimate the effects of student work separately for different years of study, which is not possible when inside instruments are used to deal with endogeneity of student work. We find predominantly negative effects of student work for all measures of academic performance (GPA, exam attempts, exams passed, and likelihood of passing a year), although many of these are economically and statistically insignificant. We supplement existing studies that do not estimate separate treatment effects for different years of study by showing that work while in college harms study outcomes mostly in the first year of study—by passing smaller number of exams and thereby increasing the likelihood of failing a year. Our results are consistent with evidence on difficulty with adjusting to college studies of first-year students, who face many uncertainties that affect finding the optimal allocation of time between studies, work and leisure.     

Opinion: ‘Learning Technologies’? ‘Convergent Technologies’? What Do These Mean?

Sometimes in curriculum development, and everyday usage, a new term, or new piece of jargon, springs into common use, and everyone seems to know what it means, yet no one can give a clear definition of just what it signifies in practical terms. Moreover, being ‘new’, we might reasonably expect that anyone using such a term would also be able to explain how it is different from any older terms, and how the ‘new’ parts of the meaning relate to older meanings. But often the sheer newness of the term is so attractive to those who use it that they do not pause and try to make any sensible connection with earlier ideas. These had once themselves been new, and had been for years taken for granted, yet are now pushed aside without a second thought, largely because the new term arrives with a sense of compelling urgency. Something needs to be done, right now, because, look...we have to deal with this new thing called ‘X’. But what is this ‘X’? And why should we do anything about it? We actually need to begin with some hard critical thinking.     

What Do We Really Know About the edTPA? Research,PACT, and Packaging a Local Teacher Performance Assessment for National Use

This article calls attention to the overreliance on research about the Performance Assessment for California Teachers (PACT)—often labeled edTPA's predecessor—as justification for the edTPA. The article argues that the distinctions between the assessments are too vast to rely on PACT data to support the edTPA, given the localized nature of PACT and the way in which it is scored.     

Is What Glitters Really Gold? A Quasi-Experimental Study of First-Year Seminars and College Student Success

First-year seminars are frequently designed to help students adjust to and succeed in college. Although considerable literature has explored this topic, many previous studies may have notable problems with self-selection, since students who choose to participate are likely more motivated academically than those who do not. Therefore, this study used quasi-experimental analyses within a large, longitudinal, multi-institutional dataset to explore the link between seminar participation and several student success outcomes. Overall, the use of propensity score analyses substantially alters the results, such that first-year seminars are positively associated with first-year college satisfaction, but they have no effect on fourth-year satisfaction, college grades, retention, or four-year graduation within the full sample. This lack of impact is largely consistent regardless of whether the seminar is designed to engage students in academic inquiry or to promote orientation and academic success. Additional analyses observed some differential effects of first-year seminars by race/ethnicity, ACT score, and sex; the most consistent finding is that first-year seminars appear to promote the college grades and college satisfaction of Black students. Implications for future research and practice are discussed.

     

How Well Do Item Parcels Represent Conceptually Defined Latent Constructs? A Two-Facet Approach

This article presents a methodology for examining the content and nature of item parcels as indicators of a conceptually defined latent construct. An essential component of this methodology is the 2-facet measurement model, which includes items and parcels as facets of construct indicators. The 2-facet model tests assumptions required for accepting parcels as aggregates of item covariation in representing the latent construct. According to this methodology, parcels are acceptable indicators of the latent construct if the 2-facet model meets parametric assumptions for unidimensionality and if items and parcels have content validity as measures of the latent construct. The proposed methodology is illustrated using a 1-factor model of the Worry construct in the test anxiety measurement tradition     

How Do Academic Achievement and Gender Affect the Earnings of STEM Majors? A Propensity Score Matching Approach

The United States government recently enacted a number of policies designed to increase the number of American born students graduating with degrees in science, technology, engineering and mathematics (STEM), especially among women and racial and ethnic minorities. This study examines how the earnings benefits of choosing a STEM major vary both by gender and across the distribution of academic achievement. I account for the selection into college major using propensity score matching. Measures of individual educational preferences based on Holland’s theory of career and educational choice provide a unique way to control for college major selection. Findings indicate that the earnings benefit to STEM major choice ranges from 5 to 28 % depending both on academic achievement and on gender and that high-achieving students benefit more from STEM major choice. Further, high achieving men benefit more from STEM majors than high-achieving women. Earnings differences in major choice may play an important role in explaining the underrepresentation of women in STEM major fields, especially among high achieving students.     

Do Local Study Environments Matter? A Multilevel Analysis of the Educational Careers of First-year University Students

This article examines the effects of local study environments on the achievements in higher education of Dutch undergraduate students. The authors find that the grade averages, the numbers of courses successfully completed, and the drop-out rates of students vary only slightly in terms of the higher education institution attended. Most of the differences observed are the results of individual factors, i.e., differences in the ways in which students select their institutions and differences in the degree of success of their academic integration into the institutions they chose. The results indicate that when students enroll in higher education there is little benefit for them in selecting the "right" institution. Academic achievement is shaped mainly by individual factors.