COURSE PLANNING AND TESTING DECISIONS IN CRITERION‐REFERENCED SITUATIONS

Norm‐referenced measurement tools — such as reliability, validity, and item analysis — are commonly used to reach and verify conclusions about criteria. Similar tools for criterion‐referenced testing situations are scant. This study examined faculty planning and testing decisions and applied formulas to arrive at numerical indices that serve as analytical tools for use with criterion‐referenced tests. The research documents the effects of applying the concept of platform unity, which has its roots in curriculum alignment theory. Alignment of curriculum occurs if the planned, the delivered, and the tested curricula are congruent. Specifically, platform unity aligns planned, domain‐referenced content with appropriate test types. Mathematical formulas were created to determine numerically if planned and tested content were congruent. In addition, four other constructs were examined. They included effectiveness and efficiency of test‐item type selection and overtesting and undertesting of course content. A chi‐square goodness‐of‐fit test was used to compare faculty planning and testing decisions. Data indicated significant differences (p < .01) between content plans and the test types used to test content. On the basis of the analysis, it was determined that faculty do not plan and test content congruently across three levels of cognitive content. Also, faculty tended to overtest content; they were effective in their selection of test types, but not efficient.     

The May Center for Early Childhood Education: description of a continuum of services model for children with autism

Children with autism have complex learning and behavioural challenges which typically require comprehensive educational and therapeutic services. In recent years there have been many developments in the philosophy, methodology, and scope of intervention for this population. Factors such as early intervention, criterion‐referenced assessment, the combination of applied behaviour analysis and developmentally appropriate practices (DAP), parent training, and public school inclusive education have been noteworthy. This article describes a comprehensive continuum of services model for children with autism developed by a human services agency in Massachusetts which incorporates these and additional empirically‐based approaches. Service components, methodologies, and program objectives are described including representative summary data. The article concludes with a discussion of educational, therapeutic, and research issues which address “best practice” approaches toward children with autism.     

SETTING OBJECTIVES AND ASSESSING COMPETENCE IN PROFESSIONAL LEGAL EDUCATION

Professional Legal Education in the Australian Capital Territory (ACT) is undertaken at the Legal Workshop. A problem‐based learning orientation is a prominent feature of their programme. In this paper the assessment of the problem‐based learning used in the Workshop is described. All stages of this assessment process, including setting the course objectives, defining the conditions under which the problems (tasks) will be undertaken, and setting the criteria which are the standards for the assessment of success in the task, are described. The importance of criterion‐referenced assessment in the stage of setting assessment standards is emphasised. The results of the first attempt to use this assessment method are then described, and problems emerging with the process are discussed. Although this first “reality test” reveals some difficulties with the assessment method, it is concluded that this novel and innovative teaching and assessment technique has been appropriately applied to Professional Legal Education. While some refinements are required in the implementation of the method, the overall assessment concept has been successfully developed.     

On Meaningful Measurement: Issues of Reliability and Validity from a Humanistic Constructivist Information‐Processing Perspective

ABSTRACT

In the past decade, there has been interest in the assessment of cognitive and affective processes and products for the purposes of meaningful learning. Meaningful measurement (MM) has been proposed which is in accordance with a humanistic constructivist information‐processing perspective. Students’ responses to the assessment tasks are now evaluated according to an item response measurement model, together with a hypothesized model detailing the progressive forms of knowing/competence under examination. There is a possibility of incorporating student errors and alternative frameworks into these evaluation procedures. Meaningful measurement leads us to examine the composite concepts of “ability” and “difficulty”. Under the rubric of meaningful measurement, validity assessment (i.e. internal and external components of construct validity) is essentially the same as an inquiry into the meanings afforded by the measurements. Concepts of reliability, expressed as a group statistics which is applied in the same way to all the examinees in the sample, have to be obviated when the precision of the trait estimates stemming from the item response measurement models can be determined at each trait level. Reliability, measured in terms of standard errors of estimates needs to be within acceptable limits when internal validity is to be secured. Further evidence of validity may be provided by in‐depth analyses of how “epistemic subjects” of different levels of competence and proficiency engage in different types of assessment tasks, where affective and metacognitive behaviours may be examined as well. These ways of undertaking MM can be codified by proposing a three‐level conceptualization of MM. It is within the rubric of this conceptualization and the MM enquiry paradigm that validity and reliability of test measures are discussed in this paper.     

THE DEVELOPMENT AND ASSESSMENT OF A NEW EXAMINATION OF THE CLINICAL COMPETENCE OF MEDICAL STUDENTS

This paper describes a new examination of the clinical competence of medical students developed because the traditional approaches used by many medical schools have been shown to be unreliable and invalid. A problem‐based criterion‐referenced approach was adopted. Studies have confirmed that the new approach achieved a satisfactory level of validity and reliability. One of the main aims of the project was to exert a positive influence on the students learning in the direction of the Faculty's clinical objectives. A questionnaire survey showed a high degree of satisfaction with the new test and a high level of perceived relevance of the content to future practice. These seemed to be the main reasons why students’ study habits were directed in the desired direction.     

USING COMPUTER CONFERENCING IN TEACHING

For 3years, I have been teaching neuroscience courses by using computer conferencing to complement the traditional lectures. Typically, the conferencing involved local, on‐campus students, although one semester the class was combined from on‐campus and off‐campus students. For most of my 33‐year teaching career, I had used the teaching approach that most professors use, which is what educational theorists call “instructivist.” Critics call that “stand and deliver.” Lecturing is an efficient way to dispense organized information, but it does not ensure learning nor is it very effective in showing students how to learn on their own.

Instructivism can be enriched by complementing it with “constructivist” approaches. Constructivists argue that there is a direct relationship between the amount of learning that occurs and the extent to which the environment provides a rich source of engaging experiences in which students construct their own knowledge and understanding. I have found that such an environment is readily provided by computer conferencing.

In my teaching of neuroscience, I have used a network software system (FORUM) for small student groups to conduct a variety of constructivist learning activities. Within weekly deadlines, students worked in groups at their own pace and time of convenience. My impression of the advantages of such conferencing for constructivist activities include the promotion of socialization in “cyberspace,” providing an environment for team learning, the reduction of social problems in face‐to‐face instruction, increased teaching and learning efficiency, more comprehensive means for assessing student learning, and improved quality of student work.     

深入贯彻“学习强国”理念需破除主体形而上学

人类的生存过程就是学习的过程。学习是文化与基因协同进化过程的催化剂,始终伴随着"人"在演化大历史中的自我理解及其突破。形成于前科学历史阶段的"人"及其主体形而上学认知框架潜移默化地束缚着人们的思维,影响着人们的学习观念、内容和机制。破除主体形而上学就要以科学为武器,通过批判性思维,构建与时俱进的"人是谁、从哪里来、到哪里去"的人类新"故事",塑造引领时代进步的新型主体,进而改善学习观念和方法;要打破主体形而上学的闭路循环,接纳新观念和新事物,切实把"学习强国"理念落到实处。     

STEAM教师学科教学知识的基本构成与建构逻辑

教师学科教学知识(PCK)的形成是STEAM教育的发展之基。STEAM教师PCK是教师面对具体的跨学科的内容主题时,所特有的将不同学科的知识和技术转化为学生易于理解的教学形式的整合性知识,具有知识范畴的跨学科性、知识来源的实践性和知识形成的融合性等特点。通过对已有PCK研究的梳理,结合STEAM教育的特征,STEAM教师PCK可分为跨学科内容知识、教学对象知识、教学情境知识和教学策略知识。在此基础上,STEAM教师PCK的建构逻辑:一是立足理论性学习为教师建构PCK打“地基”,促进公共性PCK向个体性PCK转化;二是基于经验性学习为教师建构PCK竖“框架”,推动内隐性PCK向实践性PCK转化;三是通过实践性学习为教师建构PCK添“砖瓦”,实现陈述性PCK向程序性PCK转化。     

TOWARD AN INTEGRATION OF THEORY AND METHOD FOR CRITERION-REFERENCED TESTS1,2

In this paper, an attempt has been made to synthesize some of the current thinking in the area of criterion-referenced testing as well as to provide the beginning of an integration of theory and method for such testing. Since criterion-referenced testing is viewed from a decision-theoretic point of view, approaches to reliability and validity estimation consistent with this philosophy are suggested. Also, to improve the decision-making accuracy of criterion-referenced tests, a Bayesian procedure for estimating true mastery scores has been proposed. This Bayesian procedure uses information about other members of a student's group (collateral information), but the resulting estimation is still criterion referenced rather than norm referenced in that the student is compared to a standard rather than to other students. In theory, the Bayesian procedure increases the “effective length” of the test by improving the reliability, the validity, and more importantly, the decision-making accuracy of the criterion-referenced test scores.     

中美一流大学本科生“专业课程深度学习”及其影响机制的比较研究——基于SERU(2017-2018年)调查的数据分析

“专业课程深度学习”是衡量本科专业教育质量的一个重要指标。借助“学习科学”有关“专家知识”的最新研究成果,可以构建出“事实支持性学习”“知识整合性学习”“检验判断性学习”“反思提升性学习”四个“专业课程深度学习”的构成指标,并运用中国2所和美国5所研究型大学的2017-2018年SERU调查数据进行分析,获得如下的研究结论:就“专业课程深度学习”而言,在整体、院校及专业三个层面上,中国2所大学在四个指标上都明显落后于美国5所大学,但在“反思提升性学习”上的差距较小。就影响机制的分析而言,“专业课程学习参与”和“专业课程学习要求”对于中美一流大学本科生的“专业课程深度学习”都有显著的正向促进作用;调节效应的检验则进一步表明,中国大学生的“学习参与”对于“深度学习”呈现出比美国大学生更为明显的“加速影响效应”,“学习要求”对于“深度学习”则呈现出“缓速影响效应”。在“专业课程学习要求”方面,中国这两所大学在五个层级指标上都显得相形见绌。基于研究发现的问题,文章提出了“一观两度”的改进建议:确立专业课程的学习质量观;增强专业课程的学习参与度;提高专业课程学习要求的难度。     

Developing an instrument to assess student readiness for online learning: a validation study

Given the continued growth in online learning as well as reports of high attrition rates in it, understanding student readiness for online learning is necessary. Over the years several surveys have been developed to assess student readiness as a predictor of success in online programs; however, a review of the literature yielded limited results of their translation and criterion‐referenced validity. The researchers of this article sought to develop a more rigorous survey instrument for students to self‐assess readiness for online learning. The authors report on findings from a three‐phase study during which the instrument was developed, evaluated, and validated. Through the process of validation, the researchers systematically engaged in an iterative process to refine the instrument, which resulted in not only a more rigorous instrument but one that more clearly defines ready and situates it within the literature on learner characteristics, digital divide, and information and communications technology (ICT) engagement.     

The Effects of Inattentive Responding on Construct Validity Evidence When Measuring Social–Emotional Learning Competencies

Self‐report inventories are commonly administered to measure social‐emotional learning competencies related to college and career readiness. Inattentive responding can negatively impact the validity of interpreting individual results and the accuracy of construct validity evidence. This study applied nine methods of detecting insufficient effort responding (IER) to a social‐emotional learning assessment. Individual methods identified between 0.9% and 20.3% of respondents as potentially exhibiting IER. Removing flagged respondents from the data resulted in negligible or small improvements in criterion‐related validity, coefficient alpha, concurrent validity, and confirmatory factor analysis model‐data fit. Implications for future validity studies and the operational use of IER detection for social–emotional learning assessments are discussed.     

基于一致性建构理论的高校课程思政建设路径探析

立德树人是高校的立身之本,课程思政是实现高校立德树人根本任务的途径.将课程思政融入课程建设,需要将思想政治内容贯穿于课程目标设计、教学活动安排和教学评价等各个重要环节.一致性建构理论是以学生为中心的课程设计方法,课程设计需要遵循预期学习成效、教学活动、评价任务和成绩评定四个步骤.基于一致性建构理论,遵循其四个步骤探讨了...     

Improving Learning and Teaching Through Action Learning and Action Research

The purpose of this paper is to present a theoretical framework for action learning and action research for better understanding and improving university learning and teaching. Action research is conceived as a philosophy, a theory of learning, a methodology and a technique. The philosophy includes theories of action, critical theory and personal construct theory. The learning theory encompasses adult learning, experiential learning and double‐loop learning. The methodology is based in the dialectical epistemology and the non‐positivist paradigm. Examples of action research as a technique are the nominal group technique, the repertory grid technique and other tools aiding reflection and group discussion. It is concluded that action research not only advances knowledge, but also improves practice in higher education by. developing people as professionals and “personal scientists”, and organisations as “learning organisations”.     

RECALL AND PRINTED‐WORD RECOGNITION OF RECENTLY‐TAUGHT EMOTIONALLY CHARGED WORDS AS COMPARED TO RECENTLY‐TAUGHT NEUTRAL WORDS

This study is an experimental test of the hypothesis that “emotionally charged” words will result in greater learning as measured by tests of “recall” and “printed‐word recognition” than words that are judged to be emotionally “neutral”.

Ten third‐grade subjects were pre‐tested on “printed‐word recognition” of ten words judged by two clinical psychologists to be “emotionally charged” and similarly pre‐tested on ten words judged to be emotionally “neutral.” After equal exposure to both “neutral” and “emotionally charged” words on four successive teaching days, subjects were post‐tested for “recall” and “printed‐word recognition” five days after the last teaching session.

The results strongly indicate for both the “recall” and “printed‐word recognition” that there is a significantly greater retention of the “emotionally charged” words over the “neutral” words. Despite the fact that a significantly greater number of “neutral” words was recognized in print at the outset of the study, the final performance level for “printed‐word recognition” for both the “emotionally charged” and “neutral” words is identical.

The major implications of the results obtained in this study is that the content of reading materials and the methods for teaching them must be revised so that reading deals with events, ideas, and feelings which are emotionally significant to the child, A personally relevant reading program will not only facilitate the task of learning to read, but will capture and explore the feelings and emotions of each child, as well.     

Developing generic criteria and standards for assessment in law: processes and (by)products

This paper reports the outcomes of a School of Law project undertaken in response to issues arising during the implementation of an institutional criterion‐referenced assessment policy. The project involved the development of a set of generic assessment criteria and standards, or rubric, which could be customized to the requirements of individual law subjects. Of significance are the key decisions that shaped the generic resource and additional outcomes or byproducts of the project which include the professional learning experienced by individual members of the project working party and the identification of mutually reinforcing relationships between criterion‐referenced assessment practice and other institutional policies and priorities. The paper concludes that analysis of project processes can produce findings whose value and significance are of equal interest to those resulting from implementation studies.     

WSR方法论在食品化学课程建设中的实践

食品化学课程的教学质量对培养应用型食品类专业人才至关重要。以WSR方法论为指导,从“物理”(Wuli)“事理”(Shili)“人理”(Renli)3个维度分析了食品化学课程建设过程中的影响因素。食品化学课程建设中的“物理”主要包括课程教学内容、教学条件、教学经费、教材等,是食品化学课程建设的基础。食品化学课程建设的“事理”包括课程教学目标、教学大纲、教学方法和手段、考核方式等,是食品化学课程建设的手段。食品化学课程建设的“人理”主要包括教师队伍、教学质量评价、学习需求、学生情况等,是食品化学课程建设的核心。通过“人”协调食品化学课程建设中的“物”和“事”,可以达到应用型人才培养的目标。     

初中生数学学习非智力因素调查问卷的编制

通过查阅文献资料与已有成熟问卷,征询专家意见,根据初中生数学学习特点,编制了"初中生数学学习非智力因素调查问卷".经过项目分析、探索性因素分析、验证性因素分析,对问卷进行相应修改,正式问卷包括动机、态度、意志、性格、情绪情感5个维度,以及对应的13个因子.所编问卷具有较好的信度(一致性系数,重测信度,分半信度)与效度(内容效度、结构效度、效标效度),可作为测量初中生数学学习非智力因素的有效工具.