Measuring the ROI on Knowledge Management Systems

Illustrating the return‐on‐investment (ROI) for a portal solution or knowledge management (KM) system has proven an elusive target. This paper will demonstrate that ROI can be found for portals and KM systems in measuring the ROI on improved processes and increased economic value of employee performance. Thus, rather than employing traditional notions of value and assets as noted in standard accounting practices, KM solutions are tools managers should use to support opportunities for process improvement and redesign. ROI that measures value from this perspective creates new areas of value from an organization's existing, undervalued assets. A well‐developed measurement methodology for implementing a KM system may illustrate ROI, justify expenditures for implementing the system, and provide a format to ensure that process improvement occurs. A well‐thought‐out KM system has the capability of becoming the “digital nervous system” of an organization, tying all areas to the strategic goals of an organization.     

The ROI fieldbook: Strategies for implementing ROI in HR and training by Patti Pulliam Phillips,CPT, Jack J. Phillips,Ron Drew Stone,and Holly Burkett,CPT

Patti Pulliam Phillips, CPT, PhD, CEO of the ROI Institute, Inc., consults with U.S. and international organizations on implementing the ROI Methodology. Jack J. Phillips, PhD, a world‐renowned expert on measurement and evaluation, developed the ROI Methodology; his expertise in measurement and evaluation comes from nearly 30 years of experience. Ron Drew Stone is an author, international consultant, and authority on improving and measuring performance improvement interventions. Holly Burkett, CPT, MA, SPHR, principal of Evaluation Works, has more than 20 years' experience assisting in the design and measurement of performance improvement initiatives. The ROI Fieldbook: Strategies for Implementing ROI in HR and Training (ISBN: 978‐0‐7506‐7622‐9) is published by Elsevier Butterworth‐Heinemann (price: $44.95; paperback with a CD). This publisher may be reached online at http://www.elsevier.com.     

Show me the money: The use of ROI in performance improvement,part 1

This first of a two‐part series presents the evolution of value, moving from activity‐focused value to the ultimate value, return on investment (ROI). This feature clearly shows that the ROI methodology is not only appropriate for the performance improvement field, but is designed with a variety of performance improvement solutions in mind. The next article in this series will describe issues and challenges that those using this methodology face, along with a brief case study.     

Measuring the ROI of a coaching intervention,Part 2

This article, the second of a two‐part series, describes a case study application of the ROI Methodology™. In this case, a structured coaching program was implemented as part of a comprehensive performance improvement solution designed to improve efficiency, customer satisfaction, and revenue growth for Nations Hotel Corporation. This case study provides critical insights into how a project was systemically designed, delivered, and measured to create performance value, including a return on investment.     

Competency‐Based Evaluation: A Paradigm Change

Most training units fail to evaluate training effectiveness in terms of return on investment (ROI). In this article the argument is made that this is a result of using an inadequate evaluation paradigm. The author suggests a paradigm that is based upon a common cost‐benefit framework. As such, the alternative paradigm a) will be more understood and accepted by other managers in the organization, b) will enable the evaluation of the full scope of training activities all within one framework including effectiveness, cost, quality and efficiency‐related activities. In order to measure effectiveness we have to broaden the definition of the term performance improvement. The definition should relate also to the cost of behavioral change. In addition, we have to differentiate between evaluations of training effectiveness and consulting effectiveness. These two changes will enable us to link directly, training activities to performance improvement. Training results will improve human capital and could be measured in ROI terms.     

Partnering through training and practice to achieve performance improvement

This article presents a partnership effort among managers, trainers, and employees to spring to life performance improvement using the performance templates (P‐T) approach. P‐T represents a process model as well as a method of training leading to performance improvement. Not only does it add to our repertoire of training and performance management methods, it assists trainers and managers to promote change, achieve flexibility in performance, and enhance the fluency of skill application with regard to critical performance events.     

Measuring The Business Impact of E‐Learning: An Empirical Study

This study fills a gap in the current HRD literature of return on investment (ROI) analysis of technology‐based learning interventions. Using a Type IV control group method as defined by Wang (2002), the study empirically analyzed and measured the learning effectiveness and the business impact of an e‐learning system implemented in General Electric Company. The e‐learning system under study demonstrated significant economic returns in regard to reduced learning response time and reduced project cycle time. The study shows that e‐learning, as a means of technology‐based HRD intervention, can be highly effective as a performance support learning mechanism, although the results also suggest that e‐learning alone is not sufficient to replace the traditional face‐to‐face learning platforms.     

Underdogs of Performance Improvement Transformations

A key facet of performance improvement transformation is team effectiveness. Often projects fail because we are not able to get the best out of all the team members. As a result, the effort becomes centered around a few so‐called high‐performing individuals while the others languish. The team members who often are not used are branded as ‘dysfunctional,’ ‘inconsequential,’ or ‘inoperative.’ While it is easy to negatively brand such people as ‘incompetent,’ seeking the positive contribution from them is key to success. The author refers to these people as ‘underdogs of performance improvement’ who have lot to offer. All we need to do it to look at them differently.     

Using ROI to demonstrate performance value in the public sector

The demand for accountability through measurement continues to increase. Although much progress has been made, it remains an issue that challenges even the most sophisticated and progressive performance improvement function. This article provides an overview of best practices in the return‐on‐investment process and describes how applying these measurement practices can have a positive impact on organizational performance and the credibility of performance improvement professionals in the public sector.     

Validation ROI: An HPT case study from the medical device industry

Validation is both a process and a function within Company ABC. Using the human performance technology (HPT) process, interventions were prescribed to address identified performance gaps. Forecasting an annual return on investment (ROI) based on goals yielded a ROI of 168%. Data collected for the first quarter of 2009 yielded a calculated ROI of 326%. This study discusses the HPT process and what was done to achieve the results.     

Efficacy of a template creation approach for performance improvement

This article presents the training and performance improvement approach, performance templates (P‐T), and provides empirical evidence to support the efficacy of P‐T. This approach involves a partnership among managers, trainers, and employees in the creation, use, and improvement of guides to affect the performance of critical tasks in the operating environment. The P‐T approach requires much involvement and hands‐on activities by training participants, and, as a result, training groups are not large. Given that limitation, a comparison of employees trained with P‐T methods and employees trained in more conventional methods indicates that the P‐T approach can lead to improved performance to some extent over a fairly short period.     

The Horizon Model: Human Process Improvement Model from 30,000 Feet to Below Sea Level

Incorporating macro‐ to micro‐levels of an organization, the horizon model is introduced in this article as a practical tool for organizational performance and process improvement. The horizon model is based on existing theories and models in the human performance technology and the human performance improvement industries. Application of this model is demonstrated by a real‐life example with the City Club of Boise and illustrates when, where, and how the horizon model can be used to improve performance.     

A Phenomenon We Can't Ignore: Performance Improvement Interventions to Address Workplace Bullying

Workplace bullying is a significant problem impacting both individual performance and performance of organizations as a whole. This article describes how a systematic approach was utilized by focusing on the organizational subsystem of human resources as it relates to preventing, identifying, and addressing workplace bullying. Human resource (HR) professionals play a vital role in applying anti‐bullying policies and performance improvement interventions. The article outlines the interventions and policies utilized by HR professionals to alleviate and manage bullying that have been cited in the literature.     

Questioning minds on the lookout for potential workplace research and improvement markers: Practice‐based research

In the performance improvement field, performance technologists are in an ideal position to detect and investigate unexpected effects (UEs) in the workplace: those near‐invisible root variables that, once exposed, pose either potentially performance‐harming or performance‐enhancing challenges for their organization. Working with UE triggers and trends will round out performance technologists' capabilities through the adoption of what is known as practice‐based research.     

Comprehensive Return on Investment Evaluation System for Local Self‐Government Training in Bosnia and Herzegovina. Case: Project Cycle Management Course

This article describes a case study in the use of the return on investment (ROI) evaluation methodology in local self‐government training in Bosnia and Herzegovina (BiH). Out of 20 training programs, the Project Cycle Management course was selected. The article describes pioneering work for the ROI approach in the Western Balkan public sector. A main challenge was to identify an impact measure due to difficulties in accessing relevant data. A conservative approach to data management showed the strengths of the ROI methodology and the reliability of the results.     

Of Trends and Trajectories: Searching for patterns in school improvement

The article builds on earlier research suggesting that it is fairly difficult to extrapolate current trends in school performance into the future (see Gray, Goldstein & Thomas, British Educational Research Journal, 27, 2001). It explores two areas of research on school improvement which have, to date, received relatively little attention—trends in performance over time and the incidence of time‐lagged phenomena. In responding to a critique by Pugh and Mangan (2002), it suggests that, whilst of interest, time‐trend analyses favoured by economists may be less useful than other forms of analysis which search for consistent patterns over shorter periods of time. Some strategies for thinking about their identification are put forward. The implications of these approaches for target‐setting and related exercises, which are premised on linear assumptions about school improvement, are also discussed.     

Innovation and Performance Improvement Integration

G2000's HR Execution Excellence—Retail Attendance System was one of the innovative projects to receive the ISPI Award of Excellence in 2016. It is a continuous improvement project that applies the concept of holistic human performance improvement using an ISPI human performance technology (HPT) model (ISPI, 2012) to streamline the front‐end and back‐end processes of our Retail Attendance System. As a result, it leads us to achieve one of our business goals: employment regulatory compliance. In our case, the project team was tasked with seeking solutions to ensure that the payroll process for retail staff could be performed accurately and in a timely manner. After applying the HPT model to conduct the gap analysis and identify the causes or factors that were limiting our performance, we integrated the concept of human‐centered design approach at the solution‐design phase of the project, to lead us to innovative solutions.     

Intervention Selection: An Examination of Evidence and Changes in Belief During the Decision‐Making Process

This study examines what sources of evidence are used in intervention selection and what changes in belief occur when performance improvement professionals make these decisions. Sixty‐one certified performance technologists completed a dynamic, web‐delivered questionnaire in which they provided a general assessment of intervention success (Pr₁), then responded to 12 performance improvement scenarios by selecting an intervention, providing a prior probability, receiving additional evidence, giving a posterior probability (Pr₃), indicating whether the initial intervention was still preferred, and making a subsequent choice if not. Findings bolster the long‐standing concern about the technical nature of performance improvement, and practitioners are strongly encouraged to approach intervention selection as a decision, where their intervention preferences and beliefs of likely success are carefully adjudicated on the basis of the evidence they obtain. Future research with other types of performance improvement practitioners, replication studies, longitudinal, structural equation modeling, externally verifiable probabilities, and natural environments are recommended.     

The others' values: On the importance of ethnographic ways of looking,seeing, knowing,and acting for performance technologists

The foundational assumptions of performance improvement come from a single‐mindedly economic and engineering orientation. Yet this orientation arises from a concern for maintaining and enhancing other forms of value. This other form of value, perhaps arising from cultural and social spheres, is normally unmentioned or marginalized in contemporary human performance technology. This article recovers and reintroduces this other form of value and promotes it as an important, and even essential, component of the future of performance technology.