Summary by James R. Martin, Ph.D., CMA
Professor Emeritus, University of South Florida
Balanced Scorecard Main Page | Performance
Measures Main Page | Strategy Main Page
Although an increasing number of companies have been using nonfinancial performance measurements in areas such as customer loyalty and employee satisfaction, few have realized the potential benefits of these relatively new measurement systems. This is because they fail to correctly identify, analyze and act on the right measurements. The purpose of this article is to discuss a number of mistakes that companies make when measuring nonfinancial performance and to highlight several practices that will help companies avoid theses errors and allow them to realize the potential benefits from using an appropriate mix of measurements. The authors' recommendations are based on their field research in over sixty service and manufacturing companies and survey responses from 297 senior executives.
Ittner and Larcker begin by saying that most companies have apparently adopted boilerplate versions of nonfinancial measurement frameworks such as Kaplan and Norton's Balanced Scorecard, Accenture's Performance Prism, of Skandia's Intellectual Capital Navigator, but seldom establish the cause and effect linkages between the measurements and desired outcomes. This allows self-serving managers to chose and manipulate measurements solely to enhance their own earnings and bonuses. It appears that nonfinancial measurements are just as, if not more, susceptible to manipulation as financial accounting measurements, and perhaps even more damaging to the companies because of the opportunity costs incurred.
Mistakes companies make when trying to measure Nonfinancial Performance
1. Not linking measures to strategy.
Using an off-the-shelf approach such as the Balanced Scorecard by itself does not identify which performance areas make the greatest contributions to the company's financial outcomes. Many managers have referred to the balanced scorecard as the "four bucket" or "smorgasbord" approach. Instead successful companies use causal models (causal or value driver maps) that identify the cause and effect relationships between drivers and outcomes. A fast food chain that mapped their cause and effect connections developed the causal model illustrated below. However, fewer than 30% of the companies surveyed have developed causal models.
2. Not validating the links.
Of the companies that developed causal models, only 21% validated the cause and effect relationships represented by the model. However, not validating the model leads to measuring too many things, and areas of performance that don't have much effect on what really matters. For example, one leading home finance company developed an "executive dashboard" with nearly 300 measurements. Ittner and Larcker's research revealed that only 23% of the companies in the study built and verified casual models of performance, but those that did had an average return on assets 2.95% higher than the companies that did not use causal models and an average return on equity of 5.14% higher than the non-causal model building group.
3. Not setting the right performance targets.
Outstanding nonfinancial performance does not always translate into outstanding financial performance. According the Ittner and Larcker, often the opposite is true. For example, one company set a target of 100% customer satisfaction, but discovered that customers that were 100% satisfied did not spend more than those who were 80% satisfied.
4. Measuring incorrectly.
Their research also indicated that 70% of the companies used metrics that lacked statistical validity (i.e., measures what it's supposed to measure) and reliability (i.e., the degree to which the technique reveals actual performance changes and does not introduce errors of it's own). Simplistic surveys with a 1-5 point scale of satisfaction are given as an example metric that lacks validity and reliability. Another problem mentioned is that many companies do not attempt to measure "hard to measure" qualitative areas of performance. This saves them from relying on misleading results, but prevents them from developing a comprehensive picture of performance.
Steps to follow to do it Right
1. Develop causal models.
Develop a causal model based on the hypotheses in the strategic plan. The strategic plan should be like a roadmap, not a mission or vision statement.
2. Pull together the data.
Take a careful inventory of the company's databases to avoid collecting data that already exist, and eliminate data "fiefdoms" where segments refuse to share data.
3. Turn data into information.
Use statistical (e.g., regression and correlation) and other techniques (e.g., focus groups, one-on-one interviews) to test the causal model. In this section they mention the model developed by Sears discussed in an article by Rucci, Kirn & Quinn.
4. Continually refine the model.
Key performance areas can change as the competitive environment changes, and even in stable environments continuous analysis leads to refinements in performance measurements and the company's understanding of what drives economic performance. The following quote conveys the idea. "Beneath the proven drivers of performance lie the drivers of those drivers. Since a business can't ever know whether it's gone deep enough, the effort to uncover these drivers must never cease."
5. Base actions on findings.
Use the results of the model's data analysis for decisions. For example, a major finance company allocated capital according to the relative importance of various drivers after establishing the cause and effect relationships in their causal model.
6. Assess outcomes.
Conduct post audits to determine if the actions based on the model produce the desired results. If not, revise the causal model accordingly.
Conclusion
Apparently many nonfinancial performance measurement systems have become shabby substitutes for financial performance. To be useful in guiding a company these systems need to be based more on sophisticated quantitative and qualitative methods and less on generic performance measurement frameworks and management guesswork.
____________________________________________
Related summaries:
Apgar, D. 2011. Assumption-based metrics: Recipe for success. Strategic Finance (November): 26-33. (Note).
Grojer, J. 2001. Intangibles and accounting classifications: In search of a classification strategy. Accounting, Organizations and Society 26(7-8): 695-713. (Summary).
Ittner, C. D. and D. F. Larcker. 1998. Innovations in performance measurement: Trends and research implications. Journal of Management Accounting Research (10): 205-238. (Summary).
Kaplan, R. S. and D. P. Norton. 1992. The balanced scorecard - Measures that drive performance. Harvard Business Review (January/February): 71-79. (Summary).
Kaplan, R. S. and D. P. Norton. 1996. The Balanced Scorecard: Translating Strategy into Action Boston: Harvard Business School Press. (Summary).
Kaplan, R. S. and D. P. Norton. 2001. The Strategy-Focused Organization: How Balanced Scorecard Companies Thrive in the New Business Environment. Harvard Business School Press. (Summary).
Kurtzman, J. 1997. Is your company off course? Now you can find out why. Fortune (February 17): 128-130. (Summary).
Lipe, M. and S. Salterio. 2000. The balanced scorecard: Judgmental effects of common and unique performance measures. The Accounting Review (July): 283-298. (Summary).
Lipe, M. G. and S. Salterio. 2002. A note on the judgmental effects of the balanced scorecard's information organization. Accounting, Organizations and Society 27(6): 531-540. (Summary).
Reilly, G. P. and R. R. Reilly. 2000. Using a measure network to understand and deliver value. Journal of Cost Management (November/December): 5-14. (Summary).
Schonberger, R. J. 2008. Lean performance management (Metrics don't add up). Cost Management (January/February): 5-10. (Note: Schonberger criticizes the KPI or scorecard approach from the lean enterprise perspective. Summary).
Stivers, B. P., T. J. Covin, N. G. Hall and S. W. Smalt. 1998. How nonfinancial performance measures are used. Management Accounting (February): 44, 46-49. (Summary).
Tatikonda, L. U. and R. J. Tatikonda. 1998. We need dynamic performance measures. Management Accounting (September): 49-53. (Summary).
