Dashboard Approaches – KPIs versus Operational Lists

This article was contributed by Klipfolio, the business dashboard software company that is focusing on KPI Dashboards and the topic of my previous article about the cool opportunity to be a beta tester for their new dashboard version.

KPIs or lists? Chances are your dashboards need both.

by Allan Wille, President and CEO.

At Klipfolio, experience has taught us that designing dashboards to ensure the effortless visibility of tactical, tangible, real-time operational data across departments and throughout the corporate hierarchy means distinguishing between two typical use cases: monitoring key metrics (KPIs), and providing operational lists. Both are important. Ignoring either can hurt your performance. And, when designing your dashboards, knowing which to use is critical.

KPI visualization

You’re no doubt familiar with the first case. The need to monitor a key metric – usually a numeric value expressed as a ratio or percentage – is well understood in the BI world. I’ve discussed KPIs in detail in past blogs and articles (see What is a KPI, Metric or a Measure? and The Anatomy of a KPI). And a web search for KPI dashboards or operational BI will deliver examples of KPIs that represent business metrics.

These business metrics are invariably displayed as absolute values, ratios, or percentages. If they’ve been designed properly, they’ll likely have associated contextual values, such as goals, averages, or benchmarks, that make it easier to evaluate the KPIs.

An example might be a KPI designed to help a marketing manager keep a close eye on campaign performance – one that tells them something like “Campaign A has delivered 400 leads over the past 30 days compared to our goal of 450 and our historical average of 435.”

The operational list

If you’re like many BI users, you’re not so familiar with the second use case: the operational list.

Lists might seem like old technology, but as data sources become more tactical and right-time in nature, lists make a lot of sense. Examples might include a prioritized list of leads or opportunities for a sales rep, or a list of high-priority incidents that need resolving by a support agent.

Although not KPIs in the strictest sense, these lists embody much of the same spirit – they are time sensitive, tactical in nature, actionable, and highly relevant to individuals in specific roles.

Which to use?
When should your dashboards monitor KPIs and when should they deliver operational lists? It depends on the roles of the people whose performance they are designed to maximize.

For executives, whose role is to manage, plan, and improve efficiencies and performance, KPIs are more useful.

The sales campaign KPI metric already mentioned is a case in point.

On the other hand, front-line workers, such as sales reps and support agents, typically benefit more from operational lists. They need information like that provided by the leads and opportunities lists I’ve described. And they need lists that are easily understood, so they can prioritize and align tasks, and act quickly to fix problems and take advantage of opportunities.

It’s not “either/or”

When it comes to KPIs vs. lists, it’s not always an either/or situation, and chances are good that you’ll not only need both types of operational dashboards, but that some of your dashboards will include both KPIs and lists.

KPIs and lists are, in fact, closely related, and lists will typically impact KPIs. Today’s “open-incident/closed-incident” ratio, for example, which will be derived from a list critical for helping support agents perform effectively, is an important number for managers to track. And those same managers might want to monitor a list that tells them about deals over $100K, or sales to key customers, even though they aren’t likely to take immediate action based on what they learn.

In the same way, sales reps, agents, and other workers whose day-to-day performance means focusing on key lists first and foremost, might well monitor KPIs as a valuable way to understand performance and to perform better themselves.

Use cases and dashboard design

When designing operational dashboards, it’s important to keep the two use cases – KPIs and lists – in mind. It’s essential to recognize that they address two types of needs. And it’s critical to analyze the type of need you are addressing for each person’s role, and to design your KPI or list dashboards accordingly. Dashboards for your executives and managers will likely be KPI-heavy, while those for front-line workers will tend toward the list end of the spectrum.

10 Guidelines for KPI Dashboard Design

KPI Dashboard Designers must read this excellent guide to ensuring successful KPI and dashboard design from Klipfolio KPI Dashboard.

klipfolio dashboard

1: Make sure they are important and related to the business – KPIs should be linked to strategic goals and be supported from the executive level through to departmental managers and individual employees. The KPIs have to drive the desired behaviour at the department and individual levels. Keep in mind KPIs are not only internally focused, but may be customer driven.

2: Make them actionable – Ensure that there is an owner that is accountable for each indicator who has the authority and necessary support to effect change or minimise impact. Establish accountability at the definition phase. It is not a good KPI if you are unable to take corrective action on one of your chosen KPIs.

3: They should be measured frequently – The frequency will vary on the KPI and may range from constantly, hourly, daily through to weekly monthly or even quarterly. KPIs that are only reviewed every month or quarter may be strategically important but are less critical to operational success.

4: Make them relevant to their audience – A dashboard provides relevance to its audience. Each department or individual should be provided with a view of the KPIs most relevant to their goals to focus their attention on what is most important for their jobs.

5: Keep dashboards simple, easy to interpret – Visual communication experts like Stephen Few have written extensively on how to properly design reports, charts, and graphs for quick and accurate consumption. Today, dashboard application have a myriad of visualisation options. It’s tempting, to get carried away with impressive graphical data, maps, speedometers etc. to represent performance against metrics “Hollywood style”. However, key information should be self evident and visible at all times so that exceptions can be seen and acted upon.

6: Provide context – Tracking actual performance against goals or historical averages gives insight to users and enables better decision management. The ability to alert users visually and provide drill-down views into the data provides insight and context on what corrective action may be required.

7: Education & communication – It is not enough to identify and provide visibility of KPIs. Users at all levels have to understand how they align with individual, departmental and strategic goals. Include education on the use of the dashboard application, the importance of the KPIs and how they align with strategic goals to new employees as part of the induction process.

8: The “K” in KPI stands for Key – It is important to determine if defined KPIs are really relevant to meeting strategic goals. All to often dashboards will adopt a if you can measure it, you should measure it approach where you end up looking at 20 or more competing KPIs. Are they the right metrics? Are there too many?

9: Return on investment – Following definition, ongoing measurement of KPIs can be a barrier to adoption if it is viewed as a time consuming manual process, the information is not trusted or readily available. KPI Dashboards automate the process; they enable business information to be acted upon and can be integrated into the workflow process, saving time. Improved decision making based on right time information rather than gut feel increases the productivity of individuals and departments .

10: Scope of deployment – Small pilot deployments should be used to work through definition, presentation and measurement of the selected KPI’s. Users will be advocates for adoption if they can see how the information improves their contribution to the strategic goals. Incremental steps, done really really well.

Written by Terry Brown of Klipfolio.com who is working on KPI Dashboards

Enterprise Dashboard KPI Visualization for Control Engineering

Note: Combining the science of business scorecarding and its associated key performance indicators with the technology of dashboard software results in KPI dashboards which tell you just what you need to know, when and where you need to know it.

As I mentioned recently, I do a lot of research on enterprise dashboard matters by using some very costly databases that aren’t usually available to most people. As a result, I come across some great articles that would never see the light of day. Associates always ask me to forward the good ones, and I’d thought I’d do the same for you Dashboard Spy readers. Here is a great article on using KPI visualization in the area of control engineering:

Copyright 2007 Reed Business Information, US, a division of Reed Elsevier Inc.
All Rights Reserved
Control Engineering

January 1, 2007

SECTION: FEATURES; Information Control; Pg. 42

LENGTH: 1958 words

Behind the data walls and executive dashboards, metrics increasingly link to real-time, plant-floor data. Do you know enough about key performance indicators (KPIs) and manufacturing execution systems (MES) to invest in efficient process improvements?

BYLINE: Frank O. Smith, Control Engineering


Process control is hip. Manufacturing line optimization is cool. Everybody from the CEO down wants a piece of it. And judging by the clamor around video walls, information portals, and executive dashboards, everyone wants to visualize control to drive performance.

Globalization and pressure on companies to compete at the most fundamental level of operational excellence means everyone is looking at the factory, the mill, or the plant more closely. The challenge is whether control engineering can master the strategic role implicit in this level of scrutiny.

What’s strategic in the role can be summed up in three letters: K-P-I. Key performance indicators are measures of success or compliance, and the industry is rife with talk of them. Industry analysts, organizations, and enterprises are deep in exploration and analysis of how to define, create, calculate, and disseminate KPIs. And control engineers need to understand them, so they can help in meaningful ways.

“Control engineers can no longer afford to be satisfied with a controller’s performance by viewing how straight the lines on the control chart are,” says John Snodgrass, process control engineer for Chemtura Corp.’s Morgantown, WV, specialty chemical plant.

“In today’s market, with today’s technology, typical operators aren’t typical anymore. They don’t just turn valves. Control engineers shouldn’t be typical either,” says Snodgrass. “They should be looking at the information they have and trying to determine what they can do make product more efficiently. If they don’t, it won’t be long before there won’t be a business anymore.”

MESA International, a trade organization constellated around manufacturing execution systems (MES), has actively investigated KPIs since early last year. In October 2006, it published an initial report on its efforts called “Metrics That Matter: Uncovering KPIs that Justify Operational Improvements.” But MESA concedes it’s just scratched the surface of a very complex issue. Consequently, MESA working groups are delving deeper still, particularly into the labyrinthine linkages between KPIs at different business levels within organizations. A common thread is the control layer of a plant, where the data that feeds many KPIs is generated. Another common element is MES. That’s where aggregation of the data and context typically happens before it’s passed up to enterprise systems and their dashboard and video wall outputs that executive management sees.

KPIs and MES are becoming critical elements to running a plant, because they’re critical to running a business. And, in today’s world, anyone not thinking about the business isn’t as keyed in to business performance as they should be.

Engineering to money

“Everything a control engineer does eventually rolls up into money,” says Jim Christian, product manager for Honeywell. “It’s important to know how what you do contributes to the business. Stable operations translate into greater reliability and better throughput, longer catalyst and equipment life, and reduced use of chemicals. All are measurable.”

One way for control engineers to think about KPIs is “to understand how what you do contributes to your company making money,” says Christian. “What measurements do you need to track to know you’re maximizing value?”

A vast, widely prevalent disconnect exists in most organizations today, at various levels of conceptual abstraction as well as within data flow linkages. “Many of our customers view KPI initiatives as an engineering exercise to collect data,” says Claus Abildgren, marketing program manager for production and program development for Wonderware. “They don’t understand the key needs of the business-what are the pieces of data that the rest of the business needs to make decisions.”

The disconnect is from the bottom up and from the top down. This was revealed in the survey behind the report that MESA developed. “The big surprise for us was how few felt they had a strong structure of linkage from financial goals and metrics down to the shop floor,” says Julie Fraser, principal at Industry Directions, a Boston-area industry consultancy. Fraser was contracted to help drive the MESA “Metrics That Matter” project. “Only 3% of the companies we interviewed for the study said they had strong linkage. We didn’t expect it to be huge, but we didn’t expect it to be that small either.”

Contributing to the problem is structural weaknesses in technology infrastructure-in other words, an inability to handle the vast number of applications and amounts of data now being generated. “AMR has worked with manufacturers who have over 5,000 point solutions [applications] in their portfolio,” says Colin Masson, research director for supply networks operations for Boston-based AMR Research.

Under such circumstances, “it’s not trivial to construct metrics at a single site, let alone across multiple sites. That’s not even taking into consideration performance data composed of millions of readings a day and putting that into some business context that has meaning at a higher level,” he says.

Growth of data models

Such a wealth of data possibilities is contributing to a growing emphasis on standardization of manufacturing data models that can establish commonality.

This desire for a comprehensive manufacturing model is one of the market drivers behind the surge in interest in MES, for models are core to functionality of this software. It’s also what’s prompted automation and human machine interface (HMI) vendors to invest in model-driven integration infrastructures, such as Rockwell Automation’s FactoryTalk, GE Fanuc’s Proficy, and Invensys’ Infusion platforms. These create “one common computing environment,” in the words of Peter Martin, vice president of strategic ventures for Invensys.

Other factors driving interest in MES and KPIs are increasing product and manufacturing complexity and regulatory compliance needs. Align Technology manufactures a line of individually custom-designed orthodontic appliances used to properly align teeth. Orders are acquired at its Santa Clara, CA, headquarters, then transmitted to its facility in Costa Rica where clinical technicians create 3-D representations. These are then sent back to prescribing doctors for verification, before work orders are transmitted to the company’s Mexico production facility.

Align Technology turned to MES to enable it to recover the detailed visibility of manual tracking that was lost when it implemented an enterprise resource planning (ERP) system. It uses Production Centre MES, a component of the Rockwell Automation FactoryTalk platform.

“We manufacture 50,000 custom parts every day-and every day it’s a different 50,000,” says Michael Henry, Align Technology CIO and vice president of information and technology. “The KPIs we’ve focused on initially are public facing ones-cycle times and gross margins.”

By providing greater visibility into the flow and status of work orders, MES was instrumental in attacking problems that had driven up cycle times and eaten into gross margins. Real-time visibility into the status of work orders in the geographically distributed process flow helped Align Technology identify bottlenecks and smooth the flow into a more seamless stream, from order acquisition to finishing and polishing at the end of the production line.

“Cycle times, which were two weeks, are down to five days. And gross margins that were 58% are now up to 69-70%,” Henry says.

Regulation helps?

Business metrics such as cycle times and gross margins are relatively common. Determining which manufacturing metrics are critical to track can be more challenging-unless you’re in a regulated industry. Utica, NY-based ConMed Corp., manufacturer of laparoscopic surgery products, patient support and other products, is bound by U.S. 21CFR Part 820 regulations outlining Good Manufacturing Practices (GMP) for medical device manufacturers.

“We operate in a tightly regulated industry, which determines which KPIs you must track and trend in your operations. We have to validate the process to begin with to determine the variable data, such that when I set these indicators, and the numbers are being met, I know the process is stable,” says Bill Wheatley, ConMed manager of manufacturing engineering.

“After that, we’re looking at: first pass yields, are we staying capable, is preventive maintenance effective, [and] are we preventing breakdowns.” Implementing MES was crucial to streamlining the process, Wheatley says. ConMed uses Visiprise MES.

How does a company determine which KPIs to use? Ideally, it’s a top-down decision. “In concept, you want to take top-level goals and strategy and build out departmental metrics and link those all the way down,” says Fraser of Industry Directions. “You want to take manufacturing metrics…right down to the machines.”

Eliminating boundaries

The whole issue of KPIs is exceedingly complex and, as a “science,” is still young and evolving. But progress being made is looping downward evermore inclusively into the domain of control engineering. As work in the field is revealing, everything is indeed interconnected.

“The traditional approach over the years has held that automation, MES, and IT are three individual layers,” Martin says. “But it’s an artificial separation. It’s been necessary, he says, because past technology required that problems be segmented to deal with the unique requirements at each level.

As the technology evolves, “in time that separation is going away,” Martin says. The push for KPIs linked from top floor to shop floor is driving the business case for that. And in the end, it truly is all about the business case.

That’s what control engineers need to grasp and master. Forget the flashy data walls and Web-enabled dashboards. Focus instead on the metrics that feed those broadcasts. Your company’s future will be increasingly dependent on converting control data into strategic intelligence.

KPIs in action

Frank O. Smith

Performance measurement is not static: it evolves as performance issues vary, as the market strategy changes, as technologies and the means to measure and record performance evolves over time. Sources of KPIs that others are using include research from MESA, AMR, and Aberdeen. Here are specific recommendations from users.

“We manufacture 50,000 custom parts every day…. The KPIs we’ve focused on initially are public facing ones-cycle times and gross margins.”

“We operate in a tightly regulated industry, which determines which KPIs you must track and trend in your operations. After that, we’re looking at: first-pass yields, are we staying capable, is preventive maintenance effective, [and] are we preventing breakdowns.”

“Corporate division headquarters set up the specific KPIs they wanted us to use.”

“We put no more than four [KPIs] in front of an operator at any time.”

“Our company has determined as an organization that there are four process areas that the enterprise needs to focus on. These include: financial; process excellence; quality; and environmental health, safety and security. Everything cascades down from those.”

“At the operator level, we use the real-time accounting measurement of cost-per-batch. With that an operator can drive down costs and keep quality high. The next layer up doesn’t want to see the cost of everything, but [wants instead] variance-to-budget of production, or variance-to-budget from department to department. It’s all based on the same data that flows up.”

For more information visit:









See also:

“Proving Control System Performance” under www.controleng.com/archive July 2006.

LOAD-DATE: January 17, 2007

The Dashboard Spy Business Intelligence Dashboards Blog