Why “% deviation from trailing average” indicators on dashboards aren’t helpful (book excerpt)

As I’ve discussed in a previous blog post, a dashboard that doesn’t visually flag metrics that require attention will likely flop with users. In fact, the lack of such indicators could be the number one reason why so many dashboards fail to deliver acceptable levels of user satisfaction and traction.

While not as common on dashboards as other flagging methods, participants in my Practical Dashboards workshops often ask about “% deviation from trailing average”, so I’ve written this post to illustrate why that method actually isn’t much better than the more common alternatives.

Read more

Highlight “obviously wrong” values on dashboards!

Because errors do happen, our dashboards will sometimes contain “obviously wrong” metric values, such as a “Customer Satisfaction Rating” of 12.5 on 10, or a “Manufacturing Defect Rate” of -14%. It’s essential that our dashboards be “smart” enough to detect such obviously wrong values so that we can visually flag them as incorrect on the dashboard. If we don’t flag obviously wrong values, users will likely notice them anyway, putting the accuracy of every other value on the dashboard into question.

Read more

Automatically flag metrics that require attention on dashboards using statistics (book excerpt)

In order to gain traction and acceptance among users, dashboards must visually flag metrics that are underperforming, overperforming, or behaving in other ways that warrant attention. If a dashboard doesn’t flag metrics, it becomes very time-consuming for users to review the dashboard and spot metrics that require attention among a potentially large number of metrics, and metrics that urgently require attention risk going unnoticed. In previous blog posts, I discussed several common ways to determine which metrics to flag on a dashboard, including good/satisfactory/poor ranges, % change vs. previous period, % deviation from target, and the “four-threshold” method. Most of these methods, however, require users to manually set alert levels for each metric so that the dashboard can determine when to flag it, but users rarely have the time set flags for all of the metrics on a dashboard. Techniques from the field of Statistical Process Control can be used to automatically generate reasonable default alert levels for metrics that users don’t have time to set levels for manually.

Read more

Why I now use “four-threshold” flags on dashboards (book excerpt)

In order for a dashboard to gain traction among users, it must visually flag metrics in order to draw users’ attention to metrics that require it. Unfortunately, though, the most common methods of flagging metrics on dashboards (“Vs. previous period,” “Single-threshold flags,” “% deviation from target flags,” and “Good/Satisfactory/Poor ranges”) are prone to several problems: they often flag metrics that don’t require attention, fail to flag metrics that do, and can be slow to visually scan. In this post, I discuss the “four-threshold” method that I now use, since it doesn’t have these shortcomings.

This post is an excerpt from my upcoming book, Practical Dashboards, and is the seventh in an eight-part series of posts on how to determine which metrics to visually flag on a dashboard.

Read more

Good/Satisfactory/Poor ranges on dashboards: Not as effective as they seem (book excerpt)

This excerpt from my upcoming book, Practical Dashboards, is the sixth in an eight-part series on how to determine which metrics to visually flag on a dashboard (i.e., with alert dots, different-colored text, etc.) in order to draw attention to metrics that require it. In this post, I look at the “Good/Satisfactory/Poor” method used on many dashboards. While not as problematic as the “vs. previous period,” “single-threshold,” or “% deviation from target” methods that I discussed in previous posts, this method still has several serious drawbacks that become obvious when pointed out. In the next post in this series, I’ll introduce a more useful approach called “four-threshold” visual flags.

Read more

"% Deviation from Target" flags: Confusion masquerading as context (book excerpt)

This excerpt from my upcoming book, Practical Dashboards, is the fifth in an eight-part series on determining which metrics to visually flag on a dashboard (i.e., with alert dots, different-colored text, etc.) in order to draw attention to metrics that require it. In this post, I look at what I call the “% deviation from target” method of flagging metrics on a dashboard. I explain why, despite seeming like an improvement upon single-threshold flags, and being used on many dashboards, “% deviation from target” flags can easily mislead. In a later post in this series, I’ll introduce a more useful way to flag metrics on dashboards called the “four-threshold” method.

Read more

Single-threshold flags on dashboards: Very common and very problematic (book excerpt)

This excerpt from my upcoming book, Practical Dashboards, is the fourth in an eight-part series on determining which metrics to visually flag on a dashboard (i.e., with alert dots, different-colored text, etc.) in order to draw attention to metrics that require it. In this post, I look at the “single-threshold” method of determining which metrics to flag and why, despite being extremely common, this method has several major drawbacks that become obvious when pointed out. In a later post in this series, I introduce a more useful approach called “four-threshold” visual flags.

Read more

“% Change From previous day/week/month” on dashboards: Worse than useless? (book excerpt)

This excerpt from my upcoming book, Practical Dashboards, is the third in an eight-part series on how to determine which metrics to visually flag on a dashboard (i.e., with alert dots, different-colored text, etc.) in order to draw attention to metrics that require it. In this post, I look at the “vs. previous period” method of flagging dashboard metrics and why, despite being extremely common, this method for drawing attention to metrics can be worse than useless. In a later post in this series, I’ll introduce a more useful approach called “four-threshold” visual flags.

Read more

Performance targets aren’t the same as alert thresholds (book excerpt)

This excerpt from my upcoming book, Practical Dashboards, is the second in an eight-part series of posts on how to determine which metrics to visually flag on a dashboard (i.e., with alert dots or different-colored text) in order to draw attention to metrics that require it. Determining when to flag or not flag metrics on a dashboard can be a messy process within organizations because people often disagree about what should be considered “good” or “bad” ranges for a given metric. There’s another, less obvious cause of controversy in such discussions, though, which is that people often talk about two very different types of flagging criteria without realizing it: criteria for indicating when action is required and criteria for indicating whether a metric is performing well or not. While these might sound similar, their fundamental purposes and the ways that we go about setting them are very different.

Read more

Include visual flags on dashboards!!! (book excerpt)

This excerpt from my upcoming book, Practical Dashboards, is the first in a eight-part series of posts on how to determine which metrics to visually flag on a dashboard (i.e., with alert dots, different-colored text, etc.) in order to draw attention to metrics that require it. In this post, I briefly discuss why visual flags are almost essential in order for dashboards to deliver the user traction, satisfaction and value that organizations hope for and expect, and why a lack of visual flags has contributed to the failure of many dashboards.

Read more

Video: Why dashboards for Problem Scanning AND performance monitoring don’t work

I’ve seen a lot of dashboards that failed to meet users’ and organizations’ expectations. There are a variety of reasons why this happens and, in this video and post, I focus on one of the most common ones, which is that the people who designed the dashboard didn’t fully understand the distinction between problem scanning and performance monitoring. When this happens, the dashboards that they end up designing don’t fulfill either of these needs well. This video and post are based on a chapter from my upcoming book, Practical Dashboards.

Read more

New book coming in 2020: "Practical Dashboards" – feedback needed!

Despite the fact that books and courses on information dashboard design have been available for years, many dashboards still fail to meet users’ and organizations’ expectations. Users have trouble finding answers to basic data-related questions and fail to notice urgent problems because they aren’t obvious (or possibly even displayed) on dashboards. Because of these and other issues, many dashboards end up under-used or even abandoned. Based on Nick Desbarats’ experiences designing dashboards for over 50 organizations, Practical Dashboards uncovers the real reasons why so many dashboards end up disappointing users and organizations; reasons that go far deeper than the visual design on which most dashboard books and courses focus. Readers will learn a practical, actionable framework for creating a set of purpose-specific information displays that enable users to quickly and accurately answer their data-related questions, including fundamental ones such as, “Is everything O.K. at the moment?” often for the first time.

Read more