"% Deviation from Target" flags: Confusion masquerading as context (book excerpt)

tl;dr: This excerpt from my upcoming book, Beyond Dashboards, is the fifth in an eight-part series on determining which metrics to visually flag on a dashboard (i.e., with alert dots, different-colored text, etc.) in order to draw attention to metrics that require it. In this post, I look at what I call the “% deviation from target” method of flagging metrics on a dashboard. I explain why, despite seeming like an improvement upon single-threshold flags, and being used on many dashboards, “% deviation from target” flags can easily mislead. In a later post in this series, I’ll introduce a more useful way to flag metrics on dashboards called the “four-threshold” method.

One of the most common ways to flag dashboard metrics that require attention is the “% deviation from target” method, whereby a percentage deviation from each metric’s target value is shown beside its current value:

deviation from target percentage.png

This might seem like a good idea since higher “% deviation from target” percentages should indicate metrics that require more attention. Consider, though, that a deviation of -3% from the target for a metric like “Average Payment Processing Time” may be a minor concern that requires no action, but the exact same deviation of -3% from target for “% of Payments Processed Successfully” would be an all-hands-on-deck crisis. In other words, a relatively large “% deviation from target” may require no action and a relatively low one may indicate a catastrophe, so “% deviation from target” values can’t be relied on to draw attention to metrics that require it.

This method can also get a bit confusing for users since, for some metrics on a dashboard, positive deviations from target are desirable (e.g., Sales) but, for others, positive deviations may be undesirable (e.g., Expenses). “Deviation from target” percentages can, therefore, be slow to scan and interpret on dashboards that contain a mix of “higher is better” and “lower is better” metrics which, of course, is the case for most dashboards.

Additionally, because they add more text to the dashboard, “% deviation from target” flags also reduce the user’s scanning speed for the dashboard as a whole, undermining its effectiveness. One of the reasons that we should use graphs rather than text to display information on dashboards is because we can visually process graphical information more quickly. Textual information must be read and interpreted in a slow, sequential manner, so the more text a dashboard contains, the slower it will be to use.

In the next post in this series, we’ll review the last of the four common-but-ineffective flagging methods that I see on dashboards: Good/Satisfactory/Poor ranges. After that, I’ll introduce the four-threshold flags that I now recommend since this type of visual flag doesn’t have any of the drawbacks or limitations that I list for the four common-but-ineffective types. I'll then conclude the series with a post on useful statistics for setting visual flag thresholds automatically.

To be notified of future posts like this, subscribe to the Practical Reporting email list.