Once we make one of our marketing KPIs a goal for success, we can inadvertently destroy the value of the KPI as a measure of success.
I wish I’d been taught Goodhart’s Law when I was in school. In fact, I wish everyone was taught Goodhart’s law. Understanding this axiom would improve the way we lead and measure any discipline, but it’s crucial for those seeking to improve marketing and customer experience. Goodhart’s law is:
“When a measure becomes a target, it ceases to be a good measure”
(Actually, that’s the simplified version. Since the adage comes out of eco nomics, it is stated in a more scientific and fancy way: “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” Clearly, simpler is better.)
by Augie Ray
The point is that once we make one of our marketing KPIs a goal for measuring success, we can quickly and inadvertently destroy the value of the KPI as a measure of success. If that sounds counterintuitive, let’s explore three real-world examples:
Setting targets for social media likes and followers destroyed the value of social media likes and followers
In the early days of social media, brands providing the best experience and earning the most significant loyalty also collected the most likes and followers. Had brands merely competed for authentic likes and followers by providing the best customer experience, then tallies of their followers and likes could’ve remained a valuable sign of affinity, loyalty, and interest. Instead, marketing departments set targets to increase followers and likes, each wanting more than the competition. As a result, marketers deployed strategies to deliver on the goal, such as giving away free product, Farmville items, and entries into sweepstakes in return for follows and likes. Targets were achieved, but those strategies annihilated likes and followers as a valid measure of, well, anything. And it isn’t just that brands ruined the value of those social metric metrics; the brands that bought fans with freebies also found they collected less meaningful fans–not people interested in or with an affinity for the brand, but people who just wanted free stuff. Likes soared, engagement plunged, and no one benefited. Social media marketers ignored Goodhart’s Law and learned the hard way that how we earned fans and likes mattered more than that we improved our fan and like measures.
How you increase your email subscriber list affects if your efforts help or hurt your brand
Email helps us reach our customers with lower cost and less interference than ads and social media posts, so marketers naturally want as many subscribers as possible. But, once we set targets for more extensive email lists, we inspire list-building activities that deliver on the goal while doing nothing for (or harming) our marketing objectives. Buying lists, giving rewards for subscribers, and automatically adding customers to mailing lists without consent efficiently produce larger lists, but they also encourage lower open rates, diminished deliverability, customer annoyance, and spam reports. By setting a target, we encourage strategies to hit the goal without proper consideration for whether those strategies do more to help or harm overall marketing objectives. We cannot celebrate that we added addresses to our email list without also understanding how we achieved that target.
Decreasing your Cost of Acquisition is not beneficial if acquisition strategies also decrease the lifetime value of customers acquired
One of the most critical metrics in the marketing world is also subject to Goodhart’s Law. Of course, marketers want to reduce their cost of acquisition. But when we set this as a goal, what sorts of behaviors and tactics do we encourage? A credit card company found that many acquisition strategies were, inadvertently or not, designed to collect people who game the system, switching from one credit card to the next merely to collect as many points and perks as possible. The cost of acquiring these customers was meager compared to acquiring affluent customers more likely to remain loyal and deliver higher lifetime value. When the brand made the right offer with a competitive promise of free points, the network of blogs and communities dedicated to points gamers would spread the word and deliver clicks and conversions. Without considering who we acquire, we miss that how we lower the cost of acquisition is more important than that we lower that measure.
This list could go on and on. Net promoter score (NPS) can be abused by altering surveys and sample rules. Ad cost per impression can be reduced by using cheaper, lower-quality ad buys. A bank that set new accounts as a target encouraged illegal activity to open accounts for customers who didn’t want them. An automaker targeting lower emissions manipulated its testing to show emissions 40x lower than real-world driving conditions. Even bottom-line profit itself is subject to Goodhart’s law–how many accounting scandals have we seen where company leaders altered bookkeeping to give the appearance of a higher margin (and thus receive more generous bonuses)? There is literally no measurement for which you can set a target that cannot be controlled, abused, or sabotaged, whether purposefully or unintentionally.
Before we discuss solutions, let’s make a couple of things clear:
- We’re not talking about bad metrics. These are good metrics. Great metrics, in fact. Or, at least, they start that way.
- Setting goals is not wrong. I’m sure it goes without saying that setting targets is necessary and appropriate.
So, how do we prevent good metrics from going bad and ensure our targets don’t encourage damaging behaviors and strategies? The solution is to put more due diligence into setting and monitoring goals:
- Recognize the intent of the measures. The first step is to fully understand the intent of the measure. If you understand why it’s meaningful and valuable, you can begin to understand and prevent the ways your team can meet the target while eviscerating its intended value. If authentic social media likes are a vital sign your brand is doing the right thing, then don’t accept strategies that fabricate likes in other ways. If your intent is to improve customer satisfaction and loyalty, then ensure your Voice of the Customer program doesn’t alter sampling rules in a way that improves NPS or CSAT without really affecting your customers’ experiences. In other words, ensure you deliver on the goal, not just the target.
- Target right prospects and customers. One of the greatest dangers of Goodhart’s Law to marketers is that the plans to deliver on marketing targets often shift focus from high-value to lower-value segments. If lowering the costs to acquire customers is essential, then ensure the strategies aligned to that goal don’t do even more to reduce the value of the customers acquired. If you want an email list of valuable customers and prospects, make sure you grow your list with people who show an interest and provide their explicit consent. Measure the achievement of your targets only for the right prospects and customers as defined in your marketing strategy.
- Don’t reward employees for single or limited KPI performance. We improve our marketing KPIs and strategies by avoiding a single target or narrow set of targets and implementing broader goals and success measures. If you set a target to lift the number of accounts your bank customers open, ensure you drive customer value with related targets for customer engagement in those new accounts and customer satisfaction 90 days after new account openings. If you want to grow your email list, be sure you do so without harming open, click, and delivery rates. The narrower the set of measures you use as targets, the easier it is to manipulate the results or hit targets while damaging business outcomes.
- Measure longer-term impacts. Take care not to set targets that are only short-term in nature. Don’t just measure the cost to acquire the customer–also measure if those customers are retained or attrite within a year, ensuring your marketing strategies attract more valuable prospects. Don’t just target more social media engagement–also measure if the customers engaging in social media click to your site and become leads. Think beyond the click and set targets closer to your meaningful marketing objectives.
- Audit, audit, audit. Any goal or measure can be manipulated, so make sure it isn’t. Don’t simply rely on the social media team to report on its own engagement or the ad team to report on cost per acquisition. Invest the time and resources to ensure your targets are delivering on the original intent of the measures.
If you follow these guidelines, you can ensure at the end of 2022, you will have not just achieved a set of targets but delivered on your more significant and more substantial marketing objectives. And when you do, you can thank Charles Goodhart (and me).
Augie Ray is a Vice President Analyst covering customer experience (CX) for marketing and CX leaders. His coverage topics include the ROI of CX, CX strategy and governance, how CX leaders secure and retain sponsorship, the buy/own/advocate customer journey, voice of the customer (VoC) and survey strategies, customer journey mapping, CX analytics and measurement, the role of social media and word of mouth (WOM) in CX, and persona development.
This article originally appeared on Gartner. Photo by Алекс Арцибашев on Unsplash.
