Outcomes Matter: What ‘Moneyball’ Teaches Us About Picking the Right KPI

I think Moneyball is one of the best business books ever written, but it’s not a business book. It’s actually a book about baseball.  

So in baseball there’s things like Runs Batted In, or RBIs, that’s a really important statistic. For pitchers, Earned Run Average is a really important statistic in baseball, it’s one that everyone talks about. And player salaries are by and large determined on these metrics, that are kind of the industry standard metrics that everybody measures.

In the 70s, a group came along and said, “what if all these things we’re measuring in baseball don’t really impact the outcome to games like we think they do?” And [baseball managers] were really assumptive that ERA had an impact on the outcome of a season. And RBIs, for certain players, had an impact on the outcome of the season. Well these guys said, “let’s look at some things that nobody's looking at, that maybe are undervalued in the marketplace where we can find cheaper player salaries to hit these metrics. But let’s correlate them to outcomes.”

See what happened in baseball and what happens in business, is often times we measure what we think we should measure but we don’t correlate it to any outcomes. And I’m passionate about this idea - if you’re going to measure anything in your business you have to correlate it to an outcome.

So I read Moneyball, I love the book, it's one of the best business books ever written, and I’m in the call-center space; one of the most measured and metricked industries on the planet. So we measured everything: call times, and hold times, and occupancy rates. And our industry had this really, really, complicated Award of Excellence program, where we measured 20 different parameters on mystery shopping calls. We were measuring 10 calls a month, for every single agent that worked for us. That was 5,000 calls a month we were scoring on this national Award of Excellence program. So I wake up at three in the morning, I’m in the cold sweats and I think, “what if our business is like Moneyball, and what we think is quality is not really quality. We’ve never correlated it to any outcome, what if it doesn’t matter?”

I go in the next day, I get my COO, we sit down and I say, “listen let’s take these quality scores. We’ve got 24 data points, 24 offices, and let’s correlate it to customer retention. Because we can all agree that’s the number one KPI; how long we can keep our customers." So we do, and lo and behold we are in the Moneyball trap. Our quality scores are not correlating at all to customer retention. And because the universe has a sense of humor, our lowest of 24 in the company “quality-wise” - an office in Houma, Louisiana - had the highest customer retention. And the highest quality score location, had the lowest customer retention and then there was zero correlation in between the two. So we set about to find the, “what is the Moneyball statistic?” What is the key metric that does correlate to customer retention?

We decided we were going to “Net Promote.” Fred Reichheld’s Ultimate Question, Net Promoter methodology, but for employees not for customers and we were gonna do it by office. We go and we Net Promote the employees and it was one question - “Based on your experience in the last 90 days, how happy are you working at Appletree?” And they score it on a scale of one to ten, in the Net Promoting scoring style. We found that there was an 85% correlation between the Net Promoter Score for the employees and customer retention by office.

Now I know it sounds like rocket science that happy employees meant happy customers, but nobody was looking at it that way. We were looking at all wrong. If you’re gonna measure anything in your business, make sure you’re correlating it to outcomes. And most importantly, just because your industry has measured the blank KPI or the blank metric does not necessarily mean that that metric is meaningful in your business to create the outcomes that you want.

So - outcomes. Outcomes. Outcomes. Always, if you’re going to measure KPIs.