Feedback is the life blood of any business. Once you grow your business beyond constantly having personal interactions with your customers, you need some mechanism to tell you how the service or product you provide is meeting or not meeting your customers’ satisfaction. As the owner you eventually get the answer of course when people stop buying your product or using your service, but that’s usually way too late. There have been dozens, if not hundreds, of movies based on an owner or CEO masquerading as a customer-facing employee and discovering that disconnect. In fact, I understand there’s a whole TV series where that’s the premise.
The first traditional solution is to hire managers/franchise owners/etc. who supervise the people who actually interact with your customers and have them report to you on any problems. Of course, we all know that frequently doesn’t work. Whoever enhanced their own career by telling the boss that your customers are dissatisfied? When the boss/owner asks, “Are we doing well?” there’s only three possible answers:
- “No, the customers aren’t happy with how my employees treat them.”
- “No, our service/product sucks.”
- “Yes Sir. Everything’s perfect.”
The first answer means you’re not doing your job. The second answer means your boss isn’t doing his job. Is it any surprise you, the boss, only hear the third answer?
OK, now let’s get personal. Why am I thinking about this today? When I retired last year, I had to look into personal, rather than corporate, dental insurance. I picked the cheapest plan from a carrier I’d already been with. As the year has progressed, I became dissatisfied that they paid basically nothing for my, mostly minor, treatments. In September I decided to upgrade to the next tier of coverage, so I called them up and did that thing, or so I thought. They took my credit card to charge me for the “initiation fee” and the first 2 month’s coverage that would start in mid-October. I expected the amount I was paying to go up, but I was still getting billed at the old rate. Then in early December I got a bill for the new rate, but I noticed there was something off. The bill came to my address, but was for Frank Adams rather than Frank Hood. I spent a few hours on hold to get to somebody to talk to, and then another hour to get to somebody who could fix what had probably started as just a typo.
They decided that the best solution was to cancel “Frank Adams” policy and start over at mid-December for the real me. Then they’d issue me a refund check for what I’d paid into “Frank Adams” coverage. For the next week, I got a blizzard of notices for both me and “Frank Adams” about cancelling “his” policy and starting “my” new policy. When I received those notices, it occurred to me that there might be a problem, so I called them again and told them, that the refund needed to come to Frank Hood and not “Frank Adams”. Of course the refund came on the 30th to Frank Adams, so I called them again today to correct the matter.
So, here’s the question, when I participated in the voluntary survey after the call, the recording asked me, in light of this call:
“How likely would you be to recommend this dental plan to others? Please rate on a scale of 1-10.”
The answer to that question would have to be far from high no matter how courteously I had been treated by my current customer service representative who was likely to get the blame.
Then the recording asked:
“Has your issue been resolved? Press one for yes, two for no.”
Since this was my fourth call, and 3 times my issue had been left in a mess, but, each time, I was left believing it had been resolved, how should I answer the second question? There was no, “How the hell should I know?” option, so I hung up. What could I possibly answer that wouldn’t lead to the survey’s analyst to the wrong conclusion?
They paid some nincompoop good money to create that survey, and they’re still paying some bespectacled drone good money to report on its results regularly. I understand statistics, and I’ve created and run surveys, but mine were useful.
Most people’s usual response to such surveys is to just ignore them, but then there are polls about The Important Things™ .News organizations and politicians love polls. They pay pollsters big money and almost always draw either irrelevant or wrong conclusions from these polls. Yes, they get some propaganda value out of “push polls” where they frame the questions to get the answers they want, but what if they actually want to find out what people think? Consider that public polls in America have been asking whether abortion should be legal for over 50 years, and the results have been consistent for those 50 years.
- Two-thirds of Americans believe abortion should be legal (in certain circumstances).
- Two-thirds of Americans believe abortion should be illegal (in certain circumstances).
I’m nobody’s professional mathematician, but even I know that there are not four-thirds of Americans. Those results allow advocates on one side of the issue to say the vast majority of Americans believe that abortion should be legal, and the other side to also say that the vast majority of Americans believe that abortion should be illegal, based on exactly the same polls. (Neither side mentions the certain circumstances where the vast majority disfavor their point of view.)
Statisticians are fond of saying anecdotes (meaning your own experience and that of those you know personally) are not data. The thing they neglect to say is that statistics are not science. Statistics are a very useful investigative tool. Done right, they can tell you what to look at and where to look for it. What statistics are not is science. They prove nothing. Semmelweis is statistics telling him that washing his hands between operations decreased the likelihood of his patients dying. Pasteur is science showing why that is. When I drop a hammer, it will hit the ground, even if it bounces off my foot first, every time, not 99.9999% which is essentially the same as 100% as a statistician would say. Throwing infinite time or infinite trials into the equation makes no difference to the law of gravity. It happens because it’s physics, not statistics.
So, if I don’t bother with any of those idiotic customer surveys, why should I care? Because, guess what, so-called AI is all just statistics. Statistics might tell your AI-powered program that the word “red” is followed by “hot”, 40% of the time, “dress” 20% of the time, “scare” 10% of the time, and “sun”, 8% of the time, so when you hunt-and-peck or thumb-type “red” into your text message, it suggests “hot” as the next word. More sophisticated AI merely remembers some of your earlier sentences and calculates that if you mentioned “Superman” or “white dwarf” in a previous sentence, you’re more likely to want to type “sun” after red, BUT, and here’s the big but, statistics are honey badger, and honey badger don’t care! Unlike you, no computer is curious, ever!
Statistics are only useful to the curious, and only you—and cats—are curious. So be much more worried about cats taking over the world than AI. If you’re the boss, owner, or leader, what you have to constantly ask yourself is, “Am I really measuring what I think I’m measuring?” When you quit asking that question, it’s time to quit.
BTW no AI was used in making those twisted mental connections I just pieced this essay together with.
Musings of a Restless Mind 