While brevity may be the soul of wit, when it comes to market research it can also leave you witless.
In my office (remember offices?) there’s a framed cartoon hanging over my desk. I cut it out of a magazine (remember magazines?) years ago. It was from Punch, a British humor and satire magazine, which had been around since 1841. That magazine literally coined the term “cartoon” in its modern sense, as a humorous illustration.
I had subscribed to Punch since the early 1970’s until it ceased publication in 1996. I had done graduate work in England and worked in London, so I was pretty much an avowed Anglophile. I mention that because while some of what appeared in Punch was archetypically English, a lot of it was filled with parody, wit, and Monty Pythonesque irony, from which anyone could get a good laugh. Thus, a cartoon in an 8” X 8” Birdseye maple frame ended up hanging in my office. Since a good researcher always gives attribution, I’d credit the cartoonist, but the signature is just a scrawl, so regretfully I am unable to.
It’s a simple line drawing showing a man with a shopping bag and a woman with a clipboard in what was unmistakably a central location intercept (remember central location intercepts?). That’s a research collection technique where respondents were approached, i.e., intercepted, in high-traffic locations and asked to participate in surveys. That was, of course, before everything went digital, but in our defense, we did have proper clipboards!
Anyway, the caption under the cartoon reads, “Excuse me sir, do you have 15 minutes to waste answering some fatuous questions?”
The magazine isn’t around anymore, but the cartoon endures, as prescient now as it was then. I keep it as a professional aide-memoire. A visual mnemonic, if you will. The caption is timeless in its way, suggesting that researchers should respect the consumer and their time. These days it seems a lot of “fatuous” stuff gets crammed into surveys. Digital surveys seem to have made that easier.
But just because you can, doesn’t mean you should. You shouldn’t add complexity when simplicity will do. You shouldn’t ask respondents series of never-ending questions and/or incessant scalar ratings. Or more ham-handedly, do it because you hope to unearth something. That there might be an insight in there somewhere! More often than not, all that strategy adds is time and expense. It turns out, however, time has a wonderful way of showing us what’s really important. Doubt me? You shouldn’t.
As your survey length increases, so does non-response bias. That means subjects refuse to take part in the study or drop out before completing the survey. Fun facts: Response rates of qualified respondents decrease about 10% for every 8 minutes you add to a survey’s length beyond 10 minutes. And subjects who refuse to take part in a survey or those who drop out before completing the survey? They’re systematically different than those who participate. Oops!
Years ago – meaning before online surveys became de rigueur – telephone, mail, and yes, central-location intercepts were the ways we collected answers to our questions. Sometimes – but mostly not – we offered co-op (cooperation) fees to get folks to participate. Today, people sign up to have questionnaires sent to them. And expect co-op. Points, or gift cards, some sort of incentive to entice their participation. So not an entirely altruistic exercise to begin with. And, as the famous Peter Steiner New Yorker cartoon caption goes, “On the Internet, nobody knows you’re a dog.” (His signature is a scrawl too, but I looked it up on the Internet. And yeah, hard not to see the humor in that!) Maybe today it would be, “Nobody knows you’re a bot.”
But research techniques should move with the times. Internet surveys are very popular. Scalar measures on the Internet even more so. But there’s a limit to how many you can put in front of respondents before you get, what’s known in professional vernacular as, “monkey marks.” When respondents select the first choice in every question. Or straight-line grid questions. Or simply answer at random. That kind of respondent behavior renders data pretty-much worthless. And, yes, our online authority assures me you can specify degrees of security. But reality remains steadfast: collecting lots of assessments doesn’t guarantee lots of insights.
The Thin Line Between Fatuous and Witless
And, OK, while brevity may be the soul of wit, when it comes to market research it can also leave you witless. Take Net Promotor Scores. No, seriously, take them. (Sorry. I was channeling my inner-Henny Youngman.) NPS is attractive because it’s a single number that requires no investment of intellectual capital. But single-number research can kill you.
Single numbers lack the capacity to identify what actually drives responses, or more importantly, behavior. Or what to do when you have to. It’s a lagging-indicator, about the last time someone did something. Predictive measures are always leading-indicators. The solution to these downsides? Combine NPS with techniques that do provide actionable, predictive insights. But if you have to do that, why go with a single number in the first place?
A more complicated consumer, brandscape, and marketplace imposes new demands on researchers. Including older counsel (circa 1972) about not wasting consumer’s time answering fatuous questions. Count yourself lucky if you have an expert to guide you on your research jaunts. Oh, not me. I design research and have history to rely upon. That’s important, of course, but eventually you have to execute. Our on-line field expert, Rob Luciano of IntelliSurvey, is our guru about all things digital and supports, but more importantly, understands, our philosophical and practical approach to what the rewards of a good research design should look like. You need to find a “Rob” too.
I think I’ve mentioned before, the approach we take is grounded in psychology. That allows us to keep questionnaire lengths manageable. At the same time, we can measure what consumers think, as opposed to what they say they think, and take approximately 10 minutes of the consumer’s time, which makes it efficient. We jettison fatuous questions, and don’t waste our client’s money. (See, I do follow my own advice!) Brands love emotional engagement metrics because they’re leading-indicators, predictive of future in-market behavior. So, lots of prophetic insights without having to ask lots of lackluster questions.
And the brands that rely on it? The ones that don‘t have to sift through “Big Data” to unearth an outcome, the ones that don’t have to content themselves with the ambiguities of single-number grades, or the ones that aren’t forced to guess? Well, usually they’re the ones laughing all the way to the bank!
Robert Passikoff is founder and CEO of Brand Keys. He has received several awards for market research innovation including the prestigious Gold Ogilvy Award and is the author of 3 marketing and branding books including the best-seller, Predicting Market Success. Robert is also a frequent contributor to TheCustomer.
Photo by Vincent Guth on Unsplash.
