Not so long ago, my friend Dennis posted an infographic about the misuse (accidential or wilful) of data in infographics. In a handy infographic format. I’m going to take the opportunity to embed it below. It’s worth keeping.
But Dennis’s nifty graphic only tells us about one place where we can be led into temptation — the infographic.
I happened upon a newsletter today that made me think of how easy it is to make marketing and communication decisions or take action based on information that should be questioned.
Mobile Commerce Daily reported on May 29 that “44pc of shoppers will never return to sites that are not mobile friendly: report.” The story is based entirely on a survey carried out by US software company Kentico, which makes content management systems. Kentico issued a news release about the survey on May 28, but it could be that the newsletter had an embargoed copy of the release.
The information is interesting. For example, it says that nearly 9 in 10 people with smartphones use them to compare products to competitors. And 45% do it right in the store, underlining the practice of “showrooming.”
But… in the newsletter story, there’s no information at all about the survey data. Even more frustrating is the lack of a link to the source data. I tracked down Kentico, then hit their press centre, where the news release about the survey sits. If you go to the Kentico site, you discover that the data-gathering part of this survey consisted of “More than 300 US residents 18 years old and over participated in the Kentico Mobile Experience Survey, conducted online during the month of April, 2013.”
Now, a survey sample is neither good nor bad. The point is to understand that sample. Was it a random sample? Did the participants selfselect? I couldn’t tell anything more than what I just said, because Kentico didn’t link to the survey itself or a more detailed report of its findings.
I contacted Kentico’s PR company, and Chris Blake of MSR Communications was prompt, open and detailed in his responses to my questions. He gave me demographic information that SurveyMonkey, the tool they used to do the research, provided, and a copy of the questionnaire. After a brief perusal of some USA census data, I learned that their sample of 300 people skewed only slightly more male, somewhat older, and way more educated than the US general population, for one thing. And the data provided on their sample gives me a sense of the potential sampling error rate (while Chris Blake suggests a ±5% margin of error, I’m thinking more like ±10%).
I don’t think there’s ANYTHING bogus about the survey results here. But I needed to take a fair amount of time to convince myself of that. And there are many occasions on which I find the data or survey results so problematic that I forget about using them.
There’s a flood of survey results and other materials that get published by the originators of the information, by newsletters, and by people like me every minute of every day. It’s easy to take everything at face value. But think twice. As a teacher of social media, I’m constantly looking for good data to share with students. As a consultant, I’m looking for information that I can use to help clients make sound decisions. But it is dangerous to see a newsletter article and use it to tell students or clients to base their actions on the data it contains.
Back in the days when ink and paper cost money, I understand the need for brevity and concision. But these newsletters are electronic. Pixels don’t cost anything but the time to write. And if you’re not going to disclose proprietary or competitive information, why not make as much information as you can readily available?
The more easily people like me can peruse your research, the more likely we’ll be to accept its conclusions. The more difficulty we have understanding the process behind the numbers, the more skeptical we become (or at least the more skeptical we SHOULD become).
And if you’re in business and trying to grapple with the challenges of communicating using social media, either desktop-style or mobile, make sure to ask questions EVERY time you see statistics and survey results. You don’t want to have to explain to your boss why you made a bad marketing or sales decision based on data you found in a press release and didn’t vet.
It’s too generous to assume that just because someone writes a newsletter, they’re doing your due diligence for you.
Here’s Dennis’s great graphic:
