July 8th, 2012
Key to getting high-quality, informative data from your market research is to ensure you have a solid representative sample of your consumer base – partly by having a large sample size, and partly by selecting your sample from a broad and unbiased range of demographics.
Here’s an excellent example of how it’s not to be done, from the Daily Mail, June 14th: Are X-rated bachelor parties on the way out?
The rise of low-key grooms who want pre-wedding bashes WITHOUT the strippers It seems that booze-fueled bachelor parties filled with strippers are slowly becoming a thing of the past.
A survey conducted on Tuesday by TheKnot.com, the wedding planning website, found that 61per cent of brides-to-be claim that their fiances are planning ‘low-key’ bachelor parties.
‘Tamer’ dinners and relaxed drinks are proving to be popular options for grooms-to-be these days.
Conducted on the website’s Facebook page, the survey received 61 responses from brides; 37 of them said their fiances didn’t want a wild bachelor event while 24 of them said they did.
Setting aside the ridiculous assumptions that a) stag do’s should involve strippers for them to be ‘real’, and b) stag do’s without strippers are ‘tame’, what we have here is a sweeping statement about the trends and habits of a nation… based on 61 people, surveyed via Facebook poll. This very poll, in fact:
In terms of survey sampling, this is abysmal. Not only is a sample size of 61 respondents too small to be able to assume anything at all, but given that the difference between the ‘Yes’ and ‘No’ options is just 7 people, it’s extremely unlikely to be reliable.
Added to that, we have the complications of the vague question wording – what qualifies as tame? And what if the part will involve a ‘tame’ activity and then something more raucous – which option should be ticked? Notice, in fact, that the strippers found in the headline of the Daily Mail aren’t even found in the question.
Finally, how random is our sample of 61 respondents? Bearing in mind that the survey was conducted on The Knot’s Facebook page, the audience of which is overwhelmingly female. Even the very question isn’t directed at the attendees of the stag party – instead we’re measuring the assumptions and beliefs of 61 women who already subscribe to a wedding planner’s Facebook page, and then extrapolating the response up to be a verdict on the state of the nation.
I have no idea whether in actuality men are visiting strippers more or less commonly on their stag do. But given the shoddy and weak nature of the survey by The Knot, neither do they, and neither do the newspapers running the story.