Market Research Axioms – If You Remember Anything Remember This

The value of a strong questionnaire design when complemented by the task of high quality sample development is not fully appreciated. Often these two essential building blocks of market research are relegated to the back of the line on research projects.

Research Axiom One: You can never fully recover from a poorly written questionnaire.

o No manipulation of the variables, regardless of how cleverly done
o No amount of analysis, regardless of how brilliant
o No degree of insightful interpretation, regardless of intellectual prowess
Nothing can save you from a poor research foundation. The building will collapse like a house of cards!

If there is one part of the research process that I know, it is questionnaire design. It is a task repeatedly given insufficient time and attention. Clients and research professional alike often underestimate the time it will take to develop a truly well structured and concise instrument.

What amazes me most? Project leaders relegate this task to a status depicted by the attitude of: “Once the questionnaire is done we can get on with the important stuff, like analysis and reporting.” The assumption that analysis work is the essence of the research and the expectation that interpreting the results is where the mastery of research ultimately lies is a mystery to me.

Have we not pounded the concept of garbage-in garbage-out into our heads? Can new internet tools substitute for critical thinking and the hard work of aligning the research instrument to the purpose of the study to answer the business questions that sponsors paid to learn?

If this seems like a bit of a rant, well I guess I am guilty. My own research-on-research including the use of a 25-point questionnaire audit system has shown me that even well healed researchers are less diligent about quality than one would hope. Research is not only science it is a craft [perhaps an art] and if the proper fundamentals are not applied the product is less than artful.

I will end this part of my ranting with an analogy [but don’t be surprised to hear more on this topic]. If you have not studied and then practiced writing poetry, would you expect to publish a book of poems simply because your marketing department asked you to? Designing a good quality research instrument probably takes less talent than being a good poet, but it’s close.

Wait, not so fast, we are not done, there is another mistake from which you cannot fully recover. A poor questionnaire design is one possible fatal mistake, but not the only one. Good solid sample development is also necessary. Here is another Research Axiom worth your consideration.

Research Axiom Two: You can never fully recover from a sample that lacks validity; and once again:

o No manipulation of the variables, regardless of how cleverly done
o No amount of analysis, regardless of how brilliant
o No degree of insightful interpretation, regardless of intellectual prowess
Nothing can save you from a poorly developed sample!

The value of sample development is also underappreciated, as are the skills related to creating a valid sample. Project managers, research analysts, and all those who lose sleep over the quality of the sample sources they have available and who work hard to provide the best possible sample for each research project they conduct, are worth their weight in gold.

With numerous challenges to good sample development always hovering over us, if the research team conducting the study does not pay close attention to this critically important task the chances of deriving useful results are likely to diminish rapidly. One of the worst situations to be in, is standing in front of a room full of executives and presenting the research implications when from off in the far corner an executive vice president (EVP) asks you, “Are you sure about that finding? Who were these respondents? They don’t appear to have any knowledge about the market or our products.”

If you can definitively reply, “We believe the respondents in this sample are qualified” and then give a crisp response about the quality control (QC) steps used to verify the validity of the sample, you have saved the day. If on the other hand, you hesitate and cannot defend the validity of the sample, you have lost your audience – there is nothing more they want to hear from you because in their minds the voice of the respondents do not reflect the people they are trying to reach – the day ends badly.

If you do not care about the quality of the research you conduct, well shame on you, but at least recognize that a sample of good quality is a necessity for self-preservation – enough said.