Marketing Research is a BIG field. Marketing executives generally view it as the necessary evil: you need to do your research to get your product marketed to its target audience, but the time and cost of doing research can be intimidating. Add in the language marketing researchers use, you can quickly get enveloped in a very thick set of questions you didn’t know you had. Quantitative or qualitative? What sample size do you need to ensure statistical validity? Is your question best answered by VanWestendorp or Conjoint Analysis? Is 5% an acceptable margin of error?

There’s a lot at stake when conducting any form of customer research. But it all really boils down to this: in any research effort, you need to ask the right questions of the right people.

Sounds easy, right? The basics are easy. Most executives realize that they want to get input from people that will buy their product or service. They also understand that they need to ask questions that will help market a product to those people.

Execution, however, can be trickier.

Asking the Right Questions

Recently, I was reviewing a survey that asked a question aiming to measure the perceived challenges faced by professionals living and working in the United States that were raised speaking another language. The organization wanted to measure the extent these professionals felt they were at a disadvantage professionally. By the time I had a chance to review the questions, it was clear that the survey had been sliced and diced, edited, and re-worded several times by a variety of executives. After all of this, in the pre-launch version the question was worded:

To what degree do you feel being a non-native English speaker has impacted your professional success?

The survey went out to a very small test sample, and we were finding that many professionals were indicating the top of the “extremely” range. Digging in a bit further, thanks to an open-ended qualifying question we included as part of the test run, we found that many respondents rated in this way because they felt they were impacted in a *positive* way. Being bi-lingual has helped them.

Encouraging, yes—however the organization conducting the research already had plenty measures of the positive aspects of being bi-lingual. So, to get a better sense of negative perception, the question was changed to:

To what extent do you agree with the following statement:
I feel that I’m being promoted at a slower rate than my native English speaking colleagues.

In this way of wording, the client was not only able to focus on the frustration these professionals felt, but also provided a definition around the relatively vague term ‘success.’ The question, worded in this way, was now able to capture very specifically their perceptions of professional success, as measured by the rate they’re being promoted within their companies.

Asking the Right People

One of the most memorable incidents of not asking your question of the right set of people came after the 1948 presidential election. In a perfect storm of trying to keep up with the competition in news journalism, the Chicago Tribune assembled what information they had to call the election—such that they’d be able to report accurate news first thing in the morning. They relied on early returns and the pollsters, who had been conducting surveys throughout the campaign season to predict which presidential candidate was likely to win. Dewey, the Republican challenger, was estimated to be many points ahead of the incumbent Truman. So, when the Chicago Times looked at the returns that had come in as of press deadline, combined with what poll data they had, they called it: Dewey had defeated Truman.

Needless to say, Truman won that election—by a surprisingly large margin.

The problem in the information they had was threefold: timing (polls were conducted a good 2-3 weeks before the election), a large number of undecided voters (who, as it turns out, are more likely to vote for an incumbent), and a skewed sample.

The polling organizations at the time were using two methods to obtain lists of voters to sample: telephone records and automobile registries. In 1948, ownership of these goods were limited to the relatively affluent, who at the time skewed towards the Republican Party. So, the people being polled were those that were more likely than the average to vote for Dewey, the Republican candidate. Which, in turn, provided less-than-accurate information as a result of asking the wrong set of people.

Before your next research effort, whether you decide to do it in-house or hire an outside organization, remember these basics.

• Ask the right questions. Make sure the answer will help you and not lead to more questions. Be certain that there is no ambiguity or room for interpretation within the questions. Have one or more fresh set of eyes - people removed from the development of the survey - look at the questions. Better yet, field them to a set of test respondents. Ensure the aggregated responses will provide you the insight you need to make your decisions.
• Then, make sure you’re asking the right people. Identify the types of people you need to answer these questions, and either ask questions such that you can compare two groups of people, or only look to include those that fit your target. Consider age, gender, region, the demographics and psychographics you need (or, don’t need).

Sounds simple, but these basic concepts are worth careful development, review, and critique every time.