Frequently Asked Questions

A focus group is qualitative research.  It’s a handful of people – often eight to ten – who participate in a guided discussion lead by an experienced moderator.  During a focus group, participants express their opinions on myriad topics involving, for example, products, services, advertisements, elections, and community issues.

Focus group sessions are commonly held in research facilities equipped with one-way mirrors and recording equipment.  Typically, these sessions are observed by the project’s sponsor.

In a word, yes.  There is, however, much confusion between opinion research and telemarketing.  The latter involves sales, often of products and services.  Telemarketers ask for money and make promises.

Opinion research is very different.  Companies in this field are interested only in the preferences, perceptions, and attitudes of the people they contact.  They do not sell anything and never ask for money.

If you’d like to receive fewer telemarketing calls,  contact the National Do Not Call Registry:

http://www.donotcall.gov 

Sampling error is the calculated statistical imprecision due to gathering data from a sample of respondents instead of the entire population. It provides an estimate of how the results of a survey may differ, due only to chance, from what would have been found had the entire population been interviewed.

As the size of a sample increases, sampling error decreases.  The marginal utility of larger samples, however, diminishes rapidly.  Consider this: for percentages near 50, sampling error declines from ±9.8 percentage points to ±4.4 percentage points as the sample increases from 100 interviews to 500 interviews (at the 95 percent level of confidence).  Yet, as the sample increases from 500 interviews to 1,000 interviews, the relevant error drops much less, from ±4.4 percentage points to ±3.1 percentage points.

Though it may seem counterintuitive, sampling error varies as a function of sample size, not the size of the population from which the sample is drawn. For example, the sampling error for 500 interviews is ±4.4 percentage points when the population in question is adults in Columbus, Ohio; adults in all of Ohio; or, for that matter, adults from Maine to California.

With Random-Digit Dialing, computers generate the telephone numbers used in telephone surveys.  RDD procedures commonly begin with listed telephone numbers (i.e., the numbers published in telephone directories), which are altered by randomly changing the last two digits. Random Digit sampling frames tend to produce more accurate results than do telephone directories because the former may include unlisted numbers and households established since the most recent directory was published.

Statistical significance refers to the probability that chance alone is responsible for the observed difference between two groups of survey respondents.  When samples are large, very small differences, often with no practical consequence, often are statistically significant. Similarly, when samples are small, a difference with a practical consequence may not be statistically significant.  Stated differently, observed differences may be real, but unimportant, while important differences may not be apparent because small samples obscure them.

The best tool for distinguishing between statistical significance and practical importance is the experience accumulated by research professionals.

Interviews conducted for market research:

• Are exempt from the National Do Not Call Registry

• Never, ever involve sales (interviews that do involve sales are telemarketing)

• Are voluntary – a potential respondent can refuse to participate at any time

• Unless otherwise stated, are confidential, which means that a respondent’s name is not linked to the opinions expressed during the interview

If the results of a survey are released to the public, the following questions must be addressed:

• Who conducted the survey?

• Who paid for the survey?

• When was the survey conducted?

• How was the survey conducted – by telephone, mail, online, or some other way?

• How many people were interviewed and how were they selected?

• Who was interviewed?

Scientific surveys share two characteristics:

• Respondents are chosen from a defined population according to explicit criteria developed to maximize representativeness and minimize self-selection

• The questions are worded without bias – in a way that does not lead respondents toward a particular answer

Not necessarily.  If a sample is not representative (i.e., not drawn randomly), it will not produce accurate results regardless of how many people participate in the research.  A frequently-cited example of this principle occurred during the 1936 presidential election.  A magazine, Literary Digest, sent out 10 million post cards asking people how they would vote.  Approximately 2.3 million people responded, 57% of whom favored Republican Al Landon.  Only 43% favored Democrat Franklin Roosevelt, who won in a landslide.  This poll was inaccurate despite the massive number of respondents because the post cards were sent primarily to two groups of voters: those with telephones and those with automobiles – most of whom, in that era, were Republican.  In short, though millions participated in the survey, the sample was biased, a flawed reflection of voter demographics.

A survey can do this in much the same way that a chef judges the flavor of soup as it cooks on a stove: she stirs the pot and tastes a single spoonful.  Similarly, a doctor can draw highly accurate conclusions from a small test tube of blood.

Surveys operate along the same principle: If the respondents who participate are selected at random from a defined population (the equivalent of stirring), a small number of interviews can yield reliable estimates of perceptions or attitudes, even if the defined population is composed of millions.

Think of it this way: If a chef is cooking tomato soup in a five-quart pan and chicken soup in a one-quart pan, she wouldn’t use a larger spoon to taste-test the tomato soup; she’d use the same size spoon for tasting both soups.

Opinion research provides those who participate a voice when decisions are made about the products, services, or policies that affect our lives.  Add your “two cents” and shape the future.

Each year, thousands of studies are conducted – at considerable expense. These expenditures strongly suggests that sponsors of this research truly value the opinions of the people they serve.  Though you may be skeptical, consider this: For most companies, it is much more profitable to ask consumers what products or services they are likely to buy, and offer them, than it is to foist on consumers products or services of little interest.