Friday, June 3, 2016
2:30 - 2:50 PM
Ted Talks - Marquette

Vive la Difference - Different is not wrong

Il y a des options sans fin à considérer lors de l'écriture d'une question, et même la plus petite des décisions peut avoir un impact majeur. Il n'y a pas de bonne ou mauvaise façon d'écrire une question, mais il y a une opinion éclairée ou mal informée. Cette présentation fournira une abondance d'informations pour vous aider à décider quelle question vous donnera les meilleurs résultats lors de votre prochain projet de recherche.

There are unending options to consider when writing a survey question, and even the smallest of decisions can have a major impact. There is no right or wrong way to write a question but there is an informed or uninformed opinion. This presentation will provide an abundance of information to help you decide which question format will get you the most actionable results out of your next research project. 

Interest Statement:
Most experienced survey writers have developed their craft to a point where they know the standard, clear, and concise way to write most questions. There is a standard way to write the Purchase Intent question, and there is a standard way to write a 5 point Likert scale question. Indeed, it’s easy to get in a rut and never stray from our preferred wording and formats. However, we must remember that there are many other equally valid ways to word or format every question on a survey.

Clients are great at pulling researchers out of their box and asking, or demanding, that a question be written in a different way. Other times, clients simply want to know understand what the best way to phrase a specific question is. Unfortunately, clients are often not satisfied with a response that rests solely on experience. Being part of the research industry means that clients know enough to ask to see the data for themselves.

This presentation will provide solid data that researchers can use with their clients so they can respond to questions and demands with fact-based opinions and authority.

Plus, these speakers are fun and will be sure to entertain the audience!

Learning Outcomes:
This presentation will focus on three main outcomes.

  1. How results differ based on the way a question is formatted. For instance:
    1. How do rank orders change when you ask people choose just one versus just two versus just three items from a list?
    2. How do box scores change when frequency distributions have only 3, 6, or 9 answer options?
    3. How do box scores change when an importance scale is anchored with “Very important” versus “Extremely important”
    4. How do box scores change when you ask about the importance of something using a one stage 5 point scale versus a two stage scale, e.g., Stage 1 offers answers for “Important”, “Neutral”, “Unimportant”. Stage 2 offers answers for “Very Strong opinion” and “Somewhat Strong opinion”
    5. How do box scores change when questions include versus exclude adjectives, e.g., “There are too many commercials” versus “There are far too many commercials.”
  2. Whether the differences are large enough to be practically significant. We know that all of these formatting differences will create differences in the results. However, simply finding a statistically significant difference is not sufficient. Any differences that result must be of significant size to be actionable. We will discuss which differences are small enough to ignore and which differences should cause a researcher to carefully consider which question to apply.
  3. Which types of questions are better suited to solve certain problems?  Every question has an ideal situation. While shorter questions and shorter answers are always best, some question formats are better suited for smaller mobile devices or certain survey topics, e.g., sensitive topics. We will discuss the pros and cons of using the various formats.

Mélanie Drouin
Director Client Services, Research Now

Suivant le flot de l’évolution des différentes méthodologies de collectes de données, Mélanie a commencé sa carrière en travaillant sur les études porte à porte et téléphoniques, se tournant ensuite vers la création et gestion de panels en ligne, chef des opérations en agence de recherche et maintenant directrice services à la clientèle chez Research Now.  Tout au cours de sa carrière, Mélanie s’est efforcé d'aller au-delà des meilleures pratiques de l'industrie en mettant toujours la priorité sur la qualité des données.

 


Annie Pettit
Chief Research Officer, Peanut Labs

Annie Pettit, PhD is the Chief Research Officer at Peanut Labs, a company that specializes in self-serve sample, surveys, and polling. She is also Vice President, Research Standards at Research Now. Annie specializes in data quality, sampling and survey design, and social listening. She won the MRIA Award of Outstanding Merit in 2014, Best Methodological Paper at ESOMAR in 2013, and the 2011 AMA David K. Hardin Award.