Use of Dynamic Grids

By Susan Frede, Lightspeed Research and Steve McHugh, Millward Brown

Grid or matrix questions (questions presented as a series of rows and columns) have become quite popular in online surveys because they are an efficient way to ask a series of repetitive questions. But respondents can become disengaged with grid questions and may not give careful consideration to each item. The Kantar InTouch team has developed an alternative to grid questions called Dynamic Grids.

Dynamic Grids present statements, logos, brand names, or other images on 3D cards above the answer choices. As each response is given, the top card flips to the left and the next card flips in from the right. Dynamic Grids are designed to:

• Remove the daunting experience of conventional matrix grids,
• Ensure respondents have a sense of progress and task scale,
• Offer a more interactive, appealing environment,
• Eliminate the delays of page-by-page display.

Previous research-on-research compared Dynamic Grids to a page-by-page display, which typically shows the respondent one statement per page. This research showed that Dynamic Grids are an effective tool, because they can reduce time required to answer and provide higher satisfaction than page-by-page displays. Combined with other InTouch survey best practices they are likely to positively impact survey engagement and completion rates.

Research Design

In March/April 2011 the Kantar InTouch Team fielded research-on-research to test Dynamic Grids against conventional matrix grids and to establish further evidence of the benefits of Dynamic Grids. Specifically, the research examined dropout rates, survey satisfaction, response time, straight-lining, brand endorsements and data variance.

The research also compared blank cards to faded adjacent cards in the Dynamic Grids. With the blank adjacent cards only the text on the current card being evaluated can be seen while with faded cards grayed out text can be seen on adjacent cards.

The research was conducted in collaboration with the University of Michigan, through a sponsorship by Millward Brown. The survey was scripted and fielded in the US by Millward Brown North America, with sample contributed by Lightspeed Research.

This research utilizes a test-control design to answer the research questions. The grid style and the number of grid items vary. This results in a 9-cell design:

Each cell has approximately 200 completes. The number of statements was chosen on the basis of an intuitive, expert assessment of the likely inflection points of the benefits of Dynamic Grids. The statements are a list of imagery items and respondents are asked to choose the statements they associate with brands they are aware of.

Weighting has been applied to each of the 9 cells to provide consistent demographic and category interest levels.

Blank versus Faded Dynamic Grids

There are advantages to using Dynamic Grids with blank adjacent cards over faded adjacent cards. The blank adjacent cards have lower dropout levels and better respondent satisfaction scores with 6 and 14 statements. Neither option is good with 28 statements, since with a large number of statements dropout rates are high regardless of how the statements are presented. The rest of the analysis, therefore, focuses on Dynamic Grids with blank adjacent cards.

Conventional Grids versus Blank Dynamic Grids

The number of statements has a clear impact on the comparison of conventional and blank Dynamic Grids. Specifically:

• With 14 statements, there is a large reduction in dropouts and respondent satisfaction scores are significantly higher than conventional matrix grids when dropout rates are factored in.
• With 6 statements, there appears to be slightly fewer respondent engagement benefits for Dynamic Grids. Dropout rates are nearly identical while respondent satisfaction is significantly higher than conventional grids.
• Both Conventional and Dynamic Grids produce extremely unacceptable dropout rates of nearly 20% at the image section alone with 28 statements. This provides clear evidence that the number of statements must be limited regardless of format.

With 14 and 28 statements, there are significantly quicker response times with Dynamic Grids despite having slightly fewer straight-liners. This suggests that data quality is better when using Dynamic Grids.

Although not statistically significant, the combination of greater per brand endorsements and greater variance suggests that there is better quality data when using Dynamic Grids for 14 statements. With 6 statements, variance is even greater while with 28 statements variance decreases for Dynamic Grids.

Conclusions and Recommendations

This research shows that there are advantages of using Dynamic Grids over conventional grids, but advantages are dependent on the number of statements in the grid. In this research, the sweet spot is clearly the mid-range number of statements (14). However, the speculation is that the sweet spot is likely 10-20 statements and the recommendations reflect this.

• Dynamic grids with blank adjacent cards should be used instead of conventional matrix grids where the number of statements to be rated is in the range of 10 to 20. This results in lower dropout rates, higher survey satisfaction, and less time to respond with no decrease in data quality.
• Where a question has fewer than 10 statements, either a matrix or a Dynamic Grid can be used without significantly affecting the experience. With 6 statements, survey satisfaction is higher and variance greater with Dynamic Grids. However, other measures examined are flat.
• Questions with greater than 20 statements should be avoided, because regardless of format there are significant dropout rates.

It is important to note that changing from conventional matrix to Dynamic Grids on existing studies will likely result in different data, though it will be better quality data.

About Kantar InTouch & Authors

InTouch is Kantar’s respondent engagement program, aiming to continually reinvent the research experience in the face of the ever-changing wider digital experience of consumers.  By ensuring continued engagement, the objective is to secure high quality data, lower dropout rates, and healthy panels.  InTouch focuses on three areas:  Web Surveys, Mobile, and Social Media.  The goal with Web Surveys is to provide tools, best practices and research on research on questionnaire language, questionnaire design, interactivity and look/feel.

Susan Frede is the VP of Research at Lightspeed Research. She has worked in the research field for 24 years, has published numerous research-on-research papers and is a well-respected speaker at key industry events. Some of the topics she has recently explored include questionnaire length, best practices for online research, suspicious and professional respondents and data stability. You can contact Susan at sfrede@lightspeedresearch.com..

Steve McHugh is the SVP of Research Planning at Millward Brown. He has worked in the research field for 31 years in both client servicing and operational functions. He has been a speaker and Lead Chairperson at key industry events. Some of the topics he has recently explored include drivers of and solutions to online dropouts, sampling guidelines, data trend stability, and development of an education and recognition initiative for clients and internal staff called ‘Respect our Respondents’. You can contact Steve at steve.mchugh@millwardbrown.com

Team members were:
Kantar: Brad Brown, Zoe Dowling, Susan Frede, Samantha Hare, Alex Johnson, Jake Kolb, Steve McHugh, David Starner, Karen Sychowski, Jennifer Ward
University of Michigan: Fred Conrad, Wayne McCullough, Raphael Nishimura

Copyright 2011 Kantar. Proprietary Information. All Rights Reserved.

Back to Home