Does Length of Field Matter?

By Susan Frede, VP Research

One of the main advantages of using the Internet to collect marketing research data is speed.  With that speed, there may be the temptation to review survey data after just one day or even after a few hours.  But do short field times impact the quality of the data?  Generally how long should a survey be in field to capture a truly representative sample leading to more accurate data on which to base business decisions? 

To address these questions Lightspeed Research fielded a 25-cell test.  This research-on-research study involved five concepts, each with five separate samples, one launched each day of the week from Monday through Friday.  Respondents had seven days from the day the sample was launched to respond. 

All concepts were tested monadically using identical surveys.  The demographic universes were selected as appropriate for the product target market.  Key concept measures included Purchase Intent, Uniqueness, Value, Liking, Believability, Quantity, and Frequency.  Respondents also were asked a number of category habit and brand usage questions.

Summary of Findings

Day 1 and day 2 of the field have the largest proportion of completes (see Chart).  There are very few completes on days 5 through 7.  For this research, reminders were not sent to respondents.  However, other research on research has shown that reminders don’t increase response rate significantly or improve representativeness.

There are some differences in key measures depending on the day of field respondents answer the survey (see Table 1).  Specifically, day 7 responders tend to give lower key measure scores.  Purchase intent and key measure scores tend to be very consistent on days 1 thru 3. However, the earlier responders (especially day 1 and day 2 responders and to a lesser extent, day 3 responders) tend to be slightly different demographically.  Earlier responders tend to be older and part of smaller households with no children and lower income.  They are also more likely to be Caucasian and retired.  Later responders (day 4-7) tend to be those who may have busier lives (i.e. employed full time, children/families).  These demographic differences suggest that field should remain open at least four days in order to get a representative sample. 

Purchase intent and key measures are consistent with the seven-day field period data when looking at different cumulative totals.  Table 2 shows combined data for the five concepts, but even for individual concepts differences are not observed.  There are no meaningful differences on demographics and habits across these four groups.  This data coupled with completion rates, suggests a four-day field period is sufficient to get stable results.  

Keep Surveys in Field At Least Four Days

Based on this research on research, Lightspeed Research recommends a minimum of a four-day field period.  This leads to more representative samples and ultimately more accurate data on which to base business decisions. 

Susan Frede is the VP of Research at Lightspeed Research. She has worked in the research field for 24 years, has published numerous research-on-research papers and is a well-respected speaker at key industry events. Some of the topics she has recently explored include questionnaire length, best practices for online research, suspicious and professional respondents and data stability.

You can contact Susan at sfrede@lightspeedresearch.com.