In the past 15 years, with the transition from telephone and F2F data collection to online, driven in many occasions by the search for cost reductions and quick results, we’ve seen the birth of countless “Online Panels”, providing cheap sample and online field services. Unfortunately, the digital race and the proliferation of companies offering online sample services, without minimum level of quality control and market research knowledge and expertise, have generated mistrust in regards to the quality and reputation of the online panel industry. The attractive offer of fast and cheap services summed with the poor respondent experience provided in the majority of the online surveys created, have produced major problems to the market research industry: low survey response rates, a decrease in people’s interest to participate in online panels and surveys, thus compromising the quality of the collected data.
The discussion about this issue has been a hot topic on Greenbook’s blog- one of the most popular market research blogs in the USA, with many posts and interactions covering the matter:
Heal Thyself! When Will Market Research Get Serious About Sample Quality?
Why is nobody here addressing the elephant in the room? It’s not just sample quality. It’s survey quality.
An Open Discussion on the Impact of Respondent Sourcing
In mature markets, such as in the US and Europe, where online panels have been extensively used for many years, the industry is already developing solutions to the online sample quality issue, such as programmatic sample, reduction of survey length, improvement on survey experience, mobile optimization, etc. However, in regions such as Latin America, where online research is still in early stages, we’ve been watching the arrival of some “Online Panel” companies, coming from other parts of the world, replicating their business models with extremely low and attractive prices. The main problem here is what is behind those low costs: professional respondents who complete a survey after another to get an extra-income, survey platforms attracting “prize hunters” or people who want to make easy money on the Internet, among other things.
There are great online panel companies that have been trying to maintain quality and offering competitive prices in the market. However, it will be hard to overcome the “quick & dirty” pressure, unless end clients, market research agencies and online panel companies work together in pursuit of a common goal: to protect both the quality and the reputation of the online sample industry. Here are some ideas that I believe can help us avoid the chaos that’s going on out there:
- Experience: Let’s prioritize the respondents’ experience. We need to create surveys that are pleasant to participate, in order to engage the respondents. Remember the Internet is synonym of entertainment, and while answering a survey, the user expects to have such engaging experience as the ones they have in social networks, online games, mobile apps, etc.
- Re-size & Optimization : We need to reduce the length of the questionnaires. An online survey shouldn’t be longer than 10-15 minutes. Ideally, respondents should spend no more than 5-10 minutes to respond, and the surveys should run in any type of device (“device agnostic”).
- Incentives: if people will spend their time to complete a survey, let’s compensate them with a good incentive to avoid increase in the drop-out rates. A “good” incentive non necessarily should be a monetary one. Sometimes a good incentive can be the access to an exclusive content, feedback on the survey results, or even the survey experience itself with the usage of gamification techniques and interactive resources. Online panels with loyalty programs and engagement through gamification systems, when well implemented, tend to offer high-quality sample services.
- Transparency: End clients, market research institutes and online panels should be transparent on the project goals and limitations. Many times, it will be necessary to involve more than one panel company in order to reach the desired research goals. In trackers, for example, it is necessary to use multiple panels to guarantee sample consistency over time.
- Quality: Ask for information on the quality control processes and methods used by the online panels you select: to work with:
- Recruitment methods: Multiple recruitment sources usually lead to the creation of quality online panels covering a relevant portion of the total universe of internet users of a specific country;
- Main purpose of the panel: What main message is the panel using to attract participants? (many low quality panels use messages such as “Make money by answering surveys”);
- Invitation: Request a copy of the invitation the panel will send to your potential respondents, so you can understand the filters that are in place and find out what is being offered to participants.
- Invitation Delivery Method: Does the panel only work with random sample selection? Are invitations sent using balanced out-go? Do they send email blast to the entire database? Or do they provide a list of available surveys on a website without any control regarding the amount of surveys the same user can complete?
- Incentives: ask for information, discuss about it and define a plan together with the panel provider and the market research institute in order to establish the types of incentives you will offer to your study. Some companies use multiple types of incentives, adapting them in accordance to the audience and type of project. It is also possible to work with emotional incentives, gamification and other techniques to encourage people to participate, without focusing on a monetary incentive.
ESOMAR has created a document called “28 Questions to help buyers of online samples”. Request that document to your online panel suppliers and check if they are really doing what they promise.
For 15 years, I have worked with online research and building online panels in Latin America. I may have created one of the first online panels in Brazil (www.ecglobal.com), nowadays with more than 1.2 million engaged users in all Latin America. I have seen everything in the online panel industry, from the first generations of online panels to the current mobile, social and gamified communities, and I still believe that with some fixes and improvements, we can conduct online research with quality and trust.
buy a college term paper online