The head of the agency shared a new article with the Marketers Guild on the topic: Bots and real people, how not to get inaccurate data on an online survey at the output. https://www.marketologi.ru/news/consulting/4716/
Until recently, we did not see such a high demand for online research, but now, during the pandemic, the market has completely gone online, and we are with it.
When launching a project, be sure to check with the Agency if it is ready to provide an opportunity to call from your side the respondents who participated in the online survey. During which you can call and selectively screen respondents by asking them screening questions. In this case, the sample will be of a much better quality.
Be sure to discuss with the contractor at the start-up stage the possibility of providing from the respondents supporting documents or screenshots, business cards related to the research topic. At the exit, it is important to obtain, at least selectively, confirmation from the full array of respondents who participated. This will also improve the quality of the data you receive at the output.
Check with the Agency what methods are used to select respondents for your online survey. If this is a mono-method, for example - recruiting only with the help of a panel or recruiting only with the help of a database, this will indirectly narrow the representativeness of the sample. It is good if recruiting uses a set of recruiting techniques, for example: recruiting with a snowball, recruiting in social networks, recruiting using targeting, via a mobile application, using push notifications, recruiting on the agency's own database. Such diversity will significantly increase the representativeness of the sample, which is very important in a quantitative study.
When completing an online survey, the services where the online questionnaire is developed, usually read from which IP the respondent is taking the survey, from which city the survey is taking place. Here it is important to indicate to the agency that if the respondents will take a survey from Moscow, and indicate that they live in the region, such cases should not be more than 10% of the sample. Otherwise, there are risks that people who will simply indicate false data about the city in which they live will take part in the survey. It is better to mark this item right away so as not to pay for such a sample later.
Before starting, be sure to go through the online questionnaire yourself. Let your co-workers and/or colleagues do the same. Evaluate how you can squeeze it as much as possible. In the age of technology, most people have clip-like thinking, and they will not only be lazy to answer a long questionnaire but also psychologically difficult. Ideally, do not write long questions, but it is better to reduce those questions that seemed very long to you into a scale where the respondent can indicate not a detailed answer, but a range. For example: "What did you like most in cooperation with our company?" - answer options and indicate a scale for each answer, I liked the quality of services 9-10 answers (8), I liked the communication of colleagues with you (9), etc.
the respondent of the morgue mustn't fill out the online survey immediately but in several visits. This is especially important if the survey is lengthy. In our practice, we often came across the fact that an expert began to fill out a questionnaire, then got distracted, wanted to continue filling out, but from this place, it is impossible to continue filling out, we have to return to the beginning of the Questionnaire. So you can just lose a respondent.
Include a couple of questions - "traps", or a couple of questions where you need to specify a more detailed answer. For example - "Describe your main functions", or - "State your main responsibilities at work during the day."
It is good if the company to which you apply for the selection of respondents selects participants not only for quantitative types of research but also for qualitative ones. This means that in their database, which they will connect to recruiting, among other methods, some real people participate in surveys offline or by zoom.
It would be better if you compose the questions for the research yourself. Since no one knows better than you all the realities of the company in which you work, your product and its features.
Case from our work:
Not so long ago, a government organization contacted us with a request to recruit a thousand respondents - buyers of hair clippers - to conduct an online survey. I would like to note that there is a big difference between the search for confirmed respondents (these are the respondents with whom we contacted and screened, these are respondents who can selectively send receipts or screenshots from their personal account as confirmation of purchase) and NOT confirmed, where there is no contact information, there is no possibility at least selectively ask for proof of purchase, selectively call the participants and make sure they belong to the required target audience.
Self-screening respondents, receiving confirmation of purchase and simply interviewing respondents using a panel without contact information, these are two completely different approaches, and as a result, you will receive completely different data for each of the approaches.
If your goal is to conduct research of a high quality, it will be important for you to have the possibility of additional checks on the respondents described above. If the goal is different, then of course, checking the respondents will not be something mandatory for you. Also, now there are more frequent cases of participation in research not only of not high-quality respondents but also of ordinary bots.
Let's go back to the case of buyers of hair clippers. This company was invited by a third-party agency to conduct research using a panel. Sample - one thousand respondents for 116 thousand rubles. It is several times cheaper than targeting.
We, in turn, voiced this to the client and advised him to carry out additional checks on the respondents. But the client did not ask for confirmation from the research participants about the fact of making a purchase. The company announced that it has a panel itself, but now employees are struggling and they need partners, as the volume of work increased at the end of the year. We refused to cooperate, deciding that such work would not be of high quality. Our professional advice: if the main goal for you is to conduct an online survey of a high quality, with a good sample, do not hesitate to make sure that you are ready to provide the contacts of the respondents so that you can selectively call them and ask screening questions, request confirmation from respondents, checks and screenshots yourself.
Only in this case your product will shine on the market and the management of the customer company will be grateful to you.